01
background
During this time, I was working on a code project, after the code was released, it supports editing the code and can be previewed in real time on the right side, its UI is as follows, after researching several open source (react-playground, react-live, minisandbox) online editing and running react code libraries, I will share what I want to share with you.
02
react-playground
How to use:
const files = {
'App.tsx': `import {title} from './const'
function App() {
return <h1>this is {title}</h1>
}
export default App
`,
'const.ts': {
code: 'export const title = "demo2"',
},
}
<PlaygroundSandbox
width={700}
height={400}
files={files}
theme='dark'
/> 2.1 Editor Section
开源的使用最多的编辑器：Monaco Editor、Ace 和 Code Mirror。
Monaco Editor has a rich ecosystem and powerful functions, and it is still the same editor as vscode, with code hints and other functions, and it is development-friendly.
react-playgound是用的就是 @monaco-editor/react
<MonacoEditor
width='600px'
height='800px'
onChange={(newCode) => setCode(newCode)}
defaultValue={code}
defaultLanguage='javascript'
/>
(It's relatively simple, and it's not the focus of this explanation)
2.2 Code Preview
If there are global styles and global variables in the project, it will affect the real-time code running effect, so the runtime needs a clean environment to run the code, and the solution chosen by react-playground is iframe.
- The browser runs JSX
The browser doesn't know JSX, so we need to convert JSX to JS, which is usually done with Babel, and react-playground with @babel/standalone (the browser version of Babel)
import { transform } from '@babel/standalone'
const babelTransform = (code: string) => {
return transform(code, {
presets: ['react', 'typescript'],
}).code
}
const compliedCode = babelTransform(jsxCode)
After compilation, our code becomes
- Who handles the task of compiling the code
Leave it to the main thread? It may cause the page to block and feel stuck when editing, so a good solution is to open a new web worker to handle the code compilation.
// compiler.worker.ts
self.addEventListener('message', async ({ data }) => {
// 2. 接收到源代码后编译
const compiledCode = babelTransform(jsxCode)
//3. 编译完成后，发送数据给index.tsx
self.postMessage(compiledCode)
})
// index.tsx
compilerRef.current = new CompilerWorker()
useEffect(()=>{
// 1. 源代码变更后，发送给worker去编译
compilerRef.current?.postMessage(jsxCode)
},[jsxCode])
compilerRef.current.addEventListener('message', ({ data }) => {
// 4. 接收到web worker编译后的代码，发送到iframe中
iframeRef.current?.contentWindow?.postMessage(data)
})
// iframe.html
window.addEventListener('message', ({data}) => {
// 6. 接收到编译后的代码执行
// 代码插入script标签中或者转为临时文件地址赋值给script标签
}) Third-party module packages can be introduced
Since the browser natively supports ESM, we choose to use third-party dependencies in ESM format. There are some non-ESM packages, and you can find the corresponding ESM file address in esm.sh.
The module address is then mapped via importmap.
importmap 允许您在浏览器环境中指定模块路径映射，import "react" 时，会按importmap 中指定的 URL 来加载 react 模块。
<script type="importmap">
{
"imports": {
"react": "https://esm.sh/[email protected]",
"react-dom/client": "https://esm.sh/[email protected]",
}
}
</script>
Introduce local modules
We execute a line of code in our local project import a from './A', which is usually handled by build tools and module packagers (e.g. webpack, vite, etc.)
- ESM module
Rely on the tree structure generated at compile time to find the corresponding path. Will look for A.js, A.jsx, A.ts... It is then packaged into a file or a stand-alone file.
- commonjs module
Node.js provides a file system module that allows you to access and manipulate the local file system in a server-side environment.
So in the browser, how do we parse the module corresponding to import a from './A'?
Most browsers support esm modules, which support URL files, so we can replace './A' with url addresses, and the browser can be used
import a from './A' --> import a from 'http://xxx/A' Since we don't have a service to serve the file, you can use URL.createObjectURL to generate a temporary URL file.
// 把代码编译后转换成url文件
const A = URL.createObjectURL(
new Blob([babelTransform(code)], {
type: 'application/javascript'
})
)
// 转换后的地址
// blob:https://localhost:3000/e4ef352f-1c5f-414e-8009-33514b300842
// 替换 './A'
import a from 'blob:https://localhost:3000/e4ef352f-1c5f-414e-8009-33514b300842'
According to this idea, when introducing this file, we only need to analyze the import statement and replace the corresponding module code with the local URL address at compile time.
How? With the babel plugin.
The babel plug-in adds, removes, and modifies the AST during the transform phase:
const transformImportSourcePlugin: PluginObj = {
visitor: {
ImportDeclaration(path) {
path.node.source.value = url;
}
},
}
const res = transform(code, {
presets: ['react', 'typescript'],
filename: 'test.ts',
plugins: [transformImportSourcePlugin]
}
path.node.source.value就对应ast树的如下值
至此，react-playground 的原理已经分析清楚了。 分别用 Blob + URL.createBlobURL 和 import maps 来做。
Introduce a style file
(() => {
let stylesheet = document.getElementById('style');
if (!stylesheet) {
stylesheet = document.createElement('style')
stylesheet.setAttribute('id', 'style')
document.head.appendChild(stylesheet)
}
const styles = document.createTextNode(`${css}`)
stylesheet.innerHTML = ''
stylesheet.appendChild(styles)
})() 03
react-live
Let's take a look at the react-live solution, 4.2k star on github
相比 react-playground，有两点不同：
1) The compiled js code is not made in the form of script tag insertion, but becomes executable code through new function/eval.
2) The handling of dependencies is different.
function evalCode(code: string, scope: Record<string, any>) {
const scopeKeys = Object.keys(scope)
const scopeValues = Object.values(scope)
// eslint-disable-next-line no-new-func
return new Function(...scopeKeys, code)(...scopeValues)
}
function generateNode(props) {
const { code = '', scope = {} } = props、
// 删除末尾分号，因为下边会在 code 外包装一个 return (code) 的操作，有分号会导致语法错误
const codeTrimmed = code.trim().replace(/;$/, '')
const opts = { transforms: ['jsx', 'imports'] as Transform[] }
// 前边补上一个 return，以便下边 evalCode 后能正确拿到生成的组件
const transformed = transform(`return (${codeTrimmed})`, opts).code.trim()
// 编译后只是一个字符串，我们通过 evalCode 函数将它变成可执行代码
return evalCode(transformed, { React, ...scope })
}
Let's focus on the handling of dependencies
1）import-map方案
- Pros: Simple and easy to use.
- Shortcoming:
- Only ESM module addresses are supported;
- If a package A imports package B, and only defines the mapping address of package A in the importmap, when the package A logic is actually run, an error such as Uncaught TypeError: Failed to resolve module specifier "B" will be reported, so for packages with complex dependencies, the importmap scheme is not friendly enough, and the dependencies need to be analyzed, and then each of them defines the mapping in the importmap.
2) scope scheme
What is actually passed in is a context object that defines all the external resources that the current code can access. react-live matches identifiers in your code to scope to find definitions for those identifiers.
- Pros: Variables and components are provided directly, simplifying dependency management
- Disadvantages: Complex module dependencies cannot be automatically resolved, and dependencies need to be manually managed.
This paper introduces and compares react-playground and react-live, and their main use cases are in "code snippet real-time preview". For example, component document instances, online debugging code, etc. If there is a more powerful front-end editor and real-time preview solution, the answer is yes.
比如codesandbox、stackblitz。
04
codesandbox
codesandbox open-sourced the @codesandbox/sandpack-react library, which provides a lot of codesandbox modules out of the box.
The panels corresponding to CodesanBox are as follows
Each component interacts with the SandackPreview rendered iframe via postMessage.
CodeSandbox runs in two environments:
1) Pure front-end projects (such as React projects, pure JS projects) use Browser Sandbox
2) The server-side runtime environment (such as Docker project, full-stack framework project) needs to use Cloud Sandbox (he uses MicroVM at the bottom)
For the browser sandbox, since there is no Node environment on the browser side, CodeSandbox implements a simplified version of webpack that can run on the browser side. It's called mini webpack.
4.1 Principle
There are many articles on how CodeSandbox works. I won't focus on it here
CodeSandbox build project process
The build process consists of three steps:
- Packager--npm package loading stage: Download the npm package and recursively find all the referenced files, and then provide them to the next stage for compilation
- Transpilation--Compilation phase: Compile all code, build module dependency graph
- Evaluation--Execution Phase: Use eval to run the compiled code and implement the project preview
Packager--npm包加载阶段
codesandbox is inspired by the WebpackDllPlugin. DllPlugin packages all dependencies into a single dll file (which stores the pre-packaged modules) and creates a manifest file that describes the dll's metadata (describing module mappings).
{
"name": "dll_bundle",
"content": {
"./node_modules/fbjs/lib/emptyFunction.js": 0,
"./node_modules/fbjs/lib/invariant.js": 1,
"./node_modules/fbjs/lib/warning.js": 2,
"./node_modules/fbjs/lib/react.development.js": 3,
"..."
Each path is mapped with a module ID. If I want to bring in React, I just need to call dll_bundle(3) and then I get React.
Based on this idea, CodeSandbox has built its own online packaging service, and unlike WebpackDllPlugin, CodeSandbox pre-builds the Manifest file on the server side.
This package is called, packager, which is based on the service provided by the express framework, and its process is, for example, I now have a react package version 16.8.0, first of all, the express framework receives the package name and package version in the request, react, 16.8.0. Then download react and react's dependencies to disk through yarn, find the npm package entry file by reading the module, main and other fields in the package.json file of the npm package, and then parse all the require statements in the AST, add the content of the require file to the manifest file, and recursively execute the steps just now, and finally form a dependency graph.
This achieves the purpose of transferring the contents of the npm package file to the manifest.json, and also achieves the purpose of eliminating redundant files in the npm module. Finally, it is returned to the sandbox for compilation.
manifest 长这个样子：
Transpilation--编译阶段
First, download the manifest file corresponding to the npm dependency package from the Packager service, and then start compiling the project with the entry file of the front-end project, and parse the AST recursively to compile the required file to form a dependency graph. The principle is the same as webpack.
Evaluation--执行阶段
The compiled module corresponding to the project entry file begins, and eval is recursively invoked to execute all referenced modules.
To sum up: Browser sandbox pages compile and execute code using the built-in mini webpack and other tools (such as babel).
The information about the compilation and execution of the code is also passed back to each required module through the communication protocol. For example, the console module can print messages based on information whose type is console.
05
stackblitz
StackBlitz 核心技术是Webcontainers。
5.1 什么是webcontainers？
WebContainer is a revolutionary technology developed by the StackBlitz team that allows us to run a full Node.js environment in the browser. This allows us to run modern JavaScript development tools like Webpack, Vite, and various npm packages in the browser without having to install any local dependencies.
In the past, if we wanted to localize in the browser, the mainstream idea was to use Electron as an example, using the browser's kernel engine with the node environment to achieve the localization of web applications, but with the development of wasm's technology, the computing power of the browser has been comparable to the system level, so it can support the feasibility of porting node to the browser. That technology is called webcontainers.
Applications that previously required a cloud virtual machine to execute user code can now run entirely in the browser in WebContainers.
In short: webContainer is a miniature operating system that can run in browser pages, providing a file system, the ability to run processes, and built-in package managers such as nodejs, npm/yarn/pnpm, etc.
Key features:
- Ability to run node.js and its toolchain (e.g. webpack, vite, etc.) in the browser
- Flexible: With WebContainers support, the coding experience will be greatly improved
- Secure: All content runs in the browser page, which is naturally isolated and very secure
- Fast: Spin up the entire development environment in milliseconds.
- Always open source and free
- Zero latency. Works offline.
From code writing to compilation and packaging, hot updates are completely closed in the browser, and all actions can be completed as long as a browser is opened.
5.2 了解 WebAssembly
WebAssembly is a new type of code that runs in modern web browsers and offers new performance features and effects. It is not designed to write code by hand, but to provide an efficient compilation target for low-level source languages such as C, C++, and Rust.
WebAssembly is a low-level assembly-like language. It has a compact binary format that allows it to run at speeds close to native performance.
WebAssembly is the core of WebContainers' ability to run in the browser. By compiling Node.js to WebAssembly, WebContainers is able to provide a complete development environment in the browser, including the ability to run Node.js code, install and manage npm packages, and more.
5.3 Example: Run a simple Node.js app in your browser
Step 1: Create a WebContainer instance and start it
import { WebContainer } from '@webcontainer/api';
// 启动 WebContainer 实例
const webContainerInstance = await WebContainer.boot();
Step 2: Create a file system and mount it to the webcontainer instance
await webContainerInstance.mount(projectFiles);
Step 3: Download the dependencies programmatically
const install = await webContainerInstance.spawn('npm', ['i']);
await install.exit;
Step 4: Start the service
await webContainerInstance.spawn('npm', ['run', 'dev']);
In this way, we have developed a simple node application using webcontainers.
We can experience the case of WebContainers from Stackblitz.
5.4 Principles
In a nutshell, in a service worker, implement a JS runtime with wasm.
Leverage WebAssembly to implement an API that doesn't exist in a browser environment (such as Node.js's readFile) and inject its functionality into a global object for JavaScript to use.
5.5 Usage Scenarios & Support
1）webIDE，如stackblitz；
2) Bug recurrence clips;
3. Experimental function, no need to start a new project locally, time-consuming and laborious.
WebContainers works out of the box on Chromium-based desktop browsers and is also supported on Safari 16.4 and Firefox.
Reference Links
[01] React Playground在线Dome
https://fewismuch.github.io/react-playground/#eNqNVd1u2zYUfhVOQWEbi37sxmmmJUG7bsB60Q1oL3Yx74ImjyQ2FCmQlGPX8LvvkJJsOXWBQjZAfec7
[02] @babel/Standalone( babel的浏览器版)
https://link.juejin.cn/?target=https%3A%2F%2Fwww.npmjs.com%2Fpackage%2F%40babel%2Fstandalone
[03] @codesandbox/sandpack-react库
https://www.npmjs.com/package/@codesandbox/sandpack-react
[04] express
https://expressjs.com/
[05] mini webpack
https://juejin.cn/post/6844903880652750862?from=search-suggest
[06] Works offline
https://mmbiz.qpic.cn/mmbiz_gif/meG6Vo0MevhvJfkvEYBMuCrCvjbDn6d37PcBH4tcP4c62kOhUfLHLQSOD8lhv8HR7SQDF9KA2JoIzpJkuNqgUQ/640?wx_fmt=gif&tp=webp&wxfrom=5&wx_lazy=1
[07] Stackblitz
https://stackblitz.com/edit/stackblitz-webcontainer-api-starter-4vmxqn?file=main.js
Author: Ruo Sheng
Source-WeChat public account: Ali Technology
Source: https://mp.weixin.qq.com/s/VEV6RmOdZpAg7RBQzDmfUA