In MS-DOS days programmers were limited to 640 kB of code. When their programs became bigger than that they had to implement tricky code to load / unload code chunks on demand.
In web apps we’re not limited by size but it’s still a good idea to strive for the smallest possible JavaScript bundle. We can do it by extracting rarely used pieces of functionality into separate JavaScript chunks and import them lazily. This makes the app faster to load, faster to parse and run.
For example, in
Edna I use
markdown-it and
highlight.js library only in a certain scenario. By putting it in it’s separate chunk, I save almost 1 MB of uncompressed JavaScript in main bundle. Faster to download, faster to run.
../dist/assets/markdownit-hljs-DbctGXX9.js 1,087.33 kB │ gzip: 358.42 kB
To split in chunks you configure rollup in vite.config.js:
function manualChunks(id) {
// partition files into chunks
}
export default defineConfig({
build: {
rollupOptions: {
output: {
manualChunks: manualChunks,
}
}
}
}
manualChunks() functions takes a path of the file (don’t know why everyone calls it an id). If you return a string for a given path, you tell rollup to bundle that file in a given chunk. If you return nothing (i.e. undefined) rollup will decide on how to chunk automatically, most likely putting everything into a single chunk
It gets called for .css files, .js files and probably others.
Here’s my hard won wisdom:
console.log(id) when working on manualChunks() to see what files are processed- chunk specific modules from
node_modules to be lazy loaded
- everything else from
node_modules goes into vendor chunk
- the rest is my own code and goes into lmain chunk as decided by rollup
Seems simple enough:
function manualChunks(id) {
console.log(id);
const chunksDef = [
["/@zip.js/zip.js/", "zipjs"],
["/prettier/", "prettier"],
// markdown-it and highlight.js are used together in askai.svelte
[
"/markdown-it/",
"/markdown-it-anchor/",
"/highlight.js/",
"/entities/",
"/linkify-it/",
"/mdurl/",
"/punycode.js/",
"/uc.micro/",
"markdownit-hljs",
],
];
for (let def of chunksDef) {
let n = def.length;
for (let i = 0; i < n - 1; i++) {
if (id.includes(def[i])) {
return def[n - 1];
}
}
}
// bundle all other 3rd-party modules into a single vendor.js module
if (id.includes("/node_modules/")) {
return "vendor";
}
// when we return undefined, rollup will decide
}
This is real example from Edna. I’ve put zip.js, prettier and markdown-it + markdown-it-anchor + highlight.js into their own chunks, which I lazily import.
Things to note:
- order is important. If I match
/node_modules/ first, then everything would end up in vendor bundle
id is a full path of bundled file in Unix format e.g. C:/Users/kjk/src/elaris/node_modules/prettier/standalone.mjs. People seem to match the path against just package name like prettier. I match against /prettier/ so that if some file has string prettier in it, it won’t be accidentally put in prettier chunk/entities/, /mdurl/ etc. are used by markdown-it so they should be included in its chunk. That’s where console.log(id) is helpful. I saw modules that I didn’t explicitly put in package.json which means they are implicit dependencies. I used bun.lock to see which package depends on those mysterious packages and that’s how I found what is used by markdown-it
There were 2 remaining problems.
I also had this in my code:
import "highlight.js/styles/github.css";
manualChunks was called for "highlight.js/styles/github.css" to which I returned markdownit-hljs so it was put in its own chunk. Which was too much because I didn’t want to lazy import a small CSS file, so I told rollup to put all .css files in main CSS chunk:
function manualChunks(id) {
// pack all .css files in the same chunk
if (id.endsWith(".css")) {
return;
}
// ... rest of the code
}
There was one more thing that was a big pain in the ass to debug.
To verify things are properly chunked I opened Dev Tools in Chrome and looked at network tab. To my surprise, markdownit-hljs was loaded immediately.
After lots of debugging and research: turns out that vite bundles some helper functions. Because I didn’t specify explicitly which chunk they should go into, rollup decided to put them in markdownit-hljs chunk. Because main chunk was using that function, it had to import it, defeating my cunning plan to load it lazily later.
The fix was to direct those known helper function into vendor bundle:
function manualChunks(id) {
// ... other code
// bundle all other 3rd-party modules into a single vendor.js module
if (
id.includes("/node_modules/") ||
ROLLUP_COMMON_MODULES.some((commonModule) => id.includes(commonModule))
) {
return "vendor";
}
// ... rest of the code
}
You can read more at:
Things might still go wrong. I got one invalid build that created a chunk that wouldn’t parse in the browser.
For that reason I suggest to start simple: empty manualChunks() function that packs everything in one chunk. Then add desired chunks one by one and after each change.
And how to lazy import things?
let markdownIt = (await import("markdown-it")).default;
is equivalent to static import:
import markdownIt from "markdown-it";