I have recently written a broader exposition on frontend build tooling, perhaps it will be useful: https://sunsetglow.net/posts/frontend-build-systems.html.

The performance gains in the recent past have mostly been due to moving away from single-threaded JavaScript to multi-threaded compiled languages. This requires a complete rewrite, so existing tools rarely take this step. We see this optimization in Farm alongside "partial bundling," which strikes a performance-optimal balance between full bundling (Webpack) and no bundling (Vite) in development.

Vite abstracts over a preconfigured set of "lower-level" frontend build tools consisting of a mixture of older single-threaded JavaScript tools and newer multi-threaded compiled language tools. Vite can adopt the partial bundling of Farm, but dispensing with its remaining JavaScript tools is a major breaking update.

> The performance gains in the recent past have mostly been due to moving away from single-threaded JavaScript to multi-threaded compiled languages.

This is overly simplistic. Parcel had far better performance than Webpack before they added native code or threading.

Webpack remained slow because it didn’t have a good public/private interface for its plugins, so the changes that could be made were limited.

> Vite can adopt the partial bundling of Farm, but dispensing with its remaining JavaScript tools is a major breaking update.

Turbopack and Parcel both have excellent performance without any compromises to their bundling strategy. Vite skipping this likely just simplifies it’s architecture. Bundling creates an opportunity to be slow, but it doesn’t necessitate it.

Rolldown (Rollup compatible) for Vite which is written in Rust and still in active development.

Busy reading this and it's great so far. One comment, you've referenced "tree shaking" a few times without explaining what it is. I think I know but it might help others to explain that before your reference it.

I would just use "dead code elimination" instead. They're the same but that actually tells you what it is.

Since tree-shaking is a common term across frontend build tooling documentation, I adopted it as well.

Dead code elimination in its traditional form also runs during code minification, which is a separate build step from bundler tree shaking. Having separate terms avoids ambiguity.

Tree shaking is the removal of unused exports, a very specific thing for JS. Dead code elimination is a broader term which includes tree shaking, but is usually used for the elimination of code after the compilation (or transpilation/minification in js/ts case) in the front-end world.

A practical example would be that tree-shaking wouldn't remove the variable b in "export function foo(x) { let b = 10; let a = 20; return x + a; }" but if this export isn't imported anywhere, then it would remove the whole function definition. Uglify.js, which does dead code elimination would remove the unused b variable.

Sure, tree shaking is just a very basic dead code elimination algorithm. But there's no reason to give it such a prominent and confusing name. Just call it "basic dead code elimination"! If you must be specific (why?) call it "dead export elimination".

I don't disagree with you, but on the other hand, it was really a hard problem in js (partly because functions having outside context and mainly because how mutable modules are (were, with commonjs)), so it became a huge race for optimization. Now it's really mostly dead code elimination because how much saner es modules are yet the name stays. But hey, we also don't call televisions "bigger monitors with built in spying OSS", names have a tendency to stick :)

Thanks for the feedback!

I struggled with the ordering since each section is somewhat mutually dependent; this is arranged more like a thematic history than a chronological one.

Tree-shaking naturally fits under bundling, and I'm afraid that explaining it earlier will make tree-shaking's explanation itself contextless since without bundling there is nothing to tree shake.

I can hyperlink those references to the tree-shaking section tomorrow so that there is an action for the confused.

Thanks. I did see that it was covered later as I continued reading the article and looking back I see that it's in the TOC as well so maybe I just didn't pay enough attention. I think just hyperlinking to that section would do the trick (and maybe on the first occurrence a small comment like (covered below) but that might not be necessary.

Thanks for the response and more importantly the article! It covered exactly all the points that were opaque to me about frontend build processes. I've also forwarded it to a couple of other backend devs that I know.

What helps me in writing on complex things is introducing a concept with a simple (perhaps even slightly wrong) explanation when you need it, before explaining it in greater detail and clearing up any prior simplifications when the reader is able to grok it fully.

Not to take away from the excellent writing in your original essay!

Thank you for this. This looks like just what I was looking for!

Want to understand this as well.

I was looking at using Rust for some static analysis of JavaScript files recently. My impression is that despite there being multiple parsers written in Rust -- the ones in swc, rslint and maybe others -- they are not as production ready as something like acorn, in terms of passing test262, features, documentation, and simply the amount of example code out there. There are bugs like https://github.com/swc-project/swc/issues/2038 that haven't been fixed after a few years. (I am aware that swc is "production-ready" in the sense that it is already used by Next.js, but that's not a high bar if you look at all the fallbacks and limitations)

These projects also seem to be very broad but don't have enough developers to support them. They are aiming to provide an "all-in-one" solution with the parser, transpiler, bundler etc all in one place. Which means they have perhaps too much work to do. While in the current JavaScript ecosystem, there are many projects for each part of the toolchain, the unexpected upside is that there are many people working on one thing to make it really good. Just look at webpack. It's not the best thing out there, but the sheer amount of features and the support for almost every use case is amazing. For parsers, acorn is very good, because the author can focus on the parser instead of also working on the bundler.

As an end user, I would much rather see effort concentrated to make the toolchain really good to make it more production ready, rather than seeing another tool popping up.

> There are bugs like https://github.com/swc-project/swc/issues/2038 that haven't been fixed after a few years.

I tried all those examples on the swc playground[0] and all of them pass, so the issue seems to be fixed, but maybe they forgot to close it. By the way, swc, oxc, and biome all pass test262.

[0]: https://swc.rs/playground (version 1.6.5)

Their documentation says "passes almost all tests": https://rustdoc.swc.rs/swc_ecma_parser/

Which could be outdated as well.

But if that's true, it's the problem isn't it? If some of the most important information about a critical part of the toolchain isn't up-to-date, how can I have confidence in this?

(Also for my specific use case, I need to set an ecmascript version, something that is common in JS based parsers, but it's not available in swc ecma parser. At least I didn't find it.)

For context, I contributed to acorn to fix an issue I found while using it in one of my projects. But that happened because I extensively used acorn in the first place. I don't quite have the confidence to adopt these Rust based tools yet.

> They are aiming to provide an "all-in-one" solution with the parser, transpiler, bundler etc all in one place. Which means they have perhaps too much work to do.

They seem to re-use quite a lot. I don’t think any of them, besides ESBuild, roll their own transpiler. For example, Farm uses SWC:

https://github.com/farm-fe/farm/blob/main/crates/toolkit/Car...

+1 for this.

I've just watched some videos on this but it's still nebulous to me. It has something to do with hydration, and bundling only components used in the app, but making it compatible with ES modules while also allowing for hot reloading of your app during development, ... or something like that.

I'd like to do more web dev but I find it very difficult to get started in the space as someone with 20 years dev experience in backend and data.

It's mostly a change from js to go and rust. For older tools it's harder to migrate towards rust/go because of their ecosystem (plugins etc) being build around js.

However, vite currently consists out of a different build tool for prod and dev (rollup and esbuild). And they're working on replacing both with the new rust based 'rolldown'

Rspack supports JS plugins made for Webpack. I even think there is some overlap of the dev team.

There is webpack, it's bloated, tedious to configure, slow and does breaking changes in configuration with every version. Configuration is basically write only, then don't touch and hop it still works after minor upgrade. It does solve the whole development cycle from dev server with hot reload (which on a usual fe project is subtly broken half the time). Then there are wrappers for webpack to provide sane defaults to end user. Then there are library bundlers, which don't solve the same problem as webpack.

There is also vite which doesn't bundle for dev environment, but serves individual modules separately and is fast.

Why webpack is bloated you ask? Well, because there isn't single javascript, but a mixed bag of esm, commonjs, vue, ts, css, scss and whatnot used in a combination with one another (you can have a vue file which contains ts script and scss styles and a vue file with javascript which imports ts file and css modules). Webpack tries to solve all that, plus caching and developer webserver, but relies on plugins to do a lot of stuff. So yeah, dutch tape, a lot of.

Then somebody decides it's enough and tries to rewrite the whole thing is rust because it's faster and implements 80% of their own needs on day 1, but gets burned by the long tail of annoying issues.

Since it's not a product in itself, somebody has to sponsor it or make in-house thing and opensource. In-house mainters could move to a next bigcorp and the new shiny thing will not go forward with original vision and same enthousiasm.

So yeah, welcome to frontend. It's like linked lists, buffers or unicode in C, but with a complexity or a distributed org and trying to be as cool as erlang at the same time.

I honestly think it's largely just wanting to be known as author of the popular framework/bundler/engine/linter blah. And that the JS community attaches a lot more importance/status/recognition (or uses social media to talk about the ecosystem more, and that leads to the former?) than happens in other languages.

Rust and Ruby too, to a lesser extent. But you don't see this nearly as much in Python or C++ or Go IMO. (I can name Guido van Rossum and Herb Sutter. But who wrote pytest for example? If it were Javascript and I used it I'm sure I'd know. And fwiw I've worked professionally with python for years and not really JS much at all, but I'd recognise more JS names and hear about people far more for JS than python.)

Basically everyone is redicovering the build tools and minifiers used by compiled languages, that we lost when everyone decided they had to be written in nodejs instead.

Similar to RSC rediscovering JSP, ASP.NET, PHP,...