Guillermo Rauch & Leo Lamprecht: Folding Space and Time

Guillermo Rauch: All right, after that embarrassing introduction of how we ruined JavaScript forever, we extended native prototypes, we did a lot of bad things, but we've done also some good things. So today I'm going to talk about optimization. So we call this talk Folding Space and Time because we're going to give you some tips and some patterns for how you can compress a lot of the things that your users consume every day, a lot of the processes that you deploy every day, even your own tooling, how you can see it from a perspective of minimizing latency, optimizing time, and optimizing space. So in addition to having worked for MooTools, I started this company called Zeit. Our main product now allows for seamless deployment of any any website, any application, any microservice. Leo is one of our founding members. He's gonna introduce himself.

Yeah, first of all, very great to meet you all here at AgentConf. I'm very excited to talk to you all afterwards. So what I'm doing at Scythe is basically leading product and open source, which means that I'm responsible for the product development and ensuring Everything goes flawlessly when we publish stuff. And yeah, you might know me from projects like Surf or Release or Hazel, which I built in the open source community basically. And yeah.

You can learn more about us and our company at zy.co. We actually introduced a product earlier this morning that is very related to the topic of our conversation, and we'll go into that later today, later in this presentation. If you want to use Now, you can install it via npm or our application Now Desktop, run it inside any directory. These are some of the open source projects that we created under this philosophy that we're going to talk about. So some of the ones that are relevant to today are Next.js. If— has everyone here heard about PKG? Raise your hand if you've heard about PKG. Okay, that's really great because in the topic of optimization and specifically optimizing Node.js applications, I think pkg is a critical tool to have in your toolchain. Next.js, also for those of you that use React, will apply a lot of these principles seamlessly for you. And you can find all of our open source projects under xylit.co/oss. So something that Max was talking about was sort of deciding when to open source or how to open source or what to open source. One of the things that we've done at Zeit is decide on a philosophy for the things that we want to optimize, for the processes that we want to make easy. So we started with deployment and making it a single command. We did the same for zero-configuration React projects. So this talk will first of all gonna introduce this design philosophy, how we think about it, how we visualize it. So hopefully it can also help you when you decide what projects you want to tackle, how to prioritize open source, and so on. Secondly, we're going to talk about what are some of the applicable concepts and techniques that emerge from this philosophy. And finally, we're going to give you a small case study of how we applied these principles to a recent product release. And one of our intent today is to also make you a part of this conversation about making products and focusing on the end result and the end user experience. So if you have any questions throughout the presentation about anything, so feel free to interrupt us at any time. So as I said early— earlier, part of our day-to-day job has to do with optimization. And the reason that it has to do with optimization is that if you really think about it, the operating systems The fundamental APIs and building blocks, such as Unix processes, all those interfaces and APIs have been laid out sometimes for decades. Even smartphone applications that are a relatively old concept by now, packaging those, delivering those to end users, some of these things have been well understood for decades. However, what do we do on a day-to-day basis? We still work on optimization. So we— typically the workflow involves expanding product capabilities, then taking a step back and rethinking those and seeing how we can optimize them. So something that helps a lot, I think, for modeling this into a reusable set of techniques that you can apply is to think about it as a timeline. So it all— everything starts really with a user as part of this equation. So you start with user input, and the typical flow of an application that we deal with or you might deal with on a day-to-day basis typically involves, for example, a server that takes that user input and does something as a result. Later on, you might, for example, ship a page back with 5 megabytes of JavaScript or something like that, that has to be painfully processed by your— the user's web browser. Then hopefully at the end of that, the browser starts processing the result of that computation, some rendering happens, and the user gets their result. So this is kind of what a lot of us, I think, deal with on a day-to-day basis, specifically when the pressure is on delivering a lot of product capability expansions. And then we have to take a step back and bring down that 5 megabytes of JavaScript to something more reasonable. So— and maybe, perhaps optimistically, the idea is a user input comes in and we give them something that they're happy about or that satisfies a need or a want at the very end. So you can imagine that our job is to take that, which probably already exists in some form or another, and sort of try to compress that timeline and get the result faster to that end user. And when I say end user, maybe it's a developer as well. So if you're working on styled-components, you think about your end user as a developer, and that's something that at our company we do a lot. Our end users are sometimes the end user of an end user. So we give them tools so that they can make their end users happier. So sometimes you can even think about, reversing this timeline altogether. So we enter the realm of prediction whenever we think about giving the user the result before they even input anything. So you can think about this whenever you open Google Assistant and it tells you about your flight itinerary before you come to Dornbirn. So our job is to make this timeline perhaps even nonexistent. So more realistically on a day-to-day basis and the patterns that we're gonna talk about a little bit today involve, for example, making that middle path a little faster. So you start off with an application that already exists, and you find, oh, there's 5 megabytes of JavaScript. What can I do here? Oh, I'll introduce gzip. Oh, and then you find out there is Brotli. Oh, I'll introduce that as well. So you go on and you deploy some changes to production and make things faster. It might also occur to you that in many cases you can make these timelines occur in parallel. So something that we do with Next.js, which we'll get into later, is okay, there's a lot that we can defer for later. There's a lot that we can do concurrently while the user gets their result. We can, for example, prefetch future code execution paths. So part of our job is also thinking about, okay, what's the critical timeline for that user and what are the things that I can actually parallelize so that the time that is spent is actually the most critical time for that user to get that result. So we're going to talk about what are some of the techniques that make this job— that we've found make this job a lot simpler and easier on our day-to-day basis. One of the ones that I want to dwell on a bit is doing more work ahead of time. So when we think about compilation in the native world or in a more traditional setting, we think about getting some source code in and getting some machine code out. In a day-to-day, in our day-to-day jobs, perhaps if we do a lot of JavaScript, you might think about JIT and actually that job also taking place later, sometimes even competing with what the user is trying to do or wants to do. So I think AOT is one of the things that we want to have in our toolchain in order to do more work while we're developing and deploying so that the user later on doesn't have to repeat that work. So I think this is one of the common patterns that we see on the web is continuously having the user repeat work that could have been done ahead of time. So this class of optimization I think is really important because we can ship much faster, much smaller applications. Faster— one good example is faster boot-up time. If the browser doesn't have to do a lot of work while parsing and compiling JavaScript, we'll get to the result faster. And it might result generally in smaller downloads, but actually sometimes there might be an inverse correlation here where doing too much work ahead of time could actually result in expanded code. But for the purpose of this talk, we're gonna focus on a set of optimizations that will give us both a smaller code download and some benefits in terms of execution. I think we're also witnessing a transition in our tools that have to do with— we're taking compilers more seriously as well. So recently Tom Dale had this nice soundbite that compilers are the new frameworks. So if you think about tools like Next.js or different webpack bun pre-configurations, there's some work that has to do with, okay, I brought in my favorite framework, but there's a lot of work that on a day-to-day basis we're concerning ourselves with that have to do with the actual compiler phase of putting that framework into production. A naive way of thinking about that is code splitting, for example, is we're trying to take, chunks out of the framework and not consider this framework as an, you know, one-time perfect thing that is going to remain that way forever. So back when Chris and I were working on MooTools, we would ship a bundle of JS, and then no one would ever do anything else with that. And nowadays, we think differently about these things. We think about, OK, I'm going to do— I'm going to import exactly what I want out of this framework, and Webpack will code split and get me what I need. And I want to make sure that all the code that is not executed is properly eliminated. So part of the concerns that had to do with people that would focus on compilers are also translating into people that focus on frameworks. So the first example that we're going to talk about is of how this trend sort of started is bundling JavaScript. So Leo, you can go ahead.

Yeah, basically— Like, you know, before we discovered bundling, we basically all had these nice scripts in our pages which basically told the browser to download a script, and the download could happen serially in the browser. That's possible. But what didn't happen is basically that the whole JavaScript was executed at once. It had to happen one after another. And the download was what happened at the same time. Now that we have bundling, now that we've discovered bundling, we know how to fix this problem, basically by putting all of the stuff into a single file, which basically means we only have to download one, of course, and we only have to execute one, which is much faster. So let's see. Yeah, exactly. So instead of going through all of these scripts, downloading it and then parsing it, executing it, and then jumping to the next one, we can go straight to the single one and run it. The same ideas can be applied to rendering itself. We found that, as you all know, we basically all had server rendering in the beginning. Then client-side rendering came around. Everybody said, "Okay, you only need to do client-side rendering, then you have a great app." But over time we've discovered that we need to have both. We need to have server rendering and client-side rendering. And that's also the reason why I'll talk about server rendering now, because before we had this huge problem basically of having to request the page, basically an empty page which only had a bundle referenced, and this bundle then got fetched and basically rendered the DOM, the web page. But thanks to server rendering, We only have to request the page basically, which gets rendered on the server, so we get the markup straight from the server and save this whole step of having to render in the browser and see a flicker or whatever. It's much more efficient. Yeah, in addition to bundling and rendering on the server, we noticed that minifying, of course, is a great solution to make your site much faster. But something that's very interesting, for example, with Uglify version 2 in our case, something that we've recently discovered is that uglifying the code, basically minifying it, doesn't only— doesn't just make downloading web pages faster, but it also makes the boot-up time of native native applications, for example, faster. Here, for example, you can see Hyper without minification before we started using Uglify. That's the bootup time. And that is the bootup time with minified code. As you can see, it's half the performance. It's much faster. Yeah, and together with the optimizations we brought in for the code, we also thought about increasing the boot-up time in different ways, not just by minifying the code. And what we've built is a solution called PKG, which Guillermo talked about earlier, which basically allows you to easily create Node.js binaries well, binaries that work on all platforms, in the sense that you only have to run the PKG command once on your Node scripts, and then binaries for all of the platforms that you want it to work on get created.

We have a helpful video here for those of you who are not familiar with PKG. It's technically a cross-compiler because you get all your platforms at once, like Leo said.

Oops. Video doesn't start.

Yeah, I think it started. It hasn't started in this little— Oh. There you go. Wonderful. So any Node.js application that you have, you run it through our PKG project. Very simple. And you get all your binaries for all your platforms. It's really sweet. So what Leo was talking about in terms of bundling, you can see how this compilation concerns are so cross-cutting in the sense that we were concerned with bundling for the browser. We're concerned with bundling for the server. We're concerned with boot-up time for the browser. We're concerned with boot-up— it's all one huge mess of concerns. Just kidding. No, but this is a very helpful tool.

Yeah, by the way, just by the way, we are also using this tool for now CLI, and we are planning to use it for much more of our Node CLIs, basically. And the great thing we've seen at now CLI, for example, is that now that we have created the binaries for all the platforms, we basically also serve them over npm. Over our CDN and also when you install now desktop you get it straight from the same source. The great part about that is basically even if you run, let's say, the installation of now on a Node version that's very low, let's say 0.10 or something, the great thing about that is since we use PKG you can still run now without any problems because the PKG binary actually contains Node and not only increases boot-up time heavily, but also, like I said, makes it work on much older Node versions. Yeah, exactly. And you can find it here at github.com/sightc/pkg. And that's also our GitHub org, so you can also find the other projects that we talked about earlier, Micro, Next.js, Hyper, et cetera, you can all find them there. Yeah, so previously when we did— when we shipped the source code like everybody else does over npm for now CLI, we had to of course load it from the file system. That's what Node does. But now that we have PKG, which basically puts all of these files into memory, it gets all much faster to parse and evaluate. Yeah, okay. That's a very funny topic because, as we all know, there's a huge difference between downloading one file versus a million. The great part about Node modules is that you have— that modularity. Basically, you can build anything really quickly because you only need to import a certain module and execute it. Basically, pass some arguments to it and then it's done. But it comes with a cost, and that is that all of these Node modules get downloaded, of course, separately.

And it looks like this.

Yeah, Node modules are basically the It takes infinite space. Yeah. The biggest black hole in the universe.

Yeah. So at the time when we released it, like Leo was saying, so there was a practicality side of it, which was, OK, easier delivery. Perhaps for CLI applications is not as critical sometimes. But when you're focused on certain critical boot-up time critical applications, it can make a big difference. And for example, also for the sake of auto-updating, replacing one file versus replacing others, et cetera. So at the time, it was only marginally faster to run a PKG binary as opposed to invoking Node server.js, for example. But as of version 4, we actually became faster than executing Node. And we're only— we've— we're only probably 10% in, in terms of how much more we can do with PKG to make that process faster. But I think this is one of those projects that kind of shows you the symmetry of concerns between browser and server sometimes. So as of 4.0, you get a 20% boost without doing anything. So it's beautiful because you know, you're saving a lot of milliseconds for millions of users. So that kind of goes back into that idea of folding timeline, because we did work ahead of time so the user wouldn't have to repeat it over and over and over again. Oh, yeah. So in terms of optimizations that can be done beyond— bootup time or beyond, just bundling all code together, is actually executing some of the code that you are giving to the user to execute. So an example of this, also from the native world, is constant folding. So if you write some code that performs, for example, a series of mathematical steps, and you serve it to the user, In this case, I used this really neat visualizer tool for Rust. I'm computing factorial, which is a very trivial thing to do, obviously, but it'll show you the power of this category of optimization. When you look at the intermediate representation that the Rust compiler yields, you're going to see that the actual number of the result of that calculation is precomputed and ends up finding its way into the resulting binary. So this category of optimization, I think, is very critical for the web, because there's a lot of stuff that we give browsers to compute that we could do ahead of time. So JavaScript can actually now do this same thing, thanks to a project created by someone that now works for Chris Poger, most likely. He's a great engineering manager at Facebook. So this is an example of prepack sort of doing the same exact thing with a factorial function. And I have a duplicated slide there. So one of the ideas behind Next.js, our React toolkit, was sort of compress a lot of these really good ideas and give in and present them to you sort of as a default so that, for example, when you're developing You don't have to, like, turn on hot module replacement and things like that. And conversely, when you go to production, we turn off hot module replacement and a lot of— we eliminate a lot of dev time code, and we perform a lot of optimizations on your behalf so that— trying to sort of put you into the pit of success. So we try to follow this timeline of, you know, in the beginning we thought, I need JavaScript. And that's why we had the problem with loading abcd.js in parallel, and we would bring in MooTools and jQuery. Actually, that was a big concern for us at the time is what if people want to load both MooTools and jQuery in the same page? So that's where we were at the time. Then we realized I need a framework that sort of manages all these cross-cutting concerns. So we got things like MooTools More, actually, where we put a lot of things like AJAX, and we had a little module system and so on. Later on, we realized that we were making the frameworks so big that we needed compiler-type tools to start trimming down code. And nowadays, I think we're finding out that we're gonna need compiler frameworks to sort of customize and take control of these optimizations ahead of time. So an example of that is Babel. Babel puts us in control of performing some very useful AOT optimizations. So in Next.js, an example of that is, for example, if you create a page that renders some JSX, we allow you to inline some styles that actually get compiled and isolated for you so that you don't embed global CSS all over your pages. So when you create a page like this in JSX, What Babel is doing before we give the code to the browser is we turn that style into a specific component with prefixing. So with this simple technique, we simplify a lot of issues that have to do with CSS polluting every namespace and so on. And all we had to do was register one Babel plugin. So by Next.js allowing you to extend and configure Babel, It's sort of giving you the power to not only use our tools as, as they are and as we expose them in our APIs, but also take control of how code gets compiled and modified before it gets served to your users. Another set of optimizations that you can define on your own, for example, is compiling Markdown ahead of time. So we all like to write Markdown as opposed to writing HTML and JSX in a lot of cases, but we don't want to ship a Markdown parser that converts your Markdown to HTML to the end user. So every time you find libraries like markdownjs or you go to npm and you find Markdown, a naive approach may be, oh, I'll bring in the entire Markdown library and then it finds itself into your bundle. But a technique we're applying that we've actually open sourced for our documentation is we can tell Babel, and I don't know if this is big enough, But we can tell— let me see if I can increase the size. Oh.

So what we are basically doing is adding a plugin for Babel which tells it to understand what Markdown is, basically parse it, and then output JSX.

Yeah, unfortunately it's not big enough. Yeah, and the main— and the critical difference is instead of your user parsing the Markdown, this gets compiled ahead of time into JSX, which thanks to server rendering, it actually gets compiled also into HTML so that the end user gets the result before anything else.

Yeah, and without including any libraries for actually parsing it, understanding it, and converting it to JSX in the bundle. That's the important part.

That's one of the examples where developer experience is well aligned with user experience. Because sometimes in defense of developer experience, sometimes we might hurt user experience. So this is one of those techniques that allow you to sort of get the best of both worlds. Another optimization that we've introduced that is somewhat more novel is code prefetching. So, Leo, you can give an example perhaps.

Yeah, so basically Next.js provides an argument for a so-called Link component. That Link component is basically used by us to, in Next.js, to route between different pages. So that's how it would look if you had a page called my app to which you would route to, you would want to route to. And the prefetch argument you just saw basically indicates that we need to preload the page so that when you actually hit the button, the whole page will already be there.

So an example is if I'm connected to Wi-Fi, which I think I am, if I open site.go from scratch And so typically what happens with single-page applications is you download the entire system all at once. So because we like server rendering, we decided not to do that. So each page comes pre-rendered by the server. Then on the client side, we can sort of mount the application and do more behavior. But one thing we do like about single-page applications is that every interaction on the UI feels immediate. So that's where prefetching comes in because you're gonna see that as I click these sections over here, everything is already preloaded. So you get this experience that, oh, I downloaded everything already. However, you didn't pay as an end user that cost initially. So that's one of the examples that I gave about overlapping timelines. In fact, now here I have, a series of blog posts, and we actually also prefetch those by simply adding that attribute. So Next.js not only gives you that sort of flexibility and sort of set of optimizations for production, it also gives you this new tool that you have for putting these optimizations into place that have to do with getting ahead of what the user is going to want. So we call this sort of the future timelines. So we don't know what the user is going to do. But in the background, while they're consuming some content, we can start prefetching some next potential features. And a critical insight here is that in a lot of cases, when we evaluate the performance of native versus web, we tend, I think, to conflate a lot of different things. So some have to do with we think native is better because raw performance is better. So computing factorial in Rust might indeed be faster than computing factorial, uh, uh, uh, maybe with V8, uh, maybe not. Um, but I think one of the critical things we miss is that with native applications, we pay the cost of a huge download ahead of time, which gives us all the possible future paths paths of the application. And that tends to have a huge benefit when you're, for example, offline, and you start tapping around, and everything is already there. So I think a lot of the user experience benefits that come with native apps have to do specifically with the application of this technique. So an example that I like to give to sort of drive this point is someone a long time ago disassembled the Facebook iOS app. And they enumerated all the class names. I guess they shipped debug symbols or something. So you can see that all that code is right now maybe even loaded into memory on your phone, even though maybe you're never going to do a— sometimes they even pre-download experiments that they might never even run on you. So that's why in our experience of using Next.js in production, we tend to actually prefetch quite aggressively. Once we've given the user the result of what they want, we prefetch aggressively. Some of our users in our ecosystem are also experimenting with prefetching data.

Yeah, I want to hook into that by saying, you just said we are prefetching very aggressively. On our homepage, for example, we've created a component that we've specifically made for that. It basically decides based on the href property of the link if it's an external or internal or anchor link, and then depending on that, we either directly give back the link component with the prefetch property, which means basically that all internal links on our homepage get prefetched immediately in the background once you start accessing the front page. They all get prefetched slowly in the background, and external links just get treated normally. Yeah.

So after we've sufficiently optimized the client portion of this process, I think it's also important to talk about backend. So the timeline I mentioned earlier, the timeline with user input tends to first involve server for any sufficiently interesting computation or data that the user is interested in. Typically offline acts more as a cache, and the user is perhaps more interested in getting some fresh data or performing some fresh computation. So earlier this morning we introduced a new product for finding domain names. So we decided to tackle this problem because the user experience of purchasing a domain name, which is quite frequent, tends to not be ideal with a lot of providers out there. So we decided to sort of rethink this problem. And the first thing that we did to optimize this idea of finding your perfect domain— and I'm going to share a quick demo. I think I closed. Oh, I have a tab open. Okay, great. Now I only need to find my cursor. So— Oh, wrong link. Okay, great. So actually that was prefetched, which is why the transition happened quickly. So the next thing that happens is, okay, we want to present the user with the input as soon as possible with obviously prefocusing and downloading the input as soon as possible. And putting that priority on getting the data from the user's brain into our cloud, so to speak. So the next thing is the user starts typing. One of the priorities here was to sort of make the process of typing completely blocking-free. So you sort of decide what do you want to name your next big open source project. So I'm going to call it Style Components. It's new. The idea here was the act of— this is something Google popularized with Instant Search. The act of typing in can already begin to be streamed to the server. Now, this seems obvious today, but it's quite a departure from I type in something and I press Enter and I post and I get a brand new entire page of HTML and then I get some results back. So the idea was we wanted to make this assumption that we're going to be streaming data to the server, first of all. So what we set out to do was design a streaming protocol. So HTTP is more of a request-response protocol. Sometimes you can override the content type to make it not so, but typically you send a request and you get a response. If we had to do that for, like, hundreds or thousands of domains, things would get pretty slow. So we decided to sort of— take this idea of JSON, a newline-delimited stream of sort of actions. So those of you who have used Redux, it's kind of like Redux on the server, for the server. So the idea is the connection maintains your state, kind of like if it was a remote store of sorts. And the first thing we do is we dispatch, we write an action that says search. So that search action is going to impact some remote state. Again, so just model it as there is a Redux store somewhere and you dispatch an action to it. And as a result, instead of getting an entire payload of the state that gets rendered, which would be more how Redux works, we start getting back progressively little nits and little pieces of the data. This is very important because we have a huge variety of backends to hit, all of which have distinct performance characteristics. So when we go back to the world of server rendering only, you're always bound— your page impression is always bound by the slowest piece of data that had to be fetched. So I think this is why it's so important to use tools like Next.js is sort of allow you to decide how much do I wanna prerender? How much do I wanna do on the client to sort of speed up giving the user feedback? So an example of prerendering or sort of trying to predict what we're gonna show the user is we already know what the set of possible TLDs is. So if you go to a page that has a URL like that, we already prerender it with what we anticipate is gonna be the set of results. So if I reload this page, you're gonna see that Next.js already knows all the TLDs, so it gave me a bunch of HTML with like grayed out domain names. Finally, having sufficiently optimized that idea of I ask the server something, give me back what's the most optimal, The final piece of the timeline between the user and getting that information back is actually reaching the most nearby server. So for that, we deployed our APIs via DNS Anycast to two different data centers, one in San Francisco, one in Brussels. So you can also accomplish this with one command. So when I was getting the results earlier, I was hitting api.zy.co. However, it was being intelligently resolved to the nearest point from us here in this venue, which is Belgium in this case. So it's about— we're about 40 milliseconds away. Obviously, we can continue to compress this further by having a data center here in Dornbirn, but that's coming soon.

So let's see what the future holds.

The final optimization that I sort of mentioned earlier is Something that's very important to keep in mind when using technologies like JavaScript, that's for the most part mean that our workloads are single-threaded. So one of the things that I sort of touched on earlier is this idea of parallelizing as opposed to concurrency. So for example, when I dispatch all my actions to get all my searches, that's happening concurrently to rendering, to setting up the WebSocket connection, to mouse movement, and a lot of other things. However, it's not happening in parallel, which means, for example, if I type something and the DOM gets really busy, the browser gets really busy, it's actually going to inhibit my next keystroke. So if I'm going through ideas very quickly and I'm typing like crazy, then the rendering time is actually gonna compete with my ability to transfer my next intent to the computer. So this is the key insight behind React Fiber and the big rewrite that happened recently, and we had an opportunity to apply it to great satisfaction with this project. So what it's allowing us to do essentially is intelligently gives us the ability to prioritize user input. So notice that the raw amount of time doesn't really change. So there's gonna be expensive rendering happening at some point, but the key thing is if user input comes in, we can continue to defer that rendering time until later on. So this would be the equivalent sort of thing with React Fiber. And another thing that I kind of didn't show, but I glanced over is we want to focus on what the user actually wants. So like let's say I go to the very bottom, I start giving priority to this part of the page, but similarly, just like not blocking keyboard input, I don't want to block scrolling input. So for this, we could have technically applied the same React Fiber APIs and using the regular scroll event, but we use a new API called intersection observers that is very cool. All of this work has actually not been done by me nor Leo. Is actually being done by a speaker later today. Javi is his name. So if you want to learn more about how he actually did this, we wrote a blog post that we published this morning. And I hope this sort of gave you some insight into how we think about product development and optimization, and you can apply to your own products. And we're also looking forward to answering a lot of questions. Thank you.