What's Next? Exploring the future of React, now - Guillermo Rauch

All right, so my presentation today has no slides, which is uncommon for me to do, but I just couldn't put them together in time. Hopefully I will do it for a ReactiveConf tomorrow, maybe not. Anyways, so my talk is actually about the rest of the spectrum of universal rendering, which is dynamic rendering. As a matter of fact, if anyone has heard of NOW, the tool that we create for single command, easy, simple deployment, the idea is that you're in any directory, you enter NOW, and your package.json or Dockerfile gets uploaded to the cloud, built, and served. So that, um, gives you a pretty universal way of expressing any sort of JSON API website, even like a static list of files or whatever. And we launched it initially thinking about JSON APIs. So you configure a package.json and you say, I put Express in it. You press now, we give you a link immediately with the feedback about the process. Then we— Do the build in the cloud, so we run npm install, npm start, or Docker build and whatever. And we basically reload the page and your API is live. So the initial thing we were focusing on was the new microservices opportunity. But very quickly, we realized that a lot of people wanted to just do static file hosting. So we thought about it and we thought we have two options for doing this. Is one is we special case static file hosting. So we take a directory and we say, "Oh, there's no manifest here, so therefore it's static of some sort," and the server would handle that appropriately. Instead, what we decided to do is we added a little bit of intelligence to the client and we said, "Well, actually static file hosting is just adding a package.json on the fly and using a tool like HTTP server or serve or list." to wrap a little HTTP server around that static folder or static list of HTML files and do static rendering that way. We call that Now Serve. It turns out that Now Serve, and I'll get to my main point very quickly, it's actually one of our most popular ways of deploying applications to our cloud. People use Now Serve a lot. We're very happy with this abstraction because for us static is really a subset of dynamic. Anytime you need to do something a little bit more complex, like let's say you want to write an API server, you want to write a Slack bot or whatever, you have that choice. So by supporting the spectrum of dynamic, we have static as a side effect. Another reason we created Now was we were kind of frustrated with how long builds took. On the client side, on the developer side. So there are a myriad of reasons for that. One is laptops or iPads or phones becoming developer machines are not going to be very powerful moving forward. A lot of people might start actually coding on their tablets or phones in the future, we don't know. So we were looking at a future where it's not necessary to have a very powerful machine and we defer the build to the cloud. So that was one of the main philosophies. And the other one is The internet connection of the cloud is always better than your computers. So by us sending the manifest to the cloud, we can make all these awesome optimizations. Kind of like imagine what Yarn does, but on a global scale. So, and what the other reason that we were interested in solving this problem was, the machinery that we use nowadays to make universal rendering possible is very, very robust. So I was running Yarn today, And yeah, it's faster, but I think the total number of dependencies for our project was something like 20,000. And there were all these steps, all these individual progress bars. And the reality is that this process has been becoming more and more complex and thicker over time. So, going back to the tool that we wanted to power with this, it's called Next.js. So what Next.js does is it takes this approach of using the file system as a main entry point or main API for your project. And it introduces this idea of pages. So you create a pages directory and all you have to do is have your modules export a function or a React component. And the whole process of server rendering and asset pipeline and bundling gets done automatically for you. So I, I'll basically do a, almost like a reading of this post because in this post I sort of COVID the motivations for this project and sort of the things that it enables. Hot code reloading out of the box is one of the nice things, including for route registration where if the route is missing you can register it later on. And we made a very significant, I think, contribution to the lifecycle of components by introducing what we call getInitialProps. So you can imagine that every component that Next you expose with Next is basically a static bundle of JSX. But by introducing this extra lifecycle hook, getInitialProps, you're able to access the lifecycle of the request from the server and expose properties. So the interesting thing about this getInitialProps, which I'll get later on into, is it's an async function that returns a promise. So any sort of data fetching behavior that you wanna using your applications, you can basically plug into Next.js. So, like I mentioned earlier, the getting started— it was very important to us that getting started with this was very simple, yet without trading off power and flexibility. So, we— just like we did with Now.js, we thought that more than one command was doing too much. So, basically, the main contract that we use is you add this pages directory into your file system. Then you can continue to have your components organized in any sort of file system structure that you want, like lib, server, et cetera. But within pages, everything gets registered as a public route. So zero setup and the file system API is sort of like the main point. Going more into the background of why we created this, if you look at the— or some of the reasons that Node.js became so popular was code sharing was a very big one. I was there early on. I think a lot of us became very excited about this idea of, oh, we can run code on the server and the client. But many years went by and this was actually impossible to do or very, very difficult. And what we ended up doing was we would configure some rendering pipeline on the server, output some HTML, and then some other machinery of client-side JavaScript takes over with jQuery and whatnot. So, we think that React's contribution, the main contribution that they made is solving this problem by introducing this idea of a lifecycle. So you start with a component that gets rendered on the server and then the client can take over, reconcile differences, and then execute client-side specific logic. So the fact that you separate the render function from the data is very critical because just like we exposed getInitialProps earlier, we basically wouldn't be able to do this and a lot of other interesting features like eager preloading and prefetching, which I'll get into later on, if we didn't have this separation between rendering and data. And obviously, the lifecycle is what allows us to start with a static representation rendered by the server. And for example, in the client, when componentDidMount fires, you can register, for example, real-time updates. Or you can invalidate the view completely because maybe from the moment you render on the server to the moment the client picks up, for example, they were logged out or the entire data is invalidated now. So that idea of a lifecycle is absolutely critical. So, the other really interesting constraint that we imposed on ourselves was everything had to be JavaScript. And obviously JSX is kind of like the question mark there, but essentially every route is just a function, a stateless function, stateless React component, or a full-blown React class, which you use to get those initial props. Here I'm gonna talk about two important things that we decided to build into this tool. Keep in mind, we set out to make this tool as minimalistic as possible, but there are two things that are absolutely critical to building anything in this design space, one of them being CSS. So the traditional approach to handling CSS with React is inline styles, which has some performance drawbacks and has some ergonomic drawbacks, specifically when it comes to doing lifecycle CSS stuff like hover and stuff, or media queries and stuff like that. So, that's sort of— that's certainly possible within this model. You cannot use CSS at all and you can use inline styles if you want, but we also felt that because we're setting up a server-side rendering pipeline, there are some interesting optimizations that we can do there as far as hydrating and rehydrating CSS on behalf of the user, rendering on the head as the page is rendered, et cetera. So we decided to include our— what do we think is the cleanest approach to CSS in React, which is Glamour. The cool thing about Glamour is that it exposes the full power of CSS, so media queries, hover, and whatnot. And every single definition that you create compiles down to a className that you can embed in your attributes and className and so on. So here I just have className style, and here I define this style. In the beginning, we actually started with the approach of transpiling CSS. So we have a pretty large codebase now for our web UI, and we found that our compilation pipeline, when it involves parsing and preprocessing very large amounts of CSS, and then some of our developers wanted support for more advanced CSS features like import, we actually found that constraining the subset of CSS that we consider to be really useful does wonders for performance but also the cleanliness of the codebase. This idea of just naming your rules in this way is very clean and it makes for components that tend not to get very big, whereas style sheets with CSS tend to get very big. You have the famous style.css pattern where someone dumps their entire brain into CSS and it's style.css. So, with this pattern, it's very hard for your CSS to actually get out of hand. So, we love it. That said, we mentioned in the exploration section of the framework that ideally, we would love to support pure and unadulterated style. First of all, this sort of stuff is a huge barrier to entry for a lot of people. Secondly, there are some legitimate reasons to use CSS because CSS keeps its own evolution. There are a lot of features that It's actually hard to keep up with all the stuff they're adding. So constraining it too much would also be bad. So our solution for that is instead of transpiling and setting up these complicated transformation pipelines, we would love to use Shadow DOM so that the styles that you define within your components would only be— would basically be exclusive to your components. So what we're exploring is essentially being able to compile each component to a shadow DOM instead of a global DOM. And then, me— when I mentioned the two things that I considered global concerns, the other one is affecting the head of the page. So every time you render a page, it's not sufficient to just consider it a sandbox because maybe a certain page has a different title, maybe a certain page has some mobile settings that the other one doesn't, Twitter cards, and a lot of other requirements that fall into the idea of global. So we bundle this head component that what it does is all the children are passed to it. This is similar to helmet, if you've seen it, by the NFL, no joke. The NFL team, not the NFL team, also there's not one NFL team. But the developers for NFL.com created that, but they pass attributes to it. So we would have loved to reuse it, just like we do for Glamour, but we think that the idea of passing children and appending them to the head is cleaner. The way that we use this pattern within our website is, for example, every page defines its— it describes itself entirely. There's no concept of global layout. If you've used WordPress, it's similar to how they do on every template, get header and get footer or something like that, or wp-footer. So what it does is it gives us complete control over the entire layout. So certain pages are black, like for example, I have a blog post where, for Hyper, I decided to change the layout completely for this one blog post. And the way we do this is we pass certain props to the header component, and the header component introduces side effects into the head. So it introduces different global CSS, like when I talk about global CSS, I'm talking for example about like, the color of the page, the type of logo that we include, and like now has a different triangle and whatnot. So that's where the head stuff comes in. And the other reason this is important to support built-in is that when we server render, obviously we want to create a completely different HTML representation. So when we handle this on the client side, we append dynamically to the head and whatnot. But when we server render, we wanna. We want that to be part of the head so that it can be parsed by Twitter, search engines, and whatnot. Code splitting is another very interesting thing. So, I knew from the get-go we have, for example, a page for internal statistics that uses D3 extensively. I knew from the get-go that I didn't want some of our engineers to include Ember within a page because when you create architectures like this, they evolve independently and they become chaotic. So, someone will at some point introduce some dramatic dependency on one of the pages. So we wanted to introduce this idea of automatic code splitting, and it happens at the top-level component layer. So if one top-level component includes jQuery, it wouldn't affect the other one because we produce— we separate the bundles. I'm going to talk about how this performs really well because the key ingredient of Next is that it's very well set up for anticipation of user behavior. So data fetching, like I mentioned earlier, requires that you configure a class instead of a stateless component function. You can— this is a nice pattern to use— you can define everything as a pure component. And then the top-level components— that's what I was saying, that maybe you only have a few top-level components and pages, and then you structure your components somewhere else. The majority of our components are pure components. That allows you to, for example, introduce a style guide section of your application. Imagine you could just register a route called style guide and you include all your pure components in one page with the parameters that you want. So, when you do need data from the server or even from the client, you can define getInitialProps. We've made the somewhat controversial decision to support async and await out of the box, and we rationalize it in hindsight as we're tracking V8. So, whatever V8 adds to JavaScript, we support and we transpile. So, in this case, you can use getInitialProps, you can use it— you can define it as an async function. And for example, like make— API calls and whatnot. If you paid attention earlier to the parameters that this function receives, we use this extensively within Zeit to make different calls depending on whether you're logged in or logged out so that the server-rendered representation is accurate. In the example before, you want to render the news feed statically if the user is logged in, otherwise you want to render the login form. You don't need to necessarily fetch all the data from the server, but you can make some really quick checks like, does req.headers.cookie exist? Then return server true or server false, and then depending— based on that, you render different things essentially. And here's an example precisely of that. Like, if I wanted to have something truly universal, I would have to extract the user agent from the client side and the server side. And the reason for this is that most of our transitions are actually client-side based. So this transition, for example, is client-side based. So the idea is that all of this is already prefetched by the client, and I'll get into that a little bit later. So again, if you want something that's truly universal and you don't want the React reconciliation mechanism to trigger, you basically have to, in fact, define the same exact data. Which is not always possible. And I think there is a warning that React produces that I think is not very useful in this context because sometimes you do want to have a separate server and client-side rendering. So, I mentioned earlier that the, the thing that kind of wraps all of this together is if we're code splitting in such an arbitrary fashion, we have to have a mechanism that allows us to have that snappy single-page app performance without requiring that you download everything ahead of time. The only requirement really is that you download the very page that you visited. So what we do is we wrap the link tag, which we use for the route transitions, and we prefetch it on a ser— service worker. So basically, every component has a JSON representation as well. So I think I can actually probably show you here. So this is something— so this is not— this is a 404 component. But essentially we can pre-cache this and like I mentioned earlier, we can pre-cache very aggressively because we're never fetching data directly. The data fetching mechanism is within getInitialProps. So this is our— basically our contribution to the lifecycle is we needed that extra hook to basically make determinations about what to do when we're routing around or when we're server rendering. By doing that, for example, I can go to the homepage and almost all of this is already pre-cached, so it certainly feels like it's a single-page application. This is a really neat example of why code splitting is so critical. We have this example of how you can query our API to make a dynamic deployment from the browser. So this is loading CodeMirror. CodeMirror is actually not universal, so when we server render, it's missing. I do have a missing height style that I wish it was there so that it doesn't jump, but the bottom line is the server render representation does not contain the rendered CodeMirror. And similarly, when you go to now, we're never fetching CodeMirror because we don't have to. So that's why we made that decision to make code splitting a priority of the framework is because Combined with eager prefetching, you don't really need to think that hard about what to load, what not to load. You just load everything. Because it's in a service worker, it usually doesn't have any impact whatsoever on the user's performance. They got their data, service worker kicks in, it's on a different thread, it caches the request, it's very transparent. That's why in our configuration of Next, it used to be bundled for the framework, but now what we're going to do is if you want to use that prefetch link, you basically import Link from a third-party module or a community module called Link Prefetch or Next Link Prefetch or something like that. So by doing that, you can sort of opt in into this opinionated behavior of prefetching your route transitions. And interestingly, we also thought about, you know, adding different heuristics or intelligence into this process, but it's been working fantastically even in mobile without adding— just prefetching everything. So maybe those people that have bandwidth caps hate it, but for us it's been working really well. So we're very happy about that. Finally, the thing that sort of wraps all of this together is that we think that this idea of ahead-of-time optimization is critical to performance, and that applies also when it comes to deploying to production. When it comes down to deploying to production, for example, you can build and preprocess that routing tree so that you don't have to make file system lookups. And there are a lot of optimizations that you can make when you're sort of preparing to deploy. So that's why from the get-go we wanted to have this very simple contract that you just run next build, next start, and then that's basically the production-ready version. Doesn't do hot code reloading and it certainly makes a lot of optimizations for better performance. And this can take care of a lot of things, like it takes care of minifying, gzipping, and all of that stuff. Finally, there is a future directions link here with a bike for bike shedding. So, this is the opportunity for everyone to bike shed in. So, pluggable renders is interesting. Why? Because when we looked at how big is it, so everything is 73 kilobytes gzipped. So, that's Next, the next bundle that enables CSS, head, and all that stuff that I talked about. And our stuff is actually 15%, so the rest is just like framework stuff. So I think this is the opportunity for two things to happen. One is the browser engine has opportunities here to— all those 73 kilobytes could be in the browser in some sense. I have played in the past with some proposals for extending web components to have more of a React-type feel, but I think those discussions are pretty dead. So the other opportunity is that we can have pluggable renderers that are smaller and fitted to certain workloads better. So like Preact and Inferno in certain codebases are drop-in. So the creator of Preact, for example, went ahead and replaced React with Preact for our terminal hyper, and he didn't have to change a single line of code. So this idea of pluggable renderers, I think, has a lot of interesting bike shedding potential. So, the next one is, like I mentioned earlier, we would love to support full CSS. And we can do that by— you export a function that returns a style tag, and because you are shadow domed, you don't have to worry about isolation. Better JSX is critical because right now there is an obscure bug that you probably have— you have run into in the past, which is you start writing your JSX and then you get an error that says React.createElement is undefined. And you're like, where did I write React.createElement? Nowhere. So, you sort of have to train yourself to always include import React from React because you know that the JSX is going to get compiled to React.createElement. So, this is bad for two reasons. It's obviously in the spirit of simplicity, it would be really cool if this is all you had to write to render some stuff on the client and the server universally, like just export a function. Secondly, if we want to honor pluggable renderers, we have to have a very flexible approach. So, I have two suggestions here, and then Ryan Tomasky weighed in saying that there's a very interesting opportunity to generalize JSX. To sort of just be syntax for function calls in JavaScript or even some proposals for like just a tree object definition. So with those sort of things, we would have an opportunity for sort of having JSX be a part of the language for real. Or the other solution which is kind of a non-solution is that we wrap it and then we pass it to the respective renderer, which is not too bad either. So custom server logic. So in, in Zeit, we have a lot of situations where, for example, a route is no longer useful and we have to redirect it somewhere else, or we have some fancy routing requirements. So what we came up with is a way to short-circuit the requests. So this is also up for discussion because We haven't decided fully on the API, but the idea is that you get the path of the request, and then you get the request and response objects, and then you can match. You can say, here's where you do— you would do fancy routing and route parsing and all that stuff, basically. And then you would say it returns a promise that resolves to the component wherever it is in the file system. So this is pretty important. And then finally, we don't have a notion of ejecting like create-react-app has. What we are instead considering is that you can define a Babel namespace in your package.json, or similarly to how you pass a function for the server request handler, you can pass functions to sort of decorate the webpack and Babel configuration. But also, here's— what's interesting here is that there's an opportunity also for stopping to going so crazy about customizing every single type of syntax. And sort of like with that idea of always tracking V8, we have a very significant performance boost because when you're developing, you never have to transpile, and that's really, really awesome. And similarly, when you're server rendering, you're always targeting sort of— for example, if you use Node 6, you never have to transpile for Node either. So there are some significant benefits to sort of not always promoting, like extending the Babel configuration. So that's sort of it. There are a lot of questions that I sort of imagined that people would have, and I covered all of them here. There is a very interesting project called Nextgram. So what Nextgram does is like, it's sort of like, plays with this degenerate case of routing where— I think I have a demo here. So, there is a routing scenario where you boot up a route that says photo ID 3 in this case, but then when you server render it, it looks completely different. We wanted to make sure that our routing was flexible enough to handle this and of course, it is. And like, then there are a lot of scenarios for like what happens if you go, for example, there and you come back. It needs to match. So what we do is basically we give you a URL prop to every top-level component, and when you need to make route transitions that only impact the context of that component, you do, for example, URL.push there instead of making a global push. So then Next.js is able to track what component owns each URL, and this is sort of how we enable very versatile routing scenarios. In fact, when I was testing this degenerate case, I found a bug on Instagram. I don't know if they fixed it since, where going back from server render refresh to the client stateful render or whatever was broken. I was doing nothing. So this was sort of an example where you can sort of test the limits of the framework in some sense. And this also shows you that structure that I mentioned earlier that in a lot of cases, not everything needs to be exposed. So that's why you sort of keep the top-level components there on pages. Yeah, and also a very interesting fact. So the creator of React had a very similar thing to this a long time ago called React Page. It's interesting because some of the things they mentioned, like for example, the layout component, we've sort of found that it's not such a good idea. But this is also a very interesting read about sort of like where the project started because I think a lot of this is also trying to make this React world approachable as well to people that are just getting started. So that's basically part of our intent as well, to sort of remove all the overhead and boilerplate and also experiment with what are the browser engine capabilities that a world like this would require to be extremely efficient. So we don't need to transpile, so that we don't need to execute very complex transformations. For example, I was mentioning earlier, a lot of people have this very complex CSS transformation pipelines on the server— on their developer side. And frankly it's a lot of work just to get some styles going. So that's it for today and thank you.