Episode #50: Static First Using Serverless Front-ends with Guillermo Rauch

This episode of Serverless Chats is sponsored by Amazon Web Services and Stackery. On today's episode, I speak with Guillermo Rauch about building static-first using serverless front ends. This is Serverless Chats, episode number 50.

Hi everyone, I'm Jeremy Daly and this is Serverless Chats. Today I'm speaking with Guillermo Rauch. Hey Guillermo, thanks for joining me.

Hey, thanks for having me.

So you are the CEO of Vercel, which was formerly Zeit. So I'd love it if you could tell the listeners a little bit about yourself, your background, and what Vercel is all about.

Yeah, I'm the CEO and co-creator of Next.js, which is the React framework for frontend development and Jamstack development. And Vercel is a platform for deploying projects like Next.js and many other frameworks. And Vercel focuses on making the lives of frontend developers really, really easy, allowing them to push their pages to our edge network and have a very delightful serverless development experience.

Yeah.

So that's what I want to talk to you about today. So the last time I think we saw each other in person was back in— was it back in Milan, I think, right? Almost 2 years ago at this point. Or maybe it was last year. I don't even remember. I can't remember. Quarantine has lasted so long at this point that I can't keep track of time. But the last time I saw you, I was speaking about this idea where I felt like serverless was getting harder and harder and harder. That was, or it seems to be the wrong approach. We want serverless to become easier. This is something where I think this idea of maybe, I think you call it frontend serverless or serverless frontend, is where you're trying to go with Vercel. I'd love to just get your thoughts on that, just that complexity that we're now pushing towards the backend and where you're trying to go with the frontend.

Yeah. I think you nailed it. I think the serverless world is big and complicated. I think when we first met, we really connected on this idea of what is even the right definition of it. We were both presenting at Milan trying to give a definition for it. It's a pretty silly game to play, right? To try to even like fight that fight. And when I think about serverless, I think about wanting to give people a very good recipe for leveraging that kind of technology. And I think anything that relates to serverless or infrastructure really needs to disappear. It has to be all about letting people focus on their products. Focus on their pages, focusing on the things that they're publishing to the internet. And that's why frontend really is the place where I think all the serverless action is happening. And the techniques and technologies that we're using in some ways are the original serverless, because much of what we're doing today is this idea of taking pages, generating them statically, and putting them at the edge, which means, you know, to me, the most fundamental serverless technology out there is a CDN, right? Like when they've been around for a long time, even they predate a lot of the serverless movement. And yet they had that critical idea that there is no management to do, that it accelerates you obviously because it's putting your content next to your customers. And the very technology that this accelerates is a frontend. I think what we're about to see is that a lot of what we've been advocating for in the serverless world is really starting to become much of a reality with frontend developers.

I think that that actually makes a ton of sense because whenever I was thinking of serverless, I would always think about the actual computations that were happening behind the scenes. Whether that's something where you're running a Lambda function and it's you're pushing it into SQS and you're connecting to DynamoDB and you're doing all these different things with the data. And a lot of that is still necessary, right? There's a lot of complexity that has to happen behind the scenes in order to make, you know, a full-fledged serverless application run. But I think the funny thing is, is that a vast majority of the applications you see out there are just a collection of static pages, right? And that's— With a little bit of API happening in the background, but that shift, that thinking of compute versus static pages, isn't that really where we want serverless to go to, is just this super easy pre-computed system?

Yeah. I think a lot of people in the industry have overfocused their attention on computing on demand. Which is what Lambda enables, right? You're literally firing up a VM. It's amazing how easy AWS made it. Like, it's almost like a miracle, right? That like you deploy your function so quickly and like it executes so quickly and like it's secure and in a VM sandbox and their underlying technology, uh, is absolutely incredible with Firecracker. But the question that you have to take a step back and ask yourself is that Do I really want to be computing so much? Do I want to be burning electricity and computing cycles so much? And this is where when we really sat down to analyze this problem, we realized the vast majority of pages that you visit every day on the internet can be computed once and then globally shared and distributed. So it's kind of like the technique of memoization in functional programming where like— Right. You compute once and then of course you wanna read from that intrinsic automatic cache that you get. It's different from caching because caching requires a lot of developer effort and thinking. Memoization gets closer to what I envision to be the foundation of serverless frontend, which is basically static generation where the computation happens once, probably as a result of some data pipeline, some, something that changes, computation happens, HTML is spit out. That is all in— even in the case of Vercel, is powered by functions too, by the way. But the funny thing is that the developer never even thinks about functions. They just think about building pages that then get pushed to the edge and then consumed by visitors. Now, that's not to say that the on-demand use case doesn't have any merit, right? Like, not everything can be computed statically. And like, there's lots of pages where like you sign in to a dashboard and you have to query data that could absolutely not be cached. A great example is like you log into your bank, right? And like, um, imagine that you were trying to statically generate your dashboard with your like bank account balance, but you just want to check that your payment went through for a utility and you're not sure if your— what you're reading is up to date or not. You would go crazy, right? But, uh, but the movement of frontend has also led us to where that dashboard is a single-page application, most likely, that is also served statically from the edge. And then there's JS code that runs on the client side that then queries that backend, right? So what we found is that frontend is really powered by this, like, set of static— statically computed pages that get downloaded very, very quickly to the device. —some of which have data inlined with them. And this is where, like, the leap of performance and availability just becomes really massive because you're not going to a server every time you go to your news, your e-commerce, your whatever. You're just downloading it from, like, your very own city, right? But even in the case of, like, I may have to make a strong read, not, like, a read that could be stale, You're basically also downloading static content that then runs JavaScript on the client and then that goes to a server, right? And then the question becomes, who's writing that server and how much of that server are you writing? So, this is kind of like the other big question that is, I think, coming up. We're confronting that in the serverless world. It's like, okay, I have all these amazing primitives to build everything in the world that I could imagine from scratch. But does it make sense to build everything from scratch, right? Does it make sense for you to build your own authentication function with Lambda if you could be reusing a standalone authentication service? And that's why this interesting world is coming up where like there's a rise of the frontend, but then there is a rise of the API economy. And what I mean by the API economy is that we have services like Stripe and Twilio AWS Cognito and Auth0 and Magic Link and like all the services where you're just making some quick API calls, sometimes directly from the client side, right? And that is a serverless world that seems so much more, in my mind, attuned with the actual ideal and the actual original promise of serverless. I think we kind of heard too much— you were giving the example of like SQS and DynamoDB. We earned too much on always rebuilding from scratch a little bit. So focusing on the front kind of allows you to like kind of reprogram your product strategy in a way where like, okay, I'm gonna think about the customer first. I'm gonna think about building my backend very, very low in my priority list. Right?

Yeah. No, and I think, you know, in just this idea of what could be static content versus what needs to be generated, dynamically. And I mean, I think of an e-commerce site, for example, every product page, every category page, every set of recommended products for a particular product or, you know, related products or things like that. All of that stuff could be precomputed and pushed out to the edge, right? And then the developer never has to think about, you know, processing the scale of that, right? Because if you think, and I know you used to work at WordPress, right? That was one of your— One of the things you did before. And as you know, WordPress loves to query that MySQL database on every single page load.

That is a great example. That is a great example. I think this is the difference. You just nailed it between ahead-of-time computation or generation of static pages versus just-in-time. So with the just-in-time model, which is what WordPress is doing every time you go to like index.php or blog.php, you're creating all this load. You're sometimes, you know, issuing dozens of queries. And something you and I were talking about before the show is that because we're at peak cloud, in the amazing power and capacity that we have at our fingertips, anything it seems could scale today. Like if you use, uh, the new Serverless MySQL Service I'm sure Jeff Bezos will sell you enough MySQL on-demand capacity with his incredible database engineers. They'll make MySQL scale so much that you might actually make that work. But the question is, do you actually want to? Do you actually want to put— first of all, pay all those database bills. But secondly, even it seems like we're increasing the entropy of the universe and, and we're producing all this heat and carbon emissions for no reason. Because the point is, the writes that happen are here and there to those product pages, to those blog posts, to those marketing pages. Somebody at a marketing team might go and say like, oh, I'm going to edit the— today I'm going to edit the headline of this page, you know, or today we're going to work on a blog post. So you write— and the writes are not that frequent, and then that is what creates this asymmetry. If you can take the opportunity to convert that write to the database into HTML when it happens and then share that super easy HTML stream that gets downloaded from an edge server, you can't compete with that with any other serverless architecture, you know? You can't compete with that from a speed of light perspective. But also cost-wise, you can't compete. We've talked a lot about over the years about how amazing it is that Lambda gives you 1,000 concurrency and whatever. But at the same time, just imagine 1,000 VMs in a rack firing up to respond to your blog post. Doesn't seem very appealing. And then from a developer experience standpoint, this is really what we're enabling with Next.js at a very large scale. Is that we also don't want people to necessarily have to think or remember to apply caching. And this is kind of why we, we took that idea of the CDN, but now we're like really taking it to the next level because CDNs always require this calibration of components where the frontend layer has to coordinate with several layers of caching. You know, over the years I've talked to so many people that have frontend that combine a Redis cache and then beyond the Redis cache, there is the CDN cache. And then there is all this brutal purging and invalidation strategies all over the place. And then when you peel all these layers of complexity, you remind yourself, oh, you know, I was just working on this simple page that had this simple content, right? If you think about the e-commerce example that you just talked about, like, underlying JSON data structure for that page that renders the e-commerce item and recommended products and so on, it's very simple. So, the idea that you could convert that into HTML and serve, you know, every market in the world with that precomputed HTML is extremely compelling. Yeah.

I mean, and if you think about, like you said, there's multiple layers of caching. The last big company that I worked at, every time there was a problem, the engineers were always like, it's a caching issue. It's just a caching issue, right? Because it was like, you know, CDN, and then in front of, you know, in front of all of the application servers, there was a Varnish cache. And then there was, you know, Memcached in the back and all kinds of these other things that were just layers and layers and layers. And you never knew, you never knew where it was. And It's funny that you mentioned, you know, this idea of sort of like, um, you know, these, these infrequent writes to, you know, basically massive reads. And I think about, you know, going back to the WordPress example, you know, there are people who might argue, well, but you need to keep your comments up to date. Like, if, you know, you need to see the freshest comments. Well, I think about, you know, any installation of WordPress that's getting comments, even if you were getting comments, you know, at a very rapid clip, which is probably unlikely for a WordPress installation. You could just take the write, once that write happens, and then generate the static content or the comments list or whatever it was, and push that back out to the edge. We're at a point now where I feel like the edge has become the only cache we might actually need if we do these things right.

Yes. I think, thanks for using— also reminded me of the Varnish example because the cleanest architecture you could think of is one where there's no layers of caching that need to be coordinated, right? That's why I don't think about this new wave of edge as necessarily a cache. You know, for those of you that have done versions of this by hand with S3 and CloudFront, you know, when you put content that gets generated into a bucket, And you know that this bucket is super highly available and it's super easy to think about how you could invalidate the edge once this specific writes happen to that bucket. You don't, you don't really think as, as a complicated caching scheme. You think more about it's just a simpler model for reasoning about your architecture, right? And, and let's analyze that common example for a second, right? So Let's say that you're the New York Times and you have a very high comment throughput. So first of all, there's high-value comments that they inline with their page because they contribute substantially to the narrative. So they're the highlighted comments. So there's not gonna be lots of highlighted comments. There's gonna be like maybe 5 or 6. And they're almost like an extension of the article at that point. They're like, you know, just like you would want to statically inline the paragraphs, You want to statically inline the highlighted comments. And those, again, are not subject to this strongly consistent read system. Like, if the comment that gets promoted to the highlighted ones takes a second to reflect in the global cache, in the global edge, that's totally okay. It's the right trade-off to make. And then when you paginate, when you read the long tail of trolls or whoever, You can lazy load those on the client side, right? And you can actually go against your database that has high read throughput without a cache there either. You could go to Dynamo or whatever and tell, give me the very latest comments. At this point, I want a strongly consistent read. Give me your very latest comments. At that point, by the way, also New York Times performs moderation on their comments. So they're throttling the writes anyways to make a better quality product for their pages. So the system just like fits like bread and butter, I think, you know, like at the end of the day, we're all in this business of publishing quality content that we want our visitors to consume, ideally in that first TCP packet. And what I say is that, you know, I'm a really big fan of deleting code. I don't want any code to execute anywhere, right? Like if I'm going to newyorktimes.com in— I'm from Argentina, in Buenos Aires. I don't want all this Turing-complete circuitry to be in between me and the landing page of an article. I just want to go direct to that HTML stream. Browsers are so good at rendering a stream of HTML already. And think about it, we've deleted all the code. There's no JS that needs to run on the client side to give me that first paint of the article. Everything has already been pre-computed, so there is no function execution in between me and the content. So it's this crazy combination of availability, performance, greener for the world, and just overall better.

Right, so I wanna jump back to your comment on databases because you mentioned DynamoDB there, and I'm a huge proponent of DynamoDB. I love this idea of just having these, you know, super fast single second or, you know, single millisecond, latency to retrieve back data, but you're still oftentimes querying a database, right? It's still technically a database. You still have to wait for that computation to happen to bring that back. And I like things like DAX, like the, you know, being able to put a cache in front of that. But I also find that, you know, if you do it right, you can cache, you know, whatever that GET request is, or that query to DynamoDB, you can even cache that. At the edge, right? If it's responding back from an API call. Totally. But the point that I wanted to make was, you know, people using, you know, Aurora Serverless or RDS or whatever it is, and using these, you know, using these relational databases, there is absolutely a need for relational databases, right? You can't run analytics on DynamoDB, you know. Totally. There's other BI tools and things like that. But this is something I talked about actually a while ago, where if your frontend customer, if the person accessing those comments, for example, they don't need to sort them in 10 different ways. They don't need to join them in a bunch of different ways. They don't need the power of that. So what I found is I've been able to eliminate almost all of the clusters of databases that I've had and use something as simple as Aurora Serverless with 2 or 4 ACUs. So really, really small. By simply replicating data out of DynamoDB into that, right? So now I can serve massive load with DynamoDB, but then when I need to write those queries or even run reports, or, and I'm not having, you know, hundreds of thousands of people hitting my reporting site or my analytics site. Totally. Totally. That my ability now to have a smaller footprint there is, again, I think this is what we're trying to do with serverless, right? We're just trying to reduce the amount of footprint and the amount of extra computing power that you need.

Yeah, yeah, yeah. I think you nailed it because you're going at the core of the database problem and data access problem, which is understanding how the data is being accessed. And you mentioned something there, which is like, you know, what is my throughput of queries for analytics and like complex joins and like, let's find the top 10 most active commenters on my website and like things like that. It's very rare. It fits very much systems that can respond more slowly, that can take their time to scale up and scale down. And then you have the other layer that you touched on, which is, again, like, does it matter how fast my database is, how real-time, how scale, how serverless, if my customer just wants a bunch of HTML of a certain set of comments?, right? In that case of the example that I gave of like maybe the first 5 associate or the most voted ones and so on. So I think what's important for the developer to always think about is that think about that access pattern of your data from a read perspective, from a write perspective. And I will say also not just the ratio of volume, but also the consistency. So that's, I think what's really important as well is that when I go to a, a breaking news page on COVID-19. And I'm, uh, I work at New York Times and I'm gonna push an edit to it. I can afford for 1 second to pass before I can make a strongly consistent read of the typo that I just fixed. Right. And I will have wanted that within that second to all the reads to go uninterrupted globally in the world. Because it's so much more important for that smooth line of low latency access and highly available access to my story of COVID-19, that for everybody in the world to be able to read my writes in a linearizable fashion. So I can fix my typo and I can say it's 99% likely that within a second everyone will be able to read my write. Now with DynamoDB, they give you other characteristics, right? They tell you, Well, you make your write and as soon as you query that same API that you're servicing Dynamo from, you can immediately read your write, right? But then everybody in the world has to go to that specific Dynamo cluster, right? And again, what, you know, what apps or websites need that? You have to really think hard about that because it's not gonna be e-commerce or at least for most of that front end of e-commerce, right? Like you might have some specific things where you really, really wanna read your writes in that kind of like very low latency fashion. Um, that might be the case, for example, when you go to the logged in section of your website and say like, give me my latest 5 recent orders. You know, you want— you don't want that to like be a weird stream of static generation that every time an order happens, you're custom making a static page for the logged in administrator, and then you start, you know, adding complexity to your permission system and everything becomes chaos. You know, and that's why I said, you know, when you think about pages that need kind of more granular data access, more stronger consistency, that have more complicated permission systems, more complicated queries, then that's better served still by a static page, but that runs JavaScript on the client that can query those APIs. And just to, um, give you that idea of like why the front-end economy relates to the API economy. Now, if we continue this example of the e-commerce website, maybe that e-commerce API will be a headless e-commerce API, you know, like Shopify and BigCommerce and WooCommerce and many others are now giving you very rich GraphQL APIs and REST APIs for querying this type of data as well. So you even have to wonder, if I'm making this really slick new ecommerce experience. Maybe you're starting a new microsite, maybe you're going after VR ecommerce, maybe you're like thinking of reinventing your frontend layer. The question becomes, am I going to be writing a serverless API with 10 queues, 1 million Lambdas, 4 Dynamo clusters, DAX, Aurora replication, if I could have bought an API from, from the shelf, right? Right. And that's kind of, I think, a question that a lot of people will be facing in the coming years.

Hey everyone, I just want to take a quick minute to thank our sponsor, Amazon Web Services. If you love serverless as much as I do, then you have to register for day 2 of the Serverless First Function. The Serverless First Function is a set of 2 free-of-charge virtual events offering the latest education and thought leadership material about serverless approaches on AWS. The first event just finished and focused on serverless for your organization. The second event coming up on Thursday, May 28th, is all about serverless for your application. It includes an interview with Charlie Bell about serverless operations, several sessions with senior developer advocates like Ben Smith, Eric Johnson, Julian Wood, Rob Sutter, and James Beswick, and a wrap-up by Chris Munz. The first event was incredibly educational, and this second event will go even deeper into the hows of building modern serverless applications. You can register using the link in the show notes Or you can search the web for AWS Serverless First Function.

Yeah, and you know, you made that the point about the top 10 comments, right? And this is where, this is something I think people don't get about serverless. And I'm not trying to be like, I understand it better than anybody else. It's just, for me, some of these things, or at least for me, the way I feel about serverless is a lot of it has to do with asynchronous operations, right? It's not responding immediately to a request. And that is one way in which we can get the latency down, right? So the edge is one piece of that. But like with your top 10 commenter thing, that's the kind of thing where, again, do I maintain a database cluster that has 50 instances running so that I can calculate on the fly who the top 10— Right. Who the top 10 commenters are? Or is that something I could delay and maybe run every minute if it was really needed to be that much? And just run that off of that small cluster and then push that to a cache somewhere or to the edge somewhere so that it's pre-calculated. And so anyways, I love the idea of pre-calculation. I just think that this idea of letting, you know, being able to access stuff as quickly as possible is just insane. And the point that I wanna make too, because this is something I noticed. I was on the Vercel site the other day, and I went down, I was looking at all of your edge locations, and you've got a really great page on the site that shows you all the different edge locations. It shows you where you are, assuming based on IP address or whatever, and then it gives you the ping and the latency to each one of these edge locations. And I think the closest one to me, I'm up in Massachusetts in the US, the one closest to me I think was in Montreal, And it was like 25 milliseconds was the latency or something like that. All right, here's the problem with calculations or computations. They have to run somewhere. You're not going to necessarily run your computations at the edge. So that's another huge disadvantage is that if you're running your DynamoDB cluster and all your Lambda functions in US-East-1, and you're trying to access it from, you know, from Brazil then, or from, you know, wherever. There's gonna be a huge delay. Trip. Latency there.

Yeah. And the only time where I— you can justify that trip is where you need to very strictly read the writes that happened at that origin, right? And yeah, I can't tell you, you know, like, uh, I'm gonna cache your latest 5 stock orders, uh, or your bank account balance, I'm gonna cache it in Brazil so the Brazil customers have a better time, right? No. I'm just gonna give you a static page that then, you know, gives you a skeleton placeholder of your balance and then, you know, goes and fetches it from wherever the brain is of that, you know, bookkeeping database. But that's— that goes at the heart of, like, again, like, the vast majority of pages on the internet should already be within 25 milliseconds of you. And yeah, you can accomplish this with layers of caching, but Things get really tricky because sometimes that cache gets a miss and you have to go to the origin at that point. You were talking about how there's been a rise of the usage of functions for background processing. Functions are notoriously— they have a cold problem unless you're provisioning or things get even more complicated. You have to think about what happens when, like, you go to the Montreal edge and you miss. And, again, we host customers with cardinality of pages in the orders of millions. And then we also see millions of deploys per week as well. So, that's why I went to that idea of, like, think about static as something that you put in a bucket. And Vercel is the process that kind of automates that process. And don't think about like, well, there's always a server there that needs to be hit, because you nailed it. Like, when you're doing this kind of background computation as a result of events, as a result of your data changing, we now generate a static HTML, and now we can put it in a highly available bucket that we're also able to distribute around the world. So from a perspective of performance and availability There's this idea that now every time you go to the page, no computation ever happens. We're just manipulating very basic static objects. And the fact that, like, you can run incredibly large websites with just these primitives is very reassuring from a, you know, kind of a DevOps perspective. Like, every time I talk to people and, like, I tell them that whatever they invested in that was a server could have been static. There's always this incredible desire to go toward that kind of place. Even if AWS has done this incredible job where Lambdas have 99.99% SLA and every system that AWS monitors is automatic and they have an incredible track record for reliability, you still want to pick the architecture that has basically the fewest number of moving parts. You can think of code as a moving machine.

Yeah. Well, I think your point about latency, I mean, you mentioned you're not going to cache somebody's banking records all across the world in case they happen to log in. If I happen to be in Europe, I want my banking records immediately. There is a certain amount of latency that obviously is acceptable depending on what it is that you're doing. But I don't know if you've given this example before. I think we've talked about it before in the show. For every like 100 milliseconds of added latency, you know, Amazon.com loses 1% of their sales or something like that, right? So latency is a much more important number than I think a lot of people give, you know, give credit to. And let's take that example.

Exact figure. There's this famous, you know, internal memo from Google about like numbers that every developer should know. And one of the key numbers is the coast to coast, Netherlands to California, right? It's somewhere around the ballpark of 150 milliseconds, you know, like just to like do the complete round trip. And, you know, with Improved our, you know, routing network so much that we've optimized California into the Netherlands so much that it's just close to the physical limit, right? When you think about incremental static generation and putting pages next to customers, you already have there, you know, you mentioned you from Montreal, you said 25 milliseconds. We already have, you know, a leg up of 125 milliseconds. Mm-hmm. That is unsurmountable for the traditional, you know, server, server-full or function plus CDN case that event sometimes has to go to origin. It's absolutely unsurmountable. And then we don't stop there, right? Because we make massive investments also in the Next.js layer, for example, to make sure that that content also, when received by the web browser, renders as soon as possible. So we have several integrations with Lighthouse, and now we ship the integration with Chrome Web Vitals that allows developers to measure the time to the first contentful paint. And this is where I always stress that I want to delete all the code from the world, and I don't mean no code in the you know, Webflow kind of sense. I mean, no code in that if you have a stream of HTML coming in from the Netherlands and it's some product that you want to buy, then when the browser starts interpreting it, if it has to boot up into the V8 VM to start executing JS code, then you're going to waste another 100 milliseconds for sure. You know, like 100 milliseconds, like V8 loading JS and starting to interpret it in 100 milliseconds, that sounds like nirvana, you know what I'm talking about? Like if everyone goes to their terminals right now and they run time npm --version, you can see the V8 warmup time in real time. Like you'll see it. I'm gonna run it while we're talking now just to get my own measurements here. Time. Sure. Npm --version. So what's happening here? We're executing Node, which is booting up V8, which is executing a bunch of JS. That cold hit that I just performed in my machine, which is also busy because it's executing the, the stream that we're doing— 812 milliseconds. That just sounds insane, right? 812 milliseconds for V8 to boot up npm's code to get parsed and compiled and return back to, uh, standard out. I run it again. The universe seems hotter is 174 milliseconds. Still awful, right? Yeah. This is why I don't want to run code. I don't want to run it at the edge. I don't want to run it in a worker. I don't want to run it in a function. I want, I want the function to have been executed at some point in the lifecycle but not in between my customer and the page. And then I want to have as little JS as possible also when that page runs in the web browser. So this is why, you know, we started with like why Vercel is focusing so much on frontend. I suspect that a lot of your audience, my audience, also sometimes over-indexes in measuring backend, measuring Dynamo latency, measuring ELB latency. And then they forget that there's this universe of complexity that we're shipping to the web browser that is adding that 100 milliseconds that you just talked about times 10, you know? It's not even like a couple hundred millis. Like, what we just talked about with that npm version exercise is saying, you know, the worst— the best case scenario of blocking your entire page on JS booting up, we're talking about downloading the JS. Hopefully it's a hit from a cache, right? Hopefully it's a hit from a cache from the local computer, which, by the way, a fantastic essay just came out that we all know, especially all of us that have worked extensively with AWS, that disks are pretty slow. IOPS are expensive, but also the distinction between a hard drive and an SSD and what Google Cloud calls the local SSD, right? Like the one that's like wired right into your instance. We're talking about, like, lots of milliseconds there, even in your web browser, retrieving a cached CSS asset and JS asset from the local computer. And this is why, you know, we're so obsessed also about edge pre-computation. We don't even trust— and we have the data to back it— that even if you have a stable JS resource and CSS resource has been cached from the device, we don't even think that we can afford to revive that asset, bring it alive, and execute it very quickly. And this is why I'm now endeavoring towards getting rid of computation, because if I can give my customer a stream of HTML and inline CSS for the critical parts of that page then what I end up is with something that can actually rival the performance of amazon.com when it, when it comes to their own products, right? Because what do I get? I get from the edge, I get the precise image for the size of the device that I'm serving for the product that I want to buy. I get the styling for the buy button, which is what I want my customer to press. I get that first paint in 100 milliseconds. We've altogether removed JS from the equation. JS is not being executed at the edge, is not being even executed by the page on the local device. So, you know, we can make that kind of dream happen of like your product is in front of your user's eyes in 100 milliseconds. You know, that's doable even for, you know, 2G connections or like legacy Android devices. It's It's just that we really need to shift our thinking and our obsession from backend architectures and AWS charts connecting 1 million pieces into now thinking about what we're serving to our users. That is a big shift that's happening. Yeah.

And so, I love that idea because I think there's been a movement lately of sites that run completely JS-free, right? Like you do not need JavaScript to make your dropdown menu work. You don't need it to place an order. You don't need it to do a lot of these things. It's funny, but they invented this thing called HTML and CSS that allow you to do a lot of really interesting interactivity without using JavaScript. Now obviously, Vue.js and React and all these other things add really cool features, and if you have it enabled and once you've got it loaded on your machine, Uh, you know, and interacting via the API.

And that's why, and that's why, that's why with Next.js we're, we're giving you that, but we have to find that balance, right? Like, you're right, like, for the first paint of most of the pages that you visit, there is very little need for JS, but even when it's needed, it has to be consumed in small amounts. So this is why, you know, one of the big hits that we had was that We eliminated the idea that you have to configure bundler and webpack when you use Next.js and when you produce these pages, because that's when things start to go really bad. We want the bundler to be so kind of ingrained into the system that let's say that that buy button, when you press it, you do want to use some JS because you think that, for example, just like Stripe does with their, you know, credit card modal, You think that you're a PM at e-commerce company and you think, you know, if I transition them to another page and if I didn't memorize their credit card details and if I didn't autocomplete their credit card and whatever, like, we're gonna lose sales, right? And that's a valid argument, but that's the point, is that JS needs to load just in time for that specific interaction that's going to happen., right? Like we have to be smart so that we bundle JS in minimal amounts only for the interaction that's likely to happen, which in this case is buy. Maybe as you start scrolling, it's loading product recommendations or loading the carousel of related products. And that's why like, you know, static generation also always gets combined with, you know, loading amounts of strategic amounts of code —on the client side with JavaScript so that you can bring interactive experiences. Another example is I believe I've seen that Amazon loads some 3D navigations of some of their products, right? And you don't want that bundle to be— or that feature, which is this kind of like long-tail feature that maybe some customers use— for that to be blocking what we call the time to interactive of the buy button. And that's, by the way, what's happening to every website of every visitor of most every website in the world today is that the bundler is— and we talked about that npm example that in my computer took 800 milliseconds. What's happening there, for most of the seasoned JS optimizers in the audience who probably know this, is that V8 is crunching through vast amounts of code that have nothing to do with rendering the version, right? So, like, NPM is not being bundled in such a way that, you know, all we did was execute, you know, return, uh, version 9.2.3. Instead, we're probably doing lots and lots of unnecessary stuff before. And this is what happens with the web at scale today, is that in order to— for that buy button to become interactive, we're still downloading the code for the 3D navigation carousel. Right. And again, this is what I think makes it really, really compelling for companies to stop thinking so much about full-stack development and instead focus most of their attention to this kind of problem. Yeah. Start measuring what actually happens when their products are getting— are being delivered to users.

Hey everyone, I just want to take a minute to thank our sponsor, sponsor Stackery. If you're building serverless applications on AWS, you have to give Stackery a look. They've built a next-generation SaaS platform that lets you securely develop, deliver, and manage serverless applications quickly, consistently, and at scale. It doesn't matter if you're building greenfield or refactoring existing monoliths. Stackery streamlines and organizes your AWS SAM and CloudFormation templates, environment configurations, and credentials. This creates a development workflow that not only makes sure everything is properly configured and instrumented,, but also increases engineering velocity by 60x. They provide a CLI, VS Code plugins, and IDE extensions that let you locally debug any Lambda in any language or framework, even against remote cloud resources. Stackery gives you architecture visualization, one-click access to tracing and logs, local debugging, and so much more, letting your team focus on app architecture and business logic, not YAML. Chase, Farah, Danielle, Tim, and the whole team over at Stackery are awesome people, and they've built an incredibly useful product. So for more more information or to sign up for a free developer account, check out stackery.io.

That's S-T-A-C-K-E-R-Y dot I-O. All right. So we talked a lot about frontend, right? And I think there are so many optimizations that could be made there. There's so many cool things that we can do, like the pre-rendering, the pushing stuff to the edge, getting those latencies down, not trying to load stuff from local cache even. There are a million things to think about there. And in a perfect world, we could do everything we wanted to do that way. But the reality is, is like you said, sometimes we have to load, you know, preload credit card data or something like that, you know, for a user. So obviously, sometimes we're gonna have to make an API call. We're gonna have to access a database or DynamoDB. But as you said, how much of that calculation do we wanna be doing ourselves? How much of it can we do directly from the frontend? Vercel's got the serverless function capability. I think you had mentioned this as a use case where maybe you're doing authentication and you need to send in some private key into Auth0 or something like that to trigger the workflow. That's something where I think you'd advocate build a really simple function that just triggers that thing there., but don't do all that calculation yourself.

Yes, basically I think the, the, you, you mentioned, okay, how do I execute my functions, right? Like, do I execute them as a result of pipelines like SQS, SNS, whatever, or do I execute them just in time? And both of them are absolutely amazing, right? The executing just in time case, uh, throughout the years has had, you know, ups and downs, let's call them, right? Because We know that P99, you have to be smart about the size of your function. You might want to provision your function for very, very, very amazing P99. In some ways, I feel like the background async computation usage of functions has been like the home run, and the just-in-time has been more of a incremental adoption. It's definitely gonna happen, but it's been more of an incremental path for a lot of people. And the one that we found that is awesome in the, in the just-in-time space is giving the developer team, especially the frontend developer team, a way of gluing services together. So a great example would be what you mentioned is I have to create a function that talks to API systems that already exist. So because I want to aggregate a bunch of API calls together, because I want to talk, uh, to Stripe, for example, in a private manner with an authentication token from the user, let's say that I want to commit a charge. And yeah, like Stripe gives you products that you can invoke directly from the front end, but you start hitting some limits, right, at some point, right? And like you want to customize their UI you want to do something more fancy, maybe you have some recurring charge. And this is where, like, I think the world of, of serverless infrastructure is getting really neat because we don't have to reinvent Stripe from scratch. We don't have to reinvent Auth0 from scratch or Cognito or whatever. We can now use functions as a way of mediating between the front end and these services that already exist. And that's not to say that you You can't use these functions to talk to Dynamo directly, for example, right? That's still like a use case that'll exist. But I think what's more compelling for a lot of people is to not necessarily have to reinvent the wheel and be smart about their investment into these functions because the difference broadly between functions and also in pre-generated content is that functions now have an on-demand cost, right? And like you have to be careful about their availability. As well. Now your uptime for those functions depends very much on the uptime of the services that you depend on. And we, we talked about a lot about, okay, like, how do you even ascertain that those services that you're depending on are, are functioning correctly? And why is so much more appealing for you to be interfacing with a high-level API like Auth0 for the users table rather than using Dynamo as your users table, right? Because then you start worrying a lot about rate limiting and you start worrying a lot about, like, we talked about, like, Dynamo auto-scales until the end of time, but do you actually want to let people, you know, have this function that is invoked on demand by themselves endlessly and that now you scale with them for both Dynamo and the function? And things can get hairy. So Function I see as this important tool in the toolset that has to be used when it's necessary to use, necessary from a data consistency perspective, as we talked about. Another need, by the way, that's very strong, no pun intended, is when we talk about precomputation, we're talking about vast categories of pages that are public in nature. We gave that New York Times example, that Amazon.com example. Of course you would want to push those pages to the edge. They're all public and they're all shared by lots of users, right? Even if they have some, uh, what I call page variants, which is something that we're incorporating into Next.js, which is like this page will be in a certain language for the Netherlands and this page will be— will have a built-in promotion for Texas. Uh, you know, those are like what I call like variants of pages, but what they all share in common is that they all address vast numbers of users and that from a security perspective they don't contain anything that is sensitive. Now let's think about product recommendations, right? Uh, now that are user personalized. Now let's think about your order history. Now let's think about, you know, your credit card, inputting your credit card and whatnot. Those are all things that no longer fit that neat world of precomputation from many perspectives. One of them being, you know, that it becomes prohibitive from the explosion of combinations of precomputations that you could make, but also security. And again, like, I don't want to cache personally identifiable information at the edge. From a data strong consistency perspective that again, I don't want to go to a stale cache for performance and availability reasons. I want to go directly to the data source and that's where functions come in. Now, some of those functions will be written by your team. Some of those functions will be assisted by other teams because those functions can call to all those other functions. Some can even go directly from the frontend. You know, and those three are all amazing legitimate use cases that get enabled by executing code on the client side.

Right. And I think that when you start talking about, you know, Cognito and you start talking about Lambda functions and DynamoDB, there are a lot of primitives that exist in the cloud right now, you know, that you can stitch together. And as you said, functions are great for gluing these things together. There's other ways to glue these things together. Even though there are these amazing primitives out there, though, it doesn't mean that building a serverless backend is easy. Correct.

Correct. I think what's amazing about serverless is that it's exposed the essential complexity of the problem. It stopped developers from sweeping hacks under the rug. The best example that I think from this is you can no longer do async computation as a result of invoking a function that easily anymore. You know, in the world of Node.js, I would see a lot of customers just like put lots of state in a process. When they respond, they continue doing things behind the scenes in that same process. Functions have altogether made this impossible. But for a great reason, right? They were exposing, hey, that side effect that you were computing, you should have not been doing in that same process. You should have used a primitive like a queue to put your side effect, your event there, and then use other functions that respond to that event. And then it's so smart that they also put the developer into this state of success of saying, well, if it's a side effect that now can no longer be retried by the client, right? Because the client is executing the function, the function is responding with 200, and it queued the side effect. So there's no reason for the client to retry. So now this side effect is loose in the universe of computation, and that means that we need a system that can retry it because we want that side effect to run to fruition, right? So now it forces you to put that into a queue, and queue can retry and then eventually also fail and go into a dead letter queue. So now just like going through all this in my head, I'm like going crazy about the amount of complexity. But here's the thing, and this is why I love serverless, that was to begin with the essential complexity that had to be managed to begin with. What we were doing before was chaos, was side effects that maybe sometimes run correctly and sometimes not, was, you know, unscalable systems and so on and so forth. But it is a complicated world. Yeah.

I mean, and I think, you know, you mentioned this idea of scalability and that's one of those original promises of serverless, right? Like just everything can scale infinitely, you know what I mean? And I think, you know, you mentioned, you know, that DynamoDB can just keep scaling up. But there are limits, right? Eventually, that does stop. And there's, you know, there's soft limits in place for Lambda functions. And of course, there's wallet limits, I think, for anybody out there who's eventually going to say, "This is more than I want it to be." But you mentioned that, you know, on-demand serving of data versus the static piece of things. And you had shared an example with me before about a site that was mostly frontend serverless, had a little bit of backend serverless, and even though it scaled up, I think it was, you know, tens of millions of hits or something like that, that it was able to scale gracefully. I'd love for you to tell that story, 'cause I think that is like the perfect example of how we should be thinking about building serverless applications, because even if you think your application will scale infinitely, there are a lot of reasons why it will not.

Yeah, yeah. I love to give this example because on one hand, we deal with customers that have very traditional, websites that fit under this incremental static generation umbrella. Our most recent, we onboarded a couple of weeks ago, is barstoolsports.com. And they fit under this category of the New York Times website that we're talking about. You know, they want their businesses assisted by subscriptions and ads. They want those conversions to happen very quickly. And frankly, they want their customers to get the news as soon as possible. And this was a larger team and, you know, they were proficient. They had already chosen Next.js. But I'd like to give this other example that happened that same week of a meme that went absolutely viral throughout the entire internet and where the promise of serverless really came to fruition in a very— I mean, again, this meme is weird. It's called BillClintonSwag.com. But what's incredible about this meme is that you would go to Twitter, we notice a spike on our Edge network because we get alerted when there is abnormalities, and this is a very large abnormality that normally we would confuse it even with an attack because it was like, holy, holy moly, like we went from like this, this little thing went like this and which went completely vertical. And this meme is very basic, it's a static page that presents a photo of Bill Clinton holding 3 albums, and then the visitor can autocomplete and this is where like client-side JS comes in, autocomplete and find their 3 favorite albums. BillClintonSwag.com should obviously still be around, but the meme has sort of faded a little bit, kind of like the serverless spike patterns. And that's when like computation starts happening on demand. So going back to the glue pattern, the one individual developer that, by the way, again, this is a meme that is quite controversial to begin with, so that's why he got all this traffic. But think about this, right? Like, you can create with one person a thing that dwarfs the traffic of most websites on the internet for a very short amount of time. In this case, uh, we're talking about, uh, tens of millions of hits per day, and it is scaled infinitely, and it was created by one person. But here's the thing, he could afford this because he designed it with this static-first mindset. There was no function that was getting executed when people were first going to the website. Then he didn't write a database of records, he was using the Last.fm API. Now guess what? The Last.fm API, for whatever reason, was not consumable for the— from the front end directly. So what he did is he created a serverless function in Python, also hosted on our platform, with 128 megabytes of memory, so very lean. Also caching at the edge. You mentioned this earlier, by the way, but what he discovered was that these meme makers all like the same albums in, in like very— we're talking about very large numbers here. We're talking about like trending topic on Twitter kind of large numbers, right? So they were all kind of autocompleting to the same things, you know what I mean? Like, uh, uh, you know Beatles. Well, probably actually not that one, but like, I don't know, like, uh, who's that famous, like, new, uh, Kendrick Lamar, you know I mean? K-E-N. So the function was lean, the, the infrastructure was actually Last.fm, and he also added his own, uh, caching at the edge of, you know, query parameter was K-E-N for Kendrick Lamar. So he was responding from his serverless function with cache control And we were catching that at the edge as well, meaning that you were getting the suggestions for albums for Kendrick Lamar in milliseconds. And it was really amazing to see how this kind of combination of what a lot of people call Jamstack, right? The first paint is static. It affordably went to hundreds of millions of hits that week. Then when he needed to do a computation, He was careful to pick the best provider for his data source that exposed an API to begin with. Then he also cached the computation of that. And it was, it was, yeah, it was honestly amazing because like with these two pages, he kind of like navigated the entire world. If you think about doing this with the old patterns that we had, he would have collapsed several times over, uh, or it would have been incredibly expensive, right? Like we talked about, like because your— if he had made the landing page a function, it would have been a pretty hefty bill. If he had made it a server, it would have not scaled very, uh, quickly, not very easily. Um, and frankly, like, the ease at which he also put this together, um, was just a great validation as well. He happened to be a Python developer, so he chose Python for his functions. A lot of our customers use Next.js for this, right? Like index.js is your React page that gets built statically, and then you have your functions on the side. But yeah, it was, it was kind of like a micro example of how— and I think we're gonna see this a lot at scale in the future— where like very small teams that, that uses primitives in a very smart way for front end can now go from, you know, experiment to kind of world takeover Again, this is a silly meme, but this is the same thing that can happen to your product. This is the same thing that can happen to your news story. This is the same thing that can happen to your e-commerce item. In fact, this is also a micro story of that because he was selling some product as well. Going back to the 100 millisecond thing, he created a delightful experience that was very fast to load and then he was selling something as well. This is the whole story of the internet. Publish fast, you know, make it accessible to everybody in the world, have great, uh, um, you know, uh, success— have great success story where like your backend, your frontend, everything is actually working and it's not collapsing. Um, and then find that, you know, uh, fitness function that allows you to evolve your business, right? Like I mentioned, for a lot of the, uh, publishers of, of stories that have chosen, uh, uh, the Vercel Edge For a lot of them, it's optimizing so they can render an ad or a paywall or whatever it is that they need to render to make money. So that's kind of like what you need to think about is like, okay, you mentioned that 100-millisecond rule. There is so much greatness that went into the development of Amazon.com, but for me, that was one of the biggest ones is the realization of the idea of latency in correlation with business success. When we talk about serverless primitives, you can't just over-index on wanting serverless if it's not really enabling that business success for you. When you think about 100 milliseconds, 100 milliseconds is just what it takes for a function to cold boot in a really, really good case. It's probably more than that. I think this is what the big lesson is, is that you have to really shift around your mindset to that customer finding you, whether it's in this case is Twitter or, you know, LinkedIn or ads that you're buying. And then like how quickly and effectively can you go to that page? Yeah.

And you know, it's funny, I can verify the first time a few years ago that one of my articles made it to Hacker News, it immediately killed my WordPress installation and knocked that down. It's funny because I feel like there's so many people now talking serverless first, serverless first, serverless first. I love that idea. Yes, serverless first, but static first is a really, really interesting twist on that. Because I think that— Yeah. If you just think about that for a second, it's like, what can I pre-compute? What can I put on the edge? How can I reduce the number of computations that I need to do And then anything I have to go beyond that, how do I build the infrastructure as cheaply as possible without reinventing the wheel?

Yeah, and also what's more serverless, right, than static? That's a big thing. Very good point. Very good point. Yeah.

So there are a bunch of other things that I had wanted to talk to you about, but we've already spent quite a bit of time. So the last thing I want to touch on, simply because we're in this new COVID-19 world and we've got all these people working from home,. And we've had all these big companies tell us in the last few years, like, oh, working from home doesn't work. I know that Vercel is a huge fan of this. I know you came from several companies that were also a fan of this. So I'd love to get your, your take on distributed teams and, and how effective those can be.

Yeah, I think we, we were very lucky to, you know, be a distributed-friendly company, remote-friendly company before COVID happened. And we were so well positioned to, you know, provide the best tooling possible to people that now are in this new same position. You know, we dogfooded our product very extensively. I love what you just said about like being on top of Hacker News because yesterday Deno, the new JavaScript runtime was at the very top of Hacker News.— in a very top way. I think it's one of the most upvoted things on Hacker News in a long, long time. And their website is not, you know, a Deno server. Their website is a Next.js website precomputed at the edge and hosted on Vercel. And how did they choose this? How did they start using Next.js, learn it, created their website and import it into Vercel? Well, we never talked to them. It just happened. And it's because we're out there designing this distributed systems of collaboration, especially with the open source community. People are using Vercel every day without even noticing it because they go to GitHub, they push to a repo, and guess what? That repo is connected to Vercel and it automatically builds and deploys your website to the edge, and then you get back a URL. So we're enabling this workflow for teams all over the world to collaborate in some, in some cases without even need to like teach each other anything. Like somebody comes in a team that is using GitHub, installs the Vercel app. Now all of a sudden their website is getting built for every push and you get back your deploy URL. Now you can share that deploy URL on Slack, on Zoom. You can give that deploy URL to your end-to-end testing service. So now we kind of created this new world of— it's a distributed workflow where the primitive is the URL to the front end that you're building. It's inherently incredibly shareable. It's obviously fast for everybody in the team, right? This is something that we always obsessed about a lot, was like Hey, we're building our website with Vercel. It better be fast for everybody in the team, right? Because like our, our own team is in Japan and it's in China behind the firewall. That is an amazing fitness function, by the way, I have to say, because if you try to use the internet from China, it better be very, very lean, static, pre-computed, and cached right outside of China because a lot of people escape that firewall through Singapore or Hong Kong or Tokyo. And we were so well positioned for this world. Now, I'm not celebrating COVID, but I have to say the week that COVID kind of was— the stock market was imploding, we saw the biggest peak in creation. We measured this particular metric, which is builds, right? Builds of these, uh, pages. Uh, and we have, uh, build concurrency, uh, that, uh, is quite a sophisticated system. And, and builds obviously take a lot of CPU power, so we're constantly monitoring it and so on. But that week of, of COVID mayhem was our largest week in, in, in week concurrent— in build concurrency. That same week The CEO of Slack shared that he also saw his biggest week in number of concurrent connections to Slack, number of WebSocket connections or whatever, like concurrently to their servers. So really what's happening is not a recession like a lot of people think. It's really an acceleration of teams now being exposed to the right primitives to publish pages to the internet faster, to build pages faster, to collaborate faster, to collaborate from their homes. My daughter just interrupted a meeting, you know, but that's okay, you know. Soon she'll learn what I'm talking about, you know. There's all these advantages to this new world. I'm excited about it, you know. I'm excited about like giving our tools to everybody that needs them, that, you know, can find some self in this world that is harder to collaborate in just because you are not used to it. It's not hard to be productive in a distributed manner, in a remote manner, if you install the right tools. Totally agree.

Awesome. Well, listen, Guillermo, thank you so much for spending the time with me and talking about frontend serverless, because I think this is something that is not on a lot of people's radar. They're not thinking static first, and it should be static first, Servile a second maybe is the new way we should think of it. But anyways, if people want to get a hold of you or find out more about what you're doing and what Vercel is doing, how do they do that?

Yeah, for sure. So, about me on Twitter, RauchG is my handle. I talk a lot about these topics, so it might be entertaining. Second, a lot of people ask, okay, like, I love these abstract ideas that you talked about. Static first, whatever. How do I put it into practice? So Next.js is our open source framework, right? If you go to nextjs.org/learn, we'll walk you through creating your first page of this manner. And then a lot of you are more advanced in this trajectory. Like, they already have— they're already using Next.js, already using Gatsby, Vue. They're already making single page applications. They're making— they're using static site generators. So they can deploy them to Vercel. And I think what really sets Vercel apart is this global distribution of pages that is faster for your customer. Builds are much faster than you trying to do this yourself. I talk a lot with customers that, you know, like, they've created versions of this with complicated pipelines from GitHub to Circle to S3 to CloudFront. Sometimes they purge, sometimes they forget to purge, sometimes they cache their CDN, sometimes they use their CDN as a dump pipe. So what I tell them is try importing your repo into Vercel. It might simplify your life quite significantly, might speed up your visitors quite significantly. So that's kind of the, the 3 things I recommend. Awesome.

All right, well, I will put all that in the— into the show notes. Thanks again, Guillermo. Thank you so much.

And that's this week's Serverless Chat. I want to give a huge thank you to Guillermo Rauch for being my guest this week and to our sponsors Amazon Web Services and Stackery. If you want to check out the show notes and a full transcript of this episode, you can find them at serverlesschats.com/50. For more Serverless Chats, subscribe, check us out on YouTube, and make sure you follow us on Twitter, Facebook, and Instagram. You can connect with me on Twitter @Jeremy_Daly, and if you want to stay up to date on everything serverless, make sure you subscribe to the Auth by None newsletter at authbynone.io. Thank you so much for joining me, and I look forward to chatting with all of you again next week.