Solving imaginary scaling issues - Michele Riva

Hello everyone and welcome to my talk, Solving Imaginary Scaling Issues. Before we get started, let me please introduce myself really briefly. I am Michele Riva. I work as a senior software architect in NearForm. I'm a Google Developer Expert and I'm a Microsoft MVP. I'm also the author of Real World Next.js, which as you might guess, it's a book that talks about Next.js. So if you're interested, in that topic after the talk, please feel free to reach me on Twitter. This is where I live most. So, Michele Riva Code on Twitter. And yeah, we will catch up there. So, I wanted to give this talk because while browsing on Twitter, of course, eventually I've seen this image from the Practical Dev account solving imaginary scaling issues. And I thought, "OMG, it talks about me." I must be really famous if they're writing a book about me. And eventually I found out that many people is today making mistakes I've been making when I first started writing software architectures and software in general. So, that's why today I wanna debunk a couple of myths about software architecture and scalability. But before we get there, we need to understand what scalability is. So, fullscale.io states that scalability is an attribute of a tool or a system to increase its capacity and functionalities based on, on its users' demand. Scalable software can remain stable while adapting to changes, upgrades, overhauls, and resource reduction. After we read this description, which is really good in my opinion, we should focus on the last part. Scalable software can remain stable while adapting to changes, upgrades, overhauls, and resource reduction. So, we should define what changes are, what upgrades, overhauls, and resource reductions mean. So, let's go ahead and see them in detail. Let's start with changes. So the system should remain stable when applying any kind of change to both the infrastructure and application. So for example, switching from Compute Engine on Google Cloud to Google Cloud Run to make the whole infrastructure more cost effective. So while I'm doing this change, I do expect the service to be up and running, the functionalities for my web application or website or whatever. To continue running. When we talk about upgrades, the system should remain stable when upgrading any of their components. For example, releasing a new software version or upgrading the runtime. So, we basically pull a new library from npm, we do expect everything to be working properly, or I don't know, we push a new container to our registry, pull it on Kubernetes, and then we do expect everything to go smooth. When talking about overhauls, the system should remain stable during maintenance and debugging. For example, if the system is not working properly, it should be easy to inspect production deployments. So, being able to debug, for example, a live system, that should be quite useful, of course. And last but not least, resource reduction. The system should remain stable under high load. For example, an e-commerce is running a great promotion with huge discounts. There are— there's a lot of traffic on the website and the connection should remain stable. So basically, if today we have an e-commerce website, for example, with, I don't know, let's say one Compute Engine server with medium RAM, medium-sized RAM and CPUs, If tomorrow we roll out a new promotion and we have, I don't know, we boot from 100 requests per second and we do expect to have like 1,000 requests per second, as you may know, we will need more resources. So the resource for every single interaction with the server will be reduced. And we do expect to be able to scale, so to be able to bring more computing power to our machine. Or decoupling horizontally with multiple machines. I don't know. There are different strategies. But the important thing we want to discuss right now is that every single imaginary scaling issue, it's an overreaction to just 4 scenarios. So, whatever we see and whatever we do wrong, maybe it's an overreaction to those 4 scenarios. So, let's start with the imaginary issue number 1. This programming language doesn't scale. And it's easy for people to say, hey, you know what, Facebook, it's written in PHP, so PHP can scale. And other people could also say, yeah, but they are compiling to a C++ virtual machine built for Facebook itself, so it's not really fair. And yeah, we will get back to that later. And when we say a programming language doesn't scale, Facebook is a common example for that reason because they are not using vanilla, let's say, PHP anymore. And we can say the same for Twitter, for example. They abandoned Ruby on Rails and went to Scala, if I remember correctly, because Ruby cannot scale. And then we see WhatsApp. WhatsApp, it's incredible. They manage an incredible number of connections per second, concurrent connections, and it's written in Erlang. So, we all need to adopt Erlang to be able to scale our application, right? Well, not. Let's debunk the first imaginary issue. You're not Facebook, you're not Twitter, you're not WhatsApp. And when I say that, I want to say that with numbers. So, if we see how many users Facebook has, 2.9 billion in 2021, Well, we are not likely to be competing at that level of scale. And the same applies for WhatsApp, for example, 100 billion messages sent per day. We don't really have to scale that much. There are few opportunities in an engineer lifetime to be in front of such a big challenge. So, we should ask ourselves instead, how many active users do we have today? How many active users do we expect to have in the next 5 years from now, or 5 months depending on how fast we grow? How much would it cost to teach your whole team how to code in Erlang, Rust, C++, Scala, or any other highly scalable language? When writing software architectures, you will likely see that programming language, it's a detail. It's an implementation detail. And then a cool algorithm it's way better than a fast programming language. Because there are no fast programming languages, there are just slow algorithms. That's really it. And I want to show you a tweet I found during the Black Friday, if I remember correctly, last year. And so they basically, using Shopify, they were able to address 30 terabytes per minute of traffic using a monolithic Rails application. With MySQL and Resque. So please keep in mind that tweet, because this is really important. We often compare ourselves to Facebook, to Twitter, to whatever, but we lost the focus, we lose the focus on our real needs. And that tweet reminds us that there are technologies out there that are really capable of scaling. And maybe we just have to investigate more and make, you know, more optimizations on our side. So, let's debunk the myth again. Good PHP code scales better than bad Erlang code. And good Ruby code scales better than bad Scala code. So, if you already know PHP, if you already know Ruby, if you already know Node, there are high chances that you are able to scale your codebase better than you could possibly do in a language that you don't know at all. So let's go on with the imaginary issue number 2. We need to use Cloud Functions to manage high traffic. Serverless computing scales better. So by serverless computing, I also mean, you know, AWS Lambda, Azure Functions, Cloud Functions from Google Cloud Platform. So whatever. And that is typically true, but at what cost? You know, it's— they are not cheap. So let's make an example. Let's say we have an application— oh, sorry, we will do an example later. Let's see the cost. So let's say with Google Cloud Functions, we have first 2 million invocations for free. Beyond 2 million, we pay 0.40 cents per million, which is really cheap, right? We also pay for compute time. So, not only the single invocation, we also pay for every single millisecond that we are going to use on our functions. So, if we want 128 megabytes of RAM, 200 megahertz CPU, we are gonna pay this ridiculously small number every 100 milliseconds. And I'm not even able to tell the number, the cost, because these are very small numbers. We also pay for networking. So basically we, we have the first 5 gigabytes for free of outgoing and incoming network traffic, then we pay 12 cents per gigabyte, which doesn't sound like a lot, right? But let's see with a, with a simple REST API example how much we are going to spend. So let's say we have a simple but critical REST API that gets 50 requests per second. 50 requests per second means 129,600,000— sorry, 129,600,000 requests per month, which are a lot, you know. Let's say the average execution time is 300 milliseconds, and we can use the cheapest RAM option And we will require around 100 kilobytes of bandwidth per execution. And I'm being really, let's say, this is not so much, but let's pretend this is working. So this would be the cost. So $1,600 per month to handle 50 requests per second. So let's say we want to scale and we want to go from 50 to 100 requests per second. And basically everything else remains the same, but we double, of course, the costs. You know, $3,200 per every single month. So this is Google Cloud Functions, for example, but we can do the same on AWS or Microsoft Azure. The costs are almost the same. Not really the same, but almost. If we want to see, for example, an alternative that it's capable of supporting the same level of traffic, we could see, for example, Google Cloud Run, you know. So, it's not really serverless. It's not really a typical web server, but it's way cheaper. So, $192 per month and we are able to support the same traffic with that exact configuration. But we can also save some money if we go, for example, from Cloud Run to Compute Engine. $52 per month. And there's a reason why Cloud Functions exist. And that's because Cloud Run, it's good at it. So let's go on Compute Engine. But that's also the same for EC2, for, for every single VPS, for example. So Compute Engine, for example, it's really good when we have constant traffic. So for example, we say we have a constant traffic of 100 requests per second. Once we fine-tune our machine, we know that it will go just fine. Cloud Functions, AWS Lambdas are working just fine when the traffic is spiky and unpredictable. So if we cannot predict the traffic, if it's not constant, if it's spiky, then AWS Lambda, Google Cloud Functions, they make a lot of sense. And we also save some money, I would say. I wanna give you another example I found on Twitter. So, this is an example of a business that makes $5,000 per month and costs just $7 per month on Heroku. So, yeah, don't overcomplicate things. I like that sentence. Okay, let's go ahead with the third imaginary issue. NoSQL databases scale better than relational database management systems. Okay, I want to give you an image you already seen during this talk. Okay, so we've already seen that MySQL is capable of scaling a massive amount of data. So that's not true at all, but I've read this a lot of times. So I wanted to put this architecture— sorry, this imaginary issue in the talk. There are also a large number of companies using SQL databases in production today. Just an example, we have Facebook, Uber, Netflix, Instagram, Spotify, Instacart, Twitch, Reddit, and many others. So remember what we saw, what we said previously. Good PHP code scales better than bad Erlang code. Good Ruby code scales better than bad Scala code, right? We can say the same for databases. Good MySQL code— sorry, good MySQL knowledge, it's better than no, no, any NoSQL poor knowledge. Good experience with PostgreSQL, it's better than no experience with NoSQL at all. So please don't overcomplicate just because you've heard somewhere that DynamoDB scales better. Of course it does, but it's managed, so it's easier to scale. It's also way more expensive than an average SQL database that you can host wherever you want. But that's another story. Let's go with the last imaginary issue. We need everything custom so it fits our use case. OK. I will never stop sharing this image because it's a very, very beautiful image. Ruby on Rails, MySQL, and Resque. They basically didn't need anything custom, right? But even if you need something custom, you can just customize a framework. I mean, we have GitHub running on Rails. We have Basecamp running on Rails, you know. And also we have other frameworks such as Spring, Phoenix, Laravel. You can extend those frameworks and make them fit your use case. So don't write custom frameworks for your application. Also, from a security perspective, there is a huge community behind all those frameworks. So every security issue, every— everything that could be potentially dangerous for your application, it's likely to be managed by the framework itself. So please don't go custom. Don't go, you don't need it. And that's because frameworks already provide everything we need, everything we need to implement most of the custom business logic we need in our apps, as we were saying. So, we've seen 4 different myths. There might be many others that might be worth to discuss. But How do I determine if my application is really scalable or not? I would say most of the time it's not really about the application itself. It's more about the architecture, you know, but also about the team. Is my team scalable? When I say that, I mean, are we adopting this technology? So, is my team making a choice because we are excited about it? Or are we doing this because it's the best solution for our problems? So, that's a great part of any team scalability. Do we know our app traffic? Do we know our competitors? Or are we just speculating about something that could happen? As developers, sometimes we just want to have fun, you know. We find a new technology, we want to implement everything with it. We read online that this technology is better than another one. So, yeah, it's good. I mean, we are developers. We want to have fun. But I wouldn't make, you know, I wouldn't put my company at a risk just to have fun. So, please think of your team. Think of the expertise you have in your team. And remember that good Ruby code scales better than bad Scala code. And that was all for me. Really, thank you all for participating. I leave you my contacts here. As I said, I live mostly on Twitter, so if you happen to be around on Twitter, please follow me and we can keep on chatting there about imaginary scalability issues. Thank you all.