Scaling your Website with Memcached, discussion with Couchbase cofounder

So who are you?

James Phillips, co founder and Chief Strategy Officer of nordscale.

Yeah, and what's northscale? Real quick.

So northscale is a company that is providing what we've called Elastic Data Infrastructure software. And what that means in a nutshell is the ability to deploy database and data caching technology in a manner that allows you to expand the cluster horizontally versus scaling it up. So it's a scale out versus scale up architecture.

I just told you I'm getting around to all the companies that are doing support for big data systems. When we were over at Cloudera a few weeks ago. Yes, and so you're doing memcached, right?

Correct, we're doing that.

They do Hadoop and you focus on memcached, explain where you fit in the marketplace. You were about to draw something more. That's why I came over and started turning on the camera.

So we're doing something called memcached as well as a product called Membase. And Membase is a memcached compatible from an API perspective. When the application tries to put something into memcached, the API, the network protocol, is the same on membase. The difference between memcached and membase is that memcached is a cache and it is designed to temporarily store data items, whereas Membase is a database, it will permanently store data items. In. Many organizations that have used memcached have asked for a database that's fast and scalable and simple, much like memcached, but that is in fact a database.

Now I remember talking with Mark Zuckerberg years ago and he was saying, oh yeah, I'm using memcached to scale out.

Yes.

So for a newbie, for somebody who doesn't understand this world of big databases that Facebook and Twitter and everything else builds on, can you explain what memcached is? Yeah, absolutely.

So memcached very simply. Most web applications today start with relational database technology. You'll grab MySQL or Oracle or SQL Server from Microsoft or some similar relational technology, and the database sits behind the application. The application is accessed typically via browser by a user, and the application, when it needs data, goes and gets it and puts it back in the relational database when it's modified. Now, relational database technology, a technology, frankly, was invented and optimized for the 1970s computing environment. It is a technology that provides for strict transactionality, very debit credit like transactions, guarantees around consistency and atomicity, and a bunch of other properties that collectively are called asset properties. When you put something in, you know it's going to be there when you go try and get it back in. And there will never be a time when anyone could try and get it out when it wasn't what you tried to put it in the first place. Very strict guarantees. Turns out that not all data in web applications is debit, credit, business transactions. And there's a lot of information that, you know, if I put it in there and 50 milliseconds later someone pulls it out and it hasn't quite changed yet. It's not a big deal in the grand scheme, Right, Like a tweet. Exactly like a tweet. Or a Facebook update, or frankly, even a change of address with my bank. If I open up a browser 50 milliseconds later and it doesn't look like it's quite changed yet, the world's not going to spin to a halt. Right. And the cost associated with providing those transactionality guarantees, those asset properties is extremely high. And one of the costs is that in order to do that, right, you for the most part need to run the database on a system that shares cpu, memory and disk, you know, sort of a all on one box.

Like a mainframe.

Precisely. And the technology was invented in an era when mainframe computing was the dominant form of computing. You just get bigger and bigger boxes. Now, in a web world where you've got an unprecedented number of application software users, 2000 users in 1975 was, you know, an interactive software system supporting branch banking or, you know, travel agents or ticket counter reps, there are very few organizations had software systems that were interactive that supported anything much larger than that. Today, if you've got 2,000 users of your web software, you're failing.

Right.

And you've got.

I met the kid who did chat roulette in line at the iPad launch. In three months, he's gotten 20 million unique visitors per month in the last 30 days. He had 20 million in three months.

Insane.

That's totally insane. I mean, ICQ in 96 was the hottest thing, right? And it got to 65,000 after six weeks and it sold for $400 million.

Mind blowing. So being able to support that sort of explosive user population growth is not something that this class of database technology ever really envisioned. And if you combine that with.

You can't buy enough mainframes to support that kind of growth. Not to mention you're not going to get funded for buying a whole lot of.

Exactly, right.

Some kid in Russia who wrote chat roulette, you Know what? You're not going to get funded for a couple of months at least. And even then you're not going to get that much money.

Exactly. So it turns out that as you begin to the dominant form of scaling web applications is by scaling out, if you've got more and more users coming in with more and more browsers to your application, typically you throw a load balancer into the mix and you just throw more commodity servers behind it with web server software and off you go. Relational database technology, however, is a scale up technology. Instead of just throwing more boxes, you actually have to get a bigger box with more complex SMP capabilities. There's one very well known, very successful property that has a Sun E25K 64 way multi core box supporting their database. And you really can't get bigger than that.

SmugMug showed me one of their boxes they were testing out that was all flash memory and it cost hundreds of thousands of dollars. And it was just to hold this piece.

Exactly. So what memcached does to get back to your original question in the context of this is that it allows you match the scaling model that you have at the application layer down here at the data layer. So while you have to get a bigger and bigger database, what people really want to do is just throw more and more commodity boxes at the problem. And if you look at Google and the Google File system and BigTable and things like Dynamo and Amazon, those are designed specifically to scale out.

I've been inside a Google data center and they have tons of. They don't have a big computer. They have hundreds of thousands of these little computers all in racks.

Exactly.

So they've figured out how to split this database over these little computers.

That is exactly right. And that's precisely what memcached does. So memcached comes in alongside your database. You've got these nodes that are relatively dumb, that can store your data temporarily, that can hold the data, frankly, that has been most recently accessed. So the way it works is if you've got an application that has previously just been pulling things out of the database and putting things back into the database. With memcached, there is a very, very slight change in application behavior when this tier begins to exist in the application architecture versus going and doing a query out of the database, the application will first check to see, hey, is this item I'm about to do a query around sitting in the cache and if so, I get it and I'm done. Because it's in memory and it's distributed You've got a lot more IO capability, you've got a lot faster response time. And the interface with memcached is a key value interface. It's not a complex SQL query, it's a, hey, do you have this key? If so, give me back the value associated with that key and I move on.

Now, can these be cloud computers too? Because they don't need to be physical by a server from Dell or something and put it in a rack. Absolutely it can be, because at Rackspace we have Rackspace Cloud and we can just spin up a couple more precise Linux boxes.

Yes. And so I did that in the.

Apple Store on Rackspace Cloud app on the iPad.

Exactly. So these can absolutely be virtual machines or cloud instances, if you will. And in fact, this scale out sort of elastic model that is the dominant form of architecture around web applications is, I mean, it is what cloud computing ultimately is all about. In fact, I'm just finishing a white paper on cloud computing and how this fits into that whole, you know, architectural model. It really is about elasticity. So getting back to memcached, when the application checks to see first is this data that I was about to do a database query on already in memcached. If it gets back the data, great. It's able to then render the page and present back to the user whatever it was trying to do. If it's not, then the application goes ahead and does that database query as it always had, and it puts it in the cache so that next time it comes around, the data item will be cached. And this technology is specifically designed to scale out. So if I need to support more and more users, I just throw more and more commodity servers or virtual machines or cloud instances into the cluster and I've got the ability to serve more and more users. So you're matching this scale out model at the app layer now, down here at the data layer now.

What do developers need to know about this new world? Are there architectural things that they have to think about or tell me about some of the things you would do when you're setting up a system, when you're starting a new company and thinking about.

Sure, absolutely.

The new world of data. Real time data streaming.

That's right. So, you know, it's interesting, there's this big buzz right now around NoSQL alternative databases. Right.

Like Cassandra.

Like Cassandra and Mongo. I mean, there are a number of projects that have been initiated that are in some cases clones of work that has been done. Cassandra is more or less an implementation of Amazon's Dynamo project project. And there are a number of projects that have tried to clone BigTable in the Google file system.

For example, NoSQL means there's no SQL and the databases building on this, on this elastic model. Elastic model. I almost call it a sharding model, but it's not sharding at all.

I mean, sharding is a term that is typically associated with, with relational database technology. And in a nutshell, what it means is because relational technology is designed to scale up when you do have to break a box into two, if you will, because you've scaled up too much and you can't get any bigger, or because you want to increase I O capacity across the cluster, for example, sharding allows you to say, okay, you know what? My application is going to go over there to get information about everyone west of the Mississippi and over there to get information about everyone east of the Mississippi. So you've kind of broken the database up into something that the application can deal with relative to. Now the problem with that is that the application has to change every time you kind of hit a wall, right? So, oh, okay, now we've got to change the logic in the application. Know that anything west of the Mississippi and north of this parallel needs to go to this database and ad infinitum, right? So there's a lot of impact at the application layer. What NoSQL attempts to do in is to provide a mechanism for growth that does not require substantial changes or really any changes to the application logic itself. So I throw another box in there, my data rebalances and spills over to the net new nodes. I've got more I O capability because I have more boxes listening. And I've increased aggregate data storage capability. And the NoSQL database layer or the client layer that accesses that NoSQL database layer can fix it up. It knows that if you're looking for this, I know where to go get that. And that's kind of how memcached works. It takes the key that you're looking for and it hashes that key. And that hashing algorithm gives me an answer, which is, it's on this server, so then I know where to go get it. So more or less invisible to the application logic and handle it at that layer.

There's an argument I'm seeing some blogs lately of people who are like, stop talking about NoSQL because I'm not going there. I still want a SQL database. I need a relational database. Tell me a little bit about the fight between the two approaches, when is it appropriate to go the Cassandra style route and when is it appropriate to go the memcached?

It's a very unfortunate term, the term NoSQL, because in fact, it really isn't about the SQL query language as much as it is about the architecture of the underlying data store and whether or not that underlying data store has a schema, a very strict definition of what the records are inside of those database servers, and whether or not there are operations that can occur across servers as you attempt to do things like queries. SQL itself is, is simply a language, right? It's a query language that allows an application developer or an analyst to go query a database and to get some subset of records back that meet certain criteria. It is the truth that some quote.

So sorry, repeat that because my camera goes off every four seconds.

Yeah, no problem. So it is the case that there are some. No SQL databases that actually support SQL queries or at least a subset of SQL query ability. So I think it's important to kind of break this connection between how does the underlying database get architected and how is data spread and how does sharding work, if you will? How do you grow from what happens to be the query language, the syntax of the particular query language. It does come back though, to your question about how does a developer think about this. I would suggest that it's less a question of does the developer need to be concerned about SQL itself or the query language. Although that comes into play. A bigger question is what are the guarantees? What are the application data interaction patterns that need to be supported? If you are doing strict transactions, if you are allowing someone to buy something from you, or if you are moving money from one account to another. Asset guarantees transactionality are what you need. And in that particular case you should be doing that probably through a relational database management system. For large quantities of relatively unstructured information that's sitting behind web applications, you are paying sometimes orders of magnitude more than you need to be if you choose to put that in relational database management system. So it really does boil down to cost, ultimately cost, scalability and administrative overhead. But you can get a really, really hard ROI just looking at servers, right? That box, which is exactly like that box, this one running NoSQL, that one running a relational database. I can support 10x more users on this box than that one. It's very simple mathematics.

Now, this world worked really well for the Web 2000, where it was pages and pages and not a lot of changeable data that's going on. Right. I mean even Facebook now you see it's almost like a stream of data. Right. It's not a page you don't read well on Facebook, you reload it. But friend feed or Google Buzz, it streams down your screen. So how do you have to mix the architecture so that you still get that real timeness right. As well as the scalability?

Got it. So you can almost view this as a recursive model. So if you look at specifically you're talking about the application logic to some extent that assembles these relatively complex pages that get returned back to the users. And if you look at the way that.

And they're not really even pages, they're using ajax. Right. So you don't even have to reload the browser, it's just pouring data onto the browser.

That's right. And so both the server side and sort of the AJAX side are going to benefit from the same thing, which is you may have a set of servers that are identified as the page stitchers, if you will. I'm going to then go off and call another cluster of servers which is responsible for filling in the weather, another cluster of servers which is responsible for filling in this other component. And as all of those servers sort of in parallel create all of the aggregate content that will be stitched together by this, then it's returned back to the browser. And by the way, the browser may have JavaScript calls that are then going to other clusters to get real time updates into the screen real estate they're responsible for. So it's a model that actually works. It's not fundamentally changed this whole notion of scaling out behind load balancers in the.

You just have to think it out a little bit. That's right. What data do I need to pour into that stream and how do I architect it to be scalable?

Yeah, basically how do I get multi threading of sorts, if you will, or parallel processing of page creation and distribute that across as many horizontally scalable clusters as is reasonable.

So what does your company do in this whole world?

Yeah, so what northscale does is we are very focused on this layer of the architecture building horizontally scalable or elastic data management solutions. Memcached is a technology that is an elastic distributed caching technology. Norscale Membase Server is a horizontally scalable elastic NoSQL database solution. Both of these technologies have the same application programming interface so that organizations that begin using memcached can quickly migrate over to norscale Membase server where instead can.

You go the other way Too.

Or you can. Typically you won't though. For all practical purposes, MEM based server provides a superset of the capabilities that you would get from a memcached server. So you can kind of think of it that way. So ultimately our objective is to allow organizations to build out these elastic data layers to match what they're already doing up at the application layer. And not only is it critically important for scalable web applications, it is also a much better impedance match for cloud computing in general. Cloud computing, one of its core characteristics is this notion of elasticity, the ability to, if I need to support more users or hold more data, I just clone a bunch of nodes that kind of look like the other nodes, update load balancers. And I've gotten this elastic grid growth, right, with a linear increase in cost, ideally, and constant performance. And so building these data nodes that are replicable, that are elastic, is the business of northscale. And we just introduced two products. About two weeks ago, we came out of stealth Mode and introduced NorthScale memcached server, Norscale mem based server. And we've got a lot more to come.

Now, are you running your own data centers then? Or do I use these with like Amazon or.

You can use them with Amazon, you can use them at Rackspace, you can use them at your own data center. We are ultimately a software company, right. So we provide downloadable software from our website so customers can come get the software and install it on whatever compute environment they're using.

Very cool. Anything else I need to know about this new world of data?

No. You know, we talked about Cloudera earlier and how we relate to that whole model, so just.

That was before the camera came on, so. But yeah, remind me, how do you work with them and what, what. Why would I choose your system versus Cloudera or do I need to.

You would actually choose both.

Okay.

Right. So the North Scale is focused on operational data, right. So we are, we are providing a very fast, simple, elastic, scalable data technology to support your live web applications. Because we are relatively low cost, high performance, you can collect lots and lots and lots and lots of unstructured information which you would then perhaps mine, analyze and discover Trends using Hadoop MapReduce what Cloudera provides. So Cloudera is a North Scale partner. We've worked closely together in the market and will continue to do so. And we believe, at least I believe, certainly, and I think Mike Olson at Cloudera would agree that, you know, northscale and Cloudera together provide a modern cloud friendly, distributed approach to data management and analytics, which is, you know, in contrast to more traditional, more structured OLTP and OLAP systems or even batch analytic systems that were optimized for a different type of application in IT environment.

Yeah. Well, thank you so much. This is really helpful to get people a picture of what's going on underneath Facebook and underneath all these big sites.

Absolutely.

And it's always interesting to understand the technology.

Thank you.

Thank you so much. And where do we learn more about you guys, by the way?

Northscale. Com.

Very cool. Thank you very much. Thanks.