C Summit 2013 No, Really These Companies Transformed Their Businesses Using Cassandra

Welcome and thank you for joining us for this panel today. The reason we called it— the reason I called it No Really is that I feel like I spend, as head of marketing for DataStax, I spend— and my name, by the way, is Lara Shackelford— I spend so much of my time talking to whether it's reporters, whether it's analysts, and/or other customers and saying no really because there's so There's so much hype in this marketplace. There are so many people who say they do everything for big data, and it's hard to differentiate and understand what's really true and what's really happening. At DataStax, we believe the way you differentiate, the way you understand, is through customers, real customers telling real stories. And so we constantly say, no, really, ask for 10 references. If— ask for 10 use cases when you're talking to a big data vendor. Make sure they've got real use cases and real customers. Who can prove real value. And so that's what we're here to do today. We're here to share use cases of 4 customers and talk about what big data means to them and how it's helped them transform their business. And with that, what I'd like to do is do a round of introductions and have everyone just introduce yourself, your title at your company, and give a couple of lines about the company itself, what you do. Start with you, Stephen.

This working? Oh, it is perfect. So I'm Stephen O'Sullivan. I'm currently working for a company called Silicon Valley Data Science. We're a new company, about 4, 5 weeks old, I believe. I started last week, formerly from Walmart Labs. The company I work for now is mainly an engineering consulting company, and our mission is actually to help companies solve the real big data problems, the real tough ones, as well as engineering platform, etc. So the idea is, you know, we will help you answer the data questions that you don't know the answers to. Or you haven't even figured out that you have a question for that. So that's what we're doing.

Stephen, I do have one question for you. What is the name of the company you formerly worked for that you can't mention?

Good question.

All right, we'll keep going.

You know, it's like the Fortune 1 company. Can I say that?

Got it. Is it a retailer by any chance?

I believe so, yeah, yeah. In Arkansas, someone like that. Pretty big.

Terrell.

I'm Terrell Deppy, Chief Technology Officer at Health Care Anytime, and we are primarily involved in empowering patients with their own health information so that they can be better informed and keep multiple providers on track with the their quality of care.

Okay, Terrell, I have one question for you. If you had to choose, would you be Barbara Walters or Oprah Winfrey? I ask, I encourage you all to go to our customer page on datastacks.com and look at a video for Healthcare Anytime. Terrell was in it and is a magnificent speaker, and it's a really great story. So he interviewed a few people there and really brought them to a new level of discovery. So I think he's got another career if he ever switches out of technology.

Is Walter Cronkite an op— is that an option? Can we go back in time a little bit farther?

Yeah, sure. Okay, Stephan.

All right, my name is Stephan Pichot. I'm the CTO at Constant Contact. We provide digital marketing tools and services to about half a million small businesses. Everything from email marketing, social media marketing, coupons, deals, event marketing, and the likes.

Okay, and Stefan— so Stefan says I can make fun of anything German about him, but I did get— receive some photos from your family members over the weekend. It involves Lederhosen when you were about 4, but I decided not to show them here, so being kind.

Excellent, thank you.

You're welcome.

Hi, Sean Knapp, the Chief Product Officer for Ooyala. We provide a white label cloud hosting platform for video. So we help essentially folks like ESPN, Bloomberg, and a number of others around the world stand up essentially Netflix, Hulu style competitors for their end users.

Okay, Sean, you get the easy one.

Ooh, goody.

Okay, so for those in the room who are local, there's a big game between California, University of California Berkeley and Stanford every year. So my question for you, Sean, is do you choose the Stanford tree or do you root for the Cal Bears?

Stanford tree, piece of cake.

I think there are a few others in the room who agree with you. I know I can look to see a few right now. Okay. So our next question is what business objectives were you all looking to achieve when you chose DataStax? So what was happening in your business? What was the primary driver? And why don't we start with you, Stefan?

Thank you. So for us, it was really a question of horizontal scale. When I joined Constant Contact about 4 years ago, was really business that had one major product, and the overall architecture was of course vertically scaling. And we wanted to drive the business towards horizontal scale and horizontal expansion, so going from one product to many products. And instead of buying bigger and faster database servers and larger SANs. We're looking at a technology strategy that allows for horizontal scale and horizontal expansion.

Great. Thanks. Carol.

Can you repeat the question, please?

No.

It's okay.

For us, the problem was that in healthcare, we have a kind of an ugly situation in that not all of the information is codified. So we have this structured and unstructured data that comes in, and the volume of the information, the complexity of that information when it comes in, is very difficult to deal with. So you end up with a situation where if you were to try and, you know, document, you know, the health information of everybody in this room, that document is going to be very diverse. It's going to be very different. So how do you deal with complexity? We have to obviously simplify it. And we attempted to simplify it and do it, you know, treat everything more as a free text opportunity. And, you know, see what it would look like if we were to treat it like a document management problem, because we actually receive these information as XML documents. That whole thing kind of breaks down. You lose context over— the data that's being provided to you. And so we really needed to go back and look at, you know, something a little, you know, a little closer to a relational model, but that doesn't have the overhead and the burden of a relational database. We need to be able to get information in extremely fast and get it back out. Then at the same time, we had to deal with doctor's notes, so free-form text, things, you know, doctor's shorthand, even emoticons, things like that that the doctor might type in. How do you handle that all in one place and keep that in a way that is easily manageable? And, you know, we started looking at all the tools that were necessary to put that in place, and we started out looking at, you know, typical big data solutions, you know, Cassandra open source. We're very fond of open source technologies. That's the first place we go to. And then once we had identified that, you know, this was a high-volume write problem, we wanted to go to Cassandra, then it was a question of how do we deal with the doctor's notes. And it was about that same time that we learned that DataStax was in the process of putting Solr integration into the stack, and that's just kind of— that was like the magic wand for us.

Okay. All right. Oops, I have to, like, preface this with— I can't really talk about what we did, but I can kind of talk about an idea of what we did. So the biggest thing that we had to do is look about— we needed horizontal scale. We needed multiple data centers, and we needed it worldwide. And we wanted it to be one— I could say one— say one dataset, one database, one datastore that worldwide I can access. Obviously, when we looked at this, obviously we look at the open source world. We look at what can we do with the Hadoop environment, what can we do with other open source technologies, and we really needed something that could scale really fast. It was— we had redundancy and we could actually scale in data centers worldwide. So one of the reasons we chose— one of the reasons that you would choose Cassandra let's just preference that, is because it can actually accomplish all those things. The other one as well that we were looking at is the type of data that you could put in. Sorry, it sounds like really weird that I have to say it, but it just is. So the types of data you could put in was— it would have lots of descriptions, the same as what Tara was talking about with healthcare data. You want to be able to search for it. And one of the things that first came out with, well, how do we do that? And then Literally, I think it was like 2 months later, they mentioned Solr. We actually did an implementation ourselves before Datastax actually was talking about it and made it part of the product. And that was really the key of one of the reasons— one of the use cases that we used. There was other use cases that would have been used for different types of information, like customer information you could use it for, and for the same reasons. Mainly, worldwide mobile data centers, and I don't have to worry about high availability or disaster recovery. It's already built in. Obviously, one of the things when you talk about that that you really have to think about is how do I handle the eventual consistency, which I'm sure we'll start talking about at some point during this. And what does that mean to the business? Because there are obviously interesting things that you have to solve. The other one coming from a very big company was things like data security and encryption and things like that, and how do you solve that in these different environments. That's pretty much what we did.

Thanks, Stephen.

Sure. Wait, was the question why we started using Cassandra or why we started using DataStax?

—using Cassandra and why you then chose DataStax. OK, perfect.

I can do both. We first started using Cassandra in the pre-DataStax era. We started on the 0.4 release. This was in late 2009. And this leads into why we started using DataStax was I was still trying to recruit the founders of DataStax when they were back at Rackspace. And ultimately, the reason why we settled on Cassandra was we started to really model out the volume of data that we work with. WeAlla reaches literally hundreds of millions of users on a monthly basis across our customer base, and those users, when they're playing video content, they're generating about 2 billion events on a daily basis, all of which are flowing through a real-time processing system and all now are being stored in our own Cassandra infrastructure. So when we looked back 4 years ago and started to spec out the overall growth we needed and the scalability, there was really only a few options. You know, it's— we quickly looked at and dismissed Voldemort, and then there was Cassandra and HBase. We can save the HBase versus Cassandra conversation for later, because that takes a lot of time. But, you know, ultimately we placed our bets on Cassandra. Going forward to why we ended up picking DataStax, as I mentioned, they really had all the experts, and every time we found somebody who knows Cassandra really, really well, and we try and recruit them, they end up going to DataStax. So eventually, if all the experts start going to the same company, you should probably start working with them, is the really short answer. Thank you.

Terrell, can you tell us more about your story, why you ultimately chose DataStax and Cassandra and DataStax?

In more detail in terms of—

Why us versus other technologies?

It really comes down to the fact that it was in one stack. You know, the whole— the integration of all of the components that we need, that's just— that was totally unexpected. You know, we're not in the business of building out, you know, these kinds of technology infrastructures. We want to solve real-world problems. We want to make people's lives better. And I would much rather be able to pawn this off or, you know, Like you said, you know, rely on somebody, you know, find somebody that you ought to be working with instead of trying to do what DataStax is doing. It was just a matter of, you know, finding— it's the classic buy buddy or build problem that we are faced with. And I don't want to have to build it if I can buy it or partner with somebody. And this really has turned into a purchase and partnership.

Tyrrell, there are a lot of established technologies that you could have chosen instead of us, or at least considered. Why did you not go with a traditional, as an example, relational vendor?

We started out with a relational effort to begin with. That was very difficult for us because of the dynamic complex type of data that we're working with. We're not dealing with— even though we're dealing with relationships and people, we're not dealing very much with relationships and data. If you were to take a look at a healthcare document, an HL7 document, because our, you know, individual bodies are so different, because our episodes of care are so different, everything in the document ends up having to be optional. And basically that, you know, a relational database that looks like everything is null, you know, and how do you go in and do any kind of meaningful development with it? Now that's on a purely technical perspective, you know, dealing with, you know, all of the unknown data that's in a relational environment. The biggest thing is that if you actually look at the scope of the schema for what HL7 can do, I don't know of any way to effectively put that into a relational model because it's so dynamic. So as a result, the dynamic column widths of a column family is really a boon. We can do cholesterol readings and blood pressure readings and weight readings over time, and it doesn't matter if I get, you know, 2 values collected and you get 200 values collected, I don't have to have, you know, any special relational structure behind it. I can just continue to load in the data.

So now that you've tried and successfully worked with new technology, what— does that mean that you've got the shiny DataStax hammer? Does that make every problem a nail?

Hey, you're stealing my line. No, we do continue to use relational databases. And it's a matter of using the right tool for the job. And when it comes to high-volume, high-velocity data, that's where DataStax comes in. But when it comes to, um, you know, at this point, you know, we really rely on our relational databases for a lot more of the transactional types of things. So all of— At this point, all of our financial stuff is still very relational, and all of the clinical stuff is really being pushed toward Cassandra and DataStax. Thank you.

Stefan, could you tell us a bit more about how you chose us, and then also why you were looking at a relational solution as well? So a bit more about that.

Yeah, so of course, we had a legacy install base, right? So it's not like building it from scratch. And if you go from vertical to horizontal scale, you always face the problem that— okay, let's say you use MySQL sharding as an example. We were actually DB2 customers, but MySQL is an option. Now you have to teach all the applications that assume data is in one place that it's actually in multiple shards. And so what we found, that actually that cost of touching all the applications and internal services we had to migrate to a sharded solution would be horribly expensive. And the second thing is that for us, and this actually comes down to licensing cost, actually is a big driver here, right? So we are a SaaS company and with horizontal scale, traditional RDBMS licenses, per-core licenses become cost prohibitive, to say the least. And so what attracted us to Cassandra was we found that one of the primary use cases we needed to scale out was actually perfect for a non-traditional RDBMS store, for more key-value type of store where we didn't need to form relations between the data. And then in combination with being able to get multi-data center replication out of the box and really transparent sharding, right, so Cassandra shards for you behind the scenes, if you want to say so, and for the application, it still looks like one gigantic datastore. It was a huge advantage for us. So we look at a lot of open-source solutions like Cassandra that we use. The reason why we chose Cassandra and ultimately DataStax was— so we have two 72-node clusters deployed today. And when we started looking around, we started working with Cassandra since version 0.7. So there are a lot of other MySQL solutions, a lot of other vendors out there, but if you ask them, so what's the largest cluster deployed? And you hear like 10 nodes. Okay, that's not really helpful to me. So Cassandra and DataStax said, yeah, we deploy 100-node cluster here, 200-node cluster there. It became pretty clear that larger installations across multiple data centers is something that you guys had done before. And then we always, when we adopt open source solutions, regardless of what it is, we always find an enterprise-type vendor that can help us with support, and ultimately that we can partner with. So we're looking for real technology partnerships and just vendor relationships. And that's one thing that you guys provided to us.

Great. Thank you. Stephen, can you tell us a bit more about how you ultimately chose us, why, what the benefits were to your organization. Okay. To a hypothetical organization that cannot be named. Yes, yes.

So one of the things, after having some discussions with DataStax, and again, when you're working for a very big enterprise company, you need enterprise support. You're not gonna necessarily have— well, you may have a team that can actually help and do that for you, but ideally you want enterprise support. So if there's an issue, there's someone who can help you fix the issue. Very important to an enterprise customer. The other thing as well that we found when we were speaking with DataStax is they were honest. I don't take that very lightly. When they said they could do something, they could do it. If we asked them a question that may or may not be able to do, they said, well, actually, no, that's not our skill set. We don't do that. But we could help you model it. Decide if that's the right thing to do, but we don't do that. So one of the things is honesty. And in this environment, I think as Stefan was saying, there's more than— well, Tyrell was saying, one of the two of them was saying it— there's more than one solution. If you're going to have transactions, you're going to have an RDBMS. Depending on which end, if you're an enterprise customer, there's probably one or two choices you're going to make. If you want a document store, you might go and choose something else for a document store. If you want to do real heavy data analytics, data processing, and you've got lots of data, you may not choose a Cassandra cluster. You may go with another solution. But the idea with DataStax was they were very honest. And they would actually— if we needed help to put these two things together, they would actually help us and do that. If we needed it. But my one was mainly, from the outset, was the honesty. The partnership that we had with them. Even though they were a very small company and the company I was working with obviously works with really large companies. In some cases, they were doing better at partnerships than some of the large, the bigger boys in the industry. And it was very satisfying. We asked, can we get some help on something? It happened really quickly. And there was, you know, it wasn't like, oh, you need to go and talk to such and such. It's like, well, we need this. Okay, this is the best person you need to talk to, and we would have them on the phone in 5 minutes. To us, that was very important. And to any enterprise company, that's very important. Even to smaller companies, that's also very important as well. It's the ability, the knowledge, the skill set that you can actually access. And the one as well, there is a community, and you can go and talk to that community.

Barbara Walters is speaking in my ear right now, so I'm going to convey a question here, actually, on your behalf. What we're talking about here is really kind of the difference between adopting open source and adopting commercial open source, right? And for us, we've had very mixed experiences with commercial open open source. We've had some vendors that really drop the ball and will make code contributions and will try and communicate, you know, participate in the community. And then we've had others like DataStax where they come to us and they say, you know, what's your biggest pain point? And we say, security. This is patient information. We're bound by PCI compliance. We're bound by HIPAA compliance. How do, you know, these are the things that are important to us. And within, you know, literally within months, you know, we get DSE3. Have you guys had, you know, you kind of touched on it here about the importance of that kind of support from the vendor, and you kind of touched on it with regard to, you know, the open source value. And, you know, for me, it's kind of like being able to do a code review on your vendor. So I'm kind of curious what, you know, if we can drive a little bit deeper into that.

Sure. I mean— sorry. Yeah, I mean, I can certainly provide some insight from Uyala's perspective. And this can go all the way back to when we first made the selection for Cassandra, because I think that this is even deeper in the community roots pre-DataStax. And I think they've really continued that charge. But when we evaluated HBase versus Cassandra— and I'm sure I'm going to I'm going to incite a lot of upset people here, but as we watched the Hadoop community move forward, it felt like playground rules, like elementary school. There was no unified direction for the community. Even today you have multiple commercial entities pushing the community, or I would argue pulling the community, but not oftentimes in a unified direction. When we made the bet on Cassandra, There was one community with a very clear vision, and today there's, you know, one primary commercial entity continuing to drive that forward. And I think that's one of the things that you see is when the broader community now says security is important to us, which, you know, we also share that requirement, they're able to actually make that happen, even if it's a smaller community, in much faster timeframe because it's a much more unified vision for the product needs to go. And I think that's something that we've seen over the last 3.5 years now be a huge advantage for Cassandra and DataStax over pretty much any of the other NoSQL solutions out there.

Yeah, it's pretty much the same for us, right? As I said earlier, for us it's not about a vendor relationship, it's about a technology partner relationship, right? There are always issues, things that you didn't anticipate, things where the pod has bugs. And so getting those fixed and a good relationship is paramount to being able to adopt open source at a larger scale, really, right? And I would say that most certainly DataStax has surpassed our expectations and provided better support to us than some of the enterprise vendors have in the past, where you're just one of many and they'll really say, well, you know, upgrade to the next version, or we'll get back to you with the next at least in 6 months, which still really doesn't help me. So for us, there's a real advantage in finding not enterprise vendors but open source vendors that provide enterprise-level support. I can give one more.

Go ahead.

So this is probably before most people here even started using Cassandra, We've had circumstances, for example, where we were running on an 0.6 cluster and the thing was just absolutely melting down. I'm totally going to commit DataStax to doing this for every customer going forward. But we called them and just said, look, this thing's melting down. We can't do compactions anymore. We have roughly a week based on the growth of the cluster until things literally just come crashing down and we no longer have an analytics product. We would appreciate some help, like, now. They had a guy on a plane, I think that night or the next morning. He sat with our team for 2 days. They firefight. They got us to 0.8. We got our compactions running again, and it was a zero downtime migration. And our teams dove deep in the weeds with theirs. But that's the only vendor— we work with a lot of vendors— and that's the only vendor we've had pull that level of a save. For us in 6+ years of operation.

Okay, just switching gears a bit. I have one question, actually another one for you, Sean. So NoSQL versus relational, you and I have talked about your view on it and where the future lies for both. Can you give a— share with the audience your view?

Sure, I can at least give you guys my brief view, and I think I have a session with Billy later on today we're talking about this more. Look, there's a place for both. Relational databases make a ton of sense for a lot of different use cases. I think we're all pretty aligned on that. Over time, who knows, right? But certainly for now, we use our relational databases. We run something like a 50-node Cassandra master/slave setup globally. But the sheer scale and volume of data that we have there versus what we put in Cassandra, I mean, we're talking 2 orders of magnitude, if not 3, difference between our relational and our NoSQL systems. And so as we roll forward, we will find that we'll use more and more Cassandra and probably less and less MySQL. There's certainly a long-term strategy, and we're big believers of this at Ooyala, which is, you know, look, Just because you have a big hammer, not everything should look like a nail. But on the converse of that, you always have to watch out for there's a perfect tool for every single need. And the last thing you want as a company is to have 50 different point solutions or point technologies. And so we do invest, even if Cassandra isn't the perfect technology for, say, some random use case that doesn't need hundreds of terabytes of scale. We may still use it because it has multi-geo replication and incredibly high availability, simply because it's there and the operational overhead is, you know, minimal, given that it's already in our infrastructure. So I think, you know, in short, there's absolutely room for both, but I expect our use of SQL to gradually decline over time. Thank you.

Go ahead, Terrell.

I'll throw something else in there. I'm curious, in the last 6 to 9 months, I've talked to a lot more and more people that are curious about big data and curious about Cassandra. And I'm curious whether your answer to this question is the same as mine. And the question is, what's the difference? Why is it so difficult for some people to grasp with the big data concept as opposed to, you know, the relational concept, or what's the transition that has to occur in thinking from, you know, for a DBA coming from a relational world into a big data world? And I'll let you answer, and then I'll tell you if I agree.

I can try. The way that I would look at that is, first of all, it's education. So you need to educate. What we have been finding is the coders, they kind of get it. And they understand, because normally they're coding objects and they look at it as a datastore. They don't really mind which one it is. It's higher up the chain you have to educate the people because they're so used to the traditional RDBMS transaction. Everything is a database and that's the way you should do it. And then you have the new world of like, Well, how does that work in this world? And am I gonna be secure? Which is back down to the security question that we've come up with. From an enterprise, security is important. In actual fact, security over performance in some cases is more important. That's gonna be the first question you're gonna get asked. And then the other one that I would say is when we look at the, you know, do you keep the relational database or do you go with a NoSQL solution, is more of a question is understanding the problem. And understanding the mind shift of actually having to solve that problem and understanding what it means to solve it, right? 'Cause we know at the moment is everyone's keyed up on transactions, right? I can either commit it or I can roll back. Well, if it fails, I can roll back in your traditional RDBMS world. If you look at it from a NoSQL world, well, what happens if it fails? If one node gets it and the others don't, what does that mean? What do I have to do? There's a different mindset there in thinking. On the preface of that, I come from an Oracle background, Oracle version 5, 5.1.2 back in 1990 when I first played around with Oracle. Love it. And that's changed a lot from version 5 to where we are today. And I'm seeing that in the market. It's— if you're a DBA, it's data. Regardless of where you store it, what you really want to do is store as much data as possible. I know a lot of DBAs have said, I wish we stored that, but we couldn't afford to store it. Now we can afford to store it in different ways. If you can store the fine-grained pieces of data that you have, you can take that information and figure out the problems you want to solve with that data. If you look at what we have, say, 5 or 6 years ago, 7 years ago, is you solved the problem. You made the application or the design based on a problem. You get that problem solved and it's like, oh crap. Now I have this problem but I don't have all the data for that problem so I can't go back and I have to go and redo it all again. Now in the case of, well, I have all this data coming in, all these events coming in. Let's just say sensor data, whatever it may be coming in. If you store all of it, you go through, I solved this use case. Then all of a sudden it goes, oh, there's this use case. Well, I can go back and resolve that use case. And you can do it faster. And most DBAs, it's data, they get it. And also at the same time is they're thinking job security, right? It's, it's, they've spent 10 years working Oracle or 10 years working on MySQL or whatever it may be. It's, it's change. People don't like change, but they get data and they understand that, hey, I can store more of it.

OK. Sean, do you have something?

Sure. I'm not sure if it's much to add, but yeah, I mean, this is a classic water downhill principle, right? I can tell you, so we started Ayala about 6 and a half years ago. And we picked MySQL. And the reason why was apt-get install mysql. It worked, right? It'll get easier and easier, but it's the same Same reason why— God, I can't tell you how often I do this. It's at least once a month where somebody comes to us and is like, hey, we have Mongo and it's really awesome and now it's falling down on us. Yeah, that's because it was easier to first install, but it doesn't scale anywhere near as big as Cassandra does, right? But Mongo is much easier to get started with. We will see over the course of time that that actually changes, right? And actually, I firmly believe that DataStax will probably be the one that makes Cassandra much, much easier to even get started with, probably because there's a large commercial incentive for them to do so. But yeah, it's a convenience factor. It's an education factor. We have about 10 to 15% of our engineering team is here at the conference today for that very reason. People who aren't even as part of the platform as a service or infrastructure teams are still here because they're consumers or clients of it. And is equally as valuable for them as the teams that are building and providing that service to the rest of the company. Very helpful.

So I've gotten to ask all the questions here, and now what I want to do is open it up to the audience for you to ask questions of the panel. So we've got a microphone set up in the middle. I would ask if you, as you have questions, just come up to the microphone and we'll be happy to answer them, unless you're asking us what Stephen's former company was. Yes. Oh, right there.

So two different questions. One, does anybody have an experience trying Hive, you know, and Cassandra for the same type of application? That's one. Sorry, not Hive, HBase. OK. Second question is, what kind of applications have you tried Cassandra for and that you will say, OK, I will not try use Cassandra with this kind of application or data?

OK, so the difference between HBase and Cassandra. We did. My team, we did some tests looking at it, and it depends on who's doing the test. If I get someone on the HBase base side, they always win. The Cassandra base side, they always win. But what we bring it down to is, when we were looking at it initially, one was support. You know, if you look at it 2 years ago, who could support us and the best way to support us? And the second one was, high availability and data center. If I had a data center go down, would I lose all my data and how quickly would it be to fail over to that data center? So initially, again, 2 years ago, yeah, you could do it in the HBase world with Hadoop. And a lot of people have, one being a company I used to work for. They did it pretty successfully. But the other one was, is in our use cases, we wanted to run it The idea was, and I have no idea where they got to with it, is to have it in worldwide data centers. So one data center worldwide. Because of the type of data we were looking at was things like product information. We wanted one product database. What we found, again, it was more of a case of they both did the job really well. At what point do I want high availability, which point do I want disaster recovery, and which one you're more comfortable with. And then also, dare I say it, it comes down to price. Who is the cheapest that could actually get me to the same place?

Yeah, so we are actually using Hadoop sitting on top of Cassandra, but one of the reasons is, of course, just like in your case, we have multiple data centers, we want to be fault tolerant, The most interesting aspect for us was though not having to move the data, right? So our Cassandra cluster, there's one of them, the data we run analytics on is extremely high velocity data. It's 100 terabytes, records come in at like 10,000 a second at peak and having to move that data into a Hadoop cluster is just painful, right? You talk to a lot of vendors in there and they start scratching their head, how do you get this data out of Cassandra into Hadoop cluster specifically if the velocity is so high. And so this is one of the reasons where this is really helpful. You just have another replication zone. You replicate to separate nodes. You run your Hadoop cluster on and it works. So we're at the very beginnings of that but very promising so far.

Great. And I think the second part of the question— did we capture the second part, which I think was what use cases does Cassandra has Cassandra not worked for? Does anyone have an example?

I mean, this may have changed, but we used to do a read-modify-write. This is sort of the pre-increment era. That's a terrible idea. Don't do that. That hurt. There's a lot of, I think, you know, debate today around whether Cassandra, though it works really well for real-time or batch-based or some other combination thereof, I can tell you our teams are doing a combination of all. We run real-time analytics. We actually have a new system that we're building that's coming out that unfortunately is in the room next door right now that one of our guys is presenting on, but it uses Cassandra as the underlying layer with Spark, which is all in-memory processing, with Shark on top of that, which is a Hive-type query— it's actually a Hive-compatible query language. That allows us to do remarkably fast rollups and, you know, sub-100-millisecond queries against, you know, millions of rows within Cassandra. And so we're finding that we're able to actually do both real-time and batch-based processing, everything from content recommendation algorithms that are running true, like, you know, typical Hadoop MapReduce type models to, you know, real, you know, sub-100-millisecond top-K type queries.

One more.

I'd like to answer that too, maybe from a slightly different approach. Maybe what's the— at what point does the sniff test not get passed? Or when does the radar come up and you start thinking, well, maybe this isn't the right tool for the job? A couple of the things that I think about are, from our perspective, we're looking at this from a very time-based perspective. Of what's happening to your blood pressure over the course of your life, what's happening to the growth chart for your children, that kind of thing. So when I start looking at what kind of data that I wouldn't put in, the very first one is anything transactional. My defenses immediately go up. I start thinking twice about it. Anything that is— and by that, I mean like financial that I was referring to earlier. The other thing would be for us anything that requires a delete or an update. You know, this is big data. It's about volume. And the only time that I start looking at, you know, whether a delete is appropriate is if somehow some information made it into the system and it, you know, there was a piece of medical information for somebody that got attached to the wrong account, you know, because the wrong identifier was included when the record came in. At that point, we would be looking at an absolute delete because we've got to get— we have a HIPAA violation, and we would have to get that out of there. You know, somebody fed us the wrong information. So from that perspective, I look at it and think, you know, if— as long as I'm writing and as long as I'm reading, I'm happy. As soon as I start thinking about updates and deletes, then I start getting concerned. And as soon as you add to that transactions, then I go, well, wait a minute. Now, is this really the right tool for the job?

Thanks, Terrell.

You led into my question. Can you talk about HIPAA data and HIPAA-protected PHI data in DataStax, some of your experiences with that? And using— having DataStax be the system of record for some of these medical records that you may be holding or processing and such, what are some of your feelings, experiences with that? How much of the data do you keep in DataStax? How much would you potentially put in a relational store or some other store? As you just mentioned, some of those scenarios, if you could expand on that a little bit, please, I'd appreciate that.

Sure. In terms of treating DataStax or treating Cassandra as a system of record for personal health information, or PHI as he was referring to it as, I have no problem at all using Cassandra, especially with DataStax behind it because of the encryption capabilities. Cassandra as the, you know, primary system of record for health information. Now, just, you know, we're dealing, like I said earlier, we're dealing with complex data, and what we've done is, you know, we've looked at it and said we have to bring this down to its simplest possible form, and obviously the simplest possible form is key-value pairs, right? So we're gonna throw a ton of volume at this problem, not planning on deleting or updating unless, we absolutely have to, but that means that there's also going to be duplicate data in the system. You know, we look at it from the standpoint of what is it that I want to see, and, you know, DataStax handles the encryption aspects. You know, we throw our own encryption on top of it as needed, and, you know, we do everything we possibly can to keep this information protected, but as a system It's there. Any information that we don't use, we toss it off into long-term storage, and then at some point in the future, when we realize, either because of a regulatory change in meaningful use requirements or as a result of, you know, some analytics question, you know, maybe a pharmaceutical question as to what's, you know, what the best drug is for a particular demographic, then we can go back and we can we can reprocess this information, get information that we don't have from our legacy file store. You know, think in terms of tossing the original document, this original XML document out on S3, for example. Bring it back in, give Hadoop a little process to start running against the data that we currently have in Cassandra, work on the original document, pulling data, you know, information out that we didn't already have, you have, and come up with the necessary analytics on top of that. I hope that answers your question.

All right.

Thanks, everyone. Well, we're at the end of our time. So what I'd like to do is especially thank our panelists today. Really appreciated your insights and your real-world stories. Encourage all of you, as you're considering big data solutions, ask the question, no, really? And ask for your 10 use cases. And get this level of detail from the vendors that you're talking to. And have a great rest of the conference. These gentlemen will be at the conference, I think, at least today, most of tomorrow as well. And so if you want to track them down for any further questions, I'm sure they're happy to talk with you one-on-one. Thank you.