#211 What is AIOps? With Assaf Resnick, Co-Founder & CEO of BigPanda

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

If you are a medium to large enterprise, you've been around for several decades, you've got different silos of technology, different generations of technology. The people that wrote and built your initial generation of technology might not be around anymore. Those are the kinds of environments where AIOps will really bring you outsized benefits.

Welcome to Data Framed. This is Richie. The last few years have seen a trend where the data and software worlds have been converging. Data people need to understand software development to get their code into production, and software engineers are largely responsible for implementing generative AI features. Another area of overlap is aiopsy, where machine learning and analytics are used to improve IT operations. This new field promises to improve the productivity of IT departments and improve the reliability of software platforms. Our guest is asaf Reznik, the CEO at BigPanda, an AIOps platform, and he's formerly a principal venture capitalist at Sequoia. Since he spent over a decade building out this AIOps company, he's ideally placed to tell us what the field involves and how he can best make use of AI for your IT department. Have to say I'm pleased to see that AI is expanding into yet another area of business. So let's hear what he has to say. Hi sf, thank you for joining me on the show.

My pleasure, Reggie. Thank you for having me.

Excellent. So, just to begin, what is AIOps?

AIOps is the use of AI and machine learning to scale and automate IT operations. So what does that mean? So our mission as a company is to enable the teams that keep digital services running. So what does that mean? So our customers are the world's largest enterprises. So think large banks, large stock exchanges, large airlines, so on and so forth. They have lots of digital services and and under empowering those digital services is things like clouds and servers and applications and networks, lots of moving parts that generate lots and lots of data. Our customers, their job. They are the essential workers that keep all that digital infrastructure running and When IT breaks, they have to find and fix the problem before it affects customers, before it affects the business. That's getting harder and harder to do as enterprises move to the cloud, move to increasingly complex IT operational footprints. So they're using cloud microservices containers, they're deploying software a lot more, they're using all of these new technologies that let them move faster, be more innovative. On the other hand, now they've got a lot more moving parts, generating a lot more machine data. In fact, that tsunami of machine data that the human beings in IT operations, whose job IT is to keep the business running, they can't keep up with all that data. And so that's where AI comes in. AI is a generic technology, is very good at taking large sets of fuzzy and fast moving data and turning that into insight and action. Aiops is the application of AI specifically for IT operational use cases.

Okay, so this seems like another story of AI just invading absolutely every single industry. And it seems like in this case it's helping out the IT operations and IT infrastructure people. And there are a lot of IT Ops terms floating about at the moment. So you've got DevOps, you've got MLOps, you've got DataOps, you got all these related terms. How does AIOps fit in compared to these other ops things?

Well, they all have to do with operations, but different parts of operations. So if you think of DevOps, that is a philosophy that says developers. Traditionally when you look at IT services, you had two groups. You had developers who build code and you had IT operations who ran code. The concept of DevOps is one that says, hey, developers should maintain the code base and the services they've built. You build it, you own it. That's the concept of DevOps. Nothing to do with AI. MLOps is nothing to do with IT operations. It's hey, for machine learning and AI. There's a lot of tooling and operations that needs to happen in order to deliver better ML and AI. So ML Ops is a whole branch around tooling and services to enable ML and AI developers. AI Ops is how do you use AI to scale and automate IT ops? It's a bit of a word salad. There's a lot of jargon out there, hard to keep up.

Absolutely. So three different related ideas but for different groups of people then?

Absolutely.

Okay, so to make this a bit more concrete, can you give us some examples of what an aiops project might involve?

Yeah, of course. So if you think of human IT teams whose job it is to keep digital services Running those teams. There can be hundreds, there can be thousands of people in any given enterprise. Their job is to collect lots and lots of data. And that data can live in lots of different silos of the organization. It can be data about the health of your machines and data about the health of your applications and data about the health of your customer experience. And so health of your machines typically lives in a set of data called monitoring or observability data. And so that's lots and lots of software you're using to monitor the health of your servers and networks and storage and cloud and containers. Lots and lots of data that lives in lots of different tools that all specialize in monitoring the health of different tiers of your tech stack. Then there's information about the health of your applications. So I've got lots of applications, lots of microservices. They're acting in different ways depending on what's happening. I have to get that silo of information. Then I have to get siloed of information around what's the health of my customer experience. And that just gives me enough information to know, do I have a problem? Then I need to know, well, why do I have this problem? What's the root cause of this problem? Typically, that can happen because someone changed something across my enterprise and it's leading to unpredictable situations. The problem there is, if I'm a large enterprise, I'm a global 2000 company, Fortune 500. There's thousands of people who can be changing lots and lots of things over the last two weeks that may have unintended consequences. Developers who deploy new code, security administrators who change security configurations, IT folks who change infrastructure configurations. 5,000 changes in the last two weeks that could have led to an application not working like it should and delivering a suboptimal customer experience. I have to filter through all of those 5,000 changes to figure out which is the one that led to the problem. That's a needle in the haystack issue. Then I have to filter through lots and lots of other data around who did what, who communicated what. Have we seen this problem in the past? How did we fix it in the past? And so that's why for human beings today to piece together all of that data in order to get context, it can take hours. And in the digital world, where seconds matter, if you're down for hours, that's very, very bad. There's a special place in hell, for example, think of airlines. There's this very special place in hell called getting stranded in the airport. And it turns out Customers don't like it and they'll punish you for it. If you think of most large airlines, they've got like 50 to 100 critical services that if any one of them is down for more than 10 minutes, they have to ground the entire fleet. On the one hand, you've got 10 minutes to keep this service live and back and you have to sift through a tsunami of data in order to figure out what's broken. Why is it broken? What should I do about it? Aiops says, hey, let's piece together all that data. Let's give you four full context. Let's give you, we have this concept called full context operations. Let's take all these little silos of data that are buried in lots of different places, let's piece them together and give people full context to do their job. Make decisions better, make decisions faster.

Okay, I like that you point out that it does have real consequences. When systems go down, it's going to cause genuine problems for your customers like getting stranded in airports or whatever. And certainly the idea that once systems get really big, even the developers don't understand the changes that they make. They don't know what the impact's going to be. That seems very important, quite useful for motivating why you need AI to help out there. Okay, so do you have any examples of companies that have had success stories by going in on AI ops?

Yeah, absolutely. I mean think of InterContinental Hotel groups IHG, so they're one of the largest hotel groups in the world. So this got something like 6,000 hotels representing I think something like almost 300,000 rooms. Lot of organizational complexity. They brought us in and really trying to help them a automate how they detect issues to bring down that mean time to detection and then help them get to root cause much faster. So helping them automate how long it takes them to understand why did something break and how should I fix it. And I think last year they achieved something like 99.8% availability after adopting Bigpan, which was the best they've ever done. And that obviously level availability means that customers can book hotel rooms with less problems, they can check in to their hotel with less problems, and IHD can deliver much, much better.

Customer service is the main focus, just about avoiding downtime, avoiding problems or other use cases.

For AIOps, there's a number of use cases. So one of the biggest one is improving the availability of your digital services and that's a huge one. Second one is enabling more efficient use of your resources. So instead of your people spending a Lot of their time hunting and pecking for useful information. The fact that they can get full context operations instantly lets them work a lot faster. And so you're making better use of your people and you're also not bringing in as much disruption. So I remember sitting with an executive from, you know, one of the, the world's largest insurance companies, and when they have what they call a P1 outage, which means, you know, the bells are ringing, things are on fire, they get these bridge calls from hell where 60 different people from 60 different parts of the company get on the line, including executives, and trying to figure out what the heck is happening. And they can take hours. And it's hugely frustrating, hugely disruptive, and we can help eliminate those and at the very least streamline those. So those bridge calls take minutes, not hours. And so bringing less, more efficiency and less disruption in the market or to our customers lets them continue to do their day job, which is to build services, build code, continue to innovate.

Okay, certainly not having horrible old meetings with 60 people in them does seem like a pretty good motivator. And I would say having these outages like I'm trying to debug what's going wrong. I've spoken to our engineers here at Data Camp and yeah, this is one of the more stressful aspects of the job. So I think that people would prefer to focus on building stuff rather than dealing with problems. And I'd like to talk a bit about how you go up implementing some of this stuff. So which teams or roles tend to get involved when you're setting up an aiops team?

You know, there I'd say there's three primary Personas involved. There is the users. So most of our other users who use something like a Big panda or an AIOps tool are folks who are full time firefighters. So most enterprises have what's called level one operators. They sit in an operations center looking at a bunch of screens that tell them about the health of their servers and networks and applications and so on and so forth. And the whole point of AIOps or a big part of AOPS is hey, instead of those people having to look at 30,000 different symptoms of what's going on, just want to come to work and say, hey, I really only have one or two big problems. It may have hundreds or thousands of symptoms, but I don't need to be a data scientist to connect the dots. And so those are the primary users. Then there are typically the tool owners and the users, the operators are their customers. So the tool owner is Typically someone who owns either AI initiatives for their company. So it'll be an executive or a VP who's tasked with bringing AI innovation into their company, or it will be someone who is the owner of observability and monitoring tools. And AIOps is a natural extension of that. And so you have the tool owner, you have the tool users, and typically the boss of both of those people is either the CIO or the CIO minus one or two, a VP or an SVP of IT operations. And those are typically the folks involved. When things really hit the fan and you have a big outage, then the blast radius of that outage can also bring inside developers, subject matter experts, folks whose full time job is not operations and not incident management. But when something in their domain breaks, they get called in and they want easy access to information as well.

Okay, so it seems like it's pretty limited to IT teams and engineering teams, but it's going to be the CIO who is responsible for this in most cases. I do also like the idea of having a corporate blast radius as something's going wrong, you've got an explosion rippling across your teams. Very nice. So I'm curious as to how much AI knowledge you need because often this is something that lot of skill software developers are going to have. So what do you need to know about AI in order to make use of AIOps?

00 and that, that's the whole point of AIOps where you know, I think that if you look at 10 years ago, you did see, you know, a lot of operations teams and IT teams try to kind of roll their own and create their own in home tools that took this tsunami of data and made sense of it. And most of those projects ended up in failure because it's really, really hard. I mean we've, we as big Panda as a company has raised something like $330 million in order to build this offering. Because the back end of this offering is incredibly complex. But the front end of the offering is such that you don't need to be a data scientist, you don't need to be an expert to use this thing. And like you said, most IT ops teams don't have data scientists on staff, don't have PhDs on staff. And so we've built AIOps that it's no experts required. So the interface is one where you have to be aware of what's going on in IT ops and you have to be able to speak the language of IT Ops. You don't need to be be able to speak the language of mathematics.

Okay. So in some way it's nice that all the mathematics and data is abstracted away. Since our audience, well, a lot of them care about data, but can you talk us through what sort of techniques, data and AI techniques are involved in Big Panda?

I'd say that there's three steps here. So the first step is aggregating, normalizing, enriching all of the data. So, you know, what are human beings doing when an outage happens and it takes six hours to fix the problem? They're hunting and pecking for data and they're connecting the dots in their head. That's the first thing we want to do as a programmatic piece of infrastructure. So the first step is collect all the data in the different silos where it lives. And so we have to shake hands with lots of different tools and data silos. And some of those provide very structured, very clean data. And a lot of those provide very messy data that there is no structure or every development team has a different structure. We have to come in and we have to normalize all of that data into a extraction layer that we call the unified data fabric. And so there you're taking data from 250 different systems, you're normalizing it, you're structuring it, but more importantly, you're enriching it. So the data from tool A and tool B and tool C is now all connected to each other. So you've got a ton more context. That's a big part of it. Then once you've got that kind of abstraction layer of data and unified data, then comes the AI. And there you're looking for patterns, you're looking for clusters, you're looking for time relationships, you're doing semantic processing, you're doing a lot of different, both algorithmic and AI processing to be able to glean insights out of that data. And then the last part is to be able to actually visualize that data for teams, for IT Ops folks, to be able to quickly understand what's going on. That's where a lot of the magic happens as well. We want to make sure that it's not just no experts required, but really, you know, minimal training required. We want operators to come in to Big Panda, and if you know how to use an email application, you'll naturally understand how to use Big Panda. It's just, you know, super, super easy.

Okay, that last point seems particularly important because once you've got an incredibly complex system, lots of possible things going wrong, then you want to be able to Display this information really clearly to people who don't necessarily have that much data expertise. They're in a rush to find out what the problem is. And it seems that that communication part is going to be maybe the secret sauce. Can you talk me through any sorts of techniques you've used to try and make things clearer and quickly understandable for people?

You know, it's one part data processing and one part data visualization. So the data processing is how do you not just cluster data and correlate data, but how do you explain in very easy terms what is the logic behind that correlation? When we first brought AI to market, people were very skeptical. They said, hey, this is a black box. And now you're asking me to automate some of my business's most mission critical services and activities based on the logic that your AI creates. But I don't know why the logic of the AI is making decisions. So we developed open box AI, which is kind of a very fancy way of saying we make the logic very explainable. The AI processes lots and lots of data, it finds patterns and it creates pattern logic. So when they sees this data in the future, it, it knows in a deterministic way how to correlate all these things. We make that logic, we expose that logic to our customers so they can see exactly why did the AI make the decisions that it does. And very importantly, humans can not only see it and therefore trust it, they can actually tweak it. They can say, hey, AI found some pretty good patterns, but I've got some tribal knowledge and some history in my environment that I know that this logic isn't quite right. I want to tweak it a little bit and I want to test that tweak based on historical data so I can see how it acts in the past before I release it in the wild. Being able to expose that logic goes a long way. Then two is smart data visualizations. To be able to visualize the escalation of an incident and how it started in servers and then move to network and then affected your application and then affected your user experience. To be able to see that kind of waterfall is hugely important and makes it much more intuitive for human beings to understand what's going on with my machines.

Okay, so just to make sure I've understood this correctly, it's going to learn what the standard time is to load a page on your website and it's going to learn some sort of a cut off for what's the slowest it can load. And if it goes past that Then that's when it's going to start telling you that there are problems. Is that the kind of idea of things?

That's kind of. Well, it. So customers, companies have lots and lots of tools to monitor, like I said, things like the customer experience. So they have tools that say, hey, this website or this query should be answered in less than 50 milliseconds. And if it's more than 50 milliseconds, that's a problem. And a poor user experience. Raise a flag. We get that flag, it's called an event. And then we say, hey, something over here is happening with user experience. Let's go see if something, you know, which servers support that user experience, which servers, which networks, which clouds, which applications all touch that user experience. So once we see, hey, there's a problem, that user experience, let's go look at the underlying infrastructure that we know to associate with that problem and see if there's a problem there.

Okay, so it's going beyond just alerting that there is a problem. It's going to help you triage, like where the problem came from.

Absolutely. And because oftentimes, you know, the fact that you have a page that's slow to load, that's a upstream symptom of something that might have started two, three, four hours ago. Right. And you want to understand what's going on. And that problem that started 2, 3, 4 hours ago might be because of a change someone implemented two weeks ago. And now I have to sift through hundreds of thousands of data points in order to find those breadcrumbs. That's what AI does.

Nice. That just seems like it's in a lot of pain there. So you've been through a few examples of problems. I'd like to talk about how to get started. So what do you need to do before you start adopting AIOps? How do you prepare for this?

You know, really not a lot to do. Part of the value proposition of AIOps is that you don't need perfect data. You know, you don't have to first. You know, I remember when I was a kid, my mom would have the cleaning lady come and my mom would say, hey, the cleaning lady's coming. You have to clean the house first. So I was thinking like, but the cleaning lady's coming. Why do I have to clean my room? And people often think something like that with AIOps. Hey, I have to clean up all my data before I feed it to AIOps so it can make sense of my data. And the answer is no. Part of the value prop of AIOps is, it can make sense of messy data, it can make sense of dirty data. And the fact that you have silo data and every team in your organization has a different kind of framework and schema, that's okay. Put the AI ops layer on top of that and it can help make sense of that. So there's really not a lot of change you need to do to clean up your data before you bring AI. That's what AI. That's part of the value prop is, you know, we have this unified data fabric that can come in and clean it, structure it, normalize it, enrich it. We do that work for you.

So what makes a good first project for AIOps? What are you going to hook this up to first?

There's no one rule. Typically what I tell large enterprises is start with a discrete problem. You know, pick your most painful problem and start there. Don't try to boil the ocean and say, hey, I'm a large global bank and I want to roll this across my entire footprint, you know, across, in the case of a bank, wealth management, consumer technology, trading, and on and on and on. Typically the right thing is pick one area where you have a burning, burning pain, go fix it, go get a win, go socialize that win internally, get people excited and get that snowball rolling. And so, you know, let's go get you a win within a few weeks or a few months. And that typically leads to much broader adoption because then people are coming to you and saying, hey, heard you've got this big panda thing and it's really cleaning up your data and it's really streamlining your operations. How do I get in on the action?

Okay, so I suppose the good news is if you're having a lot of outages, then you're going to get that first win a lot quicker because you can see what's happening.

It's a target rich environment. The target rich environment.

Nice. So all the examples you've talked about so far have been with very complex IT systems in large enterprises. So how big or complex do your projects need to be in order to get some benefit from AIOps?

Typically the larger you are, the messier you are, the more fragmented data you have, the more value you can get out of this. So if you're a small or mid sized business, if you're a young startup in Silicon Valley, you probably don't need AIOps. But if you are a medium to large enterprise, you've been around for several decades, you've got different silos of technology, different generations of technology the people that wrote and built your initial generation of technology might not be around anymore. Those are the kinds of environments where AIOps will really bring you outsized benefits.

Okay, can you just talk me through? Are there any particular stages to the adoption of AIOps? Is it a case of hook up one system at a time, or do you have to connect everything together at once to get the whole benefit from it? Also, can you talk about what is a data fabric? I know this is of interest to our audience.

No, you can absolutely do it in steps and iterate your way to value. So, you know, again, I recommend as a first step, don't try to boil the ocean and connect your entire enterprise. Pick one specific environment that's particularly painful and start there. Step one is to connect all the data. So you want to connect Big Panda to the different systems, the different data silos. And so we're getting that data and then we are storing that data, cleaning that data into this abstraction layer that we call the Unified Data Fabric. And that's really a data store where we're taking all your disparate data, we're structuring your unstructured data, we are normalizing all the data from the 15 different tools that speak 15 different languages, that it's speaking the same data. And then very importantly, we're enriching that data. And so we know when you have a low page load time, we know that it's related to this application, we know this application sits on the server, we know this server speaks via this network, and we're enriching all of that data. And if you just stop there, you're already going to get a ton of value because you've connected all that data, you've really cleaned it up and you've made it really way, way more actionable. That's step one typically takes several weeks. Step two is then to turn that clean data into insight. And that's where you want to unlock AI. Typically, you need, you know, enough statistically significant data for the AI to recognize patterns. The good news is in large enterprises, that can really happen very quickly because there's so much data going through large enterprises, you can get to statistical significance in typically five to ten days.

It's interesting. That first step is just bringing all your data silos together and making sure the data is in a good state before you introduce the AI level. So just to wrap up, what are you most excited about in the world of AIOps?

I'd say the most exciting is gen AI has really just really change the landscape of what we can do, I mean, if you think We've been doing AI ops for 10 years previous to Gen AI, our AI and our ML development was really based on a small handful of data scientists who are doing research. And so these are PhDs, you're competing with these people with the likes of Facebook and Google. They're very expensive, they're in high demand, and they're very far away from understanding IT Ops. They're data scientists, they're not IT Ops engineers. And so the research they're doing takes time and there's not a ton of empathy with the customers and their pain. Now along comes Genai. And it means that our customers and our employees, folks like solution architects, sales engineers, our own DevOps engineers, are technical enough to do prompt engineering on Gen AI. And so the beauty of that is you can now get folks that are deep domain expertise have lots of familiarity and empathy with the problems that we're solving. They can now leverage Gen AI to innovate and prototype. Which means that for us, it's been a huge unlock in the velocity of our innovation and the quality of our innovation because we're taking the phone, you know, we're hiring our own customers and saying, hey, now you can use Gen AI to solve the problems you've been suffering from for the last 20 years. And that's been a real game changer in the velocity and the quality of our innovation.

Okay, yeah. So bringing people with domain expertise, your business people, and getting them to be able to solve technical problems seems like incredible leap forward.

Absolutely, absolutely. The other aspect that Genai has helped unlock for us is it's dramatically expanding the surface area of the data that we can process. So up until before Genai, the data silos that we were really tapping into are things like observability, data change data, topology data, things that live within other systems. But there's a lot of data that's buried in people's heads. There's a lot of data that's buried in lots of tiny little silos across the enterprise. That's really hard to get to AI because a lot of that is in natural language semantic environments. Genai can now help you get at those silos as well. So you can really tap into the long tail of enterprise data. And that's very exciting as well.

Absolutely. It's amazing how much corporate knowledge is often just buried away in some throwaway comment in a Google Doc or something.

Instead in a Google Doc or PowerPoint or email. And now you can get at it and turn that human knowledge into enterprise knowledge. It's very exciting.

Exciting times indeed. Do you have any final advice for organizations wanting to adopt AIOps?

My final advice is just do it. Just get started. There's not a lot of prep work you need. Whatever state you're in if you're suffering from the pain of data overload, get started with AIOps. It's not a big lift.

Excellent. Just go and do it. I like that.

Just do it.

All right. Thank you for your time, Nasser.