AI Observability with Eran Grabiner

Get ready to discover some of the most actionable DevOps techniques and tooling, including performance and reliability for some of the world's smartest engineers. Hey, I'm Joe Colantonio, host of the DevOps Toolchain podcast, and my goal is to help you create DevOps toolchain awesomeness. Hey, welcome to the show. Really excited about today's topic because we're going to be exploring everything about AI observability with Iran. If you don't know. Aran is the Director of of product management at SmartBear, where he works on observability initiatives. He really knows his stuff. His journey with Smart Bear began after the acquisition of his observability startup Aspecto, a really cool solution you definitely need to check out. We'll have a link for it in the show notes after this as well. And at specto, that's where he served as CEO, he has a background that includes a bunch of different roles in cloud focused startups and he has degrees in physics and electronic engineering. So he really knows his stuff. And if you want to know anything about AI, how it's going to impact observability in DevOps, you don't want to miss this episode. Check it out.

Hey, if your app is slow, it could be worse than an error. It could be frustrating. And one thing I've Learned over my 25 years in industry is that frustrated users don't last long. But since slow performance isn't sudden, it's hard for standard error monitoring tools to catch. That's why I think you should check out bugsnag, an all in one observability solution that has a way to automatically watch for these issues. Real user monitoring. It detects and reports real user performance data in real time so you can quickly identify lags. Plus you can get the context of where the lags are and how to fix them. Don't rely on frustrated user feedback. Find out for yourself. Go to Bugsnag.com and try for free. No credit card required. Check it out. You can let me know what you think.

Hey Ron, welcome to the Guild.

Hey Drew, how are you? Thank you for having me.

Great. Great to have you. Really excited about this topic. Like I said, I guess before we dive into AI and get into the nitty gritty, just in case, for anyone that doesn't know what is observability, would you explain what is observability?

Yeah. So obviously you know it's a Y term, but for me, observability is basically your ability to understand how your software is behaving just by looking at, you know, at external metrics kind of thing. You can think about it as when we are checking if a person is okay, you know, health wise, we're taking different stuff like, you know, we are measuring their temperature and you know, we are measuring those type of measurements in order to understand what's going on inside. And in the same way in observability, we are collecting different types of data in order to understand what's going on inside our application, inside our system, in order to be able to debug it, improve it, and so on and so forth.

Nice. So we'll get you into observability. Like I said, it must be a hot Trend because like SmartBear acquired you I don't know how long ago, but yeah. So why is it, do you think observability was so important that a huge company like SmartBear would then say, we need to invest in this because this is something that's gonna be all the rage?

Yeah. So I think SmartBear, which has been around for some time now, it's a company that was always focused on delivering value for developers and developer tools across the sdlc. And obviously there are a bunch of different product lines within smartware and Observability is the newest one. It didn't start with Aspecto. There were more acquisitions that happened before that, companies like bugsnug and so on. And I think that what has changed in the last couple of years that maybe made this space more interesting for smartware is actually the fact that developers became much more responsible for these such things on how their application is actually behaving in production and more responsible to take care of issues. If in the past, you know, things were much more separated and we had like, you know, the ops folks mostly focusing on production, nowadays we see more and more developer ownership, we see that software is becoming much more complex, much more distributed and we saw a lot of changes, you know, in the last couple of years in the type of tooling. So I think a company that is focusing on developers would definitely want to expand into the developer experience also within the production environment.

So when we talk about observability, are we only talking about production? And why is that a change? Is it because back in the day we had everything under our control?

Yeah.

And now we're in the wild. We're using third party services we may not even know about, like why the rise of observability then in production?

Yeah. So I think for me personally, you shouldn't limit observability to be just in production, but it's definitely the focus because eventually the idea is to make sure your software is behaving as you plan to and making sure that you know about issues as fast as you can and that you can fix those issues as fast as you can. So the focus is definitely in production and even if you're doing something in pre production, the motivation is to have better software in production. Right. So it is around that. And I think that in the last couple of years there were changes and it actually very much connects to AI because I think AI is just, you know, a big one, but just another change in this chain of changes where software became, as I said, like much more distributed, much more, you know, as you said, third party tools, things that are managed, that you're using different types of technologies, the velocity of software delivery became much faster. It's not like, you know, every year we're releasing a new version of our product. It is happening on a daily basis. And lastly, I would say the way users are interacting with software changed a lot just because of, you know, cloud computing and mobile and so on and so forth. We are seeing much more user intensive products out there and the way we are interacting with software changed. And this has to lead to changes in how you actually maintain the software and manage it.

Nice. So let's get to the meat of the interview. AI. Everyone's talking about AI. I hear a lot about it as it relates to functional automation, but I would think observability, with all the data points it's collecting, it's probably more efficient. I don't know. So we talk about AI or machine learning or are they the same?

Yeah, so I think, you know, again, different ideas obviously on this topic, but I think, you know, it is part of the same family of technologies and eventually, you know, if you want to put a like headline on top of it, like you know, software that can basically do tasks instead of us humans and can do it in a way that would be with context and intentional and would be able to draw conclusions and act upon them and not just fulfilling a very basic task. For me, machine learning is just another method to achieve maybe AI and those type of capabilities. And when we look at observability and AI, I personally think, and maybe disclaimer here, I generally think that we humans, we are very bad at predicting the future. As we are looking more into the future, we are usually less accurate and in predicting it. So we should be very, I would say modest when we are trying to assess how things will look like, you know, near future and long term. But I do think there are some vectors and changes that are already happening and you can already see out there. But for me you can definitely, I would say, kind of like divide this conversation into short term things that are already happening and how they affect how we are currently doing, observability and maybe more long term effects on how AI is going to affect the overall software development and what developers are doing. And if we look at the past, every evolution or revolution like that that happened when we moved from machine language into more object oriented and into modern software, each one of these changes really changed how we look at software, monitor software, make sure our software is doing what we expect. So obviously in the long term AI is also going to change software and how we develop code. And this will lead to, I would say like a second degree of changes after the changes that we might see in the more of the very near future.

Absolutely. So I know you can't make predictions, but I would think as a vendor you work with a lot of customers, you see a lot of scenarios, so you may have a, a better pulse on the heartbeat of it.

Yeah.

So dumb question right here, right off the bat is, are all developers using observability and if not, will AI observability just take care of it automatically for them where they don't have to worry about implementing it?

So I think, you know, different type of developers are obviously using different tools and have different needs. But at the end of the day, if you look at observability like broader term, I think that any developer is using some sort of observability. Right. You have a code, this code is running out there, this code has an issue and now you're trying to figure out what happened. Right. Even just using a simple debugger or looking at errors and logs that you created for yourself, this is also type of observability. And again, obviously different engineers depends on the type of product they're working on and also depends if they are front end focus or backend engineers or engineers that are more focused on data and so on. They'll use different tooling, but I think each one of them will use something to the first part of your question. So yeah, I think we can try and make predictions and maybe in a year or five years I'll be here again and we'll try to meet it again. But as I said, I'm trying to divide it more short term and maybe mid long term. I think that at the short term we're Definitely seeing multiple trends that are happening with products already. And also obviously at smartware, we are thinking about a lot as well. This is where I think AI or generative AI capabilities are looking into taking stuff that we are currently doing with observability and just making them better or more efficient. I would say that broadly, when you look at observability, it's about collecting the data, a little bit of massaging it, and then storing the data and then serving the data to the developer. And I think for the first few parts, like collecting and storing the data, the stuff that we're looking at that direction as an industry, I think it's mostly about efficiency. There is a big problem in observability, relates to cost and performance. We're just collecting more and more enormous amounts of data and it's very costly and it doesn't make any sense because at the end of the day, the developer is looking at a very narrow layer of this data that we are collecting. So I think AI can be very helpful and already starting to be helpful in helping us making decisions of what data to save and what's not. Maybe being more automatic in terms of understanding which data is just a replicate of the other data is redundant, which type of data maybe we can store in a cheaper kind of storage compared to a more expensive storage that is more, you know, high performance. Being able to maybe manipulate a little bit, not manipulate, but maybe transform. Taking data that is very expensive, very rich, and compress it into formats that would be cheaper, but making, I would say, like an opinionated decision and making sure that, you know, you're choosing the right ones. So I think when we look at like data collection and sampling, we will see a bunch of stuff that, you know, trying to make the existing observability products that we see out there just a little bit cheaper, maybe.

Is that because the model you're using, the computing power gets cheaper over time.

So it's mostly, I think, like in terms of cost, it's mostly nowadays around the data processing, querying and storing the data. And if you can create, you know, different indexing or if you can make different decisions on what data you are actually saving, you can reduce, reduce the cost. But I think this is like only, you know, one part that maybe the user will not feel that much, you know, maybe the CFO will feel it a little bit. The part where I think the user is going to feel much more is all around the stuff that we are currently seeing out there that I think we can give it a headline of like, you know, kind of like observability copilot, right? Something that has generative AI capabilities that basically look at different signals of data and can make conclusions that are very complex for humans to make. And also to be conversational with the developer or engineer that is trying to debug the issue. I think that if you look nowadays or a few years back at a bunch of developers trying to debug something together, it would usually look like, hey, did you check that? Did you think about that? What did we release yesterday? It might be that. Can you look at it? I think all these type of interaction that we are feeling when we are debugging something like a production issue is going to be transformed into an interaction with a kind of like AI agent that again, can make faster conclusions and go and check things for us and bring the relevant data. And I think this can be massive. Obviously, it has a lot of challenges that comes with it. You know, challenges that relates to just the data itself. You know, how do you expose this data to your vendor? Some of this data is very proprietary for the company. And, you know, just making sure that the relevant data is getting there and that the AI algorithms can actually make a conclusion out of it is not trivial. And also there is a question of, like, you know, how much data you are showing. I think those AI tooling, they are becoming much more efficient as you are exposing them to more data. For example, if I'm just taking, just taking the logs that I have from an issue and feed these logs to an AI algorithm, the results or answers I'll get would be much different if I'm also enriching this AI model with my original code and with information about my history of releases of software and with my infrastructure as code and with my additional metrics that I collected at the same time and so on and so forth. So as we are enriching it with more information, we are getting closer to getting accurate and insightful answers from the AI model.

Absolutely. So, I mean, I'm not doing at this scale, but I created a Joe bot, which I trained on all my podcasts, blogs and webinars. And the more info I give it, I can see conversations people are having with it, and it's able to answer questions that it wasn't programmed to answer because it's making inferences from all the different data points it has that I couldn't even think of. So that sounds like, is that kind of what's happening? The more data you have, the more inferences it can make.

And then, yeah, Kind of exactly. It's the more data. And another thing that it's a big issue in the AI industry is having context and long context. Right. How this model can actually remember something that happened few months ago or happened a year ago. Right. A lot of the stuff. When there is an issue in a production environment of any company, usually the person that fix stuff and like really do it fast is the developer with the most experience in the company. Right. He knows the code the best. He knows more stuff that the team did and forgot about and so on and so forth and Eurone. I think it's going to be much more valuable if this AI agent actually has this long context. Like he knows everything that happened in the company throughout the history. He knows like different scenarios that happened to us. And this is like not trivial at all. Not just from technological perspective and the challenges that comes with that, but also as I mentioned, like just organizational security and so on. Challenges that it is not clear yet how we are going to solve.

Great point. So how does it work then, the short term? When you said generative AI, I like this idea of like a co pilot, a coworker. Yeah. So is it bubbling up insights, is it making the changes or is it you saying, hey, look at this data, tell me if you see any performance issues, like how does the generative AI piece work now in the short term?

Yeah. So I think obviously different vendors are obviously looking at differently, but I think that mainly what we see out there is like, you know, taking it step by step. You know, let's take, you know, for example, I have an error, or I have a bunch of logs that relates to a certain issue I'm seeing. And so let's take this data, maybe enrich it with some other information, feed it to this AI agent and ask him to give us an idea of what he thinks or they think is the problem and what could be a potential fix or where should I look in order to figure it out? I don't think that we will get into actually fixing or actually taking actions anytime soon. Not that the AI models would not necessarily be able to do that. It's just that like we humans will probably not feel okay with it, but definitely being like a companion that looks at the same data you are looking and being able to draw fast conclusions out of it, answer questions, even simple questions that it might be the case that the developer can answer those questions nowadays, but it's just going to take him like 20 or 30 minutes to figure it out while he can. Just ask this agent and it will bring the relevant data. So instead of going to another product right now, or sipping through other logs or looking into just a paper trail of something that happened, he can just ask this AI agent information and get it much faster. So something that took us 30 minutes will take just like two minutes.

All right, so I love this. How do we get to the long term vision? And is this a realistic long term vision where. Yeah, all right, you have observability in place, you have OpenTelemetry or whatever you're using. It's able to know what changed. It also knows your code. So it could say, all right, you're in production, this failed. I know it has to do with, I don't know, a SQL statement, whatever it is, and it knows how to fix it. Yeah, and it fixed it. At what point do we get there? Is that a realistic scenario that you see long term?

So I think long term, that's an interesting question. And I think, like me personally, when I'm trying to look at things long term, I think things will look a little bit differently because the role and the tasks that the developer is taking will change. And let's zoom out for a second. I think that every time we saw evolution or revolution with how we code and how we deliver software, we saw basically another abstraction layer, right? Every time, every revolution was another abstraction layer that happened out there. If you start from just the evolution from machine code to assembly language and from assembly language to high level languages and from that to object oriented programming and from that to the modern stuff that we see today with IDEs and frameworks, basically each one of these steps created another obstruction layer for, for the developer. So they didn't need to take care of a bottom layer and could focus more and more at the business logic and kind of like what they want the software to do. So in the early days we would need to focus on the actual performance of the machine, the CPU and memory. And our tooling came from companies like early in the days like intel, that gives you metrics about the actual CPU and so on, or even physical machines that we had in order measure how the computer is behaving. So through the days we dogged the production layer, we had different and different tooling and even the latest evolutions that we see with cloud computing and mobile and so on. So we saw the modern application performance monitoring tools and reuser monitoring and all those types of solutions that basically came out to answer those evolution. And if we look at AI, I think that it is clear that developers will focus much less on the code itself. Definitely, even in the near future, AI will be able to generate code by itself, complete blocks of codes that does a certain functionality. And the role of the developer will transform gradually into kind of like a composer that is collecting different blocks and need to combine them together into something that answer the product need. And I would even go further and claim, I think that the role of architects and designers is going to be much more interesting because they could focus more into the business logic and what they want to achieve from the system and like high level decisions. And the role of the developer will gradually become something maybe even similar to a QA engineer because you just need to validate that the code that the AI generated is actually doing what you expected and, you know, making sure that whenever you connected things together they behave as you expected. But you will less and less go into the code itself and make changes in the code itself. And yeah, I'm already talking too much about it, but we can try and think how it will affect the kind of tooling that we have and what type of tooling do we need to validate this code? Code that humans didn't write this code, what type of validation do you want to have on a code like that, how do you do that and so on.

I love it. You know, you heard it here first. It's the first time I heard someone saying developers will be replaced by qa. Usually QA people think they're going to be replaced by AI. So this is awesome. I love that point of view. But yeah, no, it's not similar than what it is now, where we have libraries, these high level libraries we're just putting together. It's almost abstracting that to even a higher level now where you're not being replaced. No one was replaced by better libraries. So hopefully that's the same here.

Yeah, you can just focus on other things. And there are definitely things that it will still be very hard definitely in the near future for the AI to complete stuff like just the innovation process of product design and architecture, like figuring out what the problem you would like to solve and how you're approaching it and what is the overall architecture. You would like to see where this thing is actually going to run, how users will interact with it, basic at least parts of algorithm development, just the overall integration between different systems and different tools. Our world is becoming more and more connected. We at SmartBear and Bugsnag, we are helping companies to trace and analyze their user interactions with your system, coming from mobile platforms and IoT platforms all the way to their backend system. This is still going to be this interaction across components, still going to be very, very complex. All the user experience part, actually, you know, UI and those type of things is going to be very, very challenging because you have a lot of, you know, human experience that comes with it and like, really understanding how humans are behaving and what they are trying to expect. Obviously, AI is going to help with it, but I don't think it's going to replace humans anytime soon. It's just going to be another tool in our toolkit that helps us do things, you know, faster and better.

Right? Absolutely. So in the short term, we're in right now as well, I would think, since someone that has a solution.

And it uses AI, how do you get AI adoption then get people to believe in it or trust it? I guess a lot of people are like, how am I going to trust this? I have scenarios where an AI is testing an LLM AI and how do you know who to trust or what's going on? How do you handle that part in order to get to the long term?

Yeah, so I think that's, obviously, that's a huge question. And I think that it's not just, you know, it's not just us developers. It's also like, you know, it's a huge question for society. You know, we see the Congress is trying to ask these questions. I think that at the end of the day, there are a few, I would say, like, key things that are happening here. First of all, if you are experiencing amazing value coming out of AI, so humans will find ways to use it. Right? And we are already seeing it. Right. It is very unlikely that we will have something that makes our life so much better and so much easier, and we will just make a conscious decision to avoid it completely. Maybe some people at some areas of the world, but I don't think that as a society it will happen. And now it comes to the question of how you're actually using it and what guardrails you will put in place. And as I mentioned at the beginning, I think this is something that organizations are going to be very concerned about and they'll look for ways, you know, sophisticated ways. How do you share data without compromising it, how you can validate that a person you're interacting with or even a machine you're interacting with is actually who they say they are, you know, questions of identity and so on. And I think, you know, it's only just. It's going to just increase our already huge cybersecurity industry, we are going to see a lot of new tooling out there that are dealing with, with those type of things. Like how do you validate that the data is actually reliable, that identity is actually. How do you add signatures to these type of things? I think it is all solvable, it will take time, but I think technology can solve most of these problems.

Absolutely. Okay, before we go, what is the most significant advantage you think of integrating AI into observability practices that you want to highlight for our listeners and maybe the best way to find or contact you?

Yeah, sure. So what's going to be the most significant? I think it's like no one likes to solve a production issue, right? It's not a task you like to do. It usually comes in a time that you don't want it to come Friday evening or in the middle of the night. And it's usually very stressful. And I think that AI can possibly reduce those events dramatically and the time we're spending on these events dramatically into something that, you know, we need to invest much less as organizations in it. I think that most big issues that are happening in our production environment eventually are human related. You know, someone released something without thinking about it, without thinking about certain integration, making a change that didn't think is going to break something, and so on and so forth. And I think that AI can be very, very helpful in identifying human errors. And I think this is going to be very, very valuable for me personally. You can reach out either way. You like LinkedIn, Twitter, whatever. You're using Smartbird themselves. I'm happy to share more thoughts and help folks that are thinking about those problems and looking into this area.

Remember, latency is the silent killer of your app. Don't rely on frustrated user feedback. You can know exactly what's happening and how to fix it with Bugsnag from SmartBear. See it for yourself. Go to Bugsnag.com and try for free. No credit card required. Check it out, let me know what you think.

And for links of everything of value we covered in this DevOps toolchain show, head on over to test guild.com p1.

So that's it for this episode of the DevOps Toolchain Show. I'm Joe. My mission is to help you succeed, creating end to end full stack DevOps toolchain awesomeness. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world. We'd love to have you join the fam@test guild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the Guild community, I'd love to hear from you. Head on over to test guild.info and let's make it happen.