AI, IoT, and the Future of DevSecOps with Hariharan Ragothaman

Get ready to discover some of the most actionable DevOps techniques and tooling, including performance and reliability with some of the world's smartest engineers. Hey, I'm Joe Colantonio, host of the DevOps Toolchain podcast, and my goal is to help you create DevOps toolchain awesomeness. Hey, today we'll be talking about Harion, all about embedded system security, AI, DevOps, all the things if you don't know, you're in luck. He's a seasoned technologist, currently leading AI server validation at his current company where he architects next generation data center solutions. He has over a decade of experience spanning embedded systems to cloud infrastructure, and he's earned recognition including induction into the Athena Health hall of fame and multiple CEO awards. He's published extensively on AI powered cybersecurity and DevSecOps, and he's served as a judge on media hackathons. From MIT to UC Berkeley, he's done it all. You don't want to miss it. Check it out. Hey, before we get into this episode, I want to quickly talk about the silent killer of most DevOps efforts, that is poor user experience. If your app is slow, it's worse than your typical bug. It's frustrating and in my experience and many others I talk to on this podcast, frustrated users don't last long. But since slow performance is as sudden, it's hard for standard error monitoring tools to catch. And that's why I dig SmartBear's insight hub. It's an all in one observability solution that offers front end performance monitoring and distributed tracing. Your developers can easily detect, fix and prevent performance bottlenecks before it affects your users. Sounds cool, right? Don't rely anymore on frustrated user feedback. But I always say, try it for yourself. Go to smartbear.com or use our special link down below and try it for free. No credit card required. Hey, welcome to the guild.

Oh, thank you. Thanks Joe. Thanks for the warm introduction.

Absolutely. I guess before we get into it, I'm always curious to know, how did you get into technology or DevOps?

That's a fairly interesting question. So I think I got into programming when I was in like I think sixth grade. I think my first language was Basic. I think it stood for Beginners All Purpose Symbolic instruction code, if I remember it right. So I started doing that and then eventually slowly leaned into C C and I think when I was in about. When I was in my 12th grade, I think that was the time when I had. I realized that I had an initial interest in Robotics. So I realized that I was into. I had some leaning towards learning electronics and then which naturally progressed to doing a bachelor's in electronics. So that's how it all started and. Or rather that's how I started getting interested in technology. I was always passionate about interfacing hardware and software. And the time when I made my first robot and when I saw it move, I was like, oh, okay, this is really fun. I thought I should, you know, pursue my interests in that domain. But how I eventually got into DevSecOps or AI, I think that's a separate story by itself. But to answer your initial question, that's how I got that into the technology space.

All right, hold on. I mean, when I was six, I was probably still playing outside with sticks and rocks. How did you get into basics? Was your parents into programming? Was your school early?

It was my school, yeah. I mean, I also definitely did play with lockstone sticks and all that stuff, but I think it was my school that, you know, introduced me to programming at a very early age and that's probably helped me out eventually.

So I know you've worked across many embedded systems at different companies. Now. Cloud native validation. What's the biggest mind shift that you had to go through from going from embedded to cloud native? And was it a big shift or was there anything you had to learn to go from one to the other?

I started off my career by doing a lot of testing in the embedded side, which I think helped me to mentally map my fundamentals that I had learned in school and college. To actually apply it in a production environments that was, that was more than. That was also really intriguing. At the same time, it was also, you know, I was able to see the code in action, you know what I'm saying? So with regard to the mindset, I felt you need to be really inquisitive and have a lot of persistence when you're dealing with, you know, embedded systems. Right. There's a lot of, like, debugging that needs to be in place. And depending on the environment that you're in, you probably, you know, also have to spend long hours, you know, even in figuring out something that may be even complete that may look completely trivial. In. In one of my projects that I had done in the embedded systems world, I had to integrate like Alexa and Google Voice APAs to my embedded systems. So that was when I realized I had to do a lot of upskilling in the cloud world. So I definitely started doing a lot of certifications that AWS provided and I started Developing a systems level approach in terms of like how I would architect systems and put things in place rather than, you know, trying to zero in on a particular problem and, you know, making it perfect. Right. So that is, it was sort of imagine it like I almost like zoomed out and looked at how I would build systems and I think that was a technical shift in the mindset in terms of like when I had to deal with cloud native architectures. I think I always go back to that. I don't remember that full code. It's almost like, you know, that there was a fantastic quote that I read online which says that you don't rise up to the level of your goals but you fall down to the level of your systems or something like that. I think I've always enjoyed that code. So I think it was more of big picture thinking and how arrange all the various LEGO pieces to get something done. I think that was a big technical shift I had to go through.

With AI now entering AI machine learning, do you see embedded system testing a bigger need for that?

I think with the influx of AI, I think there's going to be a lot more prominence in embedded systems in security, given the rapid adoption of AI. But how it actually trickles down into those sectors I think would have to be analyzed on a case to case basis. Definitely. I mean in the embedded systems world we actively use different sensors and you know, like, and for different purposes. Right. So there's definitely going to be a lot of value that you need to, you need to really care about security. But I'm not really like getting the actual question that you asked in terms of like what's the takeaway should be giving, like, should I be talking about how it affects that sector or like.

No, I'm always looking for trends. So I don't know if someone's listing and you're like, hey, here's an opportunity. AI machine learning is growing and obviously IoT is growing. The two together are probably going to cause a rise in embedded systems is my theory. I could be wrong, I could be right. Just trying to get a pulse on where you see that area.

I think that's an interesting question. I really like that. So something that I've been thinking is if you look at companies, if you look at the big tech companies, they're trying to, you know, pretty much most of their code that's been written, it's estimated that it's going to be written by a. It's going to be written by a. But when you really consider sectors like that Involve security or embedded systems that are more hardware specific. AI could be a great assist, but it also requires a lot of context on the hardware architecture or even, or like, you know, the security mindset that that is needed, which might also involve a human perspective. And I think that would take a while for AI to actually, you know, get better at. But suddenly I feel like the fields of embedded systems and security are definitely going to have a lot more prominence given that the other fields are getting automated.

So how do you get that security mindset? A lot of people just focus in on like one thing and they for some reason security, to me, I think they think it's a different group that handles that.

Yeah, that's again, once again, that's a great question. So going back to my experience, right when I started my career, I never really cared about security in the first place, to be fairly honest. I really cared more about getting stuff right and you know, maybe like pushing stuff to production and then, you know, shipping things fast. I think, I think everyone starts there. But in one of my previous employees where I worked at, I started architecting DevSecOps pipelines. I had the opportunity to, you know, manage large scale kubernetes deployments at scale. And that was when I really had to care about security. Not because I had the security mindset in the first place, but to be fairly honest, I was working in a very regulated industry where, you know, it was bad. The pipelines were subject to multiple audits, I know, in different quarters. So we really had to care about security. That's when I began to wonder why security is important and how we potentially have to care about security at an early stage of the software pipeline. And embedding security at every stage of the software development lifecycle became really crucial to ensure the overall success of the project. So in fact, some of my projects involved about shifting security towards the left. And to quote from some of my past experiences, we started like, you know, creating software bill of materials for all the software that we were shipping and we wanted to catch vulnerabilities at a fairly early stage. And that's when we started integrating several tools right from the point when a developer builds code in their IDEs to different means of scanning if when it gets built and when it gets delivered to different content registries or you know, gets deployed to different cloud environments. So I think to answer your question, I started caring about security when I started to work for a regulated employer. But then harnessing that security mindset became a skill through a bunch of certifications or like Online courses that I did. And also, I think it was also easy for me to naturally progress to have a security mindset is because when, since I'm from embedded systems and when I, whenever I had an issue that I had to debug. Right. I had to eventually, like for example, let's say I'm a memory leak, you know, when I'm building something I have to use tools like valgrind or you know, I have to use like a debugger to really figure out where the issue is stemming from. So flipping, you know, on the flipping the coin or like the looking at the other, looking at it from the other side. I had to care about like how a particular system can be breached or how a particular system can be like, you know, exploit it. So that's when I started slowly building that security mindset and started caring about like how I can, you know, build in processes and built in systems that are more secure such that, you know, I deliver like scalable and secure software.

So you mentioned tooling. Are there any particular tooling that you recommend along the stage of the pipeline? So from check in like a scanner or a sniffer is like, is that a must have to check your code for any security vulnerabilities first off, like how do you shift to left and then how do you keep it going all the way through the pipeline?

So I think if I understood right, I think you're asking for two things as to what are the steps I took to shift it towards the left and at the same time, what are some open source tools or tools that I leveraged. Right. I think I started off using tools that this company JFrog provided. I think they had a tool called X Ray which could scan for vulnerabilities. I mean it's not open source, but you kind of like you have to, it's proprietary. So if you eventually subscribe to JFROG products, that's how I started off with. And so the X A product itself could be used, could be integrated with your ide. So as you're writing code it could pretty much tell you like what your various vulnerabilities are. And you could also like scan your packages or scan your docker images after you deploy them too, and then report on them. But to code some open source images that I've actively used, I've also, I've used if you. I've used like Nuclei, which is like an offensive security scanner. And I've integrated with like tools like PagerDuty to like alert you whenever there's an whenever there's an issue. If you eventually like, you know, end up building your own like say DevSecOps a agent, you could pretty much like, you know, integrate nuclei to it and it could potentially be like a small red team that you have in your own laptop or your terminal. That's a possibility. The other tool I've also used is, I mean when you use, going back when you use the JFrog platform. JFrog and when you actually like upload your images, they have the capability to you know, generate software bill of materials. But given that's proprietary. But if you could leverage open source tools like say Sift, which is an S BOM generator, it can generate bombs both in SPDX and other formats of interest. So you can pretty much flag unknown licenses or dependencies that are vulnerable fairly very easily. And I think the understanding or like the mindset to shift left in the pipeline is about ensuring that security is not just like a gate that you need to tick off. It's something that has to be embedded at every stage of your pipeline so that the whole pipeline is secure as a whole. I think that's the overall mindset rather than, you know, trying to shift it towards the left. Yeah.

So it seems like you're always learning new things. How do you stay up to date with all the latest and greatest? Do you have a methodology you go through? Is it just something you've always done? Do you have any habits, a mindset used to help you really keep going and learning new things?

I mostly operate from a first principles approach. I try to keep my fundamentals strong. So I always like practice. I sort of divide that into say two to three sectors. Right. So one is having the proclivity or the ability to code concepts or ideas that you're able to, that you're able to formulate. The second is to have a really strong understanding of systems and able to design systems at large. The third sector would be more about having some domain specific knowledge that, that you know, maps your area of interest. So I would summarize them as coding systems approach and then a domain that you would like to associate yourself with. So I think from a coding standpoint, I think I frequently practice on all the websites that are available online, which is like lead code forces and so on. And from a systems approach I tend to follow a lot of system, a lot of creators both on YouTube and other and Substack and a few other places where they sort of deliver tidbits about how to, how to, you know, improve your systems thinking and I don't know if I can refer some books right now, but I've had, I've always like buy.

I love books any reference to books if you have them on the top of your head. But you don't have to.

Yeah, sure. I mean like I keep rereading the design of data intensive applications and I also, I'm also a big fan of the system design interviews book for both part ones and part two. I think. I don't know if I'm pronouncing the author's name right. I think it's by Alex Hsu who is. I think that's a really good book and gives you the. It gives you what you need to, you know, think from a high level view in defect in implementing software systems and think of it like design of data intensive applications is potentially like, potentially like the systems level thinking bible. And the system designs interview by Alexio is more of an abridged version of that. So and from a domain point and to move to the third piece from the domains domain aspect, I think I've been all over the place. I think in my early stages of the career I've been trying to improve my understanding of computer architecture embedded systems and I've always tried to refine my knowledge in that space. But then once I moved to DevSecOps and security I started to care more about certifications and care more about listening to a lot of tech talks and trying to, you know, be up to speed about all the latest industry trends in terms of like what's happening in that space. And that's how I got into like I think the recent past. I that is how I got interested in how AI was being applied into DevSecOps and how that could be potentially game changing in the coming years.

All right, so with AI though, do you see tech skills still being top priority or do you see more shift into now more leadership type of management skills or do you see it not as a changer either way?

So I feel, I think finding the right balance between tech skills and leadership is crucial because with the given advancements in AI, where we all have ChatGPT and Gemini and cloud and few other tools at our disposal, I feel the entry, the barrier to actually enter this field has been drastically lowered and the space has been completely democratized. You said of like you just need your phone to you know, start to code pretty much and that's definitely there. On the other hand, leadership, tech leadership or like, you know, finding really niche use cases where you could, you know, where you could you know, potentially even wipe code and find and build something innovative is also prevalent. But to answer a larger question in terms of like, how do you build scalable products or how do you build products that would be really robust and can, you know, stand the test of time? That would still demand a lot of technical progress and knowledge. It might be a little easier in certain industries and can be really hard in other industries. For example, if the product that you're building is really like, say, take a website and that's it, and that's your entire product, AI can really spin it up really fast for you and probably even, you know, take it to production and you know, it can, it can really scale, right? But if your product is going to be something really niche and you, let's say you're trying to build an edge server, you know, and you're going to use a bunch of, and it's going to, you know, cater to a specific subset that's going to demand a lot more niche knowledge. Or let's say you're trying to build something that's going to be integrated to a data center. Let's say you're building a network abstraction that is going to, you know, interface to network switches or something of that sort. That's going to require a lot of domain knowledge. So what I feel is if the problem statement that you're trying to solve requires a lot of domain knowledge and technical prowess and, you know, addresses a niche space, AI alone wouldn't solve the problem. But if, that being said, leadership is definitely important. I feel like tech leadership is sort of like a common denominator. It should always be there. But that being said, I think I'm more, more like trying to answer the question in terms of like when AI would really be great assist and when, you know, you probably should let AI, you know, rather tell AI that hey, this, you need more context and I have more context, I'm going to provide it to you. So it's more of that. And I feel like tech leadership or like, you know, really being really caring about the overall momentum and how, when products are delivered or taking, scaling things from 0 to 1, I think those are common denominators that will always be there. They are like perennial characteristics that we should always care about. But I think, I hope that painted a picture.

Yeah, I mean, I've been vibe coding with the cursor for a bit and I know AI is going to get better, but it is uber frustrating. Like it can create something really quick that looks good and then when you get into you're like, this is not what I want. So I could definitely see you get lulled into thinking AI is really doing something great until you actually start looking into it. I don't know if that makes sense.

But it definitely makes sense. Like when you have like a. Like I can be very explicit, right? When you have like a super interesting. One of the side projects that I'm working on is. I mean, it's not a big secret. I can just. I can tell this on this podcast.

You heard it first right there.

So see, one of the side projects that I'm working on is. So I was talking to my young cousin who's like, you know, just trying to, you know, read books, right? So with Influx of AI, I kind of wanted to make a side project for him. One of the side projects I wanted to do is like, you know, imagine like you're reading Harry Potter or something like that. And when you. Let's say you've read 50 to 60 pages of that book, right? And let's say you want to interact with one of the characters, right? So I kind of wanted to. One of the side projects that I'm working on is, let's say you have a Harry Potter ebook and then you flip 50 pages and then you kind of know who is Harry Potter, whose dad is, who Severus Snape is, or who Dumbledore is, and so on and so forth. And then let's say you want to talk to one of those characters, right? And so I kind of wanted to create an interface where you talk to one of those characters and then there's an avatar of that character that shows up and, you know, the child could, you know, start interact with those characters. When I tried to wipe code, this, the AI could, you know, easily generate a GitHub scaffold for me and, you know, could get me really gets. Could really get me started. But then when I had to really build out the subcomponents, like, say, you know, generate a 3D avatar of that character that I care about and show it up, show it on screen, or. Or if I had to give. Or if I had to interact with that character through some context and, you know, and if I had to care about context as he perused through different pages, those were harder problems that I could not just wipe code. Those. I needed to have a lot of domain knowledge on how to actually do that. And I also needed to care about how to actually deploy that and that those were slightly harder problems that, you know, a. Could not just solve in a Jiffy, but. But it's more of a shared partnership with AI that I have. It could definitely do a lot of trivial stuff. Similarly, I mean, imagine, I mean, I'm just talking about the problem statement. Imagine trying to test this out. When you try to write automated tests for this, that's a huge challenge by itself.

Soren. I know all the rage now is a gentic AI agents, MCP servers. Any views on that? I think there's a paper that was just released. Just curious to get your views.

Yeah, I recently read the paper. I think it was titled Deletion of Thinking. Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity. The way I see it is, I think in the recent times there's been a lot of chatter about AI agents and agentic AI. In the simple terms, AI agents are super efficient and they have. And they do things autonomously given when there is a directed goal. And agentic AI in very simple terms would be about just multiple agents interacting with each other. So I feel the whole term of agentic AI is more semantic rather than, you know, if you try to find the difference between multi agents and agent, I feel like they're more or less similar. And the recent paper that Apple researchers, you know, released, they tested the frontiers of large reasoning models. And I feel overall I feel like the paper is like fairly balanced. And I feel like they. I feel the paper has like, strong takeaways. But the larger question the paper tries to ask is like, if these models that we're actively using through, you know, cloud or open GPT, whether they have like two reasoning capabilities as a user, I feel it gives you at least the feeling that they reason. They reason really well. But I think the paper sort of like pokes into the fact that if you give it super hard synthetic puzzles, can these models actually like, generalize and, you know, you know, come up with like really novel solutions? And I feel that's just one slice of the pizza that they're looking at. And I feel like the title was suddenly meant to poke, you know, like, you know, or like, you know, trying to, you know, through a point. Right? So I think I feel that was my last takeaway from it. And I feel overall, all the AI tools that we use, they will struggle. They will struggle when you give them like niche problems or problems that they have never seen before. That's always going to be there and as always, they're going to learn and, you know, they're going to just keep getting better and better. So. So yeah, recently I've started using O3 Pro and trying to, you know, test its capabilities. But that's my overall take on it.

Yeah. So do you see, like OpenAI and Microsoft have been making some outrageous claims. They're even doing layoffs saying AI is replacing. You think that's more of a play for profits rather than actual results they're seeing because Apple seems a little more balanced. They don't seem as bombastic with AI as if Microsoft has been. I don't know if that makes sense.

But yeah, the question completely makes sense. I think from my point of view. I think I can merely speculate because I think even in the recent, one of the recent conferences that happened, I think it was a llama conference that Meta organized. I think there were takeaways on how much code is being automated through AI in Meta and how much code is being automated in Microsoft and so on. Whenever I hear those statements, I feel there is a lot of scope in code being automated for sure. For example, if you written docker files or Terraform code or any of such cloud configuration files, you do know that they could be automated down the line when you have clear specifications. For example, Google recently released an AI version of kubectl. You could pretty much give it NLP commands and it could, you know, maybe create another replica set or you know, you know, change the configuration of your Kubernetes cluster and so on. So you do know that space there's a lot of automation influx that's happening, but whether they actually translate into actual reduction force, I'm not entirely sure about that. But that being said, I think both all the big tech companies and other tech companies, as AI starts automating trivial tasks, we would end up starting to move on to other higher order problems that need the demand solving and or rather niche problems that, you know, would need a human perspective or context. So that's how I view it personally.

Yep, great advice. Okay, before we go, is there one piece of actual advice you can give to someone to help them with their DevSecOps efforts? And what's the best way to find or contact you?

Great. I think from a DevSecOps standpoint, I feel like as I mentioned, definitely get a good hold of coding. It's definitely always a great skill to have. My personal recommendation is start with Python and then move on to C or any other language of your choice. Definitely highly recommend the books such as Design or Intensive Applications and System Design Interviews by Rxu also start playing. And if you're really Interested in the DevSecope space, start playing with a lot of tools and be it open source or proprietary, every tool is going to give you a certain takeaway. That being said, also remember to zoom out and think about like, you know, okay, I have all these tools, what can I build, what can I build with it? Or how can I use these tools in the right way to orchestrate something I really care about. Kubernetes is something that's always a great thing to, you know, learn, but I feel it's like a deep dark ocean that you'd have to know how to sail through. But that's a great skill to have in the DevSecOps world. And also if you're really into like security, doing a bunch of certifications given by the various cloud providers or even OWASP or even ISE organizations as isc, they are really helpful. I think that sorts of covers covers it to an extent. But again, remember that something I keep telling myself is it's always prudent to, whenever you're trying to learn something new, it's always prudent to take both a depth first and a breadth first approach. The first. I always, you know, try to first analyze the lay of the land and try to see, okay, what are the various things that I can, what are the various tools and avenues that I have to learn or I can learn from and then pick and choose what I want to learn in depth. So that's something that I did. So when I dec, even from a programming language standpoint, I learned a few programming languages. Then I decided that I need to really get good at Python. And similarly, like when there were a lot of aws, when there were a lot of cloud certifications that were available, I picked aws. I wanted to get good at it. Similarly, I picked Kubernetes and I wanted to get better at it. So, and then even when you are trying to, you know, say, learn how large scale deployments happen and so on, there are a lot of tools that I had been reduced to like Flux cd, Argo CD and so on. I picked fluxity. Although it may not be the best of choices, I tried to become good at it and try to see how I could leverage that in the GitOps model that I had, that I had both professionally and personally and I could manage deployments. So it really comes down to having a breadth first approach first initially and then try to understand what are the various things that you could leverage to either master skills or try to build a product and then pick and choose things that you really want to get good at. I think that's the advice I would give. You could message me on LinkedIn or follow me on LinkedIn. Also email me. I think both work and we'll have.

All those links for you down below. All right, before we wrap it up, remember frustrated users quit apps. Don't rely on bad App store reviews. Use SmartBear's Insight Hub to catch, fix and prevent performance bottlenecks and crashes from affecting your users. Go to smartbear.com or use the link down below and try for free for 14 days. No credit card required. And for links of everything of value we covered in this DevOps Toolchain show, head on over to test guild.com p194 so that's it for this episode of the DevOps Toolchain Show. I'm Joe. My mission is to help you succeed creating end to end full stack DevOps toolchain awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation, testing and DevOps. If you haven't yet, join our vibrant community@test guild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit testguild.info to explore how we can create transformative experiences. Together, let's push the boundaries of what we can achieve. With lutes and lyres, the bards began their song A tune of knowledge, a melody of code through the air it spread like wildfire through the land Guiding tester showing the secrets to behold.