Exploring AI and Cloud with Microsoft’s Naveen Krishnan

Get ready to discover some of the most actionable DevOps techniques and tooling, including performance and reliability for some of the world's smartest engineers. Hey, I'm Joe Colantonio, host of the DevOps Toolchain podcast and my goal is to help you create DevOps toolchain awesomeness. Hey, today we're taking you on a grand tour of AI and DevOps with Naveen Kumar Krishnan. If you don't know Naveen Naveen is a solution architect at Microsoft. He has a deep passion for artificial intelligence and cloud computing, as you'll find out in this interview. He also brings a wealth of knowledge and experience to the table that you can get a lot of value from. He's here to share his insights on the latest trends in AI, how it helps you with development and AI and all the good things and the transformative power of Azure, and how these technologies really are shaping the future of of development in all things AI related. You don't want to miss this episode. Check it out. Hey Naveen, welcome to the guild.

Hey Joe, good to have you. I guess before we get into it, just curious to know, how did you get into AI?

Yeah, it's a long journey actually. So my interest in math and computer science brought me here so I joined. I started my career like 15 years back and I worked for several startup companies back in India and in 2015 I came to us for a consulting company and after 2015 to 2020 I was working for several enterprises. I was designing distributed systems on Prem and in 2021 is when I got into Microsoft and I started working here for cloud. Basically I started and last one and a half years I'm into AI.

Awesome. So at Microsoft, how often do you actually use AI to help you with your day to day job?

At least once in half an hour maybe.

Wow, really?

Yeah.

How?

Yeah, there are several copilots available so. So we do use Microsoft Bing Copilot as well as Office 365 copilot and yeah, maybe. Yes, that's what it is.

Nice. So normally when I think of AI, especially nowadays it's more generative AI. So I thought maybe you could just tell us what is generative AI before we dive deeper into all the other things that make it up.

Yeah, sure. So generative AI is basically. It's not just simple, so it's a big. So generative AI is not going prebuilt like if you give it any questions and it kind of does the vectorization and then it kind of maps all the relative Words where it could relate what, what we are asking for and then it could, it could run some algorithm behind the scenes and then it gets you the answers for. And there are like six or seven steps which is, which it has to go through before it can generate and transform and produce the data what you're asking for. So that's Basically the generative AIs and apart from generative AI there are a lot of other AI things. So there are like AI for vision and AI for face recognition and other stuff which is like kind of pre trained basically. So where we train the model for our use case and then it's going to help us for that specific use case. But generative AI is something which it can generate on all what you have asked for.

Right.

There are billions and billions of parameter goes to that model and that model can know like what we are asking for. It can go scrape the web and get the content for you. So these are the things what I can see from generative AI side.

Nice. So at Microsoft are you actually baking in AI to Azure? Like is that part of your activity or do you have anything to do with actually adding features to the cloud or Azure?

At Microsoft with AI no, it's a mixed responsibility actually. So work with end customers and identify use cases and I come back and then enhance work with the internals and then get this, get some features rolled. So this is. Yeah, this is what I do.

Nice. So just curious to know as a developer, as an engineer, how do you test AI to make sure it's working as you expect? Especially as a company at Microsoft I don't know how they do it before they roll out to customers. Obviously you want to test it. Is that something you do or know any insights around?

No. So if from testing perspective like how do we test AI is like there are a lot of capabilities which comes with now Azure OpenAI. So it's not just you, you do you do the model or you do the fine tuning. It's not just your responsibility stops here. Right. So first identify a model which you wanted to use and once you fix the model. Right. For my use case there are several types of models. One is SLM or LLM or multimodal models. So first try to fix the model which best suits for your use case once a model is being done and then you can go for the pattern like what pattern you are going to use this model for one is rag or retrieval, argument, generation or otherwise you go for find, function, calling or you go for any other types of pattern. What you what you want the system to be and if needed, you wanted to integrate with vision or you wanted to integrate with text to speech capabilities. So based on the use case again and once it's all finalized, then you are going to fine tune your model. And once you fine tune your model, then that's when the model can work more for your use cases. Because until you fine tune, it's going to be a generalized. So the fine tuning is a place where you will, where you kind of give the data and then see say how it should behave in these types of scenarios. And once the fine tuning is complete, maybe you'll run a result set where you will feed in the model data, not the actual live data, feed in the model data and then model data in the sense feed in the actual response. The actual response, what you are expecting and the question what you are going to ask. Right? So with those two combined together and the output, what you get from the model is what it tells you like what level of accuracy you can do that. So and with Azure OpenAI, what I have seen is like within, within like few clicks, I think you should be able to get the result of how model, how your model is going to perform. Right? So with all the data in hand.

Nice, nice. All right, so you mentioned a few words. I just want to make sure everyone understands what they are. One of them is, I think you mentioned multimodal models. So I know this year that's been kind of where generative AI went, it went more multimodal. Can you talk a little bit about what is multimodal models and maybe how it helps developers more than maybe what they have experience with maybe a year or so ago?

Yeah, sure. So from the year or so ago from like when the GPT started just does you ask the question and then it gives you the data back. So that's what, that's how it started. And after like three, four months of that got evolved into vision. So what you can do is you can upload an image and then you can ask it to do this. So giving the image and giving your data all together is going to be the multimodal model, because it can today the, it has evolved a lot and today what you see out here is three things, right? It can see or it can read and it can hear. So at the same time. So that's what the multimodal model is basically. So seeing the sense is what you, the image, whatever you upload, it can get data from it and whatever you type in, it grabs the data from. It can Read and whatever you talk it's going to get that. So all put in together and it's not you can do all these three simultaneously. So that's the beauty of having the latest recent release of GPT4O IS has got this capability and so many retails and it can help so many healthcares and other for other domains basically to get. Get a lot of problems like what are. There are so many things what these types of domains needs basically. Especially retail and especially retail. Right. So they need this particular capability. So that's how the GPT4O is going to solve this problem?

No, it's pretty powerful. I mean I use it for stupid things like someone had a letter, they didn't know what language it was written in and so I took a picture of it, I uploaded to ChatGPT and it translated it and told me what language it was. Pretty cool. Also I'm terrible with names. I'll just say how do I say this name? It'll have a voice where it actually shows me how to see a name. So how do you use it as a developer though? Like I don't do hardcore development anymore so just curious to know, do you ever use. Sometimes people want to know if you like can upload an image of your application and ask how is this from a user perspective? Any advice on ux? Can you do those types of things?

Yeah, sure, definitely. So from developer perspective what I how it works is basically I have a design diagram. So which is. Which is done like around 1998 or 2000 right. So I have the diagram with stable schema and stuff like it's huge. So and then when I just upload that and then I say like what what it is and it's going to tell you the clear picture of what technology has been used and how it was designed and how it is running and. And things like that. So and also if I have add more additional details about the product itself then it's going to tell what the current product is doing, how better it can evolve and things like that. So. So that's the beauty of it, how a developer can leverage those capabilities.

Awesome. So in your development process in DevOps how else do you use generative AI? I know you, I think you mentioned helps you with coding but when you check in your code, do you have any like AI running that can help let you know like how well the build is doing or anything like that?

Yeah, sure. So with especially a lot of integrations are with GitHub. So if you are using GitHub primarily right. Because I, I use a lot of GitHub things. So first it starts with security first it kind of identifies all the, if there are any vulnerabilities or if any dependencies that we have attached to the project has, has got some vulnerable pieces in it. So it's going to identify that. So and then as we have a lot of other bots dependabot and other things which kind of adds more value to. To this piece and code analysis, it kind of does the code analysis like whether this code is reused, like how many percentage of this code is being reused and how many percentage of this code is newly written and how this code can be improvised. Because initial days, like around two, three years back, we have spent a lot of time doing code reviews and other stuff with all this automated AI coming into picture which can, which can recommend you how the code needs to be and based on that, if you correct and then provide it the right feedback next time it's going to learn from that. So that's how I am seeing that. So it's just for the GitHub and there are a lot of other things apart from GitHub. Right. Like where you put your artifacts, it kind of does the scan and give you the report of how the artifact sizes and what other external dependencies what we are working on and how this can be cut down and how your artifact can be very small, safe and readily available. So these types of things where I can see that but for what artifactory sources like JFrog and others like I have not seen much of AI infused into that product yet, but I'm expecting a lot very soon.

Nice. So when I was automation architect I used to have to do a lot of code reviews for eight to 10 sprint teams and they always mocked me as a blocker because I had so much code to review. That's a great use case of using as for code reviews. That's really cool. There's anything you see how you can get the most out of using AI. Like maybe like I sometimes see things on LinkedIn when developers are a tester says oh, it doesn't work or it's hallucinating. But sometimes I think they're not really leveraging it as best they can. Maybe I'm wrong. But any tips for how you get the most out of generative AI? So it not only returns your information but it returns you really correct information or really useful information.

Yeah. Yes. So one thing is first it starts from the place where you, where you actually develop. So the IDE it starts from IDE so where you can add integrate with GitHub copilot and it's kind of help you with all the pieces of code like you don't have to tell it, you don't have to write all the pieces and everything right Today it can create a class functions, methods and everything for you. And so it's very more like. It's not like INTA or for B. So it's not going to do that because as you say the logic and it's. It is intelligent enough to put the right words for your variables, functions and parameters. So where and also it thinks of the reusable cases like how it can be same class can be reused and where and how. So these types of things also have been added with GitHub copilot so with the GitHub copilot with your IDE and you write the right code and then you send the code and once the code reviews and then once it scans for all other checks and everything and once the code passes all the guardrails what you have been set on that on the GitHub side. So once when it's all done it's going to be taken to the next level where it can run through sonar scans and other stuffs on your code. So which I don't think there is any new. I've not checked the sonar's latest capabilities of adding infusing AI into the product itself but if there are any I don't know their roadmap. So if that product has got a lot of use cases like where you can infuse AI on sonar and once the sonar scans does all this vera codes and every stuff X rays and other stuff. So then artifactory and then going to publish so once you publish your artifacts into your in your environment and then you do the page rollout and a can also help you there to know how the system is working well and how the users are doing and whether this application is whatever features or capabilities what you have added is it going to be reliable or is it having some additional bugs or whether it needs to be a. There needs to be a bug fix created for the project. What you have done, what you have deployed and also it can. It can also report you like how well it has went and how far so many issues and things like that. So we can see it as. It's very broad actually it's very end to end and I don't think this 30 minutes is enough to discuss the complete specifics of each and every capabilities. What is caught End to end.

Gotcha. All right, so I also have my notes. Large language models and small language models. I thought maybe you can kind of enlighten us between the difference and when you choose one over the other. So I guess what is a large language model compared to a small language model?

Yeah, sure, this is a good question. So, and I have I got this question from several of my friends, colleagues and others, right? So how do I choose and what. When should I choose LLM and when should I choose slm, right? So for example, I'll tell you some use cases, right? So specific. For example I'm creating chatbot and the chatbot current capability is for example I have my application and I have, I'm getting like around 15k calls every day for the support for my application. For example, I have a washer dryer or I have product which is running on, on the edge or on the end. So and I get a lot of support cases and how do I automate that before even customer comes in and then does ask me a question, right? So how do I first try to solve their problem before even they raise a case and even after they raise a case or if you if they want send any product details, then where should I go and check, right? So these types of things and I have my knowledge base is very huge and it's not just simple, right? So and also I have some futuristic view, right? If somebody shows me the defect product and I should say like you can go ahead and buy this part and then you can fix it by diy. So if you have these types of views and if you have these types of broad roadmap for the particular support thing and your support spend is very huge today and you wanted to cut down the support spend by enabling these capabilities and making it more user friendly. And so in that case I would strongly suggest go for LLMs because LLMs is now trained with lot of parameters and it can go get the details and it can do a lot of creative stuff so it can add more value to it, right? For example, if the knowledge base has got very minimal information on how to just tie a knot on this writer, then the LLM has this capability on based on the details what it has got and based on the user question, it can frame something which is meaningful for the end user. So in that cases you can go for LLM. But for SLM what I would say is if you have some specific use case, right? For example I have a solution where I can talk in natural language and for example reporting let's take some reporting capability. So I have this bunch of reports for my C levels and so every enhancement my C level or my marketing team wants to see on their report, they come to me and then they ask me and then I understand their use case first and then I kind of build that report for them. And till the time from the time they asked till the time they are going to see the report, it's going to be one and a half months time frame for the report to be given to the marketing team or the field team or anywhere which is too huge, too much of time doing this. So why can't I automate? So if I ask something natural language, right? How many product has been sold for this particular area for this particular on this particular event, Right. So in that case if I wanted to do some natural language to SQL queries. So in that case I don't need a LLM to do this because my task is very specific and I have my table schema and what I what it's all going to do is identify which table is the right one and it's going to frame a SQL query out of it. In that case I would say GPT4O mini should be able to solve this or else PY3 or any other small language model can help you solve this problem. And one other thing is cost, right? So when you run the LLMs large language model the cost is going to be very huge and when you come to SLM stuff it's way cheaper than the LLM for this specific task. I don't need then in that case you don't need to burn the energy what you are, what you don't really need. Right. So basically I would say echo for SLM in that cases and a lot of NGOs and other non profit organization they wanted to use AI for like email tracking and content generation and things like that. They are all more towards SLM these days for cost perspective.

Nice. So what else do people ask you about that once, once you explain the difference, did you ever get any follow ups like okay I know the difference between the two but you know any other more common questions you get asked around?

Yeah, so one other thing what I have seen is how do I run this SLMS in my local laptop? You know the SLMS is very small and can I run this in my laptop then how do I do that? And so far there is no enough documents available for few use cases. And so for those cases what I do is whatever questions I get very common, I try to block them. So it's on medium.com so AI with Naveen Krishnan is where you see all my blogs. So if you wanted to run a model called PY3 and on your local. So how do you use llama CLI to run your PY3 and how do you run GPT4 on your local. So these types of information, it's there in my blog you can get benefit out of.

And we'll have a link for your blog in the comments down below. Okay, another question I have in my notes. Microsoft fabric with AI. I actually haven't heard of this before and since you work at Microsoft, hey, you're the perfect person to ask. So what is Microsoft's fabric with AI?

Yeah, so Microsoft fabric with AI. So that's one thing like even I'm like you. So how I learned Microsoft fabric is just like six to eight months back. So I had to present to one of my team or one of my friend's team, right. So I have to explain them like what fabric is. And then so that's when I went in depth and then I started learning. So basically what fabric is analytics platform where it's built in with data warehousing and onelake and other capabilities. And one other key feature is you don't have to move your data or assets. So if it's stored on different storage system like be it S3 or be it anywhere outside of Azure and you can have your data as it is and you can have connectors and other capabilities to get the data from. So you can't, you don't have to duplicate the data and then overpay for the same data what you have. So that's the major feature of fabric what I see and when talking about AI integration. So one thing when I was trying and then one thing which impressed me a lot is as a developer and if you open a jupyter notebook on fabric and then you don't have to write any code so you just tell it like this is what I want and it can develop that for you. And you can see 10, 15 lines of code for that piece. And then for example, there is an Excel can you read that Excel and then store it in this. And I wanted to get to this so it can do this. That's a copilot what fabric has caught. So you should definitely try this once.

All right. So some people hear that might be a little scared. And I don't know if this is clickbait. I just saw something on LinkedIn. I can't find it now where a company's audio was leaked saying that they're going to replace the developers with AI next year. What do you think about that? Can developers be replaced by AI? Should people be afraid after hearing, hey, I just used fabric and it'll create an app for me or.

So what I see is. So my personal view is right, A it's just artificial intelligence. Right. So it's not AGI or something. So with this artificial intelligence you always. It's going to help you do more and from whatever you have been doing. So. Right. For example, for a feature to go out or go live, you have been spending around one and a half months with all this process, with the DevOps process and your testing and then automated testing, pen testing and other stages you have to cross. And it's going to take that long with this capability. It's going to bring that time to life, basically. That's what it's going to do to start with. So. And it's always. It's just someone you can see as something you can get help from and it's going to do. It's not going to do your job right. So you are just going to ask this something and then it's going to give it for you. Yeah. When. When the search and things rolled out and if. And it didn't take any job in the past. So yeah, it's the same.

Nice. So you know, with AI it's always moving, it's always changing. How do you keep up to date? Like how if someone's listening and they're like how do I even get started learning this? Obviously you learn this. Like do you have a process that you recommend? Do you start with a small project? How do you stay up to date all the time with what's happening with AI?

Yeah, so the first thing, so how to start getting into AI? Right. So mostly I've seen like, I don't know how many of you have not tried copilot or chatgpt yet. Right. So most of us, like 99.9% of us would have tried that. And then we all, after trying that, we all will have some assumption like how it is and what it is doing behind the scenes. So first clear those assumptions and that's the first step to get into AI. So know what is happening behind the scenes. Right. So that's where I would start basically. And once you get that understanding, then AI is very easy actually. So then first do quick certification on Azure OpenAI or Azure AI itself where it covers both the cognitive services as well as the OpenAI pieces of it. So which can gives you a glimpse of what is ML and what is AI and other stuff. So once you get that certification maybe or you can go read all the models about so what is 3.5 instructed and what the four who instructed know more about the model. So that's what I see the AI is as as soon as the new model comes out read thoroughly all the capabilities, what this model can do and which model is going to get retired as soon as this model comes out and how it is different from the previous model and if you have time and if you are a tech person then I would say do a hands on model once and then see how this model behaves for all your ask right you may have few questions like throw them on to the model and then see how that works. That's how I learned so far.

Awesome. Okay Naveen, before we go, is there one piece of actual advice you can give to a developer or someone to help them with their AI DevOps efforts and what's the best way to find or contact you?

You can find me on LinkedIn and so I'm available and I'm active on LinkedIn so that's the only place where I collaborate with my friends and colleagues. So that's one thing and one other thing small piece of advice. What I would recommend is so AI is not complex and AI is not costly so AI is very easy when compared to other stuffs around here. So learn AI and it's going to help you for sure.

And for links of everything of value we covered in this DevOps Toolchain show, head on over to test guild.com p167 so that's it for this episode of the DevOps Toolchain Show. I'm Joe. My mission is to help you succeed with creating end to end full stack DevOps toolchain awesomeness. As always, test everything and keep the good Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers throughout the four forefront of automation, testing and DevOps. If you haven't yet, join our vibrant community@testguild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges. We're excited to collaborate. Visit testguild.infox to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

With hoots and lyres, the bards began their song A tune of knowledge, a melody of code through the air it spread like wildfire from through the land Guiding tester showing the secrets to behold.