#272 The Unreasonable Effectiveness of AI in Software Development with Eran Yahav, CTO of Tabnine

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

I think generally the world is split into two camps. One that think that software development as a whole is going to be in natural language and programming languages are a thing of the past, right? Because why do you need to speak this foreign language of the machine? It seems like a waste of time. The machine should speak my language and we'll be done with it. So that's one camp. There's another camp that says these languages exist for a reason, and the reason is reduce ambiguity and be precise on what is the expected behavior without running the program.

Welcome to Data Framed. This is Ritchie. At this point it's fairly clear that one of the most important use cases for generative AI is writing code. But for software developers or data scientists, writing code is only part of the job. You also need to review and test the code and also write documentation for it. On top of this, as regular listeners will know, buying a tool is just the first step towards solving any problem. You also need to train people on how to use it and change your processes and workflows. So today I want to find out two things. Firstly, how generative AI can be used across different software development tasks, and secondly, what you need to do to get a business benefit from using AI. Our guest is Aaron Yahav. He's the Chief Technology Officer and co founder at coding assistance platform tab9 and he's also an Associate professor at the Technion. Previously he was a researcher in program analysis and synthesis for IBM Research and in 2020 he won the Robin Milner Young Researcher Award for his research into static analysis of concurrent programs. He's one of the most prominent thought leaders in Big Code, where you use machine learning to solve difficult programming tasks like code search and code completion. So with that, let's find out about how Genai can be used for working with code. Hi Aaron, welcome to the show.

Hey, thank you for having me. Brilliant.

We've talked about software development and using AI for it quite a lot. On the show. So I think a lot of our listeners are aware that AI works really well for developing software, writing code a lot of the time, but sometimes it doesn't. So a big question to begin with is like, when can you use AI assistance and when do you want to handwrite code?

Yeah, I guess it's really like, I'd like to talk about it in terms of the fundamental theorem of Genai, which is not that fundamental, but I'll play it out anyway, which says that Genai makes sense when the cost of specification plus the cost of consuming the result is much, much smaller than doing the work manually. So it really makes sense to use any of this if the cost of explaining what you want and the cost of consuming back the result is much smaller than doing it yourself. So this is maybe too abstract, but that's the nature of fundamental theories. Let me make it slightly more concrete here. It made sense to use it. If you can make a really quick snap judgment that what you got is what you wanted, then in certain domains this is easier than in others. Let me give you a concrete example. Let's say I'm writing my REACT application. Then I can see with my own eyes whether what I asked for is the UI I wanted. So I can make the snap judgment of like, I got what I wanted. So it makes sense to use that. If I'm writing very complicated, say, distributed system synchronization protocol, then it's going to be pretty hard for me to determine that what I got is really what I wanted. And therefore I may be better off not delegating this task to the AI assistant or the AI agent.

Yeah, I like that. So if it takes you a really long time to write your prompt, then that's like a big specification cost. If the output is hard to interpret or hard to work out whether it's correct or not, then that's a big sort of consumption cost. And you want those to be as small as possible, otherwise it's not worth it.

Yeah, and it's not really about Gen AI in a sense, it's about delegation. Right. It's also like I'm trying to delegate the task or to offshore a task to another entity, whether human or silicon. So I'm kind of like offshoring to silicon and it makes sense to offshore to silicon, when the boundaries of how do I define what I want and how do I make sure that what I got is sensible?

Absolutely. So maybe it's worth talking a bit more like frightening some context on. I mean, generative AI is just One tool among many different tools to help software developers write code. Can you just talk about like what are the other or what's its competition? What are the other tools that people are using in order to automatically write code?

Oh, there is a bunch, you know, of AI based solution for generating code. I think it really depends on the segment. Like are you an enterprise user who is trying to write code in an environment in which things are highly specific and incremental, in which case you will use something like Tab 9, which is more aware of the environment you work in? Or maybe you're writing smaller application, maybe you're not even in an enterprise setting. Maybe you're a hobbyist or an independent student, in which case you maybe use something that is more, let's say prompt based or more natural language based in the lower end of the complexity of the environment in which you're operating. But I think at this point there is like an assistant for every layer in space, right? And probably more to come because to be honest, the promise of gen AI is huge. But one of the areas in which this promise is already realized is software development. Right? This is almost one of the most prominent areas in which you see the value of Genai every day.

Okay, it's nice that there is a tool for like every different group of users depending on whether they're used, an individual or an enterprise or what industry. Let's talk about what you can actually do with this stuff. So is it just about having AI write code for you or are there other things you can do with generative.

AI Here AI can help you accelerate all activities across the software development life cycle. So it is about code generation, documentation generation, test generation, code review, deploying into production. On all stages of the sdlc you already see AI systems providing significant value and significant acceleration just on documentation generation, for example, something that developers really don't like to do. You can see a huge uplift there and also uplift in the actual documentation being generated or, sorry, the documentation that is present in your project at the end because previously people just not write it. But now that AI gets like from 0 to 1, people are refining it. So that's one of the effects that you're seeing is that maybe AI did not get it all the way through, but once the developer has a draft, they're going in refining that. So you get like high quality documentation by the collaboration of the AI writing the draft and the human refining that. I think that's true across the entire sdlc we also see test generation. Maybe the AI did not actually hit the full subtlety of the test that you wanted. But it provided enough scaffolding and enough of the marking, enough of the stuff around there for you to come in and do like the last mile and getting the full value out of the test.

This is interesting because I think a lot of people, if you become a software developer, you enjoy writing code. And so having AI write your code was like well, I was enjoying doing that but some of these other things you mentioned, so code review, documentation, testing, these are things where like no one got a job to really want to write tests and write documentation. So having AI work on those things, that seems like a good idea.

I think also like in, in terms of writing of code, I think there's a lot of code that people don't like. There's a lot of boilerplate, there's a lot of repetitive code, there's a lot of convoluted code, complicated APIs that people generally do not enjoy. So having AI automates the mundane stuff and the boring things. Let you the kind of the developer focus, software engineer develop focus on the creative aspects of the work is really what the Geni is making possible.

Yeah, I suppose if you just like writing a registration page like login code for a web developer, it's like you've done this a thousand times, doing it 1001 times is not going to be like that much more interesting. So yeah, maybe outsource that repetitive code stuff, but I would like to talk about some of these non code generation aspects of it. So talk me through code review. What's the kind of current workflow and how does it change once you've got generative AI involved?

First of all, it's important to understand that the bottleneck right now for really getting the full benefit of Genai is code review. Really what we are seeing right now is that generation has already accelerated using Genai. So everybody's generating more and more code, but review has not accelerated to the same speed yet. So what you're effectively getting is like a distributed denial of service attack on your review. Because all the developers are using AI to generate, say just for exaggeration, 10x more code. But the reviewer is not reviewing code at 10x speed. So if we want to leverage or get the full potential of gen across the sdlc, we will need to solve the problem of accelerating the review itself or making sure that the code we generate is of high quality to begin with. Otherwise it is really DDoS on the reviewer, which is also your senior developer effectively hammering your senior resources to review AI generated code. So that's one of the things that we are focusing on in Tab 9 is really solving this problem of the review. And the workflow is interesting because the question of what is good quality code is not a platonic question, right? It does not live in some academic sphere of just clean code book or something like that. It varies between organizations or varies even between teams. Because part of it is very, very specific to how we do things around here. Part of it is using the existing, say microservices, the existing code base, existing know how and so standards, best practices. So really the reason that experts are doing the reviews right now is because they have all this tribal knowledge in their hands typically, right. And are reflecting that as new code comes in to be reviewed. And what we're doing in Tab 9 is kind of replicating that specificity of the review by learning organization specific rules and also allowing the human expert to enter kind of organization specific rules and have the AI have topline review the code according to these specific standards. So you get the AI code review to be specific to the org, aware of the existing code base, putting in comments, but also suggesting fixes automatically. So you get the full spectrum of a human expert reviewer. What you want, the experience that you want to replicate is the expert code reviewer that has been a developer in the organization for a decade. That's the level of expertise that you want to reflect in the AR code view knows all the code in the org, knows the best practices, knows why we stopped doing certain things, why we do it the new way, and knows how to guide you to do things the right way, not just complaining all the time without remediation.

That seems smart that it has contextual knowledge about all the code that's been written before. But at the same time that also terrifies me because it's like the terrible code you wrote like two years ago. Somehow the AI is learning from that. And yeah, it's going to review things based on that. Okay, so the idea is that you've got AI that sort of knows everything about the projects you're working on, gives you good feedback. I'm wondering whether it changes the workflow now, because my sense is that code reviews normally happen towards the end of the cycle. So a software developer builds something, then Hansa opens some miles and says, take a look at this. If I'm writing a document, I can get AI feedback on what I'm writing as I go. Things like Grammarly will give me feedback in real time. Is there a chance we can move towards real time? Code review as you're writing.

Absolutely. That's part of what we're doing is shifting that code review aspect left into the ide. The nature of the problems are not always the ones that you'd like to flag as you type the code because it's much more contextual than say, Grammarly. Some of it. You'll be complaining all the time about code that has not been written yet. And that's quite annoying. We tried that. It's a bit too intrusive. But you do want some checkpoints in which you say, okay, I've seen enough. Let me tell you that you're wrong. Right. You definitely want to do that in the IDE and not in the pull request, if you can. Because also the IDE is my private space and the pull request is public space. So I'd rather get your comments privately. Right. Especially if I get this in full request. I would go, hey, top nine. If you knew this all along, why don't you tell me earlier? Why'd you wait until I push this? Like, tell me earlier. So definitely working on shifting that left into the ide. Absolutely.

That is cool that it comes sooner, but it's also interesting that you can't have that sort of instant feedback sometimes because some of the code you write relies on stuff you haven't written yet. Okay.

It's a user experience question really. I think some people like it. It's like more aggressive. Some people would like it to wait a bit and see like the full thing before it complains.

But yeah, and so related to this is the idea of documentation, having documentation being written for you. Now there have been some sort of semi automated documentation tools for a long time, but I'm hoping that Generative AI gives us better tools for this. Talk me through how does it work and how is it going to change people's approach to documenting code?

First off, it's really easy, right? You can generate docs and click off a button and you get docs, which are quite descriptive. Personally, I'm in the camp that says documentation should say why it is something and not what the code does. I mean, the code says what the code does. I don't need more of that. But as I said, I think the workflow, the real workflow is that you get like version one draft from the AI and then once it exists, people say, okay, let me add that small note here that actually explains that the important bit of why I did this way and otherwise they would just not add that bit. So I think that the workflow changes by forcing the existence of documentation. And then humans are leaning in to make that documentation more meaningful for the future. Iran, who will come in a year and will not know why he did something that way. So I think that's kind of the flow that is happening in reality. Interestingly, by the way, we're seeing a lot of success with test generation, which is another aspect that people generally don't like to do. And again, when you have a click of a button that gives you a test plan that you can then implement with another click and you can revise the plan and, you know, like manipulate what's going to be generated. So you control the process. It's not like I'll generate whatever tests come to mind. Right. You control the AI in what is the test plan that you'd like to implement? I think that is also quite successful. And another flow that is interesting is generating test cases as part of the pull request. So you push code, but in the background tabline says, hey, you know what you pushed did not have sufficient test coverage. How about I would give you like 10 additional tests. You don't have to worry about them, they all pass and they increase your code coverage from 40% to 80% code coverage. Take this additional PR if you'd like and let's move on.

That's another aspect I like that it can create a sort of test plan and then come up with the test to sort of fill things in. Are we just talking boilerplate tests here though, or can it do more sophisticated things as well?

It's doing pretty sophisticated things. Again, it's not a replacement for the human writing that super sophisticated test that will cover the extra corner case. But I've seen tests that covered cases that I missed. So it's smarter than me maybe for some cases. So I think that was good enough for me. Others may be smarter.

Yeah, I guess as a human, it's very easy to get bored writing tests and then your test coverage is low just because you can't be bothered to write the rest of the test. And AI doesn't have that problem.

No personal, I'm not a huge fan of measuring code coverage. I'm not sure exactly there are many academic words on whether that is meaningful or not, but definitely getting some tests and getting reasonable test coverage is a desirable property and definitely better than having zero tests, right?

Absolutely. I'm wondering, because of this probabilistic nature of large language models, you do have some randomness in the test that correctly. Do you have to start writing tests to test your test like Some sort of meta test situation.

That's a pretty interesting idea. I have not done that or seen that, but it's. It's a pretty cool idea. Yeah.

Okay, maybe like no one cares about test networks. They just want to maybe just manually check their own tests. All right, are there any other, I don't want to say boring, but are there sort of less fun bits of software development that AI can help out with?

Again, I think what we're seeing is any task that you can think of has assistance from AI and soon is going to have like an agentic workflow that does the majority of the work for you. The bottleneck for all this is a what is the quality of say, code that is being generated or refactored? It doesn't matter. Manufactured, right. It's not just about generating code from scratch. It's also about making changes to your code base. Right? Refactoring, modernizing, migrating between languages. All those tasks are tasks in which we're seeing increasing value, increasing productivity from AI. But really the bottleneck is what is the quality of the things that are getting generated? If I'm generating 10x more code and I do not provide any quality guarantees on what I'm generating, the fundamental question is, am I generating an asset or am I generating technical debt? And this is a very specific question as well, or specific question, because it could be very nice code, but that just doesn't use the microservice that you should be using inside the org. And so like platonically, if you look at the code, it's great, but it's just not doing the right thing. And so the specificity of what does it mean to have good quality code for my setting is really important here. And for me to trust the AI, the AI has to understand my organization very deeply. And I also need some way to control it. Because what you said is really important as well. Like let's say I have this legacy code and the AI Tabline understands that the way we do things around here is legacy code, because that's what it has seen. But I really wanted to stop creating new legacy, right? I want to somehow say from this point on, this is not the way in which we connect to Kafka, right? We do it like and different way, different microservice. So now I, the human, the expert, the architect, need a way to tell the AI need to filter mine. Starting from November 1st, we're doing it this way. You need the phase shift, so you need a the AI to know everything about your org. So it has Some implicit understanding of what is good, but also b a way for you, the architect, the expert, to say explicitly what is good and force that moving forward. Otherwise you'll never be able to escape the jupyter level gravity of legacy trapped in legacy forever.

I think transforming your code, like moving to newer languages. I know there's a lot of big enterprises where there's still code running on like cobol from like 1970 or something. And so yeah, that transformation seems like a great idea. And I love the idea of sort of getting rid of legacy code. So talk me through how does it work? I mean, I presume generative AI is good for translation between human languages, right? Does it also work with computer languages?

Yeah, translation between languages is really an easy use case. Even translating from COBOL to Java or something like that is doable. I think what people don't realize is that porting an application from COBOL to Java is not about translating procedure by procedure. It is about re architecting the application really, because it's just the concepts between a Java application and a COBOL application are different. And maybe if you're like doing this shift, you want to maybe transition from the monolith to microservices. Maybe there's a lot of stuff that you'd like to do as part of this migration and not just translating from running on COBOL runtime to running on jvm. That is not what you're trying to accomplish. Right. You're trying to get a modern application that is maintainable moving forward. And that's much more than just translating procedure to procedure. I think that's the misconception that people have. So AI can help you re architect as well, but I think we still need humans to kind of think what is the desired kind of end result of migrating the CO application. And AI can help you once you decide on this kind of the overall structure. AI can help you do the micro steps of migrating procedure by procedure or stuff like that. That works wonderfully well with Kat9 or any other system.

Okay, so that's interesting that you've got high level architectural decisions and then you've got sort of low level. Well, this is like what the code's doing on an individual. And this leads nicely to like one of the sort of the most hyped ideas of the last couple of months, which is chain of thought reasoning. So the idea is you've got AI or generative AI that can complete multiple steps at once. Can you talk me through how is chain of thought going to change all this AI Developer tooling.

It's not about chain of thought specifically or tree of thought or graph of thought or that doesn't matter, like the general structure. It's all about the agent workflows, as you said, completing multiple steps, potentially having backtracking, potentially invoking tools, and both to read and write information like sensing and transporting or manipulating. We're already seeing a lot of success in that. So we can have workflows that go from a JIRA ticket to a full implementation, right? All the steps. So I've done this cute demo in Tapline which I have, like start from JIRA ticket, doesn't matter, implement the back end of the application, implement the front end of the application, connect them, implement the database and all it is happening as a single agentic flow. You don't have to worry about anything, right? So the AI does everything for you end to end. And there are other demos on the Internet of similar things. So whether it's like baby AGI or open hands or whatever, there are a bunch of those. And these agentic flows provided with the right tools and the right abstractions can really go a long way and take you end to end. Either implement a JIRA ticket or debugging application or translate application. All those things are totally doable with the caveat that I started with, which is how do you guarantee the quality of what is it that you're getting? Maybe that loops back to your initial question of when does it make sense to use generative AI and when it doesn't. So one aspect is really easy to specify, easy to consume the result, but maybe another aspect is, is this like the maintainable code that I'd like to live with for the next five to 10 years? Or is this just no like a side project, a POC or something like that? In which case I don't mind doing the end to end flow without worrying about, you know, how any of that is really implemented. So again, in the offshoring analogy, I offshore this entire application because it's a poc and I just want to see that I got it right. And then maybe I either manually refine the components that were created or I rewrite it or I do it with some coaching rules like Topline has, which force some standard and some quality on whatever is being generated.

That's interesting that we're going from like chain of thought to agents and actually it's like two sides of the same idea. It's just being able to complete multi step tasks without having to have much human involvement.

Joeflot is like One plan, it's called planning strategy of an agent before. So it's a special case. Yeah.

We've been talking for 25 minutes and we start mentioning data once which is giving me withdrawal symptoms. So I'm curious. We talked about the software development workflow where code is basically everything. I think for a lot of like data use cases you've got much simpler, shorter data projects I'm showing for data analysts, data scientists, ways doing these short projects. How are these sort of trends in generative AI changing things?

I think data science is the perfect application for Gen because what I care about the code is really not the artifact that I'm after. Right. The code is a means to an end to get the analysis that I want, to get the visualization or data, whatever the projection that I want. And I couldn't care less about. The code is an art artifact for the most part. Right. Unless I'm building some framework. But if I'm analyzing something, the code is going to mostly be discarded at the end of that small project, as you call it. So I think that's perfect match. Also. It's a perfect match for all sorts of visualization projects because again, I can make a snap judgment of like, oh yeah, this is the histogram that I wanted, or this is the curve that I wanted, or this, the analysis that I wanted. I can look at it and say yes, this is great, or I can very easily manipulate it with a prompt with natural language because again, specifications are short, results are easy to understand, code is mostly short lived, perfect match on all three, on all three, all three counts. And also a lot of Python, which is a language that most of the models are really strong at because models are developed by researchers, they try things in Python as well. So there's some, I think very strong guarantees on Python being a very first class citizen of the models in terms of ability to generate and accuracy, et cetera.

I do like the sort of framework you have of deciding whether generative AI is going to be useful. Just like can you make that slap judgment or it just seemed like, yeah, visualization works really well here because you can look at a picture and say, well, does this make sense or not?

Pretty quickly it works really well. And complicated, complicated use cases that you've done before because I wrote this code somewhere sometime or something similar and I look at the code engineer and say that's exactly what I wanted. Move on. So it's actually interesting that for experts it may be easier to counterintuitively. Somewhat counterintuitively for experts Gen may be even more of a force multiplier in the sense that you like. Okay. I can make the snap judgment on a lot of things much faster than a junior would be able to.

Ah, so this is very fascinating because I think one of the sort of perennial questions over the last couple of years has been like, do I need to learn to code if AI can write my code for me? And you're saying that actually if you've got those skills already to be able to make that snap judgment about whether or not the code works well, that means AI is actually even more beneficial for you.

Yeah, I think that's true from what we're seeing. And I think again, in the foreseeable future, again not for all software, but some software is still going to be the combination of humans and AI and the ability to read code really well is very important. Like read code well and understand what's going on quickly because a lot of it's going to be generated by assistants or agents and you, the human will have to step in. So it's like the combination of self driving cars and human driven cars. You need to know how to operate in the mixed environment.

Absolutely. So talk me through. Like suppose you're an early stage, either developer or data scientist, or like anyone who has to write a bit of code in your job, is it worth pushing hard and becoming that expert in code or do you want to work alongside the AI tools? What would your career strategy be?

Yeah, I know career strategies is a big ask, but again I think it's inevitable. Right. It's just like IDE is a tool of the trade, so some people swear by Emacs or by Vim, but they still have some extensions that help them. Code completion or do some things that are taking them to additional productivity and AI assistance are like inevitable part of the stack. So if you haven't done it still, I think you should, whether you should learn programming, like be an expert in the nuances of Python or not, I don't know. I think a lot of the data science at least will increasingly move to a higher level of abstraction. I think again, I don't know how long will it take and definitely not for all cases. But I've seen models do, or assistants do pretty amazing things in data science from natural language alone and I think this will just increase because the sweet spot for this technology you can make the snap judgment, it's easy to specify and the code is generally short lived. So this is like really the perfect spot.

You mentioned moving to higher levels of abstraction can you just talk me through what that might mean?

Yeah. I think generally the world is split into two camps. One that think that software development as a whole is going to be in natural language and programming languages are a thing of the past. Because why do you need to speak this foreign language of the machine? It seems like a waste of time. The machine should speak my language and we'll be done with it. So that's one camp. There's another camp that says these languages exist for a reason. And the reason is reduce ambiguity and be precise on what is the expected behavior without running the program. So again, if it's a visualization or ui, you can run it, pretty much see what happens. You don't need to be super accurate or specific about the code itself. You can predict what's going to happen by running it. But let's say it's something more complicated how you, the human are going to predict how this thing is going to behave. You need some language that describes it in a non ambiguous way. And if it's not going to be English, because English is ambiguous, you need something more structured. So there have been past attempts at some, some formal English, structured English, but at the end they all deteriorate into some form of, let's call it Python, Ruby, whatever, something that does have structure that you can pretty accurately reason about what's going to happen when you run it. So camp one says inflation is the way to go, don't worry about anything else. Camp two says I need some symbolic representation that is non ambiguous and accurate. And interestingly, both camps are right because programming is not a single thing right. You don't program your React application like you program your nuclear reactor. So the level of rigor required in these two ends of the spectrum is quite different. And I suspect that therefore the languages that will be used to develop these will be different as well. So yeah, I think both camps are right. Where it lands is probably the usual hype curve. So we kind of like, yeah, everything is done in English. Oh actually we don't know how to debug this or maintain it or control it or every time I change one word in my application I get a completely different result. Because no, it's not like a continuous incremental translation. If you change the English and regenerate the app, something else may happen. So the space of mappings from the natural language and application is not like nicely continuous one. When you change one word and you get to like tiny change may be radical change from a tiny change in the stack. And so we're going to probably start with everything is English, then, oh, actually we're wrong. We need something more structured than before. Proof is somewhere in the middle and also depending on the domain, as I said, like whether it's like a UI app or like the distributed database.

Yeah, certainly. I mean, you give the example of a nuclear power plant. If you've got like nuclear power plant critical control shutdown code, you don't want that to be kind of. Well, it's about right. I think the AI generated code was. Okay, that needs to be absolutely spot on.

I think many years ago I had a conversation with someone who was working on verifying Airbus code, and he said, listen, there is a very fundamental, simple question. How much are you willing to pay for a line of code? If you're willing to pay €30, then you get Airbus level code. If you're willing to pay $0.03 per line of code, then do whatever needs to be done. It's an economical question at the end, really, because the level of trust or the level of the quality that you build in the system is at the end a question of cost as well.

Absolutely, yeah. I guess the consequences of things going wrong for a lot of code is less than like a plane crashing or nuclear power plant exploding. So, yeah, you can afford to do it a little bit cheaper. So on that subject, I'd like to talk a bit about how you go about adopting all these generative AI tools in the enterprise. So suppose your head of department, whether you're like head of data or head of software engineering, says, okay, we're going all in on AI. What do you do?

I think what I would recommend and what we're seeing is different. We're seeing people being hesitant because of the quality aspects. So I think we are kind of like in code or in software engineering. We're already past the dog playing the piano, so we're past the magic of, oh, it can write code. We're now kind of into the question of does it generate quality code that I can trust? And so people are kind of hesitant to say, take the entire department, give them tab 9 and let's roll, because they don't know what will the implications be. And as I said, the details on the reviewer is definitely one of the concerns. Like this is going to just increase the pressure on our code reviewers because they have to review the AI generated code all the time. And with war being even more careful because they know the AI can generate things that are maybe not exactly what we want. And so what we're seeing is People are taking some part of the department, letting them try it and then we see very quickly some extension once they prove the value. Right? So it's like very quickly because this thing is magic, right? Once you start using it, say oh wait, I can do so much more. And it's like a no brainer. And I think as a department head, if I look at this, I say absolutely, bring it to me today. But tell me how the code review is going to work as well. How am I not going to overwork my reviewers? Tell me, can you be explicit about what are the standards that topline maintained? Can you show me the list of rules that are being enforced? So I can say, oh yeah, these are the rules. Or I can say wait, there are two rules missing. Or I can say, oh, these rules I don't care about at all which all of the above can happen. But I think really Genai for software development is such a no brainer that the return is immediate. And I think at this point I'm guessing most of our listeners are already using Genai. I don't think they need me to tell them to use it. Right? It's been a while now. Everybody recognizes the value. I think people also recognize the potential risks and why you need code review. I think the value is inevitable, it's just coming.

I like the idea that yeah, a lot of people know what they're doing, it's just gonna happen. So in terms of areas to focus on though, like do you start with cogeneration, do you start with less do documentation and get the hang of this? Do you do like low risk projects first or high value projects first? What's the order of getting it implemented?

I would definitely start, you know, with depending on what is the project, et cetera. Definitely start with new projects because the amount of code to be generated is kind of the promise, right? I can move faster, so I would definitely start with that. Also the legacy things tend to be more subtle and more incremental changes. So we need to read much more code than you need to write. And again, it's just harder to be really successful than possible, but harder. Test generation is definitely one place to start because test code does not run in production. So you're kind of like, maybe I'm taking the risk that the test will not cover exactly what I wanted or the tests will run longer because I have more tests and now maybe not as helpful as I wanted, but the risk is zero. Right? I'm not losing anything, really not jeopardizing my production system by my junior developers accepting code from AI without understanding the implications. So test generation is definitely a place to start. Yeah, I think code review is another place that you can get a lot of value because it reviews both code written by AI and code written by humans. Right. And holds them to the same standards, which I think is also very important.

All right, so working with new projects where you're starting fresh, and then also the places where things aren't going in production directly. So, yeah, testing and code review. I like it.

I would go direct to the production. I trust this well enough. I think I'm just trying to cater for people who are trying to do some gradual adoption and are risk averse.

Very nice. There's definitely a big spectrum from people who are like, gung ho, I'll use all the latest tech for everything. And people are a bit more risk averse. I guess we're going back to the nuclear power plant example. All right, super. Okay, so just to wrap up, do you have any final advice for people wanting to make use of these AI software tools?

Resistance is futile. It's like it's coming inevitable. Start using it today if you're not already using it.

All right, Yeah. I love the Borg response there. Very nice. Okay, thank you for your time, Aaron.