Effortless Code Quality And Security For Developers with Jaime Jorge

Get ready to discover some of the most actionable DevOps techniques and tooling, including performance and reliability for some of the world's smartest engineers. Hey, I'm Joe Colantonio, host of the DevOps Toolchain podcast, and my goal is to help you create DevOps toolchain awesomeness. Hey. Today we'll be talking with Jamie George, all about effortless code quality and security for developers. If you don't know, Jamie is the CEO and co founder of Codacy, which is a leading code quality platform trusted by over 600,000 developers around the world. He's been working in the code quality and software engineering tools industry for over 10 years. It's a really cool solution. I think it's going to help a lot of people. You don't want to miss this episode to find out why code quality and security is more important now than ever. You don't want to miss it. Check it out.

Hey, if your app is slow, it could be worse than an error. It could be frustrating. And one thing I've Learned over my 25 years in industry is that frustrated users don't last long. But since slow performance isn't sudden, it's hard for standard error monitoring tools to catch. That's why I think you should check out bugsnag, an all in one observability solution that has a way to automatically watch for these issues. Real user monitoring. It detects and reports real user performance data in real time so you can quickly identify lags. Plus you can get the context of where the lags are and how to fix them. Don't rely on frustrated user feedback. Find out for yourself. Go to bugsnag.com and try it for free. No credit card required. Check it out, let me know what you think.

Hey Jamie, welcome to the Guild.

Hi Joe, thanks for having me.

Great to have you. So I came across this solution. I think I came across it through a. There was some sort of paper out there by IDC or something that talked about code quality and I thought, hey, this would be a great topic to talk about, I guess before we get into it. Obviously code standards have been important all my life. We had all weird code things that we had to go by by naming variables and things back in the day. So is, I mean the age of AI and agile. Are code standards still important?

Yeah, code centers are absolutely important. You'd be surprised by the amount of problematic code that gets shipped every day into production without people knowing. And they're absolutely invaluable. So I mean, we started a company 10 years ago with really a mission to help as many developers as we can. Kind of the mission, you know, refined over the years. But what we really try to do is create code and products that are trustworthy, that you can run and you feel safe using them. Right. And so that's really what we're all about. We've created this platform that is not only about quality, but also security and helps developers ship fantastic software and back to standardization. Essentially, many of our customers, what they use our products for is to make sure that regardless of program language, location, geography, actual spoken language, they can ship consistently with the same criteria, with the same quality requirements, and every programmer can speak the same language, so to speak. Right. So it really enhances the shipping power of a company when they ship with standards.

All right. So as I mentioned now with people shipping quicker and faster more often, it seems like this is harder to do because I used to be on code reviews for every time someone checked in code, I'd have to look at it and accept it. So how does this help get over that where we're shipping quicker and faster and we still want to adhere to our standards?

Well, we believe that in order to really help development teams deliver software quality and great standards, we have to be close to where the action happens. And that is essentially either at the pull request level, commit level, or even the IDE level. So giving feedback as early as possible so that the cost of change is the lowest. So we try to be as close as possible to where code is being written and we try to be actionable. So we have at our disposal Today more than 15,000 code analysis rules that any developer can run, plus secret scanning, plus software composition, analysis, even pen testing. Now we have also as part of our arsenal, but we try as much as possible to make sure that people choose the minimum set of rules and standards that they need to ship. So that instead of having kind of a boil the ocean quality and you know, we want to protect everything, they have a pragmatic approach to quality. Right. So that every time they ship, they don't have 100,000 issues. They have a few that are really important. And so that's for us, for example, that's why we moved into security, because we wanted to make sure that every insight is very important and crucial.

How do we know it's important? I guess it depends on context. So how do you know this security concern is probably higher priority for one software than another, if that makes sense.

Yeah. So we do two things. One, we spend time with our customers and users in configuring static analysis. As you know, static analysis is very prone to false positives. And so issues that are really not that important or should be prioritized. And so we spend time with our customers trying to configure what that really means for them to codify what quality means for each customer. That's really important. So we're investing more than what you would see in other players in the industry and trying to get to that definition of quality. And then the second part is we tried as much as possible to get the most important security levels that you need. Right. So we cover all bases from security level. Essentially how we like to say it is. You get all the security you are likely to need as you develop your company. And that allows us to focus our attention to particular issues. So for example, if we find password in your code, we're going to alert you. If we find that there's an important vulnerability to your CVE that needs to be raise at the right moment, we're going to alert you. If there's any issue that came out out of pen testing that needs to be part of your report that developers need to know, that also going to be part of that.

Nice. So I know security is a big part of it. I want to dive into that a little bit more as well. But I want to make sure this is not just a solution for security. Correct. Is it? Does this help you like with duplication? A lot of times used to find people spend a lot of time on code reviews going, wait a minute, I've seen this before. You need to change it because it's already there. Does it help you with those types of things as well?

Yeah, that's really how we started the company with metrics such as duplication, making sure that copy and pasted code is not only reduced, but managed. That's the most important part because sometimes it's the right approach to have copy pasted code, even if it's really small fragments of code. We also have other things that are really important from a quality perspective, such as unit test coverage, making sure there are tests that code has unit tests for each new functionality or previous functionality, that the percentage of code tested, that coverage is tracked so that new features are also tested and that you're keeping tabs on how that is evolving and creating and then on top of that layers such as, you know, blocking a pull request if coverage drops dramatically. Right. So all these things are part of our platform.

So, you know, developers have a lot of burden nowadays. They check in code and they get all These reports and all these actual things they need to do. How do you fight, not fight that, but how do you make it so even with security, how do they know how to even fix some of these things? And how do they know what's the most important? Where it triages it for them? Does it help with that as well?

One thing that we introduced last year with the whole AI advent was the ability for every single issue that we identify, whether that is security or just more quality or even style related, has now the ability of being fixed.

So.

So we now use AI models to suggest fixes for every single issue. That kind of lowers the barrier for people to take action on the issues that we find. We do stress the necessity of configuring what static analysis means for you.

Right?

It should mirror your quality requirements and not just a big collection of all the issues that you want to find, because that's not going to be actual, that's just going to be noise. And no one has time for that. Right. We have work to ship. And so you. Between these things and the work that we're doing, the recommendations that we do with our team, because we've seen a lot of code in a lot of companies, right? We serve not only really small individual developers and some of the largest companies out there that use us with more than 3,000 developers in certain instances as well. So by virtue of scale, we can recommend tons of things for our customers.

So that's a good point. It's almost like you can use machine learning and all the data you're collecting to find out maybe good practices or best practices. Are there any mean offenders that people, if they just watched out for this, it would cut up, you know, 50% of their issues or something like that. Or is that too simplistic?

Well, it's. If we reduce any styling issues and any other types of vulnerabilities that we might be checking as part of our libraries. One of the most important things that we are recommending people to look out are just tokens and passwords and anything that would remotely give access to anyone outside of our team that does not belong in your code base. And that's really. Those are moments where dangerous moments. And so we stress that at the very least, running tools such as Trivi, which is a great solution and essentially we also run internally for our customers, is a great recommendation. Right. So those things really cover you in a moment where something slipped into your code base and now it's potentially public.

Nice. So you mentioned pen testing other things. What are the recommendations when you're trying to integrate these type of tooling into your pipeline. Does it slow it down? Do they go into a separate environment? Do you run pen test on all the check ins or. It just depends.

So today the way we work is we partner with a firm that essentially does pen testing on the side for live environments. Right. So what we do essentially is then tie back all the findings that this firm identifies and puts it at the same level as your security or quality findings so that you have a full report centralized with just one pane of glass. We are actually diving deeper into the dast space. So that will give us another opportunity to also answer that question better in a few months. But the way that we work today is very much paralyzed. Right. We have another company that looks at your production or testing environment and tests it out very thoroughly.

Nice. So how would you explain what codacy is? If someone's listening, they're like, like how do you pitch it as like, what is this solution and who would benefit from it?

So you know, the way that we see or kind of the companies that take the most value out of this tend to be companies that have high criteria, high standards, very high quality standards. They have already a larger team being composed of 50, 100, 200, 500 developers, sometimes multiple products, sometimes different modules that need to be developed and potentially have already multiple tools. Some of them might be even more expensive. And so what we like to see ourselves or the way that we are typically best used is where we consolidate much of that security offering with a quality perspective as well and completely shift left. So a team that is shipping fast, that needs to be protected, that is for example, going through SOC2 compliance or PCI and have all these processes in place. They have auditors on their back, but they need to ship the product. That's when we're extremely useful. And so that's how our customers really take advantage of us.

How do you avoid overlap? I know like in production, sometimes people have like 50 different monitoring tools going on. They don't even realize it's. I'm sure the same is on when people are checking in code, they have all these sniffers and things running. And this sounds like you consolidate some of those things that they may be using now. So when someone buys this type of solution, do they have a consulting, does it automatically say, hey, I noticed you're using this, you don't need to use it anymore? How does that work?

If you're using open source tools, it's very likely that we're going to detect that automatically. So if you're using tools that are free to use, which discovered by talking to people is, that's many times the case. If you have a few scripts already tools running, we're going to bundle them together and I'll give you a great platform on top of that. What we see is that there's a big wave of consolidation of tooling happening and that means that just providing, unless you're providing the best insight possible, that truly is important. That's going to be very interesting for specific industries. But most companies will be looking for a tool that provides broad value. Right? So many tools, many insights, many programming languages. And that's how we see it. That's how we've been collecting arguably the best collection of code analysis rules and tools in the market also.

That's a good point. What languages does this work with? What technologies? What do people need to have in place for them to bring in codacy?

So we support more than 45 programming languages and or frameworks and we typically operate with a few cloud providers already code hosting cloud providers such as GitHub, BitBucket, GitLab, we focus strongly on our cloud offering and that's really how we have been evolving as a company is batting on cloud environments and that's really following a trend in the industry. Many of our customers have already cloud environments where they host their code and test and check for many of these things. Other than that, we support almost everything.

Nice. So obviously AI is a big deal. You've been around for 10 years. How is AI? Have you incorporated AI and is it a real deal? Is it like overhyped, underhyped? Do you see it?

We incorporated AI by essentially integrating one of the most popular models and identifying fixes for the issues that we find that has been successful. And so our customers have been taking value from that approach because it essentially allows us to find any fix for any issue for any open source analysis tool which, which is great in my personal opinion. I think it's where it should be. I think there might be a dose of overhype, but I think it's where it should be given the transformational potential of this technology, of how this could permeate every single industry known to man and change things with more productivity. I think there's sometimes an abundance of fear or essentially people are overly cautious or fearful of AI. When I think AI will first and foremost augment people and you know, when it comes, for example, for development, I find it absolutely, you know, entertaining that first of all we're running or trying to race towards trying to automate our jobs as developers. We are likely not going to succeed, at least anytime soon, I think one or two years, but we're trying really hard. If you look for last few tools that been trying to automate it, such as Devin and such, I think what we're going to be finding in the next few months or years is that this is enabling us to produce more software, more lines of code, but not necessarily more secure code, which is a problem. Right. We ran a few tests that found first of all, there's papers that show only 65% of code produced by AI models is correct. There's tons of hallucination and there does not consistency in the output that you get. So it's very difficult to get tons of consistency that otherwise you get from rules. So I think a future in terms of quality and security will likely be a combination of great rules and great AI models working in tandem with humans, not substituting humans.

Do you see your customers when they hear AI being baked into a tool, that they actually have security concerns about that? Like what's that doing? Should I be concerned about that as well?

They do, they do. Customers primarily are concerned whether companies are using their private data, their company data to train models that could essentially leak somewhere, someone in the future. So that's a really important thing. We don't do that, by the way. And so there's definitely ways around them around this training, but that's the main concern. We ran a survey in our customers about the use of AI for developing software and most companies are not yet, don't have yet a policy for using AI coding tools. But it seems GitHub is telling you this just tools such as Copilot, they're having massive adoption and developers are using that consistently every day. And so there's an interesting, it's an interesting scenario where most developers will likely use. And that's another whole thing here also, which is from conversations with development teams. Senior developers sometimes get very annoyed at tools like Copilot because it provides results or answers that otherwise they would know it's not perfect or it's not there yet or it's not 100% correct. And I'm just wasting my time writing these prompts. And so it's. We look at these things as also a projection of how they will look like in a few months or years. And they will be tremendous, they will be amazing, they will be very, very, very productive for people to produce better software.

Nice. So I guess on the flip side, would Codacy be able to tell if something was AI generated code. Because I know there's some companies that have concerns like if you just have someone that's just generating code and checking it in and they don't really know what's going on. I don't know if that makes sense, but does it?

No, it does, it does. And very frankly, that's the conversation we've been having internally is how can we help our customers write software with AI protecting themselves against the or with the use of AI while writing software. And we've been looking into how can we find AI generated code, how can we pinpoint that this piece of software might require more review because it was produced with AI assisted tooling. But the problem also is there's so many iterations that you're producing software. And so it might be that you ask copilot to generate a piece of code or you go chatgpt and get a whole functionality, whole file and you merge these two together, you merge it by, you know, manually and it's going to be very difficult for you to identify that. So I think in the future the best approach would be to just assume that there is AI generated code. Any new code will have AI generated code. That's the best assumption. We're looking into how to best help. And at the moment our approach is you should be using us, you should be using rule based code analysis and all the security protection possible so that you're safe. Because otherwise it might be you're going to be moving really fast with AI tools. As long as you're safe. Go the speed you want.

Love it. So because this is mostly an audio podcast video, but we're not showing our screenshots. If someone's trying to get their head around how Codacy works, can you talk through how does it integrate into their ide so as their writing code, it's prompting them. Because you said you want to shift to left or is it when you check in code it runs a sniffer and then it gives back a report like how does it work?

So we have a bit of both worlds, right? So we start really as a centralized tool that runs within GitHub. Right. So it runs as part of your pull request creation. Create a pull request, we come in and we analyze code for you, we tell you all these issues and then we can take other actions should you want to, such as blocking a pull request if they don't conform to your standards and such. Right. But we Also now have IDEs that allows you not only to have the information that is centralized in at the pull request level. And very soon we'll also have the ability for you to analyze code locally with the tools that you're using, but with configuration that you did on a company organization level. So we have a bit of both worlds. What we find is that for companies to achieve some sort of success, whether they use local analysis or analysis ID level, they must have at a central place, at a pull request level, some sort of guardrails. And that's where we really shine.

Nice. So as I mentioned, early on I saw an IDC survey I think you even have on your website. The top thing and something I found kind of interesting was only 50% of developers say they spend the time writing code. So how does Codacy help them to write more code then?

If we're successful, we don't necessarily need or want that developers that use Codacy write more code. If we're successful, we want developers and our customers to write better code that is right and safe the first time. That's it. That's our mission. That is creates trust for their end customers. Right. So that's the ultimate goal. If it's more code, we don't really care. Ideally it's less code, right? So it's less to maintain over time. And so it is kind of surprised us, but not really. That was that little amount of time, right? Because the function is to produce software. But developers spend a lot of time with auxiliary tasks. Whether those are meetings, whether those are estimating and doing poker and wearing shirts and different sizes for the stories they want to ship, managing, sometimes even customers, those things.

What's interesting to me is that those things today, they might be very.

Boring or not really something that developers want to do. But ultimately I think as we look forward forward in our industry for next five years, they might be what also empowers developers to continue producing software. Right? Because we might be in five years time living in a reality where code being produced is just the task of machines. And so it's not necessarily bad that we're spending this time actually taking care of the business and writing software. As a consequence of that, it might be actually interesting.

So with the types of reports like I know developers are techie, they understand these things, but obviously a CTO or you know, product manager needs to know that they're ready to ship. So does do you have any reports of dashboarding where someone that's not a developer go and say okay, should we ship, should we not ship? Like does it break things down at a higher level? Or this team always checks in security Issues, you may want to give them training. Does it give you that type of insight as well?

Codacy as a product or kind of the first product we brought to market, our first solution wasn't really designed for non technical people. We focused really on the technical audience because we wanted tools that they could understand, not necessarily non technical people. It's actually one of the reasons why we brought another solution to market which was aimed at higher level, potentially technical people that measured different things. And as a parenthesis, we brought the solution to market because we didn't want customers to be measuring progress of development teams using, for example, number of lines that could be produced or number of pull requests produced. Right. So we brought things such as medium time to recover, lead time for changes and so forth so that people can measure the right things. When we look at our dashboards, we tried as much as possible to give you the closest answer or the fastest answer possible for you to understand should I ship this pull request or not this fully configurable. So it's really up to you to define those rules. So you can say essentially everything that is within fully your control to say what stops a pull request from being merged. But as soon as that is live, those will be the rules and it's very, very easy to understand how those are defined, but also then enforced.

Awesome. Okay Jamie, before we go, is there one piece of actual advice you can give to someone to help them with their DevOps efforts? And what's the best way to find contact you and learn more about codacy?

So the best piece of advice that I would give is start small, incremental, but consistent. So I would roll out tools or processes or things that makes the lives of developers easier. I would not try to boil the ocean as you come into your, for example, digital transformation or just implementing security. I would do one at a time and see how the developers are. Developers have a lot on their shoulders. Shift left today is shifted. So it's already a lot of weight on the shoulder of giants. And I think it's very important that we have that in mind that we take it piece by piece. It can be fast, but it's incremental. That's my take. We at Codacy, we take all of that away because it allows you to adopt things very safely. Right. So the way to reach me is jamiefgeorgessentially x or essentially jamiedacy.com and happy to reach out if I can be helpful.

Remember, latency is the silent killer of your app. Don't rely on frustrated user feedback you can know exactly what's happening and how to fix it with Bug Stick Snag from Smart Bear. See it for yourself. Go to Bugsnag.com and try for free. No credit card required. Check it out, let me know what you think.

And for links of everything of value we covered in this DevOps toolchain show, head on over to test guild.com forward/p144 and while you're there, make sure to click on the Smart Bear link and learn all about Smart Bears awesome solutions to give you the visibility you need to deliver great software. That's smartbear.com so that's it for this episode of the DevOps Toolchain Show. I'm Joe. My mission is to help you succeed, creating end to end full stack DevOps toolchain awesomeness. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the small, smartest testers, DevOps and automation professionals in the world, we'd love to have you join the fam@testguild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the Guild community, I'd love to hear from you. Head ON over to testguild.info and let's make it happen.