AI Agents in QA: How to Keep Up with AI-Driven Dev Velocity

Hey, here's a question you might be asking yourself. I know I am sometimes. If AI can write the code, spin up the PR, and run the test, where do you fit in? Today we're going to break that down because the answer isn't what most people think. And we have one of the best people to speak about this because today my guest spent years building one of the earliest production LLM systems in venture capital, processing 300,000 transactions a day back when most people haven't even heard the term generative AI. So he's seen firsthand what happens when you speed up one part of the software development lifecycle, but ignore the rest. Spoiler: the bottleneck doesn't disappear. It just moves. And right now it's sitting on top of QA. So he did something about it. He co-founded QA Tech, a platform that uses autonomous AI agents and behavioral knowledge graphs to test your application the way a seasoned manual tester would without a single line of test script. And we're going to get all into it today, covering things like knowledge graphs, agentic testing, PR-level automation, bug reports from Slack, and the real role testers play when AI starts doing the clicking. Hey, I'm Joe, and this is the TestGuild Automation Podcast. My guest is Wilhelm von Ehrenheim, and he's the co-founder and chief AI officer at QA Tech. You don't want to miss it. Check it out. Before we start, I almost forgot, this episode is actually sponsored by the folks at QA Tech. Look, if your team is using AI to write code faster, your testing pipelines need to keep up. And QA Tech's Agentic Testing Platform uses a behavioral knowledge graph that we go over in this episode that helps you understand your application the way a seasoned manual tester would, and then deploys autonomous AI agents to test it at speed your dev team actually ships. So no brittle locators, no playwright hacks, no waiting for someone to write test cases after the PR is already merged. QA Tech integrates directly into GitHub workflows, analyzes incoming PRs, runs targeted tests in your preview environment, and automatically writes the review. You can even reproduce bugs reported in Slack. And we actually see some of this in the demo in today's episode. So if you want to see what agentic testing actually looks like in your real CI/CD pipelines, head to QA Tech and book a demo now using that link down below. Now on with the show. Hey Wilhelm, welcome to the Guild.

Hey, very, very glad to be here.

Really great to have you. I'm really excited to talk to you 'cause you have a lot of experience with AI I want to get into like where you think we're going and where, you know, there's a lot of hype around it and automation and testing. So I'd like to get your point of view on it. But before we get into it, I know you spent years at different companies, I think EQT building MotherBrain, I think? Yeah. One of the earlier production, yeah, LLM systems and venture capital. And before that, I think you worked on like a real-time fraud model of 300,000 transactions a day. So you have real ML work. Like it's not just, it's not like you just have hype, you've actually done it. So I'm just curious to know what made you walk away from that and then build a testing company? Like of all the things you could have done with AI and ML, why a testing company?

That's a really good question. I think first of all, I have been analyzing startups for 7 years at EQT. So I kind of wanted to do a startup myself at some point. And then of course, We were playing with early agentic solutions then. We've been using, at, at DKT, we were massaging large amounts of unstructured data continuously and then working with GenAI models before it was even called GenAI, right? But when ChatGPT was launched and we were building early kind of RAG solutions and different things like this, we were playing around a lot with agents as well because made sense in how we process information and be a bit smarter about how you do search and these things. So I was super curious to build something around agents and it felt like a really good time because that was in 2023. So it was very early for agents. It kind of barely worked with GPT-3.5. But, but I was very, very curious about this. And also I've been living and feeling the pain of testing things. MotherBrain was a lot of AI and data massaging and things, but ultimately also a CRM tool on top. So it's like a turbocharged CRM where the deal professionals can go in and work with searches and deals and kind of populated with information. And testing has always been painful. So especially when you have a lot of data as well, it's a lot of things that can go wrong. So that really resonated with me. And then my co-founders, had similar experiences at previous companies where they worked at a company called Memo where they were launching to 20 different markets in a year and had a lot of problems with testing. So that's where the idea kind of came from. And it felt very, very interesting, like an interesting area where agents should be a really good kind of application to, to, to, to address this problem with. So that's how it kind of came to be.

Very cool. So let's keep it simple right now. Like, how do you explain what an agent is? People hear LLMs, they hear AI, ML. What's the agent piece of this?

I have a somewhat alternative take on this.

Oh, good, good.

Perfect. So the concept of agents, I don't think is especially new with AI now. So when you think about where it comes from, it actually was originated way, way back in the kind of farming cultures. The word agent actually came up already at these early clay tablets where they specified rules on how you should think about agents. And the definitions of an agent that kind of stems from that time is somebody that does something on your behalf. So, and the reason why you're using that is usually two different things. Either you want to do it because of scale, like you can, if somebody does something on your behalf, then you could do it like a lot more, like you could suddenly run 20 or 100 different farms and just not just one, but it could also be that they know things that you don't know specifically yourself. So there could be like you got help with, uh, from, uh, somebody who can travel across the sea and then, you know, so, and I really like that when you think about it from this AI perspective as well, we're not talking just about like how you're structuring the calls or if it's a workflow or an agent, whatever, as long as it's something where you can offload and ask an AI to act on your behalf with your kind of interest in mind. I think that, that is a very interesting definition.

So is agentic AI and an agent interchangeable?

Yeah. I mean, agentic AI is essentially where we start to kind of let the agent take more decisions. So it's not as well defined exactly what it's supposed to do. It might have a very clear, specific goal, but how it gets there and how it gets it to the end result is less defined. And this, in order to do that, you have to have agents. It's very similar.

All right. So, uh, quick little pivot, uh, cause I actually, someone from your team sent me a book called, um, something about bottlenecks, an ebook on bottlenecks. And I think, uh, in that ebook, there was an observation on that, uh, AI coding tools, uh, didn't solve All the bottlenecks in software development, it just moved it. So with the introduction of AI in software development, it was kind of like, this is going to solve all our problems with coding. We should be able to develop quicker and faster. So where did that issue move to then if the bottleneck didn't go away completely now?

Well, software development is more than just coding. So when you think about it from the entire software development lifecycle, there's a lot of different things, all the way from discovery and product to kind of actual testing and making sure the stuff is working as intended in production. So when you make one part faster, you naturally just trying to shift the bottleneck by the old theory of constraints. We are of course seeing tools coming that try to address this in different areas of the SDLC now, but the using coding agents and coding tools to make your code faster, and especially mostly as an individual in the team, can usually put more stress on other parts of the SDLC because there is a lot of different things that needs to move and work in order for it to actually make the whole chain faster. I think also you see this quite clearly if you see there has been a lot of research from like Stanford and DX and McKinsey and a lot of different places where they see that the AI adoption and where the companies that are most successful are either kind of quite small companies or, or those that are quite large to have a lot of resources and usually very, very good engineering practices around automating the entire SDLC, like a production line, a factory essentially. And the more of that you have in place, if you, if you are a large, solid tech engineering organization and you're happy with onboarding a new junior dev and let them ship a PR the first day, then usually you have a lot of things in place that make sure that it's not breaking when it reaches production.

So one of the things you have in place, I've worked in QA testing all my life and we've always seen as a bottleneck. And I would think now with AI-assisted development, You know, your velocity is just crazy. So what are these teams doing then that allowing them to ship so quickly and still have quality, or do they just ignore it now? Does it not matter? Like, where does QA come in and how does QA keep up with the velocity of the development now?

It is obviously like, when you think about it, you know, the only way to actually speed up and keep the same pace is if you're able to codify a lot of knowledge and things that you typically have in people's brains. So if you know, like, oh, we're changing this, and then I know that we need to think about all of these different things or talk to these 5 different other teams in order to make sure that it works nicely, then it's probably not going to be easy to automate, right? Whereas if you have stuff structured in a much more kind of pipeline fashion where you have all of these checks codified as much as possible so that it's easy to to run things at a faster pace and make sure that it actually functions as it should, and the code follows contracts and like thinks about how data is flowing through your application and all of this, then it's much easier to reason about the change. And AI tooling can be kind of present in all of these different things. You see a lot of code review tools coming out that makes it easier for you to do specific kinds of reviews on the code. You also have different tools for testing and making sure that testing works nicely. So I use these tools to codify all the knowledge. I think that's very important.

I don't want to put words in your mouth. I, once again, I think I've read somewhere where you said manual testing, not as dead, but it's almost structurally impossible when you're using AI-driven dev speed, but My question is, there's a paradox I've been seeing. When I ask my audience how many people actually trust AI, it's like very low, but the number of organizations that make them use AI is very high. And so now if you're saying manual testing is impossible, like how do you test one, and how do you trust that testing then if it's out of your hands at that point?

I mean, trusting AI, it's been, that problem has been around for a long time, I think. Yeah, when you think about, other domains, like for example, my history of like, yeah, machine learning models taking automated decisions in credit risk, like there's a lot of money on the line. It needs to be like super high accuracy, right? Of course, people don't trust it from the start. You need to prove yourself and you need to make sure and validate that the models are actually performing up to par on what is required. And I think Currently, when we have had this kind of AI boom, like stuff has blown up, like very, very quickly. People are not maybe at the level where they feel they're comfortable with the performance and the structure around those processes yet, but that's something that you need to work on proving. So make sure that you collect, uh, evals and kind of make sure that you validate your, your AI pipelines and your AI agents and not just, uh,, your normal work.

So do you have any tips so that you can codify the knowledge and also do evals to make sure it's working?

Yeah. I mean, uh, when you have AI taking decisions and, and validating different things, um, there's typically a lot of data that goes into that model and there's typically some kind of decision or tool code or something that you have that, that you know, flags, say that you have like classification on your PRs, for example, or you have some kind of risk score or something like this, then the natural kind of thing to do then is to start looking at stuff that you have done historically and look at some of those things manually and make sure that you, or like, if you already have those manual processes, that's even better. Then you can just make sure that you record that data, like the input and the output, and then and then make sure that your models can do that better, or at least on par with the manual processes.

So I'm curious just to dive in now into the tech of like how to use AI with testing. Once again, I speak with a lot of people and there's usually like 3 ways test tools approach this. They either have like scripted automation or AI-assisted script writing, and then they have agentic testing. So I'm just curious to know, where do you fall, like, out of those three on the spectrum with your tooling?

We went full on agentic already from the start. That was kind of the original business idea from our end. It was a bit crazy in 2023, but it's definitely, definitely becoming more and more reasonable.

And also I think what makes you different is I saw something about a knowledge graph, that QA used some sort of behavioral knowledge graph. Of a product. I don't know if you can explain that and then maybe show that what that looks like.

Yeah, of course. So in order to understand what you're testing, first of all, I think it goes like a deployed application is so much more than what the code typically says. Like there are so many different things that can happen, so many things that should come together to form that application. If you as a manual tester are clicking around and validating that stuff is working, you typically build up your own memory. About how the application functions and how you should reason about different user journeys. What are the different kinds of risks that could potentially happen when you change a specific part? And in order for AI to be able to function in a similar manner, it needs the context. So without context, it's just like, everybody knows this now, right? Like depending on what you give ChatGPT, the better your answer is going to be. So it's, it's all about like understanding the application and the requirements and the changes over time. And the way for us to, to tackle that problem has been to build a knowledge graph. The knowledge graph is essentially then a representation about the, the deployed application, how, how the different user journeys kind of connect and how you get to different parts, also with screenshots and descriptions and things in there. And then also extracted facts and understanding about the domain and the application and how you do different things. You can also ingest documentation, for example, or API specifications and things like this that could be relevant for the agent to know about, which will also then attach to this other knowledge that we have in the knowledge graph.

Nice. So is that like a step you run before on your application? Is it exploratory while it's testing? It's figuring out itself and then writing like a .md file or some to let us know like how it learned or what it found out?

Yeah, so it's running continuously, but when we onboard a new customer, then we have this explorative phase first where we start trying to map out as much as possible and focus on some specific areas that are interesting to start with. But then we continuously scan and learn as you merge new PRs and as you run and kind of build out new tests, then it continuously updates. In order to stay updated. Otherwise it'll be old.

Yeah. So how's that validated? Does someone then look at what's written down and then say, yes, this is correct? Or do you just trust it right outta the box?

Uh, you, I, I, I do a lot of validation on our end to make sure that we deconstruct like reasonable knowledge graphs. It is quite, a knowledge graph is pretty dense and complicated to, to, to take in as a customer. So we don't actually show the entire graph to the customer, but we do show, a summary and the kind of description about what the agent knows about, like on a higher level. Uh, so it's easier for users to understand like, oh, this area is not even in the, in the context, for example. But, uh, but, uh, we are exploring how we could potentially show this as well, uh, in an informative way and not just being a fancy graph.

Nice.

Yeah, for sure.

So I'm just curious to see an example of an agent working or testing. Do you actually have anything that you can show maybe of what that looks like?

Yeah, of course. So this is Kiotec. We're kind of testing a demo application, which is a chess opening guide tool that I built out for my daughter. And doing this, obviously, if you just kind of one-shot it with Cloakode, you will get a lot of weird errors. So I thought it was an interesting case to show. First of all, like we have a few different kinds of tests. So we can kind of trigger one of these tests and see how it runs through. As we said, the agent will start by exploring the entire application and can help you through chat to kind of build out and work with these tests. But here you will see the agent as it clicks through. So it starts by kind of opening the browser and then it's clicking on the next button here and tries to validate that the functionality for this specific Italian game opening is correct. As the agent has kind of run through these different steps, it's first of all not relying on any kind of identifiers or anything to do this. It's reasoning at each step. So you would have a reasoning where it's actually looking at the results and thinking about what to do. And then, uh, um, when it has completed, it will run a more in-depth analysis about all the different things that are happening and can help you pick out issues that potentially could have happened. And in this case, you see that the kind of highlighting things here are a bit broken. So you would have the horses moving here, but it's actually highlighting the square next to it. And then you get these kind of assessment reports. So here, the test is failing because it has some kind of expected result that is given to the agent from our orchestration and Knowledge Graph knowledge. And then actual result here, after move blah, blah, blah, the origin highlight appeared kind of next to the vacated square. So it's finding that error. And it's doing this by looking at the screenshots and the recordings and the console logs and network calls and things that are happening in the application in order to validate that it actually functions as it should.

Did someone write this test or you're just completely just saying, look, my Agentic AI, just validate. Here's a chessboard, validate, and it creates its own and new, like, how does that all work?

No, it's, so you don't need to write any code or any specific kind of test. So we, you can kind of ask it to test something by just requesting it in a chat, or if you have English written up instructions for what you want to test, which you typically do when you're working with manual testing, then you can kind of ask the agent to work with you to create these test cases. And the test cases are defined on a higher level, so they're much more focused on things that should be possible to do in the platform. Preferably, I recommend creating things that doesn't necessarily change. So instead of relying on like clicking on this specific identifier, then this identifier, then wait 300 milliseconds, which tends to be very brittle. We're focusing much more on like, it should be possible to go through the Italian game opening, uh, um, uh, end to end.

So that's interesting because rather than verifying clicks, then you almost, you're validating intent, which we think would be better, right? For a tester.

Yes, exactly. Because that's what we really care about.

Right.

Like, uh, and it's also much more similar to what you do with manual testing. You don't necessarily care about these specific clicks on identifiers. You care much more about the actual user intent and experience as a user, uh, using the platform. So we can kind of, for example, we can, uh, uh, let me test the, for something new. Um, so then it starts by, uh, searching through the the knowledge graph and understanding that it has about the application so that you can actually test things and make sure that stuff exists and the user journeys and how they function. And then when it's done doing that, it will create a test and start running it through. It takes a little bit of time, but yeah.

I would think also with this knowledge graph, it probably could find things that you're, you normally may not be able to find. Maybe a tester doesn't know about a a feature or function that's being used in a workflow, this would be able to find all those permutations, I would guess then, of user journeys.

Yeah, exactly. So it can analyze and find different kinds of permutations in user journeys automatically. And you could also run it more in an exp— kind of as changes are arriving. So you can trigger it to, on a pull request, when you have a new change and adding a new feature, then you can kind of automatically explore and create tests for those things without having to, wait for somebody to code them as well.

Also, that would help, like, say I had a hotfix and someone touched a piece of code. I don't have to guess what test would cover that code to verify. I would know that there's a test that would test that PR that was checked in for the hotfix. Does that make sense? Yeah. So rather than run all the tests, you, it's all targeted.

Very nice. Yeah, exactly. I mean, it does have context about the user journeys in your application. So maybe you're changing some small specific feature, but that actually kind of affects other user journeys, then, then it will pick up and test out those things as well. But then you can run much more targeted things and then you can automatically write a review on the PR for it. So yeah, now it's kind of generating this thing. So it started out just analyzing and getting the information that it needs and then it started creating a test. Typically, maybe you run work in more than one test in order to actually make sure that it runs more things. Yeah. And we can run this. Oh, I think that opens it in the window. Okay, I'll just go in and say here. So now it starts and then it has Given the information that it has about the application and the knowledge that you have provided, it can then generate the goals and help you with these kind of expected results and things automatically. We also have the concept of steps, which is like a guiding, essentially like a guide to the agent, right? So, and with these steps, you can make sure that it takes a specific journey, for example, or that it does some things in a specific order. If you want to. The chat can help you, of course, work with those things, but so you can be very specific if you want to. Yeah, it clicks through and you have the same experience as before, clicking through this kind of chess opening things. I presume it will catch the same error since it's present in this opening as well.

So for some reason, testers always want to be able to see the code. Is that a mindset shift? Are you able to see the code with this? Does it even matter nowadays?

Well, there is definitely a mind shift in there. And first of all, there isn't any code, so there's not any code to look at. There is definitions and you can look at the definitions, but there is no, there is no kind of actual code. We don't write code underneath. The agent is actually analyzing and doing things as it moves through, which is also what makes it much less brittle. If you have hardcoded things on how it should actually behave exactly and click on the screen, then it will, you know, be more brittle when you change that. So there isn't any code to look at, per se. I think that is changing. When you talk with developers today as well, of course, there is different schools here, but the code becomes less and less important. I think the behavior and the intent and building the right thing is much more important. And also because I would say, argue that another bottleneck in the SDLC is definitely the review process now. We used to do code review for a lot of different reasons, not only to find errors and bugs and things, but also to do knowledge sharing and all this. Which I think is also more encoded in the agents now, like when you write different MD files and make sure that you update and kind of have a compounding project that actually updates over time as you're building out your features, then the AI agents will have the context. So first of all, like the normal, uh, uh, kind of knowledge transfer thing is, is a bit broken. And then also I think if you, if I get a PR for under 3,000 lines, like, do I actually look at all the different lines anymore? Like when we produce so many PRs, if you produce like 10, 15 PRs per day, who's going to look at all the code? Like it's just creating a new bottleneck. So I think we will have a shift here in making, moving out of actually reviewing each line of the code, but rather like making sure that we have processes in place to ensure that the code is good and safe and kind of lives up to standards so that it will not block us to develop other things in the future. And then also that the intent, you know, that the stuff that we have, uh, we want to add from a product perspective actually makes sense and looks good in production.

How does it determine what passed or failed? Is there like, you said just have like a verification point that shows up the past, like, like obviously it could change. So how does it know if it changes, like what, what, what is a pass and what's a fail?

So a pass and a fail is encoded in the expected result. So you do have an expected result here, like what it kind of validates that all these moves are functioning. As if your platform is changing in some way, say that I kind of change how this behaves compared to what we knew about the application when we created the test, then the Agent can help you analyze this and learn as you give it more feedback and similar to what you would do with a coding agent, you would kind of ask it to update its context about how we're developing things or which, how we're structuring our project and things like this. So you could just ask it to update it and given the new requirements that you have and we can automatically pull in that from Linear or Jira or whatever you're building with as well.

Another thing I think testers get hung up on is they don't like self-healing. They say it's covering up a real issue. And I'm like, I don't know if that really applies now, if this is self-healing or is it just adopting, almost adapting to the situation.

I think, I think it is important to be mindful and look at the results and like think about the structure and how you're testing and and what your kind of strategy is, because just generating a lot of tests, regardless if you're using agents or if you're just generating code, right? Like just generating a lot of stuff and you don't really know what it's doing and you don't really know if it's tested well in the end, like then you're, you're back in the kind of position of not really trusting it and you don't really get value from it, right? And then you have to do a lot of manual work on top anyway. So. I think you need to work together with your agents to make sure that they produce good stuff. It's the same when you're coding. I think you need to make sure that you structure your, your work in a way so that it's easy for the agents to, to perform the tasks that you want them to do. And you should talk with your agents to make sure that they are aligned with what your testing strategy is and that you're actually validating those things in a good way.

All right. So if someone sees this, they're like, well, my job's replaced. Like, is, is, is there still a tester in this type of workflow? Is it just an engineer that codes and tests? Or is it like where, you know, what's happening here with the tester role?

I think there is definitely roles for testers still. I think all of the engineering roles in, uh, are changing a little bit. And we also see this with some, some more kind of AI-pilled teams where you tend to be slightly smaller teams and. And share responsibilities and be more kind of broad in your capabilities. So the same with product and design and front-end, back-end, and like data engineers and like all this, like it gets to be a little bit more spread out. And I think that is actually quite good. And I think from the tester's perspective, it's the same, right? Like you, there's definitely need to make sure that stuff is landing in production, that it has high quality. And how you kind of structure that testing and how you make sure that you have an automated pipeline where all of these steps are kind of fulfilled. Everything, it could be anything from, you know, brand guidelines to accessibility and other things like it depends on very much on your product and your setup. But I think it's super important to have somebody think about how you actually validate that the stuff that you're building is good. Like that doesn't go away, I think.

So another thing a lot of people have been concerned with is, uh, token cost, uh, because it has a knowledge graph, is the token consumption less? Does it matter? How's it work with their product? Do they have like a set number of tokens or can they point to their own LLM models?

No, it is actually, uh, it's like you would think like, okay, we need a lot of context. It's a lot of context in the graph. It would be much more expensive. It's actually helping. Because we can be much more specific about the context that we give the agent. And also we can identify things that we have done before in a, in a quite sophisticated manner where we don't have to even do an LLM call, but just kind of repeat the same actions.

Another, maybe a resistance or pushback you might get, I would think is, someone can use like Playwright and Claude. Code and say, hey, test this and it could figure out the application as well. But what's the difference between a solution like this compared to like, like someone just hacking it with Playwright and an LLM?

I think, I think there's a lot of differences. First of all, I think you get much more nuanced feedback similar to what you would get with a normal tester if you're, if you're actually looking at the results and not just having code. Even if it's automated or not, like just clicking through things. I think the risk of doing it with Claude without being very specific and mindful about what you're doing is that you're just going to have a lot of code. Like it's going to be very much harder to actually have a clear understanding and intent with all the stuff that you have done. So we help you a bit with that. And I think this knowledge and understanding and reasoning about the changes in the platform as you're developing and as you're moving through with your changes is also going to be very valuable.

So, what you showed me in your point of view sounds like it's the way things are going to go in the future eventually. How do you get people to adopt this? Like QA tech, how do you have the pipeline ready to go and how do you incorporate it into an existing pipeline? Is it a lot of work? Like how does that happen?

It is a bit, it is a little bit of configuration, but it's not that hard. So it depends on your CI/CD and your different kind of things that you're set up. But typically what we, since we host all the infrastructure around like running the agents and everything like this, you don't need to think about like, you know, how it should scale and how it should run and things. So that makes it a lot easier. And then apart from that, we need to scan the application and start kind of building the knowledge and understanding. So typically what we say, the only thing that you need to kind of get started is like a URL and some authentication or kind of credentials. But then of course we need to scan and make sure that it actually works and set up the context. And that is an important part for anything that's using agents so that you make sure that you test the right things. Yeah, do the right things.

Nice, nice. All right, really quick, you did mention something about a PR. So how this normally would work then, someone would check in a PR developer, QA Tech would be kind of called and it would take action on that PR and then alert someone, hey, I took this action, here are the results, okay, not okay?

Yeah, exactly. So there is a GitHub integration that you can just install, which will then listen on webhooks from GitHub when a new PR is open. It needs to have some environment, like preview environment attached to the PR so that there is something to go into and click around in, right? So if that is in place, then our agents can kind of analyze the changes on the PR against the knowledge that we have about the application, go in and test that specific new feature, and then write the review findings that it's found. So maybe something behaves weird or it's unintuitive or, you know, whatever.

Nice. Another use case I think I saw was something about bug reproduction. So if someone complains in a chat, the agent can then take action on that? Sounds almost too good to be true. Is that right?

Definitely depends on the complexity of the bug, like how far in the backend it's happening. But if a user complains about a bug, it's usually user-facing, which means that if you have an environment where the agent can run, then you should be able to reproduce it. So if somebody kind of mentions like a problem, so say that you have a message in Slack from somebody internally, then you can mention our bot on the Slack thread, or if you have it somewhere else, copy paste it into the chat. And then given the context that we have about the error or what the user has experienced and the knowledge we have about the application, the agent is usually quite capable on kind of reproducing those things, but it could be a weird intermittent bug that only happens every thousandth time, then you need to run it a lot to actually find it and things. So it depends on the bug, definitely.

Awesome. Arvid, before we go, is there one piece of actionable advice you can give to someone to how to get started with AI and what's the best way to find or contact you?

Well, I think the best way is to get your hands dirty. Like don't be afraid, don't kind of get stuck in this, uh, oh, it doesn't work for this specific thing. I tried it once and then it wasn't good, but rather like think about what needs to be there in order for it to actually have the capability and the knowledge to actually, uh, do it well. Try to record a few different scenarios so that you can, you know, have something to test your prompts with and then, uh, and then move, move from there. I think there's a lot of things that you can do. With very little means today. So say that you have a contract between two services, for example, add an agent, like, uh, that actually makes sure that that is kind of valid each deploy, for example, that's totally doable. So we, we have a booking form on the, on the website. If you want to contact me directly, then I'm available on LinkedIn or X. So, um, just search for my name and you will find me. And there is not that many Von Neumann names in the world.

Awesome. So if someone's listening, they want to see the future, where the future of testing is going with AI, you definitely want to get your hands dirty by booking a demo using that link down below.