AI Codebase Discovery for Testers with Ben Fellows

Hey, what if understanding a code base was no longer a blocker for great testing? Most testers were trained to work around the code, like clicking through the UIs, guessing selectors and relying on outdated docs or developer explanations. But in this episode, Playwright expert Ben Fellows flips the model on its head using a real world example of how to use AI tools like Cursor. So testers can now explore the code base directly by asking questions, uncovering APIs, understanding data relationships, and spotting risk before a single test is written. This isn't about becoming a developer. It's about using AI to finally see how the system really works and using that insight to test smarter earlier and with more confidence. If you're joining a new team, inherited a legacy app, or struggling to understand what really changed and released, this episode is for you. You don't want to miss it. Check it out. Hey, before we get into it, quick heads up. If what Ben is sharing is really vibing with you, you want to miss it. Ben is actually joining us at Automation Guild 2026 for a 90 minute hands on workshop on getting started with Playwright using cursor and AI. This is in theory, he's going to walk you through practical step by step ways to use AI to understand your code base faster, generate Playwright tests and avoid the trial and error most teams get stuck in. If you want to stop guessing and start testing with real insight, register for Automation Guild now using that special link down below. Hope to see you there. Hey, Ben, welcome back to the Guild.

Thanks for having me back, Joe.

Yeah, we got a lot of awesome feedback on our last show that I'll have a link for down below. But it was your expertise of using Cursor with Playwright and since then we've been talking and. And we also talked in the green room and you were telling me how I think as testers with AI and cursor, we need to almost have a mind shift now. Where before it was almost like we had to use a product, almost like a third party product. It was almost like a black box. So we'd have to go to a browser, we'd have to look for elements and see which is the best locator. And then you did a mind shift on me. You said because we have access to Cursa now and all the code, we no longer need to do that. We have access directly to the source. So I thought this would be a great episode to maybe talk about what this mind shift is, where people could get benefit of this new technique and cursor and maybe I think you also have an awesome new tool that can actually unlock a lot of this as well. So I thought we'd just jump right into it. So first off, yeah, happy to. Yeah, Ben, so maybe a little bit about my summary. Is it a correct summary of what you've been seeing as well?

Yeah, 100%. So to recap, you and I had a conversation involving cursor, and really at that point was focusing on. Okay, so then this world of test automation, you hear all of these devs using augmented coding, and they're using it. But how does that help us as QA engineers? And I think we had a really great conversation around what are some of the techniques and introductions for cursor and how to use it from a QA engineer. I've. Since then. Because that was almost a month ago now or two months ago now, I, since then have pushed this thought process further to say, wait a second. Cursor's value is not just in writing code like an augmented code, but how do we as QA engineers unlock a lot of the hidden information that would make our life easier? And, yeah, excited to show you it. Like, you mentioned have been building some interesting things that honestly, I just want feedback on from folks. So, yeah, happy to dive into it.

Awesome. Yes. I thought maybe you'd share your screen, actually show us and tell us as we go along, because some of this is audio as well.

Yeah, 100%. So let me just turn this on really quick. Okay. So a lot of this, I think conversation is going to be prefaced on a mindset shift for a lot of QA engineers. And this is where I'd love to answer your questions as you go, but for me, as a QA engineer, the code base always scared me. It just was terrifying. Am I allowed to swear, Joe? I can never remember. Probably not. Sorry. Bad habit of mine.

Yeah, I'd rather not.

I'll hold off.

I'll hold off.

Okay. Sorry. Not my fault entirely. So to recap, for me, as a QA engineer, the code base was always, like, a really scary thing, and it was intimidating. It was just so much I didn't know where to start. And because of that, I never really tried to approach the idea of using it as a resource. Right. Like, all of this information that I'm going to show you and talk through, and ironically, it's been in code bases for the last 15 years. There's nothing new about this. What I think is the mindset shift for QA engineers, and I would argue product engineers. This other stuff is we now have this accessibility layer, which is cursor, copilot, these other tools to be able to interact with this code base in a way that's been previously, just never really something that we're able to do. And happy to dive into a demo, Happy to answer questions like how do you want to set this conversation up? And what can I do to help the listener understand what we're going to talk about a little bit better.

Absolutely. But I think a little more context. I think you've been doing tons of demos of real people and so this is from real world experience from those conversations as well. Right. Is what you see them struggle with.

So the most important takeaway that I've gotten from people is fascinating because it happens in the first five minutes of a lot of these hour long conversations, which is I start asking questions of the code base and it's like this light bulb goes off in their mind where they go, oh, wait a second, I've been viewing this tool as a tool to produce code for me and I actually view the tool as an information gathering tool just as much as a tool to produce code. So like, I'm asking questions three or four times about how does the code base work? What are some of these things? What are these things? And then I'm producing code. I want to address a really important thing though, early on in this conversation, which is a lot of people in the QA space are so used to thinking, wait a second, code bases have bugs in them. So if we're using AI to ask questions of the code base, won't it give us polluted answers back if there's a bug in it? And to me, the answer emphatically that I have is no, if you do the prompting correctly. If you just think about this as an agentic sort of magic solution where you suddenly go read the code base and produce everything I need to know as a QA engineer, then yes, you're probably going to introduce some things. If you engage with it in a thoughtful way and you say, hey, tell me about trials in the code base. Tell me about this stuff. And it gives you information back you as a, you need to think. And that's where I'm hoping what I show you today is the best of both worlds, where it's techniques that empower QA engineers with new information, but it doesn't try to replace them. I'm not trying to sell the magic solution where AI tries to do everything.

Gotcha, gotcha. And it's interesting because we've always, back in the day we'd have Eclipse and we'd have our tests within the code base, but we still almost treated it like a black box. What's the element locator or selector for this, all that information. And right now, because of AI, then you're saying because it's a cursor, you're able to prompt it directly and it can give you the information without having to go through a developer or begging a developer for help. It sounds like.

Yeah, exactly. So let's do a couple examples. So basically what you're looking at is a code base that has a front end, it has a backend, it has a schema, and it's actually the code base behind this tool that I want to show you here. But basically you have all your test runs, your dashboards, stuff like that. The beautiful thing about that is you think about some of these information gates that have always happened for QA engineers, the API endpoints, the swagger is never updated. Some of this other stuff. So I might say something like, find me the endpoint that is used for creating test runs and I'm going to ask Ursafra this piece of information. I never would have opened a code base to go find the file where the API endpoint was for test runs. It was just. I wouldn't even know where to start. But now as a QA engineer, I get to just ask and every time will find me the exact file, right? It's going to find me that test run crud happens in this file. It's even telling me where it's hooked up in the front end. And I might say, what are the foreign key relationships that need to be set up for this before I can hit it with a test? So really important question from a QA engineer. What data, what test data needs to be in place for this? And the beautiful thing about this is it's going to just give me the answer. And this goes back to that first moment where a lot of QA engineers fight me at first because they say, wait a second, aren't like, what if it gives you the wrong information? And this is where like you have to be able to think a little bit, adapt, and then compare what it's giving you to against your expectations, right? So what it's going to do is it's going to give us an answer, it's going to allow us to go look. So here we have an answer, right? Required data. So it's saying that this API endpoint needs organization. It needs test plan files as a foreign key relationship here. Test cases it has a foreign key relationship and it's going through and it's going to basically give us setup wording for testing. And the really fun thing is this is confusing. So I'm going to say that is too much info. Simplify it for me, right? Because I'm constantly just doing this. I'm not even writing a test. I'm just trying to think and understand my upcoming strategy, right? So now it's simplified. A foreign keychain, it needs organizations, test plan files, test cases, test steps, minimum setup, create an organization, create a test plan, so forth. Now here's the question. Am I somehow ceding my control as a QA engineer to AI in this moment? And I'm going to argue I'm not. I'm going to argue that because I am getting this information from the code base, I still should have a pretty good understanding of how the API in general works, how the user flow works. I have a pretty good understanding of what these requirements are. What I'm doing though is I'm getting this information so much faster. It's always that last 20%, Joe, that really slows you down, right? So the first 80% and then you're like, ah, what about this last? I can't remember this last 20%. What this is so good at is taking the first 80% that you have as a human and your knowledge of it and filling in that last 20% quicker as opposed to you going to slack, asking a dev, waiting for three hours to get a response back. Right? What are your thoughts so far looking.

At this complete shift left, right. And so I'm just thinking off the top of my head, this could be completely wrong. Can you query and say, hey, give me the latest code that was checked in the last week and then you can planning how you can look at it.

You can look at commit histories, you can have it. So one of the things that I'm doing all the time is someone makes a commitment and they're like, hey, these test cases are that need to be updated. I'm not taking their word for it, I'm just pulling their commit in. And I'm saying using AI to say, hey, based on this commit, look at my automation suite and tell me what automation tests need to be refactored. And I know before even running the automation, what test will fail because of that commit in the sense of not bugs, right? Like, but what test would fail because it needed to be refactored? And once again, the point of this is that it's not, it's not this agentic sort of world that there's so many companies trying to do. Instead, it's just taking the tool that devs are already using and instead saying, how do we use it for qa? And I don't want to get too far ahead of it, but I will say, if you're not a QA person listening to this, rather than just turning this off and thinking, okay, this only applies to qa, think about the application from like Tickets, for example. So if I'm adding a new feature and I'm saying I am adding a new feature related to adding tags to test cases, look at the current implementation and give me some ideas for the scope of it. Think about a world in which we don't have bad dev tickets anymore because a product engineer or project engineer is thinking, okay, I'm going to talk to the AI about my feature. I'm going to engage with it. It's going to look. AI is looking. Are there any tags in here? Not tags in here, so forth. And then it's going to be able to produce a test case or project ticket that's exponentially more useful for devs and stuff like that. It'll have language that actually matches the code base as opposed to what do they mean by this? And I don't want to get too sidetracked, but I just want to keep the audience a little bit. Like, this goes well beyond just qa. And I would advocate that this will continue to be useful if you listen to it. Where do you want to go next? Sorry, because I've took us off on.

A little bit of a tangent there. What would stop anyone from not doing this if they're using Cursor? This would be a no, no brainer. What is it like? It's a total mindset shift, right? Because it seems obvious, but it's not obvious. I don't know if that makes sense.

No, it's what you just said. There was a LinkedIn post. Actually, that was what I was like. I saw a LinkedIn post from someone in my workshop saying the most obvious thing, the best takeaway from one of my workshops was the most obvious thing ever, which was just this one thing. And I was right. I should reach out to Joe about this because I think other people need to hear this. Once you get it, there's no reason. You're sitting there just being like, what? Wait a second, why am I not doing this? Because it goes back to. It is so far, right? So I write pom files, I write page object files. I don't open the browser anymore. I just say, go find me the front end files for the test run pages. It'll find the files. I'll say, write me my page objective class. Is it perfect? Absolutely. Not in the sense of I end up tweaking locators and stuff like that. But the best use case for Cursor that you can think of is to summarize this entire conversation into two core use cases. You should know more about your product and the implementation in the code base than you have ever known before simply because you can query it. And then two is it is a keyboard that can type exponentially faster than you. That is the core use case. You are smarter as a QA engineer about your product and about the implementation and you have a keyboard that can just implement faster than you could possibly type. And I'll give you one more example which is we have a client that is a large legacy app and they have a bunch of different clients and they over the years have done a lot of custom tenant work for all these different clients. So my QA engineer was coming to me and saying, I have no real visibility into where all this custom logic is. So I think what we just need to do is write a lot of end to end tests for all all these different flows. And I was like, wait a second, did you ask Cursor where the customization was? He pulls up the code base, he says, where is the customization layer for each tenant? Cursor answers in 30 seconds. All the customization happens within the components. The components, the front end components expect a 10 ton tenant ID and then they behave differently depending on the tenant ID. Completely changed our test strategy. We then focused on components and saying, okay, let's just mount the different components, let's give it the different tenant IDs. And now all of our individual tenant testing is isolated. And those smoke and sanity tests are just about the all applicable things. In just that one moment of did you go ask? No. Completely changed his trajectory of what the next few months would have looked like in terms of test strategy for him.

Wow. I used to work for Healthcare System. They had legacy Systems of code 30, 30 years old. Seriously. So this could help as well where you could say, all right, you added new function features. A lot of people were afraid to touch the code because they didn't understand it. To help them not only understand, but you also can understand this new code you added is going to affect these other codes. So let's write some tests around that as well that you may not have thought of.

It goes Back to it is the worst case of AI right now is the sort of set it and forget it and hope it ends up the best use case. And everyone's looking for what's real. What's real. What is absolutely real right now is you have a tool that can just answer questions for you. It could research, it can grab. And so to your point, like, I am considering a new feature where I can save. Save. How do I want to say this test runs two folders? I completely am making this up. Look at the current implementation and tell me what you think likely might be needing to be touched. The point here is that it's not going to give you perfect information. The point is that you should not treat this as like, absolutely. But what it is going to give you in the course of four or five prompts is a much better idea and plan. And so it goes back to. It's going to say files that need to be changed. We have a database schema. We probably need to add a folder, right? And we can take any of these answers and we can dig. And maybe it's just because I have ADD or something, but I love to dig. So I read this answer and I'm like, oh, interesting. It's going to touch a lot of things. And one of these things stands out to me of, oh, I'm interested. So I'm going to dig and I'm going to dig and I'm going to dig. So then I'm going to maybe say, okay, I don't really know the schema. So let's just say give me more detail about the schema. I don't worry about spelling errors anymore because it can fill it out. So we said, give me more details. And what we're doing is we're just digging and we're digging and digging until we feel like we've wrapped our mind, run it. And then what we're going to do is we're going to shift to thinking about testing and stuff like that. So it's going to give us option A, which is a simple string field, right? And so it's saying, here's the test run and here's our new folder. None of this has changed anything. We're just digging, right? Option B, and now we can look through it, right? And it's giving us really good analysis and then it's even giving us some migration considerations, right? And all we're trying to do is we're just trying to think and understand. This absolutely applies to qa, right? If you take this from a New feature conversation to. Hey, a dev just did this. How should I think about testing it? How is this going to impact my test automation? How could I think about refactoring and yeah, sorry, I get a little worked up. I'll take a breath. Where's your head at?

My head at is a lot of people think AI is going to replace it, but will you just show that it almost unlocks your inner testing geek? Because you could just really, like you said, go down rabbit holes, prompting it, asking the right questions, judging the value that it's giving it. Giving you and like just interacting with it as a tester, using your mind. Yes, more. And rather than getting bogged down, I have to create a palm file. Things are not really. You're not really testing at that point, are you? So this way you are.

So I think on the other hand, the danger, right, the danger, and this is just being completely honest about capitalism, is it makes people explosively more effective, which does mean that there is the world that like, it's not going to replace QA engineers, but if one QA engineer using these techniques can do the work of three, there is some level of that now I have all. We could have a beer and have a conversation around. I think there's going to be an explosion in entrepreneurship as people realize you can do stuff without the need for VC capital and stuff like that. So I'm hoping that we're going to see a dip where there's going to be a lot more of this pain and suffering that we're going to go through and then we're going to see a swing back as a lot more startups are coming and they need these really great QA engineers in the most cynical possible way you could do it is the person doing this is going to end up being the person who survives. So that's gonna. That is the most cynical way of thinking about, yes, there are tremendous. I get excited. We can spend the rest of the time being excited about it. And I hope people are excited about it. But the problem for people who are not excited about this is these techniques just make people that much better and that will impact some of that.

I think maybe this is a weird way of looking at it because we're senior and we know how to test like this is I don't think this is gonna replace senior people. I think it's going to replace people that are new to testing, people that don't know how to test because can I take care of. You need to know how to take your thoughts.

Let's ask slightly different direction. I think it's going to replace not curious people. I didn't know code four years ago. Didn't know a little bit plugging in my phone. I was in politics 15 years ago. I worked on Capitol Hill. I joined a startup knowing nothing about tech. I did operations. I started a QA company knowing nothing about test automation. Did not know code. Maybe five years ago. I can build full stack apps now like it's with no problem. I can write front end now. Do I write all of the code? Do I do all the squiggly lines? Do I do all that? No. But as an information system, there's all these people saying it's going to replace junior engineers. What it's going to do is it's going to shorten the life cycle that someone is junior. I think the right person, a curious mind is only going to be junior not for 10 years, but for two years because they have a thousand dollars an hour tutor that's sitting over their shoulder that they can ask why to over and over again. I've learned more in the last four or five years than I've learned in the last 20 years. Because you're not going to stack, overflow and try and find something that kind of does it. It's what I call static learning versus dynamic learning. You are no longer basically relying on static information. So that was so pompous and high horsey. But I've been thinking a lot about this, which is basically I don't think, I don't think it replaces junior engineers. I think what it does is it should shorten the life cycle that someone is considered junior if they have a curious mind. I'm curious if you want to push back on that. You think it's wrong?

No. Now that you say it's true. But I think that's definitely a valid point you made. As long as you're curious, regardless of your level, it's going to help for sure. Not replacement.

Now, whether or not CFOs and job titles and all the other things survive, like that's a different topic.

Totally, totally. All right, what else can this do? Like I said, can it help with performance issues before they become performance issues? This code was just checked in.

Yeah. Security, performance and everything. It does so much. If it's all right with you though, I'd love to show you one or two examples of just like implementation level things of like examples of how it can help. Does that sound reasonable?

Yeah, absolutely.

Okay, so this is where I just want to bring in A tool that, that we've produced that I, I think just.

What's this called? This tool?

This is called Conti Test. I'm not a. I'm not a marketer, so don't get too upset with me, please, about it. But four tools called the Continuous Suite, all based around this idea of how can Cursor revolutionize things that were previously disconnected from the code base. So the four tools are test planning, help websites, documentation, and project management. Kind of like a GR ripoff.

Okay. Yeah, yeah.

And all of these concepts basically apply to those four tools. And if you think about all four of those things, what you think about is manual work. Right. Think about how much time and energy goes into manual test planning of, write out the regression, keep it up to date, trying to figure out this stuff. Right. What's really cool about Cursor is you can actually just layer things on top of Cursor. And I'm sharing these tools with you, not to try to sell them. I'm hoping to inspire people to rethink how you can engage with some of this stuff. So one of the really cool things about Cursor is that you basically have something that has existed in VS code forever, which is add folder to workspace. So we're going to add a folder to the workspace and what we're going to do is we're going to go into our test plans and we're going to add our test plan folder to our workspace. Now, here's what's really powerful about that. I have completely separate repos that AI can interact with across both repos. So if you think about this, I now have, and we're doing this all of the time with our Test Automation repos, where you have a test automation repo in one repo and you have your code bases in the other repo. I can ask questions of my code base and then put that information in a separate folder structure. Right. So you don't have to be able to commit to the actual core repo and you can still use it as a resource. And so in this example, what we're doing is we're trying to say, what if you can do all your test planning in a folder structure that Cursor can interact with? So all these are our test cases for the app. And then all we're doing is having Cursor read the app and write them. And so, for example, we could say, tell me about the test coverage related to Stripe. We're going to pull in our Stripe test folder. And what this is going to do is it's going to read all of our test stripe coverage. It's then going to go read the Stripe feature and it's going to give us analysis of what is our manual test coverage against Stripe. And so if you think about those regression sweeps, like how long was that a disconnected process, right? Like the regression was over here, it was constantly out of date. And there's these layers of communication and in one little sort of mechanic, which is push the information into cursor, my belief is that you completely get rid of that friction and the time and energy it goes into it. And so you can see it's reading all of our tests related to stripe right now. It's then going to go read Stripe and it's going to give us basically really good analysis of the test coverage. It also sometimes does some very weird things. So I don't know necessarily what it's doing right now. And it's, that's where it's, it's sometimes hard to look through it. Okay, great, Stripe test coverage. You have seven test plans, stuff like that. But you might say to me, okay, wait a second, I don't want to have to look through these massive docs in cursor. Like that feels like that's a bad user experience. And so that's where our tool is basically focused on bridging the gaps. You have this folder and you have a web app. And this web app just displays all the test cases, right? So those same Stripe test cases that are in this, those are all here you have your test case and this is much more readable. You have filters, you have reports, stuff like that. And how you connect the two is actually a fairly simple thing. You basically have a small NPM package that you basically just install in the test plan folder. So then none of this touches the code base. There's no security issue at all with the code base. You simply put a little NPM package into the test plan folder, you authenticate with a login token, and then you simply scan, push and through the cli, you've basically pushed all of this into your web app. This mechanic is challenging, I think, for QA engineers to think about because they're still scared of code bases, scared of git, stuff like that. But what I'm finding is that the engineers that are using this or the project managers that are using this, it is a time savings, nothing else. It goes from a four or five hour long task to a one hour long task and then you still get the benefit of the web app and Pushing it up into the web app and you can still go and look or the manager can go see what test plans do we have. You have all your search and stuff like that. But the time and energy it takes to maintain things, regardless of whether you use this tool or any tool, I would push towards try to think about what other things help docs. Other things could just be their own folder and cursor and you use the code base to help maintain it. I feel like this entire conversation I've been so high horsey, but I get very excited about this and I'm just like, if there's one thing I want to like preach to the world, it is trying to unlock because I think we're scratching the surface.

Absolutely. So when you made that claim, four hours to one hours, you've worked with a lot of people, so is that something you would actually see in the wild as realistic?

So we did a couple just fun test examples. So we took a couple examples where basically, and I'm gonna try to remember the specifics, I made a LinkedIn post about this, but we did a couple examples where we had a regression suite. And that regression suite was being managed. It was being managed in a private tool. And I don't wanna say the name, but you could probably guess one of the private tools where this regression suite was being managed. Right? Like one of the big manual private tools tools. And so we, we made a fake feature update. We told two QA engineers, hey, we've added this feature, so you need to update the test cases. The first person using the traditional model had to sit down. They opened up the third party test system. They looked through the existing test cases. They tried to decide which ones needed to be updated and refactored, which new ones needed to be added. It took about 2 hours, 30 total minutes. And they were heads down working. Maybe it was like 2 hours, 20 minutes, but it was that range. The second person took that same thing with access to the code base and access to their test case. They actually just copy and pasted the ticket in. Said what test cases need to be updated? Within 30 seconds they had a list of the test cases that needed to be updated. They then updated those test cases accordingly in here. And then basically they used the CLI to push it and they were done in about 15 minutes because they spent the next 12 minutes like reviewing it, making sure there was nothing that was missed and stuff like that. So what you're talking about is a difference in 2 hours and 30 minutes to 15 minutes. That's a pretty Substantial difference. And you could think about that in so many other ways that have nothing to do with me. Right? It's okay. The time and energy it takes to write a good dev ticket versus a bad dev ticket. Imagine a good dev ticket and it's not only is it faster, but it's a much better ticket. Like how many dev tickets are just the title saying add, yes, add a submit button. What does that mean? What are the criteria? Stuff like that. Right. So I guess that's the most explicit way I could, I could add it and just to give one more kind of really good explicit. So, like what we can do. So for example, let's just, let's go to our test cases here and let's just say we are missing test cases related to settings. Evaluate the settings feature and get back to me. No coding. So what we're going to do is we're going to tell it, hey, I'm going to write test cases related to Settings. So I want you to do it. In the meantime, we're going to basically go ahead and we'll add a new folder in our test structure called Settings. It's going and it's reading through our app right now about like how settings work, stuff like that. And then we will say add test cases for the settings. Now you can see that I'm not giving it much context. I tend to trust AI pretty good as a first draft tool at this point. A lot of people don't trust it probably to the level that I trust it. But like, my experience with it is it's quite a good first draft tool. So I'm not going to spend a ton of time and energy doing this. What this is going to do is it's probably going to produce, I'm going to guess something in the range of 30 to 50 test cases around settings in this app. So if we go and we look at the settings, you probably have stuff like Manage Account, you have your billing, your team management, some of these other things. My guess is what it's doing is it's going to see all that and it's continuing to read through them and coming with it. Side note, I still don't know how to fill time when AI is doing stuff. So if you have any secret to that.

So, Ben, while this is working, it's back. All right?

Oh, yeah. All right. So we're just gonna give the next prompt. So what it's doing is it's gone and it's basically come up with all of our settings. So what we're doing is saying, hey, go and add test cases. And it, by the way, it's gonna. It's guessing. That's what it wants us to do. So it's already come up with what test cases it thinks it wants to add. So what we're now having it do is writing test cases. And what we've done is we've used cursor rules to give it a specific format. Now it's producing test cases. Right. And once again, I don't want to claim these are perfect. I don't want to claim that you should just blindly trust these. You should not blindly trust these. What I want to claim is that this is just an exponentially faster keyboard than you. And it goes back to like, our job is going from writing things to reviewing things. Our job is going from spending four hours of writing something to one hour of reviewing it. That's the working thing, I would say, all right, this is going to generate for probably 30, 45 more seconds. So what was that question you had?

The question was you said about something about context. So do you see, if you make assumptions and you make the context too much, like you said, you don't give it a lot of context, that it can actually sway the AI of what it's going to give you. Okay. If you give it an image of something, you say, this is a wire for Internet, and it's not a wire for Internet. It goes, oh, yeah, that's definitely a wire for Internet. If you didn't give it the context, just said, what's this wire? Are you bound to get better results?

It's gotten better at not just agreeing with you. There was a long time when I would just agree with you. Right. It's gotten much better at that. I tend to limit the context to just verbally vomiting what I want. A lot of people spend a bunch of time and energy on. There's two aspects. There's your prompt and there's your context. My context tends to be files and folders and stuff. I want to limit where it's looking. So my context is the files and folders, where it's looking, and then my prompting. I don't write beautiful prompts. I tend to just verbally say what I want it to do. Most importantly, though, and I think this is what most people get wrong, I watch what it's producing, and I'm actively critically evaluating why is it doing that and how is it doing that and deciding if it's good. And I'm not willing to. I'm going to go back probably every Five or six times it does something I don't want to do. I go back and I catch it, I fix it, I go in a different direction. And I think this is where maybe there is something that goes wrong with a lot of people, is they're not doing that critical evaluation and then resetting their prompt and so forth. Okay, not to jump too much. We've now produced these test cases, right? And it's continuing to check its work. So once again, you might say, okay, cool, let's produce test cases. How is that valuable to us? And this is where it just gets really fun. So now all we have to do is we have these test cases in our folder. You can see our settings folder. We're going to do scan, push. And now in our web app, in a matter of five minutes, we can search settings. We have test cases, right? And we have all of our new test cases. COI token generation. And now I'm going to spend the next 10 minutes going through it and saying, okay, successful token generation, navigate to settings, verify workspace name, verify generated, click button. And these are all spot on. A lot of people think, oh, it's not good enough. And this is a scary thing for QA engineers is it's very good at test case writing at this point. Like, it's very good with the proper engineer of, like, test case two, copy the command function, generate a cli, click copy, verify that clipboard works, Wait two seconds, paste it, perfect test case. If I don't like, go ahead.

Do you have someone on the back end using cursor? And then you have all your Sprint teams using this dashboard.

So how it works right now is from a tool perspective is basically, we are treating test plans as essentially almost dev projects. So basically, like the test plan itself has its own git repo in GitHub and the team, all of the QA engineers have git branches. And this goes back to something that it's hard to train people on sort of new things. We're trying to get QA engineers to think about a test plan repository almost as a dev project, where it's okay, so you're responsible for adding these test cases. You're going to make a new branch, you're going to add the test cases, you're going to merge branch into main, right? And what they're doing is they're able to basically have that. And then from main, they're just merging up into the web app. The web app's value primarily is just for doing manual test runs. Right now I have a Million different ideas of how you could use this connection of the mechanism of cursor through a CLI to the web app. In the web app, right? Say they say, okay, guys, time to do a test run. Or, hey, let's verify a specific feature. What you could do is you could go in here and you could say, cool, I want to run the test. And then it'll say, yep, I want to run this test. And you can create the test run. This is all manual right now. The next stage, maybe in the next month or two would be like, could you somehow map all your automation tests along with all your manual tests and try to use cursor to better do it? And I think there's a lot of possibilities in terms of like, traceability, code coverage, all these other things to really bring these independent, disparate sets of information into more of a unified tool. Now that we all can use the code base as our tool. But that's the short answer is basically, you have different QA engineers, each with their own branch of your test. They're all in their own branches adding it, they're merging it into main. And then from main, someone's just using the CLI to push it up into the web app. And then the manager, anyone else, or anyone else who wants to look at it, can just see it in the web app.

Is there any pushback from companies that we don't want our QA in our code base, or is that not.

Yes, and it's the thing that I most fundamentally do not understand. It is shockingly high. So if you post about this, you'll have 50% of the devs say, that's the dumbest thing ever. And then you'll have 50% of the devs be like, I post about this and there are devs that think that I am, like, polluting the world because this exists. I was like, no, this is literally 50% of the companies I work with. Devs are, for whatever reason, just not allowed even view only access to their code base. Everything I'm showing you right now hinges on access to the code base. If your QA engineers do not have access to the code base, none of this is useful.

But another random idea is we're normally notoriously bad at estimates. We have a hotfix, code checks in. We don't know what test to run, if it's in. Can we use something like this to help us estimate, oh, here's the new code that was checked in. Here's the test run, here's how long it should take you is that an.

Option it goes back to? I think it can do so many things. I use it for evaluation, stuff like that. Where I want to be careful is saying it's like the magic bullet to do everything. What I think it can do very well is it can take an engineer who has already given a task, who knows what that task should be able to do and it should be able to do it faster and things. So like when it comes to something like that. So let's say a job of an engineer is to evaluate a code, check in or do a pr. On one hand there is the risk that they will ask AI PR this, check it. The AI says it's good, they're never going to lay eyes on the code and it's going to ship one hand. I can't fight human laziness. That will always exist. To a certain extent it has always existed that devs say they PR things and I don't know how many devs are actually looking through like the PR before they approve it. Right. Like they're just like oh it looks good, ship it right. On the other hand, the dev that cares, the dev that's curious, they can get a lot more information about that PR much faster. What does it touch? How's the scope of it? They can engage with that PR in a way that's much more interactive and quick. And so I would absolutely agree with the point that you made. I think this with the caveat of with great power comes great responsibility.

But so what you should is 80% of it can be done just through cursor. Right now the benefit of the dashboard is huge as well. If someone want to try the dashboard that you create here or any of the other three tools you mentioned, how do they get their hands on it?

Yeah, so what I want to quickly show, assuming I can type is I'll just pull up the site. So basically the tool is called Conti Test. Never write a manual test plan again. So you can go here, from here you can do a get started. It'll take you to do a trial. We have one other tool that we've released right now which is Conti Help. The same exact concept applies to help docs. So you know how we've been writing all these test plans. You can do the same thing with help docs where actually this help center that I have is all written with AI and the help docs are great and you basically have all your stuff. Are they perfect? No, but are they help docs that I would ever otherwise have written by myself in 20 minutes. Absolutely not. So that startup that doesn't have the help docs because everyone's underwater. Same thing applies to that. So Conti Test, Conti Help, and then we're hoping to release Conti docs and Conti Plan in the future here. But this is what. What I'd have people check out.

All right, hold on. So those Conti docs, it's almost like a wiki Wikipedia page for the team, but it's updated automatically so you don't have to go.

Just 100% think about confluence and how much people hate updating Confluence. Now imagine Confluence. And this goes back to my. My idea of the continuous suite. One, if Confluence was continuously updated, what if your help website docs were continuously updated? What if your test plan was continuously updated? Right. That's what I want people to be able to think about is like, it's. It's trying to. It's like my, my. And I'm horrible at marketing, but Zapier made a company by trying to say we're connecting things, we're doing Zapier. Right. Zapier is this connection. Cursor can be our connection. Cursor can be the thing that connects all of these different disparate pieces and drives a much more effective output. Absolutely.

All right, Ben, I think we should wrap things up. Before we go though. One piece of actual advice you can give to someone to help them with their AI automation testing efforts. And once again, the best way to find or contact you.

Yeah, best way to contact me or find me is on LinkedIn. It's Ben Fellows on LinkedIn. Basically you can just find me right here. I never could remember my URL in terms of actionable. I think that the biggest thing for me is I think ask why. I think that's the biggest thing for me is too many people, they open up AI and they start doing. Don't open up cursor and start doing. Open up cursor and ask why. And just learn about the code base, about the project you're on about, anything. And then start doing. That's my one actionable piece.

Awesome. You can find links to all this awesomeness down below. Thanks again for your automation awesomeness. The links to everything of value we covered in this episode. Head over to test guild.com forward/A 5, 7, 1. And if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show. And I read each and every one of them. So that's it. For this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation testing and DevOps. If you haven't yet, join our vibrant community@testguild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit testguild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve. With Hoots and Liars. The bottom began the song A tune of knowledge, a melody of code through the air it spread like wildfire through the land Guiding tester showing the secrets to behold.