AI QA Agents Explained: How Amikoo Helps Testers

Hey, before I bring in today's guest, I wanna ask you something. How many pull requests did your team ship last week? Now double it. Now multiply that by 10. That is the reality a lot of engineering teams are telling me they're waking up to right now, and QA is being asked to keep pace with the code velocity that was unimaginable even 2 years ago. Today's guest has been working on this exact problem since before it was a problem. He spent over 6 years running thousands of testing teams through a no-code AI automation platform, and he saw the cracks forming before most of us did. So instead of patching the old tool, he went back to zero and built something new from first principles, specifically for the world we're testing in right now. His name is Ivan, and he is the co-founder of Amico, a QA agent designed to help testers, SDETs, and dev teams move faster without letting quality slip. And I think you're gonna walk away from this one having a really clear picture of where the tester role is actually heading and what you should be doing right now to stay ahead. You don't wanna miss it. Check it out. Hey, hey, hey, before we get into it, this episode is brought to you by the awesome folks at Amico.ai. Amico is a QA agent built from the ground up for modern software teams, not an AI bolted onto an old test framework. Amico was designed with 12 specialized agents and 43 purpose-built tools, and it writes test and playwright with proper page object model structure right outta the box. And the idea is simple. Your dev team is shipping more code than ever. Amico helps your QA team keep up without burning out or falling behind on coverage. There's even a freemium account so you could try it for yourself with no commitment. And here's the deal: sign up and mention you heard about it on the TestGuild, and they'll double your free usage. More tests, more runaway, no credit card drama at all. So go to amico.io and tell them Joe sent you, or use that link down below. Hey Ivan, welcome to The Guild. Thank you, Joe. Great to have you. Really excited to have you on the show today, 'cause I think you're really starting to create something really cool with Amico. Before we get into Amico though, just curious to know why Amico? I assume you talked to a lot of different people. What's changed that you think maybe QA needs this solution?

Yeah, no, first, thank you for the invite. I really, really appreciate it. I think it's pretty fun. I have been watching many of the episodes. So it's great to be here today. And yeah, regarding Amico, we launched MOOC Labs, MOOC Test was a free product 6 years ago, early 2020, before COVID And it was a no-code test automation platform with AI. From the beginning, we were using expert systems and symbolic reasoning to generate tests. But I mean, as you have seen and everyone has seen, like, The last couple of years and mainly the last probably 9 months have been crazy with the advance of cloud code and CodeX. So we saw the market moving and shifting and the needs also changing. So what we did is like we decided to launch Amico, which by the way means friend in Mayan. So we decided to launch it like a QA agent, specifically trained as a tester for testing, to help testers, to help a developer to test, to help a tester to test faster. So practically it's like their counterpart. We saw the progress of AI and how companies were generating 10x more code with Claude Code and Codex. We saw there was a need for faster testing and we saw how the technology now enabled that. So what we did is we put together all the learnings we had with our first product, Muktesh, and we put our heads together and say, okay, now that the world is changing, the user experience expectations are changing, what good and what would we create now in this new world and reusing learnings and things that were, that are still working in our first product. So practically we started with a beginner's mind and with all the learning, we said, okay, what a new QA helper powered by AI would need to do in this year, 2026. Actually, we started 2025, December 1st, the talk process, and went ahead and building it. So practically, the quick market shifts and the quick acceleration of technology pushed us to think what else was possible and what is helpful in this new world.

Well, so you did mention symbolic logic. I've been reading that symbolic logic for non-deterministic systems is better the generative AI for certain testing tasks. So with the Miko, like, is it, do you still use symbolic logic at all? Symbolic AI, I mean, versus generative AI. Is there a difference when you use one or the other?

Yeah, we use it in a sense. So what we did is we created our own agents, trained, and you could say a structure based on what we learned from MuukTest, which was using symbolic reasoning, a lot of symbolic reasoning and rules. So we reuse what was pretty valuable in MuukTest and the agents now, they decide when they decide what to do with all that information. So yeah, they use it in some sense when it makes sense. Yeah.

Totally cool. So I know a lot of people think Well, at least not think of— maybe people that don't know about testing say that AI is going to shrink the need for testing, but it seems like based on your solution, you're betting on the opposite. What do you think? Does this—

Oh, yeah.

More AI in testing or less? What's the deal?

Oh, super. More testing. More testing.

Okay.

Testing becomes more critical, like more critical. Actually, right now, I mean, Amico, we launched Amico 2 months ago, 7 weeks ago, 8 weeks ago. The main conversations we are having with potential customers are customers that are deploying 10x more code, so they are desperate to test faster. So, yeah, definitely more testing because companies are deploying 10x more lines of code, which needs more testing, which introduces more regressions, which introduces more complexity. And yeah, I see the level of the tester elevated in this AI world. Because also tools like Amico, like, they are very powerful, but you still need someone smart polishing, verifying, making sure that the right things are getting done, the master of the puppets, the master of the AI agents. You still need someone. So the role of the tester is augmented and more needed these days.

Nice. So, so what I love about your background with MoocTest as well is you've probably spoken with hundreds, maybe thousands, I don't know, of testing teams. So, you've seen a bunch of different testing problems. So, is there certain patterns you saw that you're like, okay, we need to move to create something like Amico to solve this or have these features?

Yeah, I think the main purpose of testing inside of QA has not changed, but it has been accelerated. Kind of, we started We started the company Mooklabs in 2019 with the vision of providing instantaneous, trustable feedback on the quality of the software as soon as possible. So what changed is now you need more feedback because there is more code and faster because the agents are growing faster. So the main mission of testing remains the same. But the expectations and the possibilities are larger and faster. So that's why we needed to create a new product designed for the new world of software development. Yeah.

So in this new world, who does the testing? Who do you see Amico, who did you build it for? Are there still SDETs or QA engineers, or do you see more like a morphing of Developers doing everything or like, where's the role and where does Amico fit in?

Yeah, we built it and it adjusts for QAs and SDTs, depending QAs, if they are non-technical, we obviously see it as non-technical users and technical users. We designed it for them, but these first weeks in the market, we are seeing a big traction for also development teams using it, but more like a collaboration, like, okay, developers have a PR. Now Amico takes that information and with all the context decides, suggests what should be tested in that PR. And then those tests go to the regression bucket that now QAs or SDEs, depending who is managing the, the whole set, they make decisions about it and they add it. So we designed it for non-technical QAs and technical QAs or SDEs. But we are seeing collaboration happening. So now developers and even sometimes product managers are participating in the flow. That's something pretty interesting that we, we have learned in the first weeks.

Oh, cool. So I guess another thing is the market. I talk to a lot of different vendors, a lot of different people. They all coming out with AI testing tools. So how, how does Amico use AI differently? I know you've done, like I said, like you mentioned, used AI before it was cool, before generative AI. And so I think you probably have a better footing on what it could do, but how is this different than maybe some other tools you've been seeing?

So we didn't try to add AI to something existing. Like our first product was AI, the symbolic reasoning expert system. We designed it from first principles and new learnings again. So something that really is, it's designed from zero. And what we did, because we already had years of experience and knowledge and data on how a test automation product is used, we use those principles to design it. So at the end, we design a harness of agents and tools. We have 12 agents and 43 tools, specific tools. That they manage the agents and the LLMs as needed. So in the backend, I mean, we use different LLMs, just giving you some of the technical context, but we build a set of 12 agents and 43 tools right now, and we keep adding to make the purpose of the agent more precise. By that I mean, we have one agent that just specializes on test design, like understanding the context, the criticality to suggest test cases. One agent that is an expert test automation engineer, an expert SDT to help to design test cases in a scalable way with the right, we write them in Playwright, but with Page Object Model, or you can select other types of frameworks to, or models to set it up. So. At the end, what we did is we are leveraging the power of LLMs, but we added a harness of experience and insights to polish what can be done and make it more efficient, which also is very important. Like right now with tokens getting crazy, like we are seeing 10x more efficiency versus Claude using the opus, for example. Yeah.

So you've actually seen that. You've seen companies that are using this saying, we've actually achieved x amount of efficiency over Claude or something, or using that directly?

Yes, yes, it's about 8 to 10x in the test that we have done, more efficient in tokens. And that's achieved because we created a special purpose agents that manage Claude, Gemini, OpenAI, depending on the task, but they are special purpose agents that are very specific. So that makes the token utilization super efficient.

So that's a big deal because I've been seeing in the news lately that companies have blown through their token usage. It's like a lot, lot, like a lot more than they thought. And this sounds like this could help solve that because like you said, you have an agent on the context that's specific to that particular task, so therefore the token usage would be less?

Correct, correct. And I mean, yeah, the token problem is huge. Like, I mean, It seems like the only viable solution that many are proposing, SpaceX wants to put the data centers in the space, right? So that's kind of, that will take a while. So yeah, the token problem is going to be, it's becoming larger and larger. And yeah, how we solve it is we decided to build special purpose agents and special purpose tools. For instance, like we built a lot of our own MCP servers that integrate with different tools, GitHub or others, because the general, in general, the NCP servers that exist, they are not efficient. They bring all of the data. They bring a lot of things that are not needed. So we figured it out. We figured out how to make it more efficient by building our own NCP servers for what makes sense. But again, like we— We were able to make that decision because we have been in testing now for 6 years and test automation. So we know which things are important and that has allowed us to be efficient.

Yeah. All right. So how does this work? Is it all MCP? So if I'm in Cursor, I can go like a Miko test case agent or executing agent. I don't know what the agents are. I'm making it up. How does that work?

So actually, I mean, you're interested, I can show you.

Yeah, great. Let's see it.

The MCP connection is coming later this month, so you can use it from VS Code, from Git Code, from cloud. But right now, let me share my screen and show you quickly actually. So right now, a user can go to qa.amico.ai and they can create their account. And the first thing they need to do is just add project context. So testing repository in whatever it is, Selenium, Cypress, Playwright. We generate Playwright, we train it for Playwright, which is the more powerful, the most powerful right now. So, but we can take test repositories with anything our customers have. That's context. Then app repository, some customers, this make it super powerful, but we understand some customers don't feel so confident and give access to that repository. We have all the legal framework and paperwork necessary, but yeah. If a customer adds the app repository, it's super powerful. Actually, you can see how D'Amico goes from entry mode to legend mode when you add more context. So as you add context that is meaningful, it starts growing and it gets to legend mode.

So really quick, what I know a lot of people are worried about is, is it actually being trained on your test and your repository? Or is it just using it just to to generate things. I don't know if that makes sense. It's not being trained, your LLM model behind the scenes, is it?

It's just, you see, we already train it with other data and other knowledge we have, other scenarios, but we just use it to make decisions. Yeah, we don't train it with the data. So yeah, you could, in 15 minutes, you create account, you add all the context you want. You can add more context later. In this case, I'm testing this Odoo free page. You can add context on PDFs, videos, any type of document. You can integrate with Jira, Azure, GitHub. With Jira, you take the context from user stories. So practically, 15 minutes set up, and now it's ready to go. And you can ask anything regarding testing, like, what is my current test coverage. So this is kind of the experience after you have the context. What is my current test coverage? And suggest 5 critical negative test cases I'm missing. So after this, this Amico is the QA helper that can help you to do anything, right? If you need to automate, for example, or test, automate email verification, you can ask, hey, amigo, I don't know how to do this, help me. It will give you the instructions. It can suggest test cases. It can check the test coverage, make suggestions. When the code changes, if a pull request changes the code, you can ask it to suggest the best regression for this change. So practically, it's a QA helper. Yeah. That then a tester, SDE can use to accelerate their work, yeah. Yeah, we can see here how it's pulling the different tools we created, most of them specific. And now it suggested what current coverage snapshot for 4 parts of this software, create product, sales order, fulfillment, returns. So it mentions how strong are the tests, how stable, flaky, and suggests 5 negative test cases. Create a product with no name, confirm a sales order with no customer selected, sales order with quantity exceeding available stock, showing forecast warning. So yeah, that's in a nutshell. So in 15 minutes, you can add the context and then— you can start creating all the coverage you need, including negative test cases.

Yeah. So, what I like about this is it doesn't seem like you're trying to replace testers. It's like a real friend. Like, you're like using it as a friend, say, "Hey, I'm trying to do this. Any suggestions?" Obviously, it's not saying, "Thou shalt do this." You can give or take what it gives you and use it or not use it. I like it. Yeah, exactly.

Even at the end, he's asking you to, "Hey, do you want to commit these ones or want to make changes?" Right, right, yeah. So, yeah, and that comes to the point that we mentioned about the role of the tester is augmented practically. And also, it's a world of collaboration. So, we are seeing developers, product managers helping with the testing because they have some visibility, but at the end, a tester with experience, they are in control of making the decisions of what has been tested, what is right, no?

Yeah. Yeah, they're just getting better information. And being a tester, they know whether or not that works or doesn't work or apply or doesn't apply. So, the approach is really great. And you also mentioned something, you just glossed over it. Like, when I used to work for healthcare, we used to have to do hot fixes. And, but we wouldn't know, you know, they checked in code B, we didn't know test Z or D did anything. We didn't know what tests run against it. So, they said, just run all the tests. And so it was really bad. It looked like this could help you know when someone checked in code, here are the 4 or 5 high-risk tests you can run to cover that change. Am I understanding that correctly?

Correct, yeah. After there is a code check-in, you can ask Amico like, "Hey, check the last 3 pull requests and suggest the best regression testing for those." And it will tell you, okay, run these 23 tests, 18 are already automated. Do you want to automate the other 5? Yes, boom. And it can run them live in the cloud or it can automate them if needed. So for example, we are seeing development teams like, okay, I just push my PR, test it quickly, not even need to automate, we call it ephemeral, and then it goes back. So, but yes, it can suggest based on the regression. And right now, you have to go to the Amico platform and tell, oh yeah, check my latest pull request and suggest tests. But what is coming in 2 weeks is automatically Amico will say, oh, I detected this pull request, these are the tests, want to run them? So even more proactive, yeah.

Nice, and obviously the user has the option to switch it on or off to do things automatically, so.

It's never out of the room. Yes, yes.

Correct. Nice. So, say Nestdat's listening right now and they're drowning in, say, flaky tests or maintenance overhead. How would Amico change their day-to-day?

Ah, yeah, great question. So, one of the agents, it's, we call it repair agent. So, it's always running to detect flaky tests and suggest improvements. So I mentioned that we have 12 specific agents. One of them is the repair agent and it's always running. You can ask it for, show me the flakiest test, or if one test breaks, just you can tell it, fix the test and it'll show the code and you can approve the code, or it can show just in plain English if a user isn't technical. So, That agent will suggest repairs as the tests break. So in short then, if the tests are breaking, it can fix tests in bulk. Let's say that you have 153 tests and 50 broke because you changed one button of position or something like that, classic selector issues. You can bulk improve all of those. You can tell, hey, Amico, this test broke, it seems they are the selector. Okay, you want me to change for the 53? Yes, boom, and it will update the test right away. So again, to your good point about being a helper, a counterpart, it can accelerate how you fix flaky tests, suggesting the fix and applying it in bulk. Also in that release of 2 weeks that I'm talking about, the proactiveness of Amico, It will proactively, for a failed test, it's starting in 2 weeks, it will proactively tell you this test broke and it's 85%, it's flaky, do you want me to fix it? Or this test broke or failed and this looks like 93% chance is a defect. So it's going to the next level to tell you exactly ask you, "Hey, do you want me to fix it or want to file the defect?" So, that's the next level we are releasing in 2 weeks. Yeah.

All right. So, for some reason, testers get hung up on this, you know, flaky tests, they're always complaining about it, but then people say you shouldn't ignore flaky tests, or they get worried if AI just automatically updates them. So, what do you think it is? Is it a tooling problem, a test design problem, a real application problem we're hiding? Why do teams do that?

All of the above. All of the above. All of the above. And that's why we designed this with Amico will give a confidence level, but still ask the person in control, the tester, like, hey, want to check the video and make a decision or just want to go directly and file the defect or fix the test? So all of the above. And that's why a human reviewing It's still needed. So in that scenario, now that you will have, instead of hundreds, thousands of tests because of the level of code, something that we are optimizing is quick review, no? So you will need a tester is still the brain managing the whole thing. So you will need to see very quickly videos of the failure part and check up, check. Thumbs up, thumbs down, thumbs down. So to accelerate the review part, right? So to your question, yeah, it's all of the above and still a human might need to be involved to make the tough decisions. At least when there is no certainty, we are trying to measure certainty on flakiness or potential defects. Still, we want, we recommend a human taking the look and making, ensuring that Everything is going as expected. Yeah.

Awesome approach. So, another thing I think people struggle with, besides flakiness, is scaling their automation. So, is there any, like, big mistakes you see QA teams maybe making when trying to scale the test automation? Maybe, and how does Amico maybe help with that?

Yes, right now, what we are seeing is, um, LLMs are super powerful, no? Codex, Code, um, Cloud Code or Cursor, and we see many teams trying to automate testing and the checks with LLMs, and that works at low scale usually. Like, it's easy to get 10, 20 tests running, but when you are trying to scale it to hundreds or thousands, now it's a— software, a complete software, a serious software project, no? So we see many teams breaking in that transition from a few dozens of tests to hundreds of thousands, because they saw early results with a few scripts, but they never talked about the architectural design you need to scale at the end. What happened with AI is coding is cheaper, coding is easier, But still, you need the structure around coding and the thinking to ensure that your software will scale, no? And test automation at hundreds or thousands of tests, it's a formal software project, yeah.

Sure. So, you know, let's be honest, are there anything in AI or AI in testing that still falls short today? Like what should not absolutely not be handed to an agent yet?

I think what I would do now if I was a Romanian, for example, even with your example with Amico, like, oh, let's automate everything. I would be thoughtful about which scenarios are critical for the user and those ones always a human checking. Maybe you can let the agents test the scenarios that are not critical, no P2, P3 scenarios, but P0 and P1, I would still have a human always checking those for two reasons. One is just— You can see a QA agent like Amico, like an intern or a new software engineer right out of school. So you need to keep training it. So if it makes a mistake, one AI agent or LLM in one critical test, then you can train it and tell it, no, this was not a mistake. Next time it might get it right. But then the context of your software might change. You might need, you might add new features, you might change the functionality of some features, what is expected. So you need to retrain it, right? So I, if I had to make massive testing with agents, I would delegate to the agents the non-critical test cases, still the critical subset. I would always have the human reviewing, so I wouldn't give that to the AI on its own.

Nice. I guess another thing that a lot of people worry about is they have a lot of skills. So what skills should a Nestat maybe deliberately be building right now to stay relevant? You know, when agents, as you show, can write and maintain tests, what gets more valuable? What becomes obsolete? I know, like I said, you talk to a lot of different companies. I'm just curious to get your insight on that.

For a, for a, for a SDET, I think they, I mean, if they are junior, I would study a lot about how to create test automation frameworks at scale, because you still need to manage the LLMs. It's easier to do it now, but you need to know the concepts. You need to know the patterns of when to do X or Y. So if you are thinking about being an SDET, I would train more to be an SDET architect because writing scripts is cheap and fast now, but thinking about how to handle an architecture of test automation is still a challenge. You need to tell the LLM what to do. And one of the main cases that we see that still needs a lot of architecture is data. So data management of data, depending on a specific use case, that's other thing. If I was a manual QA or non-technical QA, now coding is cheap. I mean, LLMs or AI helpers like Amico, now they are the new no-code tools because practically you can manage and automate with no-code. Is it not? The new no-code generation. But the same, then if you want to grow test automation, you might need to learn some concepts of architecture for test automation. And for both of them, they have to keep developing that sensibility for product and for how humans, for user experience, for UX, they have to continue developing that UX experience because Now there is more software being created and still we need humans that validate what makes sense and what doesn't make sense. Yeah.

Yeah, absolutely. All right, so you've mentioned a few things already, MCPs and things. So what else is coming next for Amico? I know you've, it's fairly new, but you're like in 6 months, 12 months, you have like a roadmap, give a little teaser for anything that It's not 2 weeks out, but maybe a little further out?

Yeah, the main thing the next couple of months is a lot of productivity. So right now, you can be a QA, SDT, managing, but the next few months is a lot of productivity and reacting to productivity and integrated in the development workflow that customers are already using. Let's say you have Jira and you move a user story from working to done. Amico automatically runs the test, and then you can review and suggest more. You have a pull request, the example we were saying earlier. So now Amico decides which is the right regression test set. It runs it and reports to you, and then you can say, okay, run others or fix it, but productivity. Yeah. Some changes happen in the pull request, automatically fix the automated test that could be broken so you don't waste time. So right now, the next couple of months, we are super focused on productivity and living where the user lives. This means, I show you the demo right now of the Amico platform. Coming in 3 weeks, we have the Slack integration so you can interact there, interact from PRs. So we want to live where the user is living, Slack, Jira, GitHub, VS Code. So that's what is coming in the next 2 months.

Very cool. All right, so forget about product for one second. What's the one thing you wish every testing team understood about working alongside AI agents since this seems like it's going to be the new norm?

Okay, I think related to my earlier point about think of them as an intern or junior helper. Be patient. I think I see a lot of times that we make the mistake of wanting them to be perfect and they learn, they learn, so it's learn. So I could try to see it as a, intern, junior helper that maybe you hired to help you, no? So you can start seeing yourself as a manager and manage and treat them as a helper. That will help you to create the right mental model to work along agents. So see it as a helper and have patience in that sense and try to give it the right context. As you would to a new teammate, yeah.

Awesome, and I guess where could people learn more or try Amico for themselves?

Yeah, so go to amico.ai, so it's Amico with two Os, .ai. We have a freemium account, then paid, but yeah, for anyone that is interested after this podcast, when you sign up and mention that you heard the podcast, and we will increase your freemium to double. So you can create more tests and use it for longer. Yeah.

Love it. Love it. And as I always say, seeing is believing. Try the free account yourself. Tell them Joe sent you and you can use it. Check them out using that link down below.