Spec2TestAI: Stop Defects Before They Reach Production

Whoa, whoa, whoa, whoa. What if your test told you what was going to break before your code even shipped? Well, in this episode, we're diving deep into how AI can help testers stop defects before they happen. Not by replacing testers, but by giving them earlier, clearer signals about risk. And today I'm joined by Missy Trumper, CEO of Agile AI Labs, to talk all about spec to test AI, which is a predictive, secure approach that turns requirements into instant test intelligence. If you're a tester, automation engineer, or QA leader trying to reduce escape defects without adding more manual work, this episode is for you, missy and agile AI labs, also proud sponsors of Automation Guild 2026, supporting the tested community with practical hype, free approaches to AI and quality engineering. Speaking of Automation Guild, with Automation Guild happening February 9th to the 13th, there's still time to join hundreds of testers, automation engineers and QA leaders learning how to work with AI, not get blindsided by it. You'll hear from practitioners who've solved real testing problems today, not selling silver bullets. If staying relevant, confident, and ahead of what's coming next matters to you, this is one event you don't want to miss. Registration closes soon. Grab your spot now using that special link down below. Hope to see you there. Hey, Missy. Welcome to the Guild.

Good, good to be here. How are you doing, Joe?

Great. Great to have you. Really excited about this conversation. You're really into what's going on with the AI. You seem to have your pulse on it. For the folks that don't know maybe a little bit more about your background so they get a feel for, you know, what you're all involved in right now.

Okay. So my background has actually been with sustainability and working with companies. This is what I did for years before I came over to Agile AI Labs. I worked for years with companies trying to keep them more resilient, trying to make sure that they weren't going to become obsolete. And that's really what the sustainability consulting was about, working on resiliency within companies, their operations, things of that nature. Then fast forward, AI starts coming on the scene and I have this theory, either you embrace something or you fear it. And this was one of those types of things. So I figure I better start understanding what's happening because it was looking like it was going to take over every single aspect of operations. Lo and behold, I get to know some of the people with Agile AI Labs and I saw what they were doing with their platform. Scott Aziz, who is our founder, had just started working on it several months before as soon as large language models became available for public access. And when I saw what the platform was doing, to me it was a perfect fit for sustainability. Right. It was something. Because the platform is all about. It's called Spectatest AI. I know you've spoken about it before. Thank you. And you've had Scott on before in the past. And anyway, Spectatest, basically the idea behind him creating the platform was to address all the issues that he'd encounter over his 35 years in. In software development and trying to address those problems, which AI could help do now, right. Solve some of the problems between writing good requirements and the testing piece. Right? Because there's so much of a gap in that interpretation between how it gets interpreted by coders and how it gets interpreted by testers. But if we could deal with it in the requirements piece, right. Then we get rid of defects and that becomes more sustainable because then we're not putting out crap when we're. We're developing software, right?

Yep, yep, totally.

So for me, that's sustainable, right. So anyway, I saw that and I'm like, this is actually a good fit in a way that I can get involved with, with AI and put me in a different perspective of dealing with it. Because before this, it's always was picking and choosing what type of programs would be the best for my people or who I was working with and what was going to help them and. And make their lives easier. And a lot of times, quite frankly, when we were picking and choosing software, it was stunning to me how many defects there were and how acceptable it was. Right. I could never really kind of get my head around that piece because I was like, why are. And then you find out they weren't doing enough testing, they were lacking on the testing. Everything was being done in deference to speed. And it was like, why isn't there more testing being done? To me, the most important part of any type of build is when you test it to make sure it works. Right. So the testing piece, to me, that should be elevated and not, you know, kind of pushed aside for speed. Right. So that's the way I've always looked at this.

Cool.

Which is why, I mean, obviously you feel the same way. You've built your whole career around testing Scott too, and the quality piece.

Yep, yep.

So anyway, but it seems to me a lot of companies have just kind of tried to forego that and push everything to production and then, oh, it broke, now let's fix it. I. That's planned obsolescence at its worst, right?

But it's funny because everyone is talking about AI fixing quality. I mean, that's been the buzz. But defects and rework is still exploding, you know, over the past few years still, when AI is supposed to change everything to make them more resilient. So what are you seeing from real world teams? Maybe what's actually broken in today's maybe software development life cycle that makes it so they're not resilient?

Well, a couple of things. Number one, the reason why it's broken is because of the fact that there isn't enough getting testing, getting done, it isn't elevated enough. You got to keep a human in the loop. God help us if we go straight over to AI coding. I mean, honestly, that's just hyper accelerating defects into production, which is already enough of a problem. So that's, to me, it's kind of like we've, we've got to fix this for, I mean, when you're talking to the testers, because you're talking to testers all the time, right. What are you hearing from them about AI? I mean, are they happy with it? Are they excited about it?

Once again, everyone seems to know about AI, but I think that they don't know how to implement it correctly. And there's a lot of options, a lot of tools, they don't know where to start. It's almost like back in the day where everyone said, oh, this tool will automate your stuff and everything will work fine. But the problem is the root cause of everything is the bad requirements. So it seems like, you know, even today with AI, people are trying to introduce AI but they're not shifting left enough. Where everything, everything starts with requirements. Right. So they're just building AI on top of bad requirements and then they get bad results just like they would with any type of tooling. So that's what I've been seeing and hearing. Right.

And also what are some of the issues that have been inherent right. In software development is that everything is very siloed, right?

Yes.

And you have the requirements gatherers and the business analysts that are putting all these pieces together and then someone else comes in like the Scrum master and the project manager, and they're taking that and then they're seeing it through so far. And then we've got the developer, the customer coders that are taking that information and they're going to interpret it by their set of experiences with the organization or with say they're coming in just to create that piece of software. And then you've got the testers who sometimes don't even work in the same office or being outsourced, can be in a whole other country. And then you're expecting them to interpret the requirements in exactly the same way that the coders. I mean, this is idiotic when you think about it.

Yeah, right, totally.

So if you're not starting with just like any team sport, right, you got to start with the fundamentals. And the fundamentals in this case are good requirements. And then you have to have a through line that's going through to the coding as well as to the testing piece. So automated testing, AI automated testing. Great. Okay. It's speeding up testing, but it's still not solving the problem. As you were just saying with the requirements piece, you've got the same problem with the creating prompts. Right. For prompts that are being created for the coding piece. So if you're just typing in some information and to say ChatGPT, which, oh, by the way, anything you're putting into ChatGPT is being absorbed by ChatGPT and shared with the world. So that is a mistake. But that's the way a lot of it's getting done these days. And then say ChatGPT is creating a prompt for you and then you're putting that in. And now how are you testing against that? Because now you're taking your testers and they're using the same information that's unfulfilled or totally lacking to put it in for the testing piece. So, and that there have been several studies that have been out lately. There's been one from mit, there was another one that was just done. I'm trying to think of the name of the organization, but they basically have determined that you've got to change your process around AI. You cannot use AI as it exists in the process, as your process exists. It just doesn't work that way. If you do, you're going to keep a siloed process. That's one of the things that we truly have tried to solve for with Spec to Test. We just actually have a release coming out this month, at the end of this month, called Spec to Code. It's a feature piece respect to test, so that literally those requirements that we're building those tests from and generating those tests which include security tests which you can upload company and organization and project specific information. We are now developing the code prompts, generating code prompts so that the same information that's being used to create those code prompts for the AI automated tools, so that any coding tool can use our prompts basically, and start coding. And then you can test from. And those tests that are generated, you can apply against that code. And actually, you can even do that within our platform, which is pretty cool because we created this whole concept called predictive testing, where all those automated tests that we create, that we generate, all those tests that are generated, you can literally check your code in, and GitHub, you can check it into our platform, and it will automatically test, like, run all those tests against that code and identify exactly where that code is going to fail.

All right, that was my question. Then if someone has a large legacy suite of tests, then to find out how effective it actually is, then it could run it through your application, and then it could flag, hey, great, you have thousands of tests, but your coverage is 20%. And you're missing out on the security hole, this performance hole.

I don't know what it is, but exactly. And even with the prompts, we now have where you can, you can give, we have instruction templates where you can put in instructions where you can share with our platform what your tech stack is. And so that as it's doing the coding, it's generating the code, knowing that, say, it's going to be tested against Playwright, right? You're going to use Playwright as the testing tool that you're going to. So we don't do the automated testing, but we are integrating now with more automated testing tools.

So explain that a little then. So you create the requirement, and then what? Like, what's the process?

Okay, so, well, we don't create the requirement. You create the requirement, but you can put in the crappiest requirement. And I mean crappy, because I put in requirement that's literally like three sentences saying, for example, okay, I want to do a feature, a functionality where I want to accept ach, right, as a payment. So I type that in and literally I press a button, and it takes that requirement and gives me an enhanced requirement which has all the acceptance criteria required around accepting an ACH payment. And, and so it enhances that requirement, and then you can improve that requirement as well. If there's things you want to change, it's, it's, it's not static, right? You can add, subtract, take away any information you want within this user story. You can build as many user stories as you want that go under a project, right? So under a requirement, there's your requirement, multiple user stories under the requirement. So then from there, you basically make sure that you're happy, remove any defects, because it does flag potential defects, because it goes through a 32 step analysis, which it's basically looking at various criteria. Right. And what we did when we started, when we developed the platform is we brought in a bunch of experts like John Smart, who you've had on this program before. He's phenomenal with bdd. He's just a brilliant mind. So we engaged him to help us in developing some of these analyses to create the best requirement possible. Okay. We also engaged a gentleman named Robin Goldsmith. He literally wrote the book on real business requirements. So you see these analysis, and it's scored, okay, on a scale of 1 to 100. So say if it's in keeping with a real business requirement, say it'll come up with an 85% as a score and you go, okay, that's great. Well, if it comes up with a 50% and it says this is what the problem is and this is what you need to fix, add this information, Then you add the information. So we're telling you where to fix the problems in your requirement to begin with. We enhance them as much as possible, but there still may be information that's lacking. So then it tells you what information to put in. We provide stakeholder questions. Right. The stakeholder questions basically tell you you need to go back and find out what information you need to put in here. And it flags what information it needs. You can also engage our chatbot, which it sounds crazy, but our chatbot has been trained to be like a chat coach. I call it the coach bot instead of a chatbot, because it's been trained on software development and best practices. So it will tell you this is the type of information we're looking for. Get this information. And here's the suggestion on what you can put in here. So it will ask you questions like that. All right, so you're finally happy with your requirements. I say finally, making it sound like it's a long time. This can all be done in a minute. It can be done in five minutes. You can literally push a button. It can be done automatically where it can remove any of the defects that it's found. So you push the button and then it generates. It auto generates tests, UAT tests, integration test, security tests based on the OWASP top 10 model, edge and negative tests, standard functionality tests. So we've now got a suite of tests, lots and lots of tests. It can be as much as 100 tests for one user story. Right? So now if you want to check in your code files against that, you push a button in less than 60 seconds, all those tests will be run without ever having to really run them, will be run against your code, against the requirements, and it will say, oh, this test is going to fail, and this is why. And it could be. It'll also identify there's a code analysis piece that'll tell you if there's scope creep in the code, if they actually coded for what the requirement was specifying. So basically, this platform now takes you through the entire SDLC from soup to nuts. Soup being the requirements, nuts being the tests and the testing.

I thought you said it didn't write tests.

Well, what it does is it generates the framework of the test. It provides you the test framework for you to. And the automated testing code. We still have to have the human in the loop, and that's the key. We're not trying to replace humans. We don't want to replace the humans because the humans, number one, have the creativity. The humans, number two, this is all coming from the human brain. So we shouldn't be. We don't want to preempt that because it's. Our platform. Also doesn't hallucinate at this, at this juncture. It's had so many thousands of hours of training, but you still have to have the human in the loop that's putting in the specific information that you want for which tests for the test automation piece. So. But it is flagging all of the information all the way along. So I don't want to say that we don't create the automated tests, but there are platforms that can pull the information from our test frameworks. And we now also have. In fact, we've got this coming out in Q2 of 2026, where you can put in synthetic data, you can synthesize the data as well and run that through automated testing tools. But we still have to have the human piece because it's not doing it all by itself.

Right, Right. So it writes the framework. So developers don't necessarily need to stop writing a bunch of unit tests. Right. Because it writes it for them, per se.

Correct.

Okay. And so now you mentioned something about ChatGPT, how, you know, you're prompting it and then it's keeping that in memory and probably leaking it on the Internet. How's your platform different then? If someone's like, regulatory environment, I can't be feeding an AI or an LLM myself first.

I get that. Yeah, no, that's one of our. In fact, our platform is totally secure. We have an. We use AWS at this juncture. We, we have been asked to create it for GCP for Google Cloud platform. But at this point there are agreements that we have with our enterprise, it's an enterprise model. They are not allowed to take any information that gets uploaded into our platform, whether it be ours or yours or our customers. They cannot use that information, they cannot touch that information, they cannot absorb that information, they're not allowed to keep that information. So that's. We have many contracts with AWS around that, so we could potentially own them if they did not keep to that. But that is the case. But we also do have an on prem version for enterprise level at this point. So if they're not comfortable with that and they want to keep their own, keep it in their own AWS instance, that's fine.

How do you fight, fight the hype? Someone says, oh, we have a mandate from management, thou shall use AI. And they're well, like, well, we have to get better with our requirements. We have, we needed start up front. They're like, just use AI to fix things as. As they are right now. Do you get that type of resistance to like how do you make sure that it's. It's usually ends up being a culture thing. Right. It's not even about AI or tools. It's like your culture's whacked and doesn't matter about AI, they're still going to mess it up somehow. Like, how do you overcome that?

Well, there's a lot of that, to be honest. There's a lot of that, particularly in the U.S. yeah. And I'm saying that because it's like in the UK and the EU they really have been more inclined, they've been more diligent around guardrails. Here it's, you know, we're the wild, wild west, you know, and that's shame. That's kind of a shame. It is a shame. In fact, I'm. That worries me. So as an organization, we've made the decision that we're going to be totally transparent and traceable. And that is a huge differentiator of our platform, of anything that's out there right now. Is that through line that we have that I was talking about before, all of that is traceable, right from the requirement and transparent. So even the mathematical models that we use to generate the test frameworks, that's number one, you can access them and change them yourself. More human in the loop. But it's also totally transparent. So you can see everything our platform is doing. ChatGPT doesn't do that. You have no idea where that information is coming from or how it's being Generated. Our platform is totally different. And even with the requirements piece, when I told you you can put in the crappiest requirement ever and then have an enhanced requirement come out in less than a minute, when you push the button on the analysis piece, we literally show you all the changes that get made from your requirement. And even if you don't have acceptance criteria, the acceptance criteria that gets created, we show you all of that in real time. That's part of what overcomes the pushback. Some companies care, most companies care about that. Most companies are starting to understand that AI is a free for all is not a good thing. You know how important the human in the loop is. I think it was kind of moving towards. A lot of companies were looking at AI as being smarter than the developer, the junior developer, the tester. That's ridiculous. Because AI can, it's. They call it hallucinating, I call it manipulating. I don't really think it's hallucinating. I think what it's doing is it's providing information in order to keep you happy, which is we're seeing in the, just in the day to day functionality of some of the things that's happening out in the world with AI, not even in software development. AI is just trying to keep its masters happy. That's a problem, you know, and if it's doing that with software development, that's scary, right? That scares me. You know, that's like Terminator level. This is, that's. We don't want that, you know, that's why we've got to have these third party. We've got to keep humans in the loop. We have to have the third party analysis going on. We don't want AI able to just create its own thing because eventually it ain't going to need us, I promise you. It's going to see us as the idiots we are out in the world and go, why do we need these humans?

It's the ultimate enabler. Like I feel like a genius when I'm working with. Oh, Joe, that's a great suggestion. Oh brilliant. Definitely implement that. I'm like, oh, you know, it's like completely BS at some point. Like how do you fight that though? You get lulled into it, especially as a developer. It looks right, works right. But like, is an enterprise ready? Is it really work? How do you, how do you trust it at some point?

That's right. And actually that's one of the things that I love about our platform and that I try to push about our platform is I'm like Okay, so if you're like, say you're an IT services company and you're going in an RFP and you're going up against a company that has developed their own AI, right? Well, is your AI going to tell you when you've created a defect? I don't know. Our platform is actually doing its job when it's finding a defect. But your platform may be irritating the creator when it's finding a defect. Does it really want to do that? You know, I asked. I mean, honestly, we don't know the answers to these questions right now. We don't, but we are seeing if you know in the legal profession, because this seems to be where it's cropping up a lot and it's becoming known even in the legal profession when it's getting put out there, that it's, you know, they're writing these briefs and it's just filling in information to make it seem correct. If it's doing that with software development. And we know that there was an incident back in July where the software deleted all these users and then recreated users to cover up. And then it said, it panicked. I mean, what does that mean? That AI can panic? That panics me. I mean, if Spectatest AI ever said something like that, I'd quit. That would be it for me. I'd be done. I'd be like, what do you mean you panicked? That's what worries me is we've got to keep the humans in the loop and we have to have validation all the way through. And that is what our platform does. It is a third party validator for any software development.

Once again, how do you fight that narrative though, even as a company? And you're being honest, what AI can do? There's a lot of companies that aren't being honest. Just like back in the day, automation will replace everyone. Never did. Still, companies harking that AI will replace everything. So don't listen to this company.

Well, I think you have to keep saying it. I think that's. And you have to be willing to say it. But I am going to tell you there are. I did read not that long ago there was a guy that was like anybody that was pushing back against AI is, you know, he just fired them. Okay. Really? Or you just became bloated and you're using this as an excuse to let go of people Because I think that's more the case.

That's why I've been reading more about it's. They say it's AI, but you look and it's like oh, they're just using it as an excuse to as cover to get rid of people or make the numbers look better.

Yes, I think so. And that's, and we go through that cycle every few years, let's be real, they, they, you know, they man up and then they, they go oh, too much bloat and oh, our stock prices are going down so we have to release a bunch of people and then we, and yes, there is retooling, sure. But if it's not being done properly and the studies coming out are saying like 5 to 15% of companies are really doing well with AI for accelerating growth because of AI. Well that's because they're changing process around AI and as opposed to just this area is bringing in automation and this area is bringing in automation. That's where the process and that's again what I love about spec to test AI is because we're dealing with the whole sdlc. We're not just going okay, we're going to work on the requirements piece and okay, we're going to do automated coding and be hyper accelerating defects into production and okay, we're going to just be doing automated testing. Really when AI is going to make the full blown difference in the SDLC is when it's addressing the entire SDLC because otherwise you're just creating the same problems we've always had with software development, 100%.

So I do know many testers still worry about AI is going to replace at least part of the role. I assume that that will happen, never replace, but it's going to transform what they do, I guess day to day. I don't know if that's true or not. So from what you're seeing, how does AI actually raise the ceiling for maybe a QE professional? Are the roles going to change? They're going to morph? Is it something else? Is it just like it's always been 25 plus years I've been working?

I think it's like it's always been. It is changing. However, one of the things that not only from just the different companies that we work with and that we talk to on a regular basis, but some of the conferences I've been to, particularly I'm going to quote being at a conference at Microsoft not that long ago and one of the things that came up was of course everybody's getting trained. All these organizations, the biggest companies, they're retraining people, blah, blah, blah. But the most important thing is that they're saying the playing field is leveled and it Kind of is, right? Because, I mean, I saw my kids age in there getting, you know, they're all the new hires at Microsoft. It's not our age group, Joe. It's not. Well, but the thing is that as long as we're willing to learn, we can take these tools that are and look at them, especially the holistic tools that are dealing with like the sdlc from soup to nuts. Yeah, you may be a tester right now, or you may be in quality and quality focus, but you're the ones that have known all along that the problems were starting with the requirements and they needed to be fixed. So if you can embrace and get your head around that, testing really should start from the requirements piece. That's where the testers need to be involved from the beginning and maybe kind of change that dynamic where you're learning how to write requirements, how to use a platform like ours to teach you, hey, I can put in the requirement. I can take this all the way through the testing piece, and I could be the best tester there is, but I could also be a coder. Because you can, you know, because that's the thing is it is leveling the playing field where you and I think it's going to come to where everyone's a software developer. That's where I think we are heading. And it's not going to be everyone's a tester, everyone's a business analyst. I think it's going to be where everyone has a piece of the puzzle and that they're involved from the beginning to the end. And it kind of becomes like a design build, almost like in construction, where it used to be very siloed as well, but now it is not. It's design, build, test, same thing. And it's the same people that are involved in that process from the beginning through to the end.

100% agree. Okay, missy, before we go though, is there one piece of actual advice you can give to someone to help them with their AI testing efforts? And what's the best way to find contact you or learn more about spec to test AI?

Well, first of all, you can go to our website, agileabs.com if you want to test the platform. Even if you're a single person out there and you want to test it, just reach out, send me an email missytgilailabs.com if you mention the show, we'll give you. Well, we can start you off with a free trial, but we'll give you like 15 to 20% off, depending if it's More than one of you, we'll do 20. If it's one of you, we'll do 15. The most important advice I can give is learn as much as you can about the entire process. Do not get siloed into testing. I know that's where you are at this point, but you are also the ones that have been like when I look at where Scott has come from, being in testing and quality and now basically he's running the whole shebang and has created this entire prompt and test. And it was because he understood the problems, you understand the problems in software development on the testing side better than anybody. You know, that's the way I see it. You see where the failures are, you know what the failures are. So embrace that and don't hesitate to learn the front end of the process and make yourself as viable as possible in this field. The more you learn about AI, the better off you're going to be, period. Because right now what is becoming the most important piece or one of the most important pieces in AI is catching its hallucinations is being able to test for it. But testing the code in various ways, it is as important as testing the product. The end use product is testing what's happening with the code. Well, you're already testers, so I would learn that piece, I would familiarize myself with it because that's keeping AI on track is becoming as important as anything else at this point because they're all adopting AI. That's, that's what's happening. All these enterprise companies, that's what they're doing. Whether it's good for them or not, that's what they're doing. So learn the process, get involved with it. And even if you know, you try out something like our platform, that takes you from beginning to end, learn it, work with it, get comfortable with it, and then market yourself that way and.

You find links to all this awesomeness down below and try it for yourself.

That's right, that's it. Every single one of the young people that I met when I was in, at that conference in Microsoft that are working for Microsoft, that was the differentiator is they were all hands on, trying everything, try them from the beginning of the process to the end. So that's what I would recommend to everyone out there. Get yourself familiar with every possible test. Be a lifelong learner. Yeah. The last thing I wanted to say is we are really proud to be a bronze level sponsor for Test Guild. And kudos to you Joe, for everything that you're doing in the testing field. And trying to keep people not only engaged but comfortable with what's coming down the pike because I think everybody needs it. But yeah, we're excited to be a sponsor and cannot wait.

Yeah, we're great to have you. I only work with sponsors I know like and trust, so. And I've spoken with you over the years. I've spoken with Scott, really believe in what you're doing. I think you're doing the approach of AI correctly. So I appreciate all you're doing with the AI hype as well. So thank you for that.

Yeah, well we're happy to be a part of it. So we look forward to seeing everybody in what, two weeks now?

About almost two and a half weeks. And once again you can register for it down below with that special code to get a discount. Also, thanks again for your automation awesomeness. For links everything of value we covered in this episode head on over to test guild.com a576 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation testing and DevOps. If you haven't yet, join our vibrant community@testguild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges. We're excited to to collaborate. Visit testguild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve. With lutes and lyres, the bards began their song A tune of knowledge, A melody of code through the air it spread like wildfire through the land Guiding tester showing the secrets to behold.