Kevin Surace explains how software QA can be a competitive advantage startups often overlook. GENIE treats test cases as data-driven, running thousands of input rows from a single script, and includes machine learning for script optimization and self-healing element recognition. Most AI-for-QA demos only handle simple clicks, missing the real work of testing APIs and backend systems.
Chapters
I also like this idea that I bizarrely heard from the head of Software QA just the other day, which is, hey, look, software QA is actually a source of competitive advantage for us. Like, I don't want to populate the world with our methods, our scripts, our knowledge, our smarts, because we can do stuff much faster than our competitors can. And so that idea of being able to have a private instance almost that's secure, I think it's going to really appeal to people. Can we just maybe go down even a level in some of the, I don't know, the geeky stuff, for want of a better description. So if I've got an app with heaps of variable rows of data, right, that all need to be validated, right? And I've got hyper variability in the app itself, but hyper dependency. So in other words, I've got a version over here that does this, this and this. It's all a central version of an app, but it has to manifest itself as 6,000 users or customers. I make a change here, it affects a customer over here. The complexity of that is qu. Quite immense. How does GENIE handle that environment where you've got an app with hypervariable rows of data? How does it think about that?
Yeah, so genie assumes that all test cases are data driven. So it can write a test case to a test script, that script ties to a data source, and that one script would be expected to run when it's done maybe a thousand times with a thousand different rows of inputs and certain expected outcomes. That's all built into the system. So it's not these, I see these demos sometimes where people, well, we have AI and QA and they do this very simple thing that's not data driven, it's it, you know, it clicks on this and it goes here and it's done. And you go, okay, well, you know.
I mean, I think that is exactly where I'm going, is that when I look at the expanse of people talking about AI and sqa, most of what I'm seeing is UX or CX pattern level testing, not data level testing.
Right?
Yeah.
You know, here's a funny story. I've seen, I don't know, maybe half a dozen startups in the last three years that got funded. They got their Series A and all of them said, we've got AI for qa. And of course I know how many millions of lines of code we have to make AI script generation in genie work. So. And it was 12 years and, you know, 70 people working on it or Something like that. So I know how many man hours went into it. And we didn't do everything right and some of it got tossed away. But here's four kids, nice people, but they're not kids. They're in their 20s, but you know, there's like four or five people there and they say, we've cracked the code. And I just go, you'll just be dead in a year. You have no idea. Because what they did is they got in and said, oh, you know, we can test a website. Well, yeah, anyone can do that. That's not where the money is. When we are working with a customer, the website is one application out of 10,000, the other 9,900 and some odd face the inside world and run the company. The website was the easiest bit and there's hardly any business there. They test it and that's it. Okay. These internal applications are getting updated all the time. So I think that if you are new to this business, you don't realize that all of the action is internal facing running companies. Highly data centric, huge data sets, lots of APIs, the back end systems that have to work, lots of coordination between systems. That's what people are testing. When we talk about end to end functional testing.
Yeah, no, sorry, keep going.
When you have a new startup and you go, hey, I've worked with websites, this can't be that hard. You're right. And then you get your first customer and they say, well, we use SAP and we built 12 apps on Salesforce. Let's go. And they go, you did what? What is that? How's it work? And of course what they built with, you know, 10,000 lines of code has no hope of testing those systems even in its basic core. The other thing that we've seen is lots of this. Oh, we've got AI and you know, AI and qa. And when you dig, dig, dig, what they either have is self healing, some kind of self healing for scripts which we invented in 2016. So that's, you know, that's fine. I would expect that's table stakes today. It's old and the second it's nine years old. Right. And the second thing they may have is kind of a co pilot. And I'll give you, there's one big company that has been touting their copilot and we've looked at that and we said, how does it improve productivity if at all? Well, it doesn't reduce headcount at all and it really doesn't improve productivity. What it's doing is when you're recording and the recorder comes up and says, hey, you know, you chose this X path. Here are some other choices. And I, the copilot would recommend this thing, or I would recommend doing this, or I'd recommend. And in fact, it may be slowing down people on the recorder. I don't know if it makes them more productive or not. It's not making them faster. So we took a different stand some years ago and we built in quite sophisticated machine learning. It's not a transformer, but machine learning that says go ahead and record what you will, and we will fix whatever you've done in a script optimization routine that takes that script, runs it multiple times and tunes the script so that it's going to run perfectly in the future, builds out a whole library of the ways to recognize this element. And, and you can add in visuals as a recognition of part of that element, as well as we call a locator or an accessor. So that is pretty interesting because we just skipped right over the copilot. I'm not here to help you. I'm here to just actually do the work because ML or AI can actually do it better than involving the human whatsoever. So just record what you're going to record in our recording web designer and we will make the script the most resilient it can be. And I don't need to ask you on every line, what do you think of this? I don't actually care what you think, with all due respect. Just do what you're going to do. We'll let ML fix it. That's what you want. You want this stuff working for you. You want to be the robot overlord. So we did that years ago. And so I just kind of think it's laughable when people introduce this little co pilot. And I go, we don't need a co pilot. You know, I have an AI pilot.
