Kevin Surace explains how Digital Twins generate test scripts for regression testing without automatically changing them on every build. Scripts run at each release candidate and only regenerate when user flows are intentionally redesigned. When tests fail, humans decide whether to edit the script or create a new Digital Twin, replacing manual offshore maintenance.
A part of the argument that I'm hearing here which resonates with me is that right now, if I'm doing a fraction of the test environment with a fraction of the accuracy, and now I'm doing all of the test environment and it's 80 or 90% accuracy, I'm so exponentially better off than I was before. It's sort of interesting. But one of the tricks you're alluding to here is, I think, the intelligence component. Let's talk a little bit about the automation component. So, for instance, if I'm sitting there and I'm going, okay, my app just, just changed. Some developer, he updated the app, he's published it, hasn't followed the process. Is the Digital Twin able to monitor and detect that and go, hey, look, we've just detected a change in the app and we've run the test case. You've got a problem. Or are we a little bit of a ways off that? And it requires the human to instigate it, but nevertheless, the degree of automation is high. Do I have to regenerate the test case or update the script, or does that all just happen?
Remember what we're trying to do with regression testing. Ideally, we've got a set of test cases that don't change because that's the user. Those are the user flows that must be checked with outcomes, right? And regardless of what someone has done to the application, regression means whatever used to run in production last time had better still run this time or it can't release it, right? So arguably, at every build or every release candidate, I want to rerun the exact same regression test cases, which are now scripts every time to a T. I don't want those to change. If they change, it's because I literally redesigned part of the application on purpose, okay? And if that's the case, then I've changed my test cases. And if that's the case, then I probably need a new Digital Twin to have those test cases be accurate. But nine times out of 10, you've changed something in the stack. You fix some bugs, you've done a variety of things, and it better darn well run exactly the same or better than it did before, but those user flows had better still run. And again, regression testing ideally means I've taken exactly the same tests I ran last time, that 100% passed on production. And before I released this to production, this 100% passes. That's what we do. So we don't do anything with Digital Twin. You know, the engineer went in and changed something Whatever, who knows? Might have broken something, might not. You rerun the scripts that were already generated a year ago from it, and that's the normal part of your cicd. Anyway. People ask me, where does this fit in our process? So. Well, when do you run, you know, when do you do your tests? While at every release candidate. Good, keep doing them. What do you mean? Just run them now, they're automated. You don't run the digital tool, you don't do anything with that. Right. You just run these and you only regenerate those scripts if you really need to change the user flows. Otherwise, no change.
The user flow becomes the trigger point for recreating a digital twin and regenerating new scripts. But until then, no change in the user flow. But I've maybe tweaked a bit of the code to alter performance, to target my database differently. I can go and test that and run it.
Yes, you've changed the APIs. There's something changed in the backend system. Who knows what changed. Something changed in the stack for security reasons, which happens every few weeks now. The whole idea of regression is, again, I'm going to take those scripts that I've got and rerun them. And if one fails, then a human gets involved and says, why did this fail? What changed in the application that either should have or shouldn't have? And they may say, this change is correct. Now, you could take that script, put it in our ide, and literally just edit maybe that one line of the script to do something different. Right. So you can still edit these scripts and they're yours. You don't have to go back and have them regenerated. They're editable. You can bring them up in our ide. You could test it one by one, run it through the ide, put them back in your test scenario and push them out. So this doesn't change anything. I was talking to someone this morning again and they said, well, how does this change our entire flow? It doesn't. All you do is put. Every time you were going to do manual tests or write scripts this now, or maintain scripts, this just goes in the place of that. Well, at this point I wasn't going to do any testing. Still, you don't do any testing at this point. I always test and I go to that manual team offshore and I take three weeks. Great. This replaces that. Those scripts that are already written are now automated. You'll be done.
Well, I mean, and what for me is so critical in that is as someone who has to constantly bother software teams to update things, fix things, or correct things. A lot of what I'm asking to be implemented from a code standpoint is critical. It's a customer issue. It's affecting a customer's footprint in the market. And we need to get something tuned and tweaked very quickly. And the holdup is just working across all these time zones and getting to people and trying to hit release windows. Suddenly, something that should have taken an hour is now taking four to five days.
