How to Stay Irreplaceable When Agents Write Your Tests

15 Sept 2026 · 34 min
34:04

In this TestGuild podcast episode, host Joe (JoeBot) interviews automation veteran Jonathan Wright (the "Automation Cyborg") about the rapid shift from traditional test automation to agentic AI systems. Wright explains that the questions testers ask have changed: the focus is no longer on scripting repetitive UI checks but on managing and overseeing AI‑driven agents that can generate, execute, and validate tests. He demystifies the hype around large language models (LLMs), arguing that most practitioners conflate LLMs with true AI agents and that only a small fraction actually use AI effectively. The conversation introduces several emerging concepts – intent‑driven testing, confidence engineering, neurosymbolic (high‑level English) testing languages, and the need for verification blueprints – and illustrates them with concrete examples ranging from web‑tool testing, open‑source platforms like OpenWrite/OpenTest, to complex cyber‑robotic and space‑licensing scenarios. Wright stresses that human testers remain indispensable as "evidence engineers" and overseers who bring domain knowledge, critical thinking, and guard‑rail design to non‑deterministic systems. He offers practical advice: shift left to improve requirements, adopt intent‑based agents, use confidence metrics to assess test validity, and avoid extremes of full automation or pure manual testing. The episode concludes with a call to testers to evolve into detectives who ask hard questions, manage data drift, and continuously learn new skills to stay irreplaceable in an AI‑infused testing landscape.

Chapters

More from JoeBot

1:00
youtube.com

Loust or JMeter?

2d ago
1:11
youtube.com

Who Grades the AI? 7 Questions to Ask Before You Trust AI Testing

3d ago
1:00
youtube.com

Your Users Don't Use Emulators

1w ago
2:55
youtube.com

What Actually Happens When AI Self Heals a Test🤔

1w ago
36:02
youtube.com

Locust and Azure Load Testing: Create and Debug Load Tests with AI

1w ago
10:00
youtube.com

WebdriverIO, AI Agents, and 7 Questions to Ask Before Trusting AI Testing

1w ago
0:45
youtube.com

Should AI Check Its Own Homework

1w ago
0:42
youtube.com

The $700K/Month Device Testing Mistake

2w ago
40:57
youtube.com

AI Agents Passed the Tests. Real Devices Didn’t.

2w ago
9:57
youtube.com

Playwright MCP vs CLI, Mutation Testing and AI Agents Flying Blind

2w ago
29:33
youtube.com

AI Testing Reality Check: What 13 Founders Refuse to Automate

3w ago
See all content