MCP Servers, Microcks, and the New AI Testing Stack

Are you shipping code faster with AI but quietly skipping the tests that matter most? Do you know that GitHub Actions went down 57 times in 12 months? And does your MCP server pass every test but your AI agent still fails in production? Find out the answers to these and other top news for the week of May 10th in this episode of the TestGuild News Show. So grab your favorite cup of coffee or tea and let's do this. Before we dive into the news, a quick heads up on something you don't want to miss. Quick reminder, the TestGuild IRL is one week away in Nashville on May 19th and Atlanta on the 21st. As always, our IRLs are free events, only 100 seats, and they're going fast. Don't wait. If you're in those areas, register now using that link down below. Hope to see you in real life. First up is all about WebDriver Bidi. So this article was written by Pooja, who talks about how WebDriver Bidi is the next generation of the WebDriver protocol, enables bidirectional communication between test automation tools and browsers via WebSocket. Because unlike the traditional one-way request-response model, this allows browsers to send events and data back to testing tools in real time. But in this article, Pooja breaks down the specific features with direct implications for test speed and stability. Preloaded scripts— she goes over what a preloaded script is, which is a JavaScript function that runs every time a new browser window, tab, or iframe is created, right at the start before anything else. Critically, that script can send messages back to the test client, meaning the page under test and your tests can communicate with each other. And the practical payoffs is significant. She goes over without preloaded scripts, a single test requiring login might need 6 to 9 WebDriver commands over the wire. Multiply that across 50 tests and you're looking at 300 to 450 setup commands, each traveling across the network. With preloaded scripts handling the setup and signaling completion, the same 50 tests require only 50 wire calls. The second problem preloaded script addresses is flaky waits. Rather than using arbitrary sleeps or polling, a preloaded script lets testers define the app's actual readiness state, checking whether the product grid is visible, products exist, and no spinner is showing, then signaling back to the test exactly when it's safe to proceed. All right, someone tagged me on LinkedIn all about a new tool called Test Orbit. What is it? Well, let's check it out. All right, so Saktivel posted on LinkedIn that after 4 months, He's created a fully open source browser native platform built for mobile test automation with Appium. Every line of code is public with the frontend and backend repositories available on GitHub. And this project grew outta frustration that he describes from his own experience as a QA engineer spending 3 days setting up Appium before running a single test, waiting 4 hours for sequential regression suites to complete and repeatedly pushing parallel execution to the backlog. And Test Orbit runs entirely in Chrome. The only local requirement is Docker, which manages test execution workers. Appium, Java, Node, and Android or iOS tooling are handled inside the worker image with no local installation needed. According to the project site, teams can run their first automation tests in under 10 minutes. The platform supports parallel execution across USB-connected, wireless, and cloud devices and includes live log streaming via WebSocket, a built-in Appium Inspector, a built-in web IDE, a dashboard, email alerts, on run completion, a flakiness leaderboard, and failure pattern detection grouped by exception type. Next up is our webinar of the week. So the TestGuild is hosting a webinar sponsored by Perforce, which is scheduled for May 14th at 1 PM, which includes awesome speakers like Sheen Evans, Director of Product Management at Perforce, and the one and only awesome Scott Moore. So you definitely want to check this out. This one is squarely aimed at problems most testing teams know too well: performance issues, broken user journeys, and last-minute surprises that show up not because of what teams plan to test, but because of what was skipped or pushed too late in the cycle. So the webinar is going to map out what high-performing teams test months in advance, including core user flows, performance baselines, third-party dependencies, and test data strategies. It's going to cover what typically gets squeezed into the final stretch, why that creates release risk, and how to prevent critical gaps from slipping through entirely. As I mentioned, Shane Evans and Scott bring a ton of experience to this area. You don't want to miss it. Check it out. Register now using the link down below. Hope to see you there and hear your awesome questions. Next up is all about an API tool that can also help you test microservices. Let's check it out. And this is all about how the CNCF Technical Oversight Committee has voted to accept MicroRocks as a CNCF incubator project. Software testers working in microservices or API-heavy environments, this is worth paying attention to. And I actually never heard of this tool before, but apparently this is an open-source cloud-native platform for API mocking and testing that lets teams turn existing API contract documents, whether those are OpenAPI specs, AsyncAPI specs, gRPC or GraphQL schemas, Postman collections, or even SOAP or WSDL projects into live mock servers. Those same assets then power automated contract conformance tests against real implementations. And the article goes on to say, looking ahead, maintainers are focusing on integrating with AI and Model Context Protocol, which will position this as a testing and simulation layer for AI-powered AIs and AI agents. All right. Speaking of MCP and AI, let's shift to AI testing. What happens when MCP server passes tests but your agent still fails? Well, I just caught this article published on LinkedIn by the Agentic AI Foundation, which covers a talk by Red Hat engineers Wesley Chun and Kayla Murray at MCP Dev Summit North America 2026. and this is a scenario that's becoming increasingly common for teams building AI-powered systems. The MCP server passes every test, yet your agent still fails in ways you can't trace. The core issue is that traditional software testing assumes deterministic code while LLMs behave differently on every run. Agent evals have emerged to handle this, but they focus on what a specific agent did and require instrumentation inside the agent itself, which makes them impractical when you don't control the agent. But MCP's Checker fills this gap by testing whether a model can correctly use your MCP service tools regardless of which agent is running it. The tool sits between your agent and your MCP server, logs every interaction through a proxy, runs evaluation scenarios defined in your YAML, and produces structured results you can use in CI. It works with any agent without requiring code changes. There's even a demo on the site which shows how this works in a concrete manner. So if you're building a testing system that depends on MCP servers, this is a must-try-out tool and you can try it out using that link down below. So in the next post I found on LinkedIn, AJ Gore makes a great point that the industry is finally adopting testing practices that many of us have been advocating for decades but have largely been ignored, but not because teams were convinced, because AI made skipping them too costly to hide now. So AJ breaks it down that when AI ships a feature in 3 hours, the bug shipped in 3 hours too. Compressing the blast radius from a sprint to an afternoon. Practices that teams deferred for years, paying the cost slowly through manual QA, production incidents, and 2 AM rollouts are now impossible to defer. He identifies what he calls the PR rubber stamp ritual. Senior engineers approve pull requests after 10 or 15 minutes reviews, rarely checking out the branch or running tests, creating a false transfer of responsibility that left nobody actually owning the outcome when something broke in production. And he talks about how this is happening even more in the AI era. His core point is really direct: every harness built to govern AI agents, eval suites, approval gates, sandboxes, executable specs works equally well for human engineers. Teams aren't building infrastructure for AI, they're building the infrastructure they should have had all along. Next up is something that Jason Arbon shared with me. I think he created it, and it's a post about a new manifesto called the Manifesto for Engineering with Confidence. And it makes a pointed argument: the existing models for software quality are broken, and rule that replaces them needs a new name. That new name is confidence engineer. This manifesto draws a direct line from waterfall to agile to the AI era, arguing that neither model anticipates software that changes continuously, generating faster than any team can review, deploy, or validate using practices designed for a slower era. The confidence engineer though, as defined here, is not a tester who writes scripts or a QA analyst checking requirements. They're an engineer who owns the question every team is now forced to answer: do we actually know enough about the system to ship it? They own the evaluation infrastructure, the behavioral benchmarks, the observability pipelines, and treat that infrastructure as a product, not an afterthought. So definitely check it out down below. Let me know your thoughts. Last up is all about what's going wrong with GitHub between May 2025 and April 2026. Incident monitoring services tracked 257 separate instances on GitHub 48 of which were classified as major outages, roughly one significant disruption per week. GitHub Actions alone suffered 57 outages in the same 12-month period. But testers, that last number is one that stings. An Actions outage doesn't just slow you down, it freezes your entire CI/CD pipeline. What do you think the issue is? Well, GitHub CTO Vlad attributes the broader reliability issues to a rapidly scaling problem Probably because why? AI is probably generating a bunch more code. All right, for links of everything of value we covered in this news episode, head on over to the links down below. So that's it for this episode of the TestGuild News Show. I'm Joe. My mission is to help you succeed in creating end-to-end full-stack pipeline automation awesomeness. And as always, test everything and keep the good. Cheers.