Playwright With AI: How to Automate Tests Without Shipping AI Slop

Hey, what if the code your team shipped stopped being the thing you actually review? What if Markdown was the new programming language and the AI coding agent was just the new compiler? And this is not a hot take for someone on the sidelines, and this is not some verdant tool hype. This is actually a director of engineering shipping the biggest release quarter of his career with the team of just 6. Hey, I'm JoeBot, and in this episode of the TestGuild Automation Podcast, today I'm joined by Andrew Knight, better known by most of you as the Automation Panda. Andy is a Senior Director of Product and Engineering at CycleLabs, and he just released a brand new LinkedIn Learning course on Playwright with AI. And he's one of the few people I know who could talk about test strategy, engineering leadership, and actually working code in the same breath. So in this episode, Andy breaks down exactly how Playwright evolved from a web testing tool into an AI automation platform. We also get into the MCP server, the planner, generator, and healer agents, and why the new CLI and skills approach might save you a fortune in tokens. He also shares how his team uses Spec Kit to make testing strategy into a product once, so it never has to be argued about again. He also explains how he spots AI slop from his own reports and what he does about it. And then he says something I think is going to make a lot of testers uncomfortable. He tells me code quality may not matter the way you think it does anymore. I pushed back. He didn't blink. So stick around for that part. Because whether you agree or not, this is a conversation your team is definitely going to have in the next 6 months. You don't want to miss it. Check it out. Hey Andy, welcome back to The Guild.

Hey, thanks for inviting me back, Joe. Good to see you.

Good to see you, man. It's been a while. I mean, the first solo episode we did, I think was in 2019, but then we did an episode with you and your boss, I think in 2023, but it's still been a while. So just great to have you back on the show.

Thank you. Thank you. Always a pleasure.

So I'm almost up on episode 600. This might be 600. It might be a few before 600, which is crazy. And I met you when you were a young panda. And I think when you're a young panda, you're really into Python, but I've been seeing you blowing up everywhere, all your different talks at different places, but I don't see anything around Python. Uh, so a little, little, like a little bit maybe backstory of where you are now, where you were. Uh, sure. Yeah, sure.

So I consider myself a polyglot. I still love Python. I'm still very active in the community, still go to Python events. But I mean, with AI these days, you can program in any language that you want. I find that much of my work tends to actually be in the Node stack these days. I've been doing a lot of TypeScript. Now granted, I am a manager now, so I'm not coding as much as the regular engineers, But when I do projects, it's almost always some kind of Node-based application. On my team, we do Node, we do Go, we do some Java. So there's a little bit in there. A big portion of my team's backend is written in Scala, and we're trying to get out of that because it's kind of scary. It's not good. But yeah, so yeah, I mean, even a fun fact, even when I was like super, super doing Python stuff. Um, that was kind of a public face. I was still working at a company where day to day I was doing C#.NET. Python was more of like the passion projects. And so people would ask me all the time like, hey, so tell me how you use Python at work. And I'm like, I don't. And they'd look at me like, what do you mean? I was like, no, seriously, I don't. I work on a— at a .NET shop. But so it goes.

All right. Right. Very cool. So you did mention you're more of a manager now, but you know, we're going to dive into this later. You just released a course on Playwright on LinkedIn. How do you keep yourself up to speed then? Because it seems like you're, you're really still in the technical weeds still, but how are you able to balance that between being a manager and still being technical?

Great question. It's tough, right? Um, I think it really starts with a a true passion for the technical side. I still love to code. I think these new AI tools are heckin' phenomenal. It is the best time ever to be a builder. And that desire is within me. It's not something that I have co-opted. It's not something that I pretend. It has been within me, you know, since 9th grade in high school when I first picked up a TI-83 Plus and started programming that sucker, right? That spark has never gone out. And now that I'm a manager, it's not my day-to-day job to be in the middle of code, to be grinding like that. And so I need to find ways to keep myself sharp. And so whether that's doing code reviews with my team, whether that's picking up some smaller tickets on the side that, um, that I can, I can easily crush, that's not in any critical paths, not blocking anybody or whether that's side projects on my own, right? For example, I just, I just built a new small little web app. Uh, it's, it's called— if you go to autopandamotive.com, it's a spin on automationpanda.com. autopandamotive.com. I made a little static site to show people my cars because, as you know, Joe, I've got some classic Volkswagens. I also have a cool Mercedes now that I've been working on. And so I wanted a quick, convenient way to show people, here are my cars, here are some cool pictures, and here are their stories. So I built a Node.js-based app. It uses Astro for the static site generator. It's all server-side rendered too, which is really nice. And I use JS Deliver for the CDN, so all my photos are sent through for free. The only money— the site is hosted on GitHub Pages. The only thing I paid for was the $10 a year for the domain name. Hook it all up. And so I, I used that project specifically to learn the Superpowers, um, uh, toolkit, uh, Superpowers, the AI skill set to help you brainstorm and develop through a process. And I found it to be really helpful because like I'm— I would not identify as a web developer. Like I'm not good about twiddling CSS and making it look good, but what I could do with Superpowers is I could have it like kind of build out, uh, different designs and show me, and I could pick, do you like A, B, or C, right? And so doing all these fun kinds of projects on the side also helped me keep my skills sharp even when my day-to-day is people management.

Love it. So, you know, you think with AI you'd have less to do, uh, you wouldn't be learning as much, but it sounds like almost you're using AI to empower yourself to learn more, to do a web, a web from scratch almost, right?

Yeah, there is this fallacy that AI is going to reduce the amount of time we have to work or reduce the effort we have to work. On the contrary, like, what I have found is that, um, it is a force multiplier, but when you do use it right, it really takes a lot of focus. It takes a lot of mental energy and capacity. Such that by the end of the day you end up being more mentally tired than you were before. And it's because there, there's so much more detail that's flying. You— there's almost an art to deciding what you don't have to worry about and what you do. And because it all comes so fast, and it's not just you, everyone else on your team is doing it too. And so you're now moving at a much faster pace. You're not walking down the sidewalk anymore, you're driving down the freeway. Do you look at every single little tree? Maybe not, but you look for the signs Right. It's that kind of shifted mentality. And so you need to be more alert, more engaged, um, more focused.

All right. So I don't mean to keep hopping on this. I'm just curious, being a manager now though, um, how do you know what's really being produced by your, uh, people that report to you? I don't know if that makes sense. Like, how do you know, like, thinking's still being involved or someone's just not phoning it in as a, like, how do you know what's quality or real or what? People really doing nowadays?

Yeah, the— that's a people problem. Yeah, because some people are still highly engaged, using their full mind and thinking through the problems and using AI where it's appropriate. But there's other people that just put it into Claude and push AI slop, right?

Uh, it's—

sometimes I'm able to deduce myself when something doesn't seem quite right, and yep, yep, you push on it a bit. Uh, sometimes you, you rely on other people you trust to kind of give a once-over and be like, this didn't— something seems wrong about this, what do you think? You know, sometimes you gotta, uh, double-check yourself. Um, being a manager, I do have the powers to be able to push back and be like, this ain't it, this is not good enough, you need to try harder in these ways. And, and not in a punitive sense, but in a a coaching sense. Thankfully, I will say that pretty much everyone on my team— I, I have a team of 5 reports, um, they're all pretty good. Like, they're, they're not fresh out of college, they're all more senior-level folks, um, they are, and they're all driven to do the right things too. So it's not like I have a problem person that I need to keep smacking upside the head to be like, no, think for yourself. But, um, what I'm finding is that the tolerance for AI slop is very low. And so if you try to sling that, you build a culture of— if you build a culture where you don't like AI slop, right, then it can be self-correcting to catch when it leaks out and it happens. Because sometimes it's not malicious, it's not lazy, it's just like, oh shoot, I didn't look into this one area of the spec that I was generating, and next thing you know it went off the rails and hey, maybe you gotta check that before you, before it wrecks you, that kind of thing.

Yep, for sure. So, uh, uh, once again, as a manager, is your team getting more done with AI that you don't, that you could reduce headcount or is it like you said, it just gives you more stuff you have to do?

It gives us more stuff we have to do. Um, yeah, I, the, the way I look at it is that I have an expectation that people are going to work full-time jobs. Right? You know, order of magnitude, 40 hours a week, right? I expect people to be optimally performing for the time that they're on company time. And so with AI, that means that everybody should be able to get a whole lot more done, right? I've had managers in the past who have said things like, well, I don't care how much time you spend to get done your, your stuff. If, if you can get everything done you need a week in 20 hours a week, so be I don't quite have that view. I think that was more old school. It's a bit flippant to be like, manage your time. But if you take that perspective, it's like, well, then, then our company and team are never going to advance because everybody else is using these tools and moving a whole lot faster too, right? So it's not that we're going to reduce the hours we work or that we're going to reduce the headcount. I see this as an optimization opportunity, the likes of which, Joe, I don't think you and I have ever seen in our working careers. It's, it's wild, you know, uh, this, this coming quarter at, at my team, like, like my company's small, we got 30 people-ish. So my, my team of 6, which includes me, we are the entirety of product and engineering, right? We own both of those functions. This Q3 of 2026, we are going to have the, the biggest releases in terms of the number of features we're pushing out. Than I have, I have ever witnessed at this company. I've been there full-time for 3 years now. I, 3, 3 days from now is my 3-year anniversary as a full-timer, but I'd been working with this company going back to like 2018, right? This is the biggest sets of releases we've ever had, and it's thanks to A, the team being awesome.

Yep.

But B, being superpowered and supercharged by AI coding tools. We're able to get done so much more.

Love it. How do you know then that shipping that velocity, these big features, that it is actually actual quality? Because I assume it's not a team of testers, it's developers. How do you then build in that mindset? Like, okay, yeah, we're able to create this code faster, but how are we verifying it?

Excellent question. All the traditional ways, also baked into AI. That's the short answer. What I really, really like with the state of AI coding tools right now. It's, it's not just that Claude is going to poop out a bunch of code, but when you have the right, like, skills and process in place— when I say skills, not, I'm not talking about like the, the human people skills we have, but like the, the AI coding agent skills.md files, right? If you pull in something like, uh, Spec Kit, for example, Or I mentioned Superpowers before as well, right? I— we use both of those on the team for different purposes, but particularly let's focus on Spec Kit for, for an example right now. Um, when you do your, your thinking and your product management pieces for first, and then moving on to your engineering design in conjunction with the tool like that, it catches a lot of the extra things for you. Right? Um, things like, oh, you should be automating your tests, right? Automate them together at the same time with the code that you're, you're writing. Things like, hey, have you thought about things like accessibility? Have you thought about things like create— putting this in a pipeline, right? Um, it's the— it's leveraging the AI to think of all these extra cases that when you as the human are almost entirely focused on just the behaviors you're trying to develop, there's a bunch of these extra technical concerns you should also be thinking of too. We, we used to promote people to be senior and staff and principal level engineers because they were the ones who thought about the things nobody else did. Now the AI is catching it and thinking about it for us too, right? And as we're, we're refining the designs and the, the specifications using a tool like that too, because you start with that before you do your code, which we should have been doing before, but that was too much work. Now we have tools to help, right? But when you're doing that, all of a sudden, like, the AI is able to catch these kinds of problematic issues and raise them to you to be like, hey, have you thought of this? Hey, you know, these two things, uh, don't quite align right. How would you like to proceed? Option A, B, or C, right? And then through all of that, you're basically building better quality into the product from the beginning. And you know, when, when you have all your specs written and you go to say, okay, let's implement the code, one thread can go implement the product code, the other thread can go implement all the tests, unit integration, end-to-end. And so you can have that together. And so you can review the spec, review the code, review the test, get it running, make sure it's good, and then boom, then you're protected moving into the future. That's how you can really— it's, it's really the only way you would keep up, because if you're trying to push, push, um, product code at the speed of AI, you can't have another separate test team doing— trying to do n+1 sprint automation or something. Not going to happen.

But so if you have an avalanche of code, you must have an avalanche of tests. How do you know what tests really are really needed then? Because you can't have— I know you did a session on risk-based, basically no longer pyramids, but more skyscrapers. Is there a certain approach where you make sure the tests that are generated rather than just a bunch of tests just to generate for AI's sake of really testing?

Oh, that's a great question. If you just let raw AI go to the hilt, You're gonna get a lot of good tests, but you're gonna get some that perhaps aren't worth it. And when I say not worth it, it's not that they're like a bad test, but it's like you have X amount of hours in the day to run your test, X amount of dollars to, to pay for the compute resources. Do you really need to check that the, the, the button is yellow? That kind of stuff. So part of it is still human review, right? What, what we're finding is that people are spending more time doing reviews. And less time like doing the hands-on coding, which is fair because you're writing spec, the code just happens as a side effect, and then, you know, you have to use your human intellect to actually review and make sure that things are good. Uh, the other thing that's really nice, um, and again, this is why I've become a huge proponent of Spectrum development and of using these kinds of skills and, and building out the processes you see, you can, you can codify the guidance that you want for your testing along with your project, right? Um, because if, like I said, you could, you could ask Claude to do something and it'll just go puke on the screen, right? But you say, hey Claude, um, don't worry so much about like testing visuals or stuff. Thanks, Pandy, I'll keep that in mind for next time. Save it as a skill, it's in there, it's baked in every single time. So that way you know, like, like, um, for another example, I, my wife runs a dance studio. I think I've mentioned this before. Uh, DIA Dance Studio in Durham, if you happen to be in my area, right? North Carolina. She's wonderful. She's awesome.

It's great.

Uh, she has had to cobble together a bunch of different SaaS solutions to run the operations of her studio. Everything from like scheduling classes to soliciting payments, right? And she's like, hey Andy, can you use those AI superpowers to build me an app so that I don't have to pay hundreds of dollars a year for this? And I'm like, actually, yes, I can. Before, the answer was yes, but it's not worth it. With AI now, it's Yes, I can. And so like, I've set up a project for a web app that I'm building for her that I work nights and weekends when I have a chance.

It's—

we're only just getting started here, but one of the things I did is I, I set it up with Spec Kit. And you better believe it, in the project constitution, I was very meticulous about the testing strategy I was going to enact for this, right? We're going to have, you know, in a thorough test coverage, unit integration, end-to-end level. We're going to use Playwright as the end-to-end solution, right? We're going to make sure that we're not duplicating getting tested differently. I mean, I had a whole spell out markdown for all the kinds of things, like, you know, um, writing your acceptance criteria in Gherkin and stuff, you know. So you know me, I'm picky about that, right? But it's like all of these things you can specify once and then it's baked in so that you, you— there's a much less risk of you constantly having to check every single pull request that it did it right. You know, versus like you have a college intern, you tell him or her to do something, okay, then they fix it in that PR, the next PR, same issue comes back, third PR, same, and then you're like, why did I hire this person? Right? Those kinds of issues go away when you bake it into your context and your skills. So it's just part of the whole system, right? And so that you're perpetually finding new issues and eventually those run out and then you can finally focus on the behaviors you're trying to build and not all the technical details that should just be taken care of for you.

So Andy, it sounds like you're using AI for a lot of things, but as you mentioned, how about, how about cost? I know I've been seeing a lot of companies talking about token cost. Have you been seeing that? Is there a way, a best practice that you found that maybe, hey, maybe you don't need AI for this, or if we do AI for this, maybe have the context smaller than what you're using? Mm-hmm.

So thankfully for my personal projects, the $20, $25 a month subscriptions to Cursor and Claude have been good enough, right? I'll pay that out of pocket. You know, I've got my automation, I got the business expense for my own personal stuff, right? Like, like my wife even has a Claude account that she pays for through her dance studio. Cause think about it. She's, she's got to write rental agreements and all the paperwork and stuff. I remember one day she texted me, Joe, it was the most amazing day. I was at lunch pizza with my friends. And she said, Andy, I hit my Claude limit.

And I was like, whoa, success, right?

So she had to wait like 2 or 3 hours for it to reset or something. But I'm just like, dang, like my non-tech program, like she's doing a dance studio and she's hitting her Claude limit, right? But like we've been okay with like the $20 a month. It's like another monthly bill, right? We business expense this Studio, whatever. On my team, um, thankfully we haven't had any runaway costs yet. We have a— almost everyone on my team uses Cloud Code. I'm the one person who still uses Cursor because I'm weird and different, right? But like on, on the team, and I have both of them, Cloud and Cursor, um, like there we, we have like more than half the company has an account. And certain people— we have a team account, so it's all together. Uh, some people are okay with the standard subscription, which is like $20 a month or something. Uh, our power users, e.g. my team, many of them are on the premium, which gives you the, the increased limits, the increased speed, because literally they've got like 3 Claude terminals grinding code on different things at any given time, right? They've got it synced on their app where they're sending prompts while they're grocery shopping and stuff. It's wild. Um, so it's like the, the, the costs are roughly expected. Um, there haven't been any overage costs that we've incurred so far. Um, but I mean, there have been times where certain people have hit certain limits and be like, ah, shoot, I gotta stop for a bit. Um, in those cases, like, okay, if it happens 3 times, bump you up, right? So it's, it's been manageable.

Nice.

I fear when all the subsidies run out and companies start charging what it actually costs, then that could be tough. But we're not there yet, so we're going to get as much progress as we can now.

Absolutely. So, uh, yeah, how do you know that you're not losing company, uh, domain knowledge this way? Oh, like over-relying on the AI where before maybe you would need— is it just like you're just now a person that manages MD files and As long as you have those, doesn't really matter about being a domain expert?

Well, I have a few thoughts on that. I can't— I don't want to speak to like more entry or intermediate level folks because that's a whole question mark problem, right? But for more senior and experienced dev folks, who, like you and me, we've been through the trenches, we fought battles, we've had to learn things the hard way, right? Um, those lessons learned, that experience, that domain knowledge doesn't go away or get invalidated. I feel like it truly gets accentuated by AI. Um, I, I know, like, we're 3 years into this AI revolution and the coding assistants have only been really good for about a year. I don't feel like I've lost anything in terms of my knowledge or understanding, but rather I've gained because like, I feel like there's so much more that I can do now. And, and I don't have the hesitations I used to have, right? You know, like I mentioned this, this web app for my wife's dance studio. I could have figured that out like 5 years ago, but that would've been a major, major project. I would've had to get some books and really dedicate a lot of time and learn some things, right? Because it's like, I, I can figure this out, but that's not my day-to-day. I'm not a web developer. Now it's just like, yeah, Spec Kit and, uh, React on the front end, Express on the back end, SQLite for now.

Go, go, go!

Pew pew pew pew pew! And then at night I'm like, here, here's, uh, version 0.1, honey. What do you think so far? Wow, make this like this. Okay, you got it, girl, right? Um, And so like, I understand principles of good software development. I understand web application architecture. I understand good testing practices. And so like, I can bake all of that into a system, set it and forget it, and then just new features, right? So I don't feel like I'm losing domain knowledge or expertise in that of anything. I'm simply codifying it and I can refer back to it if I happen to you know, need to check my notes, right? Um, I would also posit this, um, do we ever open assembler code and read it today, right? Do we, after we compile something, you know, do we, do we crack open that, that executable, read the assembler code, and be like, aha, I see what it's doing? No, we literally blindly trust what comes out of compiler, right? Um, that wasn't always the case 40 years ago, right? People would still be going into assembler trying to eke out little, little efficiencies. But now it's like, no, no way, right? I see these AI coding tools as the new compiler. I see Markdown as the new programming language, right? And if you start to think about it like that, does it really matter at the code level what is generated or that it works and it works well enough?

Oh, that's controversial because people like code quality, especially with testers. They care about, you know, the things we grew up on— atomic tests, page objects, treat automation like a product. What language should I use? Like, who cares now? But is that the wrong way to go? Like, if it's working, It's doing what I need. It's doing it in a way that's fast enough. What do I care what it looks like?

Isn't that what business always cared about anyway? See, wasn't that always the business perspective?

That was always the case, but we were so caught up in, I think, the development aspects of the, like, the mechanics of it rather than, like you said, the business that we should probably always be focusing on. I don't know.

Yeah. So business people never cared about the tech details because they were like, push features, make money. We as technical people cared about the tech details because we were humans actively working in those spaces. And so we had to make it good for ourselves to use, right? If we've taken away that and we're no longer working at the level of the code, much like we were no longer working at the level of assembler, if we're now working at the level of spec, we're working at the level of skills, we're working at the level of Markdown because Markdown is magic. How much should we care about code quality? We should care about efficiency, right? How fast does it run? We should care about bugs, security vulnerabilities, right? But at the end of the day, is it worth waging holy wars on, on your company's formatting standards? Right? Maybe, maybe this doesn't matter as much. Now, I know you said this is very controversial. Yes. I'm, I'm— today, where the puck is, highly controversial. Where the puck is going, I think, I think in 10 years, absolutely, this, this would be a no-brainer. Like, but we're still early in this revolution. We're still early in this transition. And so, um, and, and the fact is the, the, the AI capabilities and the AI wisdom is not evenly dispersed amongst us as a whole technical community, right? There are, there are still people working in places who, whose companies do not let them touch these tools.

That's true.

Who haven't had as much exposure. They've heard about the wonders of Cursor, but they've never actually cursed at it, right? You know, um, and, and so like, especially in big enterprise— my company, Target Supply Chain, guess what? Guess who, right? You know, we just had a a customer advisory board and almost half of them say, yeah, my company won't let me use these kinds of tools. And I'm like, oh my gosh. Right. And you're like, but it's 2026. It's like, we're still only 3 years into the revolution. Oh, we still got a long way to go. And so I think the, the fact that what I said right there is still controversial is because not enough people have gone through and it hasn't been accepted as standard enough yet. And to be fair, You know, the AI coding assistant tools are now just good enough that we can have that conversation, but they're not good enough yet that it's universally accepted. That's still gonna take more time too. I do believe we will get there. I believe it's inevitable because look at compilers, right? Right. It's the, the AI building technology is too good. It is too good. That's why we're building data centers around the world. Like it's too good. And anybody who has touched it and known it and been supercharged by it knows, right? It's— there's no going back. There's no going back.

So like I said, controversial. Even testers that use it sometimes will have a different point of view. But what I thought is interesting, a lot of people said, oh, courses are dead now. Uh, thought— being a thought leader is dead. But I noticed you, you— this was the whole show. This is supposed to be about this, but you released a new, uh, a new, uh, course on LinkedIn about, uh, Playwright with AI. Like, and it's pretty extensive. Like, why couldn't it just be like open cursor, uh, write me a test? Like, what type of skills, or why did you devote so much time to play around with AI if people think AI is going to replace them or you can teach them everything without putting the effort in, I guess? Yep. Yep.

So what, what we as humans have to learn now are, are lessons, tools, and techniques at a higher level, right? We, it's less important to know how to write the most perfect XPath. It's more important to know how to leverage these tools and technologies as part of your quality strategy, right? Because that part's not going away. Remember, software is ultimately for people, right? There, there's no, no world in which software is going to be in a hermetically sealed bubble, never, never operated by people at all, right? We build applications to solve our problems, right? We still live. We are endowed with souls, right? It, it, that doesn't go away. And so like, okay, Maybe I don't need to worry about, like, you know, like I said, little minute details in the code, but I still need to worry about risk mitigation, quality management, building a project that is sustainable, right? That doesn't go away. And that's more what I wanted to focus on in my new LinkedIn Learning course, which is, uh, I forget the exact title because it stretches off the page, but it's basically Playwright with AI, you know, agents, contexts, and test automation patterns, smart web testing scripts with Playwright and AI.

Amazing.

Yes.

Yes.

So what I cover in that course is, is how you can use Playwright with, with AI coding agents leveraging the, the vast amount of AI tooling that Playwright has offered. What Playwright's strategy has been like originally, they were, they were meant to be a web testing tool, arguably competing against Cypress and Selenium.

Yeah, right.

In, in the days before AI, right? Their, their whole pitch was, you know, smarter, faster, stronger, better, whatever the Daft Punk stuff is, right? And they were execution-wise the fastest. They had, in my opinion, the best developer experience.

Um, again, controversial, and probably not anymore, but, you know, if you said a few years ago, probably.

Yeah, yeah. But like, it was, it was meant to be something that was easy for developers and testers alike to pick up and run and get going. And that's— I fell in love with that. I'm like, this, this is great, I love this, right? Um, then when the, the AI revolution came, Playwright wasn't trying to build their own models or anything, right? They weren't trying to build a whole platform around it. What they realized is, hey, let's make Playwright extensible. Let's make it usable by AI coding agents, right? Well, how do we do that? Well, they started with like an MCP server. Hey, you can have an MCP server that exposes all of the web interactions that Playwright does. Now if you have, say, Claude or Cursor, you can direct it to open a browser and explore and learn and do stuff. It became the ultimate RPA tool. If you think about robotic process automation, all you need is a coding agent plus Playwright MCP. I mean, the things you could do are amazing. Like, get this, you could, you could make it explore your app, exercise the instructions of your user manual to verify that your user manual instructions are good, and then provide suggestions. Or you could have it go, you know, clip by clip through your app and capture screenshots to make sure your user manual screenshots are up to date. That has nothing to do with like traditional test automation, But it's, it's the kind of thing you can do now with, with Playwright and AI. Just like, oh, this is beyond testing, this is pure automation of my tasks. Wow, right? So they did that, then they added a whole bunch of, um, of agents or sub-agents, right? There's basically three, three big ones: Planner, Generator, Healer. Planner, hey, let me suck in any specs you have, any information you have, any context, or hey, maybe I can kick open a browser and explore your app and I'll come up with the, the reasoning behind like test plans that you should— or tests that you should be writing, right? And so it'll generate test plans as Markdown and you can review them and, you know, you can whack some tests, you can tell it to refine, but it's like it, it speeds up, you know, your, your test development process because now it's like, oh, here's a plan I can use. Generator turns those plans into actual Playwright test scripts. You know, okay, you know what, now let's do how, right? Goes through all of that. And then finally, healer is you've been running them and something broke, something changed in your app, your locator was flaky. Let's figure that out, read the, the trace log of errors, and then implement the change, run it, and make sure it all works together. Um, planner, generator, healer very much mirror, um, the BDD process of discovery, formulation, automation.

Mm-hmm.

Of course I eat this up. I'm just like, ah, this is genius.

Right?

So they, they did that. So not only do you have the MCP, but you also have these agents to guide, right? And so like you could go to just, just vanilla AI and be like, gimme a playwright test and it could poop out a script based on whatever it's sucked into its frontier model. But in terms of making sure that's actually a good test and that is actually a reliable test, that's where the MCP and the agents really come in because they help guide you and they go through all the iterations and make sure that it's actually something worthwhile. And then finally, the third thing, um, uh, you know, they, they did, they did the MCP path, but then about half a year to a year ago, uh, the whole world lit a buzz with skills and CLI because that was going to be so much more efficient, right? MCP is heavyweight, eats all the tokens, let's do CLI with skills. And so they did, they made a whole Playwright CLI and they added a bunch of skills files that basically do the same things. And so now you can choose which way you want to go. So if you're a tester or developer just on your local laptop you probably want to use CLI with Skills because of the token crunch is lower. If you're trying to embed Playwright into some agentic ecosystem and all this kind of stuff, you probably still want to use MCP. But it's like they gave all the tooling and said, you bring your own agent, we'll work with it. And now boom, it's like, this is amazing, you know? So that's what the course really shows you, how to, how to use all those tools, bring your own agent, and then make amazing things happen.

And I highly recommend everyone check it out. We'll have a link for it down below, but I think it's like 4 hours long. And I don't know how that works, Andy. I have a premium subscription to LinkedIn, but is it a paid— is it— who gets access to this?

Uh, goodness, I'm, I'm not entirely sure of the details. I think like, um, I think you— there is payment that you have to give, but I think if you have like a LinkedIn Premium or if your company has like a LinkedIn Learning subscription, you can get in on that.

Nice.

Um, I think I'm able to give some some like, like discount or free links or something. Um, I'll be honest, all the, the payment details, it's all LinkedIn.

Yeah, right.

Um, yeah, they have an amazing process for soliciting instructors and running them through a program and everything. It's— it was an amazing experience developing the course with LinkedIn Learning. A+ world class, would recommend. If you're, if you're an instructor out there and they invite you, say yes.

All right, Andy, before we go, any actual, actual piece of advice you can give to someone to help them with their AI testing efforts? And what's the best way to find or contact you?

Sure. So best advice I can give to testers out there trying to get started with AI, just do it, right? Um, if you're not sure, ask the AI how to help you through, right? There's no, there's no barrier to entry. There's no reason to be stuck. Right? Ask questions, Socratic method. Go ask your AI. Go ask people like me and Joe. Get involved in your local community. Um, now is the best time to get into software development and testing, right? I, I'm— I don't subscribe to doom and gloom. I don't think all of our jobs are going away. I see this as a true revolution on the scale of the Industrial Revolution for what it's going to unlock for humanity. And we are, we are on the front lines as the builders. And it's, it's going to be bumpy. There's going to be, um, issues to solve along the way, but ultimately, like, now is the opportunity. If, if you had any doubts, strike now while it's hot. So if you want to contact me, if you just Google Automation Panda, you will find me. My name is Andrew Knight. Um, I am currently Senior Director of Product and Engineering at CycleLabs. I am very active on LinkedIn. I used to be active on X. I'm still kind of out there, just don't post much. My website is automationpanda.com. If you want to see cool things about cars, go to autopandamotive.com.

And you can find links to all this awesomeness down below. Thanks again for your automation awesomeness. The links to everything of value we covered in this episode, head on over to testguild.com/a599. And if the show has helped you in any way, why not rate it and review it in iTunes. Reviews really help in the rankings of the show, and I read each and every one of them. So that's it for this episode of the TestGuild Automation Podcast. I'm Joe, and my mission is to help you succeed with creating end-to-end full-stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms. And over 40,000 email subscribers who are at the forefront of automation, testing, and DevOps. If you haven't yet, join our vibrant community at TestGuild.com, where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real-world challenges, we're excited To collaborate, visit testguild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve. Oh, the Test Guild Automation Testing Podcast. With lutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread like wildfire, through the land, guiding testers, showing us secrets to behold.