Combining AI and Playwright using Autify Genesis with Ryo Chikazawa

In a land of testers far and widely journey seeking answers, seeking skills, seeking a better way. O the Test Illumination Testing Podcast Guiding testers with our nation awesomeness. From ancient realms to modern days, they lead the way. Oh the Testicle Automation Testing Podcast.

Want to know more about the power of AI Enhanced Playwright scripts? Well, you're in for a treat because today our expert guest is Rio, the visionary behind the AI automation testing solution Autofy Genesis. So in this episode we'll dive deep into the innovative AI driven solution that Audify is bringing to the world of software testing. We explore how their integration with Playwright makes it uber flexible for both technical and and non technical users. Real will share insights on his journey from a no code solution to their latest product which leverages gen AI to bridge the gap in the testing lifecycle. We'll also discuss the powerful feature of their AI agent which can generate Playwright scripts, interact with applications autonomously and adapt to changing requirements with ease, all while ensuring user avoids a lock in with a vendor. We'll discuss the powerful features of their AI agent which can generate Playwright scripts, interact with applications autonomously and adapt to changing requirements with ease, all while ensuring the user avoids a vendor lock in. So tune in as we explore how Autify is making strides with local and cloud based solutions in the importance of AI in helping with software development and testing and also what sets their offering apart in this ever growing space. So don't miss this episode packed with awesome revelations and practical insights for anyone involved in QA and software testing. You don't want to miss it. Check it out. And to actually see this all in action, be sure to check out the webinar replay of the session Rio did with us a few weeks ago on our Test Guild webinar series on how this all works and you could see it for yourself, so don't miss it. You can check that out using the link down below. Hey Rio, welcome to the Guild.

Hi Joe, great to have you.

Really excited about this. I'm always excited when I learn about new solutions, especially around AI. So the first question I always ask founders is probably the first thing is how did you get into AI first off? And then we'll dive into what the solution is.

Yeah, sure. So OTFI actually has started since 2019, so we've been investing into the AI since the beginning of the company. So the first solution we built at our original flagship product is the odefy no code which is the record and playback the no code local solution so that anyone can automate then AI to maintain the scenario you created by the record and playback. So yeah, based on the UI changes. So that's how we get into the AI in the first place. So like back in the original product we employed the classical like machine learning technique to identify the same element in the page like even though the UI has been changed. So like how we did that was like you know, collecting the all the so called features of the element like IDs, a class name, like positions and colors and like a text parent element, siblings and all that. And then we like calculate the similarity score between those two screens like a different screens and then pick the other code element from there. So that was the adapt. Like you know how we started in terms of the AI. Then from there we eventually gradually like evolved our AI technologies. Like now we implemented the kind of image recognition to identify the element. Like that is actually applied for our mobile solution. So like mobile one of the biggest challenges, you know, finding the you know the right element as well. But you know the mobile sometimes you, it's difficult to access to the source code. So that's why we use the image recognition technique to identify the element in the mobile screen. So we have the visual recognition kind of like the machine learning engine there then. Yeah, so we invested a lot areas in AI. Then now gen comes. So we decided to launch the new product which is the OT5 genesis. Yeah, so the OT5 genesis takes the earlier part of the software testing journey. So which does the test case creation as well as test script generation by understanding the requirement. So requirement can be like anything like a PDF word or spreadsheet or figma, Jira ticket, whatever like you write as a requirement can be taken by the our, you know, the Genesis Geni engine that can create the test cases with a wide range of coverage. Then also we build the like AI agent that actually accesses to your application to like generally like interact with your application and then generate the playlist script in real time. So one of the biggest challenges of like generating the test case or like the play, I would say playlist script by just making use of the ChatGPT or like just other LLM is it can create a very nice look kind of playlist script. It looks works but it doesn't because it doesn't know your, the structure of your application. So what we did is implemented the AI agent to you know, actually interact with your site and then it analyzes the structure of the pages and you know, get the accolade element locator and so on. So like with that we could, we now can create the, the very, very accurate Playline script because actually interacted with the application and then we actually run the, the generated predi script at the same time. So the, the outcome is actually very accurate. It runs. Yeah. So anyways, that is kind of like, you know, a long story short. This is the. Our kind of AI journey since the beginning.

So interesting. I want to dive in a little bit more on how this all works. But the first thing is you already had a solution in 2019 before. I mean that's about when gen AI started to take off. But obviously you were developing before then. How did you know AI was going to be as big as it was going to be? Like how did you see that then? Because it seemed like you got in right before everyone else jumped in. Slap on AI. You started AI first way before other companies started coming into that space.

Yeah. So actually since the beginning we truly believe in AI. We think AI is going to be definitely revolutionized software testing as well as software like development itself. So actually at the time of starting Odify, there's some reports, I think the report is from Gartner, the Gartner repo mentions that I forgot the detail but maybe we can paste it on somewhere on the note maybe after that.

But sure, yeah, we'll have in the comments down below.

Yeah, so it says like.

In few years maybe like a 2025 or something like a lot of companies going to employ AI engineers. So AI engineer means like AI's right. India that they wrote a software development teams. So then when I read that I thought like yeah, this should be the future. Right. So the AI should definitely evolve in next five to 10 years then that's going to definitely increase the productivity. And then I think it's not going to replace our job, it's more like helping us to make us more productive. So that was the kind of the vision, like our vision from the beginning. So we really truly believe in AI is going to definitely eventually revolutionize the testing and then software development then. Actually our first idea is something like OT5 Genesis. Like yeah, we kind of tried to like generalize the test case from like you know, the bdd Garkin. Yeah. But we actually failed because you know, it was too early. Like the technology was not really there. So that's why we decided to pivot it into like a record and playback type of classical approach plus AI. So that was more realistic solution back then. So yeah, that's why. But you know, still AI then you know that the more we invest into the AI, the more we believe that oh like you know, the AI now can capable of understanding the, the element from the other screenshot or can maintain you know, the scripting the scenario by the UI changes. So yeah, the more we do, the more we believe. Then now finally gen AI comes and then I was like yeah, this is definitely the future we envisioned from the beginning.

So I'm just a little confused. The original solution seemed like it was using image based AI or machine learning to figure out everything in a no coded way. And that to me seems like it would be especially now with like multimodal AI the way forward. So why create a new. You make a clear distinction now between that and modify genesis. Why is that?

Yeah, so the first of all for the original product employ the, the image recognition techniques as well as the, the classical, I mean this is classical and the machine learning approach by the way. But we also like use the HTML like you know, both sides like HTML to extract the features then also understand the image to like. So we employ those two techniques, use those together, use both approach to identify the element. So but yeah, to answer to your question, yeah, so why we started, right, because like there's a few reasons. So first of all like what no code, low code product. What our original product can solve is kind of limited in the testing life cycle. So that's the kind of later part of the testing cycle. Like you know, if you know what to automate, you can easily use record and playback to start creating the automated scenario. So but like you know, like we've been supporting a lot of different types of customers and what we found is lots of people is like spending their time on like creating the test case as well as designing the you know, automated test case and so on. So I do see the like larger problem is actually outside of that scope that we've been solving. So like what I realized is like we need to capture that, we need to support, we need to help the customers to solve that earlier part of this problem as well so that we can provide you the whole value chain for the entire testing life cycle. So that's why we started thinking about like you know, started brainstorming like how we can solve that you know, earlier state challenges like you know, the other test case creation and so on. Yeah, then you know, LLM comes in and I was like, yeah, so this is actually like you know, the very good solution, the problem to be solved by Genai because you know, understand the requirement and then generating the test case as well as generating the Tesco. This is actually like you know, what the Other gen AI is really good at. So like you know the piece get together got together for me then yeah. So let's employ Genai to tackle this like you know, test case creation problem then. Yeah, build yet a whole platform to cover the entire problems in the testing life cycle.

Awesome. So how does artify Genesis work? It sounds like I can feed it figma file. I could feed it requirement file. Someone else gave me like all these files. It's almost like I'm training it on my own context based LLM that it then will understand what it needs to be done to test it.

That's right, that's right. So it's very important to let our AI agent understand the context of your application. So not only just putting the requirement but also if you have any other like a supplemental documentation or like a supplemental information, you should definitely feed it. Like you know what we did for our internal use case was like not only just put it our PRD but also like put it at our entire help center documentation. So the our helpers and the documentation has the old information about the our application's behavior. So like with that like if you have that kind of like you know help help doc or user manual or something like that, that is going to be also the great input. So you put all that then the hour AI try to create the other test cases based on that like a specific requirement. Yeah, so that's basically how it works. And first of all general idea the list of the test cases and then you review it. Then after that it general idea the garking scenarios. So we decided to go with the garking format because the garking format is kind of widely used then you know, it's well known format. Then like how I like Garken is Garken is very abstract like you need to write it a garking scenario in an abstract way. Right. So you basically don't really put yet a lot of granular information like how you log in or what kind of credential is going to be used or something like that. So because one of the biggest challenge of the end to end test is the maintainability. Right. So the more grlanya you go then the more difficult the maintenance becomes. So that's why we decided to go with garking in terms of maintainability. So like kind of higher layer. Right. Then so to let you regenerate the test script if something changes on the UI side. Right. So one of the biggest challenges of the end to end every time the UI changes or like a page Changes or some requirement changes. Like although the other whole, the higher level operation won't be changed, a lot of detail can be changed. So why we decided to go with Garken is Garken can be maintained in the higher layer, the abstract way so that you can regenerate the playlist script. Like when the requirement, when some detail changes. Yeah, so that's basically how we create the test cases first of all and then the latter part is going to be the Playlite script generation. But yeah, I think I can pause here then if you have any questions I can answer.

Yeah, so say the requirement does change. How, how does it know? Is it plugged into to your code base? Do you know? Is there a process that developers need to start feeding in what they're going to do for the Sprint or something? Like how does it know it changed? And it really did change.

Yeah, yeah. So like our approach is not really approaches to not to access to your code repository because like we go with the black box testing, we basically rely on the other requirement from. From you. Yeah, so like whenever the requirement like the spec spec is changed, you upload the new requirement. Yeah. If the figma is changed like you know we are now building the integration to figma so like you know, when the something changed we can automatically feed it to the our model to regenerate the other test cases and also we building the integration to JIRA so when like a new requirement is created or the, the the JIRA ticket has been updated and we can automatically feed it. Right. So anyways like you give the recommend again so that we can understand the changes. You can understand the changes. Then you can regenerate the other test cases that will be affected and by.

Doing so it regenerates the test then so a developer doesn't need to oh, I need to change this ID and this function because it will. Or there's a new page with this flow, it automatically would know that and update all the existing tests.

That's right. That's right. So what we generate is as I mentioned, is a higher layer. Right, Higher layer of the garking scenario. Then with that garking definition our AI agent actually interact with your application then by following the other garkening scenarios. Right. So if the other garkening scenario says like log into the system, then the agent tried to figure out a way to log into your site. So you don't really need to put a lot of details in the garking because the AI agent can figure it out. Yes. While you can maintain the abstract Way in the garking scenario, the AI agent can figure out the details. So you don't really need to do a lot of detailed ground up time consuming work. So AI to figure out. Then you can also break the garcking, step down into the very actionable executable steps while interact with your application and they figure out the operation. Then yeah, create the playlist script. Then that you know what you need to do is like a reviewing the other what the AI agent does. So it's kind of autopilot like when you start a session, like the session kicks in, then the AI agent started accessing, interacting with your site and then try to figure out the other way to log in. Then while do that it also creates the play that script in the screen. Right. Then you can just monitor how our AI agent figure out a way to log in. Then if something goes wrong and you can just pause the session, then give the agent another like input to fix the operation. So you know, it's very interactive. So we have the chat window on the left side of application where you can communicate with the AI agent so you can tell the. So sometimes the AI agent is going to ask you like you know, how to log in, like you know, what's the other user's credential or oh, like I couldn't actually pick the right element from the screen. Could you like specify on behalf of me? So like you know, AI agent is actually going to ask you. Yeah. Then you give the input to the agent so that you know, it can fill in the, you know, the gap. Then create the oculate playwright script. So that's how it works.

Gotcha. So you mentioned Playwright a bunch of times. So you start off as a node code company. Why are you generating Playwright code then? Yeah, what is, how much does people need to know about coding? Why are you using this approach?

Yeah, yeah, yeah, great question. Thanks a lot. So first of all, like I was pretty amazed by Playlite. I mean I used to be the software developer by myself, so I write the Playlite script by myself too. And then I was pretty amazed. Playlite is like very amazing comparing to the like the other old solutions. Yeah, it's pretty fast, very light and then lots of great element locators and so on and so forth. Very stable. It can automatically wait until the element gets rolled or so so on, so forth. So the Playlite is like very good. And you know, like now I think like everyone is like really interested in the, the Playlite and then you know, the community has grown a lot so yeah, that's why the Playlite comes then why we are doing this right, as a no code local company. Because like one of the biggest challenge I've been seeing is like, you know, there's a two types of audiences, like you know, the engineers, like a very technical people who writes the script by themselves. So the other side is like, you know, like, let's say manual testers or like, you know, the QA managers who doesn't really write the code. Then like, you know, I do see the kind of like, you know, the gap between those two audiences. And then you know, there's no right solution to fill in the gap. So like, while no code works really well for the people who don't really write the code by themselves, but I do see the like, you know, lots of people writing like let's say Selenium or Cycle and Spreadlight to do the automation as well. So like, you know, we've been kind of thinking about like how we can help the other that kind of the technical side of audiences as well. Then those people think no code local is not their thing because they can code. Then if you go with a no code low code product, then sometimes you can export the scripts or you cannot, you would lose the flexibility. So I was thinking of how we can kind of bridge the gap between those two audiences and then also provide the right solution to the technical audiences as well. Then. Yeah, so now we build it at OT5 Genesis. So which can definitely increase the productivity of those, you know, people who write the end to end script by themselves while giving them the freedom flexibility of the solution. So that's why then. So like what we do is like, you know, if you are not very technical, right. The script, you can still use the no code by connecting with Genesis. So Genesis and Notify, no code is integrated so that while Genesis generally the Playlist script, no code can convert the Playlist script into the no code scenario. So like our new version of no code is going to be based on top of Playlite. So if you are technical, you can like, you don't want to write the lots of, you know, the script like for the, the very basic operations as well. So in that case you can just use the other local no code solution then you know, maintaining the easier way. But if you go with like, you know, the very technical complicated flow, then you can still maintain the Playlite on the back end. So yeah. So anyways, yeah, so that's why we decided to support the playwright.

Does that also mean then because it's generating Playwright that it's not vendor locked. I mean, obviously if someone uses it without your tool, they wouldn't get all the AI, but would it they at least be able to rerun the scripts if it doesn't change or anything like that?

Yeah, that's right. So what we do for otfile Genesis is completely flee from the vendor lock in because we generate the Playlite script. So you can do whatever you want. Yeah, so you can just use the Playdyh script. You can run this anywhere you want. So we are completely free from the advent of lockdown.

Now you mentioned agent a few times. When you're running your test, are you able to run in the cloud? Do you have to have an agent on all the machines that you run on? How does that work?

Yeah, so Odify Genesis is the form of the desktop app. So you install there a desktop app then. So yeah. So this is also like, you know, the other side of the election from the other cloud version. Right. So like the reason why we decided to make this as the other desktop is like, you know, like you can help your. Let's shift left. So you might want to test against the other local host and then something that you're dropping. So I think like, you know, that should be in the form of desktop. So anyways, that is a desktop app. Then agent is going to access to your. Either your local host or staging or production, whatever you specify. Yeah. Then the AI model itself is on the cloud. Yeah, if you have your internal AI model. Of course, you know, we are like AI model agnostic. So you can specify your internal model as well. But yeah, that is on the cloud. But yeah. So the agent works inside of your application or on your desktop app.

Gotcha. And do you have a VS code plugin? Is that different than the desktop?

Yes, we have just implemented the VS code plugin. So inside of the VS code plugin it does the same thing. Yeah. Then it generates the Playlite code to your.

VS code then where you can edit from there.

So there are a lot of players coming out in this exact space. Obviously you had a head start. How would you say that your solutions may be different? Not to say other solutions are bad, but is there like a key differentiator that if someone's listening and they meet these criteria, like this is the tool for you or maybe it's not the right tool for you?

Yeah, so I think there's two aspects. First of all, we generate a playlist script. Some of the players is kind of like no code, low code, plasti the scenario generation, meaning that you still kind of like go with the vendor locking situation and then you cannot, you know, like have the flexibility of the solution. Yeah, so first of all, like, it's very flexible. It generates the playlist script. I think that is one of the uniqueness part. Then the other thing is like we provide the end to end solution from the test script creation to the execution and the maintenance. So if you are interested in that part, you can also employ that solution too. So like, you know, I think we are pretty unique on providing the, the whole end to end support for the entire testing life cycle. Then. Yeah, no code part is also unique. It's. So our new version is the desktop app based on top of Playlight. Then you can still access studio Playlight. Yeah, so like a no code scenario is actually, you know, essentially saying playlist scripts. So yeah, I think that part is also pretty unique.

Nice. And if people are listening and they're trying to visualize it, don't worry. I'm gonna have a link down below where you can see a replay of a webinar we did on Audify Genesis, partnering with Genai to build test cases and playwright scripts. It's a must watch webinar where you can see this all in action and you'll find a link for it down below. All right, Arielle, before we go, is there one piece of actual advice you can give to someone to help them with their AI automation testing efforts? And what's the best way to find or contact you or learn more about Genesis?

Oh yeah, you can access to our website and then like contact us or like you can sign up for the better access for defi Genesis from the website.

Thanks again for your automation awesomeness. For links to everything of value we covered in this episode, head on over to testguild.com a517 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation podcast, Joe. And my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of all automation testing and DevOps. If you haven't yet, join our vibrant community@testguild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit testguild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

With lutes and lyres. The bards began their song A tune of knowledge, a melody of code through the air it spread like wildfire through the the land guiding tester showing the secrets to behold.