INGenious Playwright Studio with Ashish Ghosh

In a land of testers far and widely journey seeking answers, seeking skills, seeking a better way O the Test Illumination Testing Podcast Guiding testers with our nation awesomeness from ancient realms to modern days, they lead the way. Oh, the testicle of nature Testing Podcast.

Foreign.

Hey, do you want to learn how to really take your Playwright Automation to the next level? Well, you're in for a special treat today because we'll be talking with Ashish all about Ingenious Playwright Studio and quality assurance at a large enterprise bank. If you don't know, Ashish is a seasoned test automation architect with over 12 years of experience. He has been instrumental in taking Testing to the next level ING bank, where he leads the Test center of Excellence. I haven't spoken to someone at a Test center of Excellence in a while, so really excited about that as well. Also, he is the creator of Ingenious, a low code test automation framework which I think you're going to get a lot of value from and is really passionate about advocating and sharing his knowledge, which you'll see in this episode. You don't want to miss it. Check it out. Hey Ashish, welcome to the Guild.

Thanks Joe. Thanks for having me.

Really excited about this. I saw this on a few weeks back and had it on my new show. Got a lot, a lot of interest in it. So I guess we'll just start off with no, could you go give maybe an overview on what is ingenious? What are the key problems it's designed to solve?

Yeah, sure. So effectively, when we were looking at the testing problems within ING bank, one of the things that came out was the lack of involvement of business users. Now technically testing is all about business requirements. I cannot just go around testing any random stuff of the software unless I know what the business that the software is supposed to fulfill. Now the problem with coded solutions was that a lot of customer Journey experts, as we call them in ing, did not find a lot of interest. They could not contribute, they could not collaborate towards creating good quality tests, automated tests. The other problem that we had was in ing having a commercial solution like Tosca for example, or Catalon is a bit of a nightmare. Of course with the license cost and ING is quite heavy on open sourcing. So what we thought of doing is there's a very big community within in the outside world with Selenium. How about we build a framework on top of it which makes automation with Selenium and let's say for APIs, HTTP client and then wrap it around so that the end users or the Customer journey experts could really understand this. We did that. And that was the inception of Ingenious. And this happened some four years ago. But then as technology evolved, as modern web came into picture ings, web front ends had special qualities which, even though I'm a very big fan of Selenium, I have to say we were facing troubles with it. And then came Playwright. And four years ago, in fact, I was the one who introduced Playwright within ing. And we thought, let's tweak this a little bit. Let's give Playwright a chance into the Ingenious framework rather than Selenium. And it worked wonders. So now we have a situation where our business users, our customer journey experts, are able to write good quality automated tests and at the same time leverage the power of Playwright. And that's a humongous thing for us.

All right, so I know a lot of people start off with Selenium. We're not saying Selenium is a bad tool. Depends on your context and your situation. But I'm just curious, in your context, what were the problems or what were the areas that Playwright did better than your implementation of Selenium?

Yeah, so one of the things that immediately comes to mind is piercing shadow roots. For example, now we have a lot of web components which are, you know, a lot of security layers are built on top of it. And it's a reusable set of components templatized, let's say, and it's used across ing these shadow root piercing within Selenium. Well, I have done this with, with jspots, but it's a nightmare if you look at how a customer journey expert would look at it. There's so much of dynamic IDs that get generated which had to be parsed into this JavaScript. And that was horrible with Playwright using something like a label or a text, it just pierced it seamlessly. So that was number one. Number two, we have a lot of intranet applications, so homegrown applications. Of course, we have applications which support customers. So customer facing applications. Yes. But 80% of the applications are homegrown. So there is a decent amount of latency, slowness in rendering. And with Playwright's automating capabilities, that was like. It was like moving a knife through butter. So these were two major aspects that I would say Playwright won over Selenium.

All right, so I guess the next question is, a lot of times people just go to creating a new framework. There are a lot of frameworks out there like webdriver, IO that I think helps with the piercing of Shadow Dom's Earth and the Serenity. So why go the way you did with creating your own framework then?

Yeah. So if you look at Serenity, or for that matter, Webdriver I.O. and you. And you build a layer on top of it through, I don't know, something that helps communicating with Customer Journey Expert, you would probably think of BDD Behavior Driven Development. So you'd have a Kirkin layer. But then somebody had to write the glue code. Somebody had to still come in and write the glue code for you to be able to make it more effective. So here we were looking at a setup where, what if we eliminate the need for writing glue code? What if the Customer Journey Expert just comes in, simply writes, okay, click on this button, click on that button and do an assertion, and an actual Playwright script comes in and does the assertion for you. Then it eliminates the need of an additional hand holding. So there is a very seamless integration between requirements straight out of the business, and then it is immediately translated into good quality automated tests. And of course, it's too good to be true if you speak it in that way. There has to be a decent amount of hand holding initially, maybe the first couple of weeks, but after that they're on their own. In fact, just now, before I was coming into this building for this interview, there was a specific segment that was dedicated where Customer Journey experts were actually sharing their experiences of using Ingenious. So that was really awesome.

That's cool. All right, so I want to dive into that. I just wanted to follow up. A lot of people also say, I want to go to Playwright. I'm on Selenium. Obviously, you're at a large bank. You must have a ton. I would dig a ton of Selenium scripts. How were you able to migrate from Selenium to Playwright, and was that baked into the original framework to make it so you could just swap out libraries behind the scenes and no one would even know?

Yeah, exactly. That's exactly what we did. So the layer on top remained as is. 80 to 90% of the work was just reused. So the way we have created the framework is engine underneath that changed. So, for example, if I say browser open and then a URL instead of Selenium with driver get, we now have Playwright launch. So, yeah, so nobody notices the difference, to be honest. But I won't say that is the case for 100% of the cases, 20% we had to alter.

Gotcha. All right, so you mentioned how it's easier for business users, so how a lot of times People like you said, use like cucumber, BDD as the front end. Are you doing the same? What's different with this framework?

This framework has the capability of importing a Kirkin file and then creating reusable components. But then when these reusable components, or step definitions as we call it, right. Come into the ingenious ide, somebody, as I explained, right, somebody in a traditional framework would still have to come and do the coding, create that glue code. Here we have eliminated that. Of course. We strongly advocate the use of reusable components, but do we really necessarily have to code those components? I don't think so. And that's what Enginius shows you, that you can just drag and drop data, parameterize it and then you have it an enterprise grade automated tests. But I must tell you something, Joe. The use of bdd, I mean not just bdd, BDD is very good, but the use of cucumber was or has, is not very mature within ing. And for that matter, I've seen a lot of different implementations everywhere. There's a lot of implicit usage, explicit usage and it's. Yeah, it's not uniform. They treat it as a test automation framework, but it is not supposed to be one. So it was all about collaboration, it was all about having business requirements brought in, converted into tests. Yes, and that is the philosophy that we wanted to propagate through Enginius.

Yeah, I agree. It's just extra overhead when you're just using as an automation framework rather than a communication collaboration. So is this more like keyword driven then? Yes, like I don't know what a banking feature is, but like ad account.blah blah blah or something.

Yeah, so it is hybrid, basically. So it is keyword driven. You can just simply treat it like a keyword driven framework. You don't have to use Kirkin overlay. That's perfectly fine. That's one way of using it. The second way is you have the Kirkin overlay and then you treat it as a behavior driven. And the third overlay, third type, is a data driven format. So where you have all these automated tests, you just automate it once and then manipulate the objects, manipulate the data entry by making it data driven so that all your different business test cases are then fed into the script using these CSV files which are already preloaded into the ingenious ide.

Nice. So now you did mention like tools like Catalan, Tosca when you work for a large enterprise, and this is huge. ING bank. I worked for healthcare, usually testing more than a web Application like it's not like a rinky dinky. I'm not putting down any other things testers test, but like this is like an enterprise grade mission critical audited type of software you need to test. So how did you get over that? Because like things like Tosca or UFT have they automate more than a web browser? So how did you, I assume you're testing more than web browser, I guess. And if you are, how are you getting over using just Playwright then?

So to answer your question Joe, if you look at the documentation of engineers Playwright Studio and if you scroll down to the architecture, it basically says that it uses Playwright for browser testing, but it also uses Java HTTP client for API testing. But this was built at the time when Enginius was still having Selenium as the browser testing tool. But now we are also going to use Playwright for API testing and it's still in works and it's going to be released very soon. So we also use JDBC and ODBC connections for databases. We are also doing accessibility testing and we are also integrating Appium for mobile testing. So within a bank it's not just the browser, the browser is just the interface. And to be honest, we have multiple such interfaces of the API interface, the browser interface. We have FAT clients and we are using WinAppDriver which is also a Microsoft library for automating FAT clients. We have Java messaging queues, we have Apache, Kafka and all these different technologies in addition to databases and traditional file based validations. So we needed a framework which was able to handle all of this. So all of these different capabilities had to be plugged in. Additionally, I would like to say we are not using the JavaScript or the TypeScript version of playwright, which is hugely popular. We are using the Java version of Playwright. We had to have Java as the programming language because we were catering to different tech. Now good luck with testing a Java messaging queue with a JavaScript based framework. That's not going to happen. So that is why it was a very conscious decision to go with Java.

All right, so if someone wants to. This is open source.

Yes.

If someone want to use this. So do they need to use Java or is that just. If they need to create their own methods, they would have to code it in Java behind the scenes for their users then to.

That is exactly what it is. So if there is any special method that has to be coded, they have to just open up the Java engine, which is nothing but a maven engine, just simply write your custom code or whatever. And then do a maven clean install and there you have it. All of Playwright's actions are basically plugged into it. So they don't have to use any special coding for Playwright, but they might have to write something for their program for their application, which may use some special logic.

Nice. Is there anything that would prohibit someone from running this at scale? Can you run the framework like against, like a browser solution?

Yeah. So this is very easily integratable with Lambda Test with the browser stack and Sauce Labs. And we have done these experiments over and over again within ING. We have our own CI solution. We are using Azure, DevOps and then we just run these tests at scale. The whole purpose was to make sure that this is a Java solution. It runs via command line and it can be triggered into as many parallel threads as much as possible. But then the only limitation is the agent on which you're running. If the agent allows you to spin up ten threads, then yeah, very good. But it only allows you to spin up two threads, then that's where you're limited.

Gotcha. So you mentioned the different types of technologies happening under the covers, and I assume it is all under the covers. Like someone would just use the same keyword driven thing to. I don't know what the keywords are for accessibility, but whatever it is, you can mix and match.

Yes.

The web and then do an accessibility type action all in one flow.

Yes. We even have built a setup where you write your functional tests and then within the functional test you can actually use a keyword like capture page timings, which basically captures the complete load dome load time and resource load time and everything, and generates very nice visual representation. All of that is documented in the documentation that I showed you. And we have tried to make it very user friendly, of course, open for feedback. And if the community feels that there is more that can be added or something can be made better, more than happy to look into it and work together with the community to make it better.

All right, so I guess like how, how do you envision the open source piece? It is all open source, it's not a piece. How do you envision the open source community contributing to this then?

Yeah, One of the interesting things that happened very recently was Andrei Lushnikov, who's the founder of Playwright, was in the ING office and we caught up for coffee and then he said, hey Ashish, show me Ingenious. And I was a little nervous. And this is like two days, barely two days after Ingenious went open source. And I was hoping and praying that, okay, if it has to go wrong, please don't let this be the occasion that it goes wrong. But in one hour he was able to just navigate through it, create his own tests, run it via GitHub Actions, and then during that process he shared a few feedback. For example, making some options like browser capabilities like set headless or Start maximized as dropdowns. So he is our very first contributor to the open Source project. What I genuinely hope from the open source community is to try it out. Open GitHub issues, create pull requests if you have to. There are two projects, basically, the documentation project and the engineers project in its totality, just feel free to contribute to it, suggest ideas. This is one solution that you don't have to pay for and yet you're getting the perks of different types of testing.

So yeah, so you brought up a good point. A lot of times people think contributing to a project is just coding, but actually opening up bug defects and things like that is helping the project because you let people know there's an issue and getting them resolved that way.

Absolutely. We have created a discussion forum on GitHub as well. So if there's anything that you would want to discuss, even just create a new discussion point and then we start talking about it. One thing that I would definitely like to add for the folks out there, this has been used within ING for four years now. Of course, four years back, this did not have the same level of maturity that it has now. So over a period of time there has been a lot of improvements from the feedback that has come from the internal folks. And I think this is a product that you would really enjoy working with.

And that's a good point because like I said, you're not testing toy applications as a true enterprise app.

Exactly.

And I just asked ChatGPT how big is ING? It has 38 million individuals and corporate clients across more than 40 countries ING serves. So it's like I said, it's been tested against real world applications. It sounds like.

Exactly. I was doing a talk over in a conference earlier this year. At that point in time, I was not at a liberty to share the name of this framework. Now I can, because now it's open source. And one of the slides basically showed where are the places within the world where I Ingenious has been implemented. So it ranges from Belgium, Netherlands, all these different countries in Europe to the us, Singapore, Philippines. Yeah, everywhere.

That's awesome. How did you get them to do open source? I would think a big company like this would have so Much red tape. I'd be like, I'm not even going to go there. Like, why? First, how and why did you get it? Open source?

Yeah, I think this is a very good question and I think it also has to do with the amazing leadership that ING has and the culture within ING is a culture of collaboration. When I first pitched that idea to the head of engineering and culture, he just jumped onto that idea saying, hey, this is one solution that if it has already added so much of value within ing, then this can only grow bigger. This can also add a lot of value to the open source community. But at the same time contributions will help make this a better solution within ING itself. Because now we don't have two different forks, we just have one solution which is used by ING and used by the outside community. And it also shows a very open mindedness that is basically the essence of ing, to be honest, collaboration and open mindedness. And the other factor was the process was not an easy fact. Easy process. I had to go through a legal review, I had to go through multiple security scans, I had to go through, yeah, a lot of reviews. Eventually the open source body basically approved it, but that's not the end of the story. I couldn't have just released it without a proper documentation. And that documentation also took a lot of effort to create, to have the right ING flavors, the colors, the fonts and then eventually it was out.

Awesome. So I know it's been around for a while, but you kind of, I think you just opened source to maybe a few weeks ago, if I understand correctly, a month or so.

Yeah, that's right.

So I know it may be too early to talk about future roadmaps, but a lot of, you know, AI is the rage everywhere. Any plans to incorporate AI driven automation? Or is that because you have to probably rely on a model that's paid that is not on the roadmap?

Yeah, that is. No, that is true. So as you mentioned in the intro, I'm leading a center of excellence. One of the pillars within the center of excellence is the innovation pillar and where we are basically strongly looking at the usage of AI, there have been few use cases within the bank which give us a very good confidence that very soon we will be able to leverage the AI capabilities within Enginius. We have the roadmap, we have the plans, we have the different use cases mapped out. But because it's not official yet, I am not at liberty to discuss that. But it's definitely on the cards. It's going to be very exciting. Joe.

Very, very cool. All right, so we mentioned one of the key features here. It leverages other technologies to help you do other types of testing. And we'll have this in the show notes, a link to the documentation. But does it also help you with DevOps in general? Like does it do any type of monitoring or is that outside the scope of what you want this to be used for?

No. So this is strictly meant for functional testing. As of now, we are basically leveraging the capabilities towards generating performance tests as well from functional tests. So we are creating a feature now where you just right click generate K6 scripts and then it'll generate performance test scripts for K6 runners. But it is not meant at this point in time, it doesn't do any kind of a monitoring. But we also have a feature where it basically links it directly to Azure DevOps test plans, which is meant for the test management. It's also documented in the link that you're going to be sharing. So yeah, no monitoring now, but that's a project that's also, yeah, being built in parallel.

Starting to sound like a Jill Wishlist. But how about test data? I assume as a banking company that's something you probably like. Oh my gosh. We have to create data and it has to be compliant and not using real information. Anything built in around that or is that something you could see happening?

Yeah, yeah, yeah. So there are very nice questions. So there are two things already. There are a few capabilities which are built in, so generating random strings, random numbers, but those are basic utilities in terms of test data. We have so far always been reliant on relevant business data. So we never gave it a thought. But since the day we have open sourced it, we are, we gave it a real hard thought and we thought, hey, we want to integrate Java Data Faker library so that, you know, the end users could generate synthetic test data directly just by creating some keywords. So it's there. It's also going to be released soon. And because you talked about data, that is one of the use cases that is being discussed for the AI.

All right, so you have really good documentation. You could tell you worked a lot on it. Is there any other resources you're working on to help people maybe get up to speed quicker with your framework?

Yeah, sure. In fact, one of the feedback from Andrej was can we have video tutorials in addition to documentation? And that is something that we are working on. We have created a very nice playlist and very soon it will also be made available to the general audience.

Really really excited. All right, Ashish, before we go, is there one piece of actual advice you can give to someone to help them with their Playwright automation testing efforts? And what's the best way to find contact to or get our hands on Ingenious.

Yeah, so one of the only thing that I would say is if you at this point in time are not using Playwright or are using Playwright and are not very, you know, savvy with coding, but you want to write real good automated test cases contributing to your business, try out Ingenious. It's super easy, super fun and if you have any questions just please reach out to the GitHub page. Just create an issue or just start a new discussion and we are really active there. We are really looking forward towards your contribution and we would immediately jump in and start a nice discussion.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode head on over to testguild.com a512 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast, Joe and my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation testing and DevOps. If you haven't yet, join our vibrant community@testguild.com where you become part of our elite circle driving innovation, software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges. We're excited to collaborate. Visit testguild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

With lutes and lyres, the bards began their song A tune of knowledge, a melody of code through the air it spread like wildfire through the land Guiding tester showing the secrets to behold.