Digitial Testing Equals Testing of Digitalized Processes with Tobias Müller

In a land of testers far and widely journey seeking answers, seeking skills, seeking a better way O the Test Illumination Testing Podcast Guiding testers with our nation awesomeness. From ancient realms to modern days, they lead the way. Oh, the Testicle Automation Testing Podcast.

Hey, do you want to discover a new way of thinking about automation while you're in for a treat? Because today we're going to be talking all about digital testing, which equals testing of digital processes and a bunch more. I think this episode is going to blow your mind if you don't know. We're joined by Tobias Muller, who is the CTO and co founder of test results IO. He's a pioneer with over 25 years in software development and a decade specialized in software testing for high stakes regulated environments. Which is critical because the proof is in the pudding when it comes to automation, especially at enterprise level. So he really knows his stuff. Tobias has led projects, they get up to 160 million and redefine the tested market with test results IO which brings a fresh technology driven approach that sets a, I think a new standard for really automation and the way they approach it. If you haven't checked it out, you definitely want to check it out using the link down below. But but let's dive into his unique insights and vision for the future of testing. And what is digital testing? You don't want to miss it. Check it out. Hey, before we dive into this interview, I wanted to share with you something that Tobias mentioned. He said he's going to be hosting a webinar that takes a deep dive on everything we talk about in this episode. The name of the webinar is called Digital Testing Applied and it's going to address the growing complexity in testing full digital business processes from initial user authentication to to API calls, OS specific interactions, and even PDF and file operations. This 30 minute webinar offers a structured approach for software testers and automation engineers aiming to manage these complexities or challenges across web, mobile and desktop applications. And during the webinar Tobias mentioned, he's going to share strategies to reduce test maintenance costs, detect bugs earlier, and create more efficient test processes. The presentation will emphasize customer centric testing, showing participants how to implement approaches that prioritize end user experience, reduce testing times from weeks to hours, and ultimately lead to more reliable software releases. I highly recommend you register for this webinar now using the link down below and also take a chance to check out Test Results IO while you're there. Hey Tobias, welcome back to the Guild.

Hi Joe, thanks for having me back.

Yeah, it's been a while. You're always working on things, you're always trying to rethink different things with automation. So I was really intrigued by this concept of digital testing. It's one of the first times I've heard of it. So I guess before we get into it, what is digital testing?

Yeah, digital testing is. I think it's a new term. You haven't heard that a lot beforehand. So it's about testing software in context, like in processes. I mean, there's a lot of digitalization still ongoing in enterprises. So they are still discovering their processes and trying to get them coined in software in a way. And a lot of people still test software these days in silos. So they test individual software elements and individual software, but they never test the full process, like the value stream, what you typically call in enterprise and in banks. And that is actually a problem. And we said like, hey, if we do automation in testing, so what would be the challenge if we switch that over to digital testing and say, like, okay, how could we test multiple different software pieces that are along the process chain and how would we do that from a technical point of view?

Nice. So how's this different than what people conceive of automation already?

Yeah, the thing is, if you look at automation today, I mean, I just saw a posting about selenium being 20 years old, right. And that is more or less the thing to automate if you have web pages. But that is exactly also the limitation, like you have technology and you automate a web page. So we invested a lot in learning on automation with Selenium. And then you can test webpages. Lovely. But we all know that there are still desktop applications, native applications out there even they are getting replaced by webpage applications. They are still out there. And there are databases and there are mobile applications. I mean, if you think about just a stupid login, if you think about that, I mean, most of them are already two factor authentications anyway. So you log in on the web, but then you get your second kind of code on a different device. So you only need to take on a different device. If it's your mobile phone, you would need to switch to Appium. So it's already a second tool in the chain. Then have some desktop applications as well. Then you need to, I don't know, come up with another tool or can use Appium for that. And I think that is part of the problem. Right? That is what automation today is like. You do have an application. So developers are creating an application and they are doing unit Tests, they are already doing automated testing. And there's a lot of thoughts going into how can we do like end to end tests, but in the sense of like, okay, how can we test from the front end to the backend and then just to the database. But that is only affecting this one single application. But nobody is using a single application these days. Just nobody. I mean even if you use winword for example, like 20 years ago you used it and you just printed out a sheet of paper afterwards. Today you actually create a PDF file so there's already a second piece involved and then you send it, I guess via email. So there's Outlook or Gmail or whatever involved. So there's just already three different kinds of software involved in a little process that is writing a document and sending it to a friend.

Absolutely. And like you said, Selenium was made just for browser automation, which is cool. But I'm always confused when people say that's all they use. Because I'm used to working at enterprise companies like insurance and healthcare and they have all these applications, more than just a web browser. And like you say, they jump context a lot. So why is it or what industries do you think more probably see this type of challenge? Or do you think a lot of people just try to cobble something together and give up on really automating the full end to end workflow?

That is what we actually see. It's like they give up. I mean they try to automate everything but then they fail and then they are, they go back to exploratory testing. They sometimes call it manual testing and they openly state that as well. They say like we did automation, we tried it two times, three times already, it never worked out. So our business users have to test again. That's the only way that works because we lost a lot of money and that is not what we are going to proceed anymore. So we do have the business users, they just spent a few days on testing that. So we are back to happy pass testing. It's like this is the functionality that I urgently use. Does that still work? And I think that is not where testing should go and that is not where testing should be. And I fully get the differentiation between testing and checking and the automation and the stuff. But nevertheless there will be more and more software out there and it just needs more manpower. The manpower is not existing. So we need to automate. There's no way around automation. And then we trap ourselves in silos like just being able to test web pages. And I think that is the biggest Pain for all of those because management sees like, okay, automation is possible. I mean, look at robotic process automation. It has blown everybody out of the water. So everybody's aware that automation apparently can work in a way. Testing is a completely different beast. I agree to that. But nevertheless, people expect that automation just works and it works about software like Borders, that you can just automate what you see in front of you. Like, why can I not automate what I'm using today? I'm just back from a prospect, actually. And they use like a core banking application and then they have the trading application and the trades go into the core banking application and you know what I mean? It's 2024 and they test manually in one of the biggest banks over here in Switzerland.

So is it an education thing or is it a convincing thing? Because I see a lot of things on LinkedIn that there's no such thing as automation. It's not testing all this. I don't know, I never followed it. Maybe because I'm an old dude and it's like, I hate when people change words and meetings, but yeah, exactly. Is there confusion of what is automation then or.

I don't think there's a confusion. I think there's just a lot of. I mean, it's like a philosophy thing. It's like we had big projects, like there are 16 people in a room and there was specific question and you had like 20 different opinions, but none of them gave an answer because they were not educated on that. So I don't mind if it's automation or if it's checking or if it's testing or whatever it is. The problem is people are out there, are not able to automate their workflows. That is the pure problem. And they have different software, so they have different frameworks. They tried it on their own with Selenium and Appium because they are open source and naturally they are free and you don't have to invest anything. We all know that that's not the case, but that is what people actually perceive. And then there are software vendors that actually add on top of that and have their studio solutions or whatever to make it even more easy and they still have the same problems. And then there's stuff like Helenium that auto heals itself, which apparently doesn't fulfill the promises all the time. And that is actually where you have to ask yourself, is that really where we are in technology, in testing in 2024? And my answer to that is actually since 2018 already, no, it, it can't be that cannot be state of the art in testing. It just cannot be. I mean, we have, for developers, we have millions of different tools that actually make the life easier. And in testing it seems like it's ridiculous. It's like it's. You need more time in automation than you would actually need to just execute those test cases manually or just do exploratory testing. And that cannot be the state of the art. And that is what I'm fighting against, to be honest.

No, it's a good fight for sure. So, you know, since 2022, AI has been around for a bit, but like it really took off. Has AI really accelerated this issue for need for more automation? Because more code is being written by AI and therefore you're going to need to test more. So even to keep up, you're going to need to stop automating a lot more of these processes.

I saw that statement from Jason as well, where he says, like, hey, we need more testing because there's more AI. AI will generate more software. So we need more testing because there will be more software. But in reality that's not the case, right? AI is not writing software. I mean, AI is like extending the line that I've just written in the code or is adding some fragments that are easy to add, but it's not that it's generating software in that sense. So I know that is what everybody is telling everyone else, but that is actually not true. If you're using all of those tools, what you do is like you get a little help on being faster, so you get more efficient in writing software. But writing software is still for humans right now. That might change in the future, but right now it's for humans only. And so there is more software. True, because everything, everything is software these days, right? I mean, every company more or less is a software company. I mean, if your core processes rely on software, then you are a software company because you need to be able to manage that software. It's pretty simple. And that means there needs to be more testing, but there needs to be more reliable testing. Because what we see is automation is not reliable. And I think that is the fundamental problem is people trust in testing because they understand meanwhile that is required. Not everybody, but most of them understand that's required. And then they see the burden that's coming on them and they need to find a solution to get all of that work done. And that is actually automation. And then they start with automation and then they have a setback on that because it breaks every two weeks. So you automate and then it doesn't work anymore. And I know everybody will not agree with me and they have all successful automation projects but I'm, I'm going to those sites, I'm going to those customers and I see what they have and I see that they do have like just a prospect. Today had a whole management team on keeping up with the, with the updates on the different testing systems, like keeping up with the newest version of Selenium, migrating everything, keeping up with Appium, keeping up with Tosca, all of the different players. So they have their own team just to keep that stuff up. So there's a lot of challenges in automation and that is what we are changing right now because we say like hey, if a user can use an application then we should be able to automate that, automate it easily and get rid of all of this locator stuff that we need to go into code and need to be able to identify how to get to that element. Because you, Joe and I, if we see a field that is labeled with email address we enter our email address. It might be a fake one, that's a given, but we will enter our email address. And why can we not have test automation or automation in testing that does exactly that. Like there is an email address label and just find the right field where you would enter the text. Like you as a Joe would, would go there and would identify the right field. And it's pretty simple actually if you understand the basics of human computer interaction. I don't get why in test automation that is still not the case. People think like it's still the 80s but. And we don't have processing powers but we do have enough processing power for all of those different AI models. If you think about those large language models they consume tremendous CPU power. So we can use small computer vision models that are already good enough for testing and can just put that into place where we need it in automation and just rely on that and have far more stable automation than we have today.

Nice. So I agree with the visual part. I just know maybe some people tried back in the day something like Sakuli and they're like eh, not really reliable or some sort of OCR type of thing. So maybe, maybe this is a good point to talk maybe about maybe technologies and maybe methodologies that you as a founder of Test Results IO use to help, you know, really digitalize the processes more effectively or efficiently.

I mean in the end we just have pixels right? Like everybody else and SQLLY failed in multiple ways. Actually it just did an Image search and tried to match that and yeah, over promised in the end, more or less. And what we do is completely different to that. We still use pixels, so in that sense we still have images. But the idea is that we don't try to find an element or that we try to identify an element based on what you tell us. It's more or less like we understand how humans work and how they actually behave in interacting with software. And one simple example is like if you have a label that says like first name and you have a text box to the right where you can enter it and you have it below the label, you will always enter it on the right side. So you will always choose the element on the right. Why? Because you learned so in the culture that you grew up, you learned to read from the left to the bottom right. It's pretty simple. So that is your priority in how to fill in stuff. You learn that you're not aware of it, but it's how you work. And it's pretty simple to model stuff like that these days in actually small models. And you can just interact with applications in a natural way. And there's a lot of different stuff if you think about like how much problems tables still face, like for test automation. And it's extremely simple. As a human you understand even without having any kind of frame around that you see by proximity, ah, there are different columns, those are rows and you can easily handle tables without any training at all. And automation should act the same. We do have enough processing, we do have enough computing power. So it's actually, it's not simple, but it's doable from a technical point of view. And we just need to do it, that's all. And we just need to step up a bit from what we know from. Yeah, we need to have those locators and I need to find an XPath to identify the text field that is there for the first name and it's in an iframe or whatever. Okay, congratulations. Now you found a way to get to the text field. The next application you test is a mobile application and it doesn't work at all. So find a universal approach. And the universal approach is actually like how your end user would automate.

Yeah, great point. And especially when people need to test like these package based applications like Salesforce or SAP where they generate like random IDs or someone uses a shadow DOM and they're spending all their time developing a way to get at these fields to make it more reliable. This sounds like a way that maybe gets around that. Am I hearing that correctly?

Yeah, completely, completely. Specifically Salesforce. I mean Salesforce is an application where a lot of people claim like, yeah, we do a frameworks for. For Salesforce. Yeah, but the question is why? Why do you even need a framework for Salesforce? It's so natural for people just for humans to use it actually. So you just enter the stuff and that's it. So why do you need a dedicated framework? The same for SAP, the same for other technologies like that where you always say like, hey, they cannot be automated, easy. We need to have a dedicated connector to that. We need to connect to those systems and understand how the UI is created and stuff like that. And you are right, Joe. The thing is in UI specifically in the web, you do have a lot of different frameworks and they are generating those IDs on the fly because they are used for styling and for different stuff, but they are not used for testing. Yeah, true, you can add testing IDs, but most of the time that's not the case. It's not done. And even so it is done. Sometimes it gets forgotten and then you have the next problem. So it's always like, it's always a catch up game, right? You always try to find a solution to a problem that you initially created by yourself. So we say like, hey, test automation needs to rely on the code. Okay, fine. And now we created a problem because we cannot navigate in that code. So why can we not get rid of the approach of relying on the code? Because that makes us accessible universally so we can access any application. If you find an approach that doesn't rely on the code, we can find an approach that works for any application. And that brings us to the point where we are in digital testing that you can actually test the whole value stream. So you start in one application, you just continue naturally in the next application. You switch over to some kind of web application, you switch back to a mobile application that sends you some notifications where you need to check the stock or whatever and then you continue with the next application like just in a natural way, like you are using it. I mean if you go to a trading floor and you look at those 8 12, 16 screens they have in front of them, it's a lot of different applications. Or like the railway control system, I mean they have eight different applications running at the same time and they just work in cooperation. So you need to test all of them together.

All right, so we touched on it a few times. How though, how does test results IO's approach to testing go through these interconnected workflows. So because it's image based, it's able to run on Linux and Mac and like, how does it help though? I know it's vision based. Is that why? Because it's vision based, it's able then to interact with all these other systems because it doesn't matter on the OS it's running on or any of that type of information.

Yeah, that's the point. The thing is, even if you're using vision or if you're using visuals, it still matters on which operating system you run on. You just need to look at browsers, right? Firefox, for example, renders text with grayscales around it, and Edge and Chrome renders it with rainbow colors around it and stuff like that. You can just abstract on that because then you don't notice it. So there are simple tricks in perception how you can actually make it perceive easier in a computer vision model. So you just like, you do subsampling and stuff like that. You do super sampling. You have a lot of different techniques that are out there for game development. You can use all of the modern techniques in test automation and that is actually what we apply. So I won't tell you exactly how it works.

I can just tell you it's a combination of different algorithms and different computer vision models, actually. And in the end it's pretty simple. If you're thinking of examples, like most of the time you will interact with something that has some text around it, so near proximity. So you need some kind of computer vision model that is able to detect text. So to make text distinguishable so that you can see, okay, there's text and here's text, and there's text as well. And afterwards you apply a different computer vision model to identify, okay, what kind of elements can I detect close to that text? And then you apply some kind of behavioral model where you say, okay, but for somebody from that culture, what would be the most probable element to choose? And all of that combined, you can actually call it a multi. How's it, how's it called a multi mode AI agent? Stuff like that. That's just a new terminology for that. In the end, it's just different computer vision models that are specifically trained on those tasks.

All right, so that's interesting. I know Claude just came out with the multimodal for automation. Is that basically trying to use visual. Is that what it is? It's called an AI agent, but all it's doing is using a visual type of algorithm to get through the workflow.

I Don't know what they do, to be honest. I just saw that they have a success rate of 23%. And that is where I said, like, okay, I'm not interested right now. And if you look at their demos, it's pretty interesting because they are happy if they can open an Excel file, read the text in that Excel file, and then put that in some form. It's pretty nice. If you look at those demos closely, they don't need to scroll, they don't need to do any selection, nothing. That is a complicated interaction. Right. And they also, they. And the next part is they don't position it as a, as a testing tool. They position it as an alternative to robotic process automation. And as all of the big RPA players learned, Testing, Yeah, robotic process automation is part of that, but testing is a lot more around it.

All right, so you just opened up a memory pocket with me. At one time I needed to test somehow for some reason, an Excel file to go to a browser or do something. I forgot what it was, but it was almost impossible. It was a weird like Office API type thing that never worked. So this type of technology, once again, it's not just for browsers. It could go to PDFs because it's visual. It go to PDFs, it can even go to, you know, Microsoft Excel file, Microsoft Word, am I hearing that correctly, is able to interact with the text and things like that also and validate it.

That's what they typically do because that's what's typically in the workflow. Right? You do something, I mean, entering leads in, like for example, you mentioned Salesforce. Entering leads in there is pretty interesting. But in the end you have some kind of dashboard that shows you actually how many leads you converted successfully. And just numbers like that, like KPIs like that, and they are all represented graphically so that you as a human can easily understand that need to test that as well. And there are most of the time you have PDF reports and stuff like that. And yeah, naturally, I mean, the approach, if you go visually, then you can test all of that stuff. And that's what we do. So typically if I give a demo, actually I showed this prospect exactly that. Like doing a login on the web page, then doing the two factor authentication via confirmation code on a mobile phone, then going back to a desktop application and then opening some kind of PDF invoice and showing actually how to check the invoice.

All right, so once again, I just want to make sure people aren't getting confused here. When they hear visual, they may think of other software, other vendors that use visual validation, but they're not really able to interact with elements. So if I had a webpage and I had a chart and I needed to validate the chart, but also interact with the chart, this is different because this sounds like not only will it validate the chart, but also you can interact with the chart because it's using visuals. Am I hearing that correctly as well?

Yeah. To be specific on that, it is not visual testing, just to mention that one. And there is a big player, we all know them, they are doing visual testing. And it's exactly the same problem, right? You compare screenshots in the end, but you need to get some way to get to the corresponding screenshot first. And I guess that vendor is doing it meanwhile as well. I mean, they do have their compute platform and stuff like that. So I think they got the limitation and are fixing that. But that's it. Yeah, we are not doing visual testing. It is not about comparing screenshots in some kind of clever way. It is really like having visual cues, having visual hints. Then use those visual hints to actually navigate applications and do verifications within these applications. And the verifications can be that we are checking for text in there, we are checking for other visual hints in here and stuff like that. So we can change all of them up. And the interesting part is because you mentioned that ocr and that is also a topic that is always coming up if you're doing ocr. I mean, OCR just doesn't work, right? It just doesn't work. You can see that all the time. You can even see that on your iPhone. I mean, if you go to an Apple store today with one of their gift cards and I did that to buy a new iPhone, actually they need to scan that text on the gift card. Apparently they don't have QR codes on those gift cards for whatever reason. And if they start scanning that, actually you will notice that sometimes the scanner just doesn't get the correct code. So if you don't get the white space around the code correct. So about the text that represents the code and you scan a bit more, actually the scanner is completely confused. So that is the state of the art ocr. It's just not meant for low resolution, colorful text. It just doesn't work like that. And there is also something where you just need to come up and see like, hey, you as a human, how would you do that? So if you read some text and it might be that you read it wrong, right? I mean, it might be that the villain in the, in the background here that you, that you, you confuse that with an like whatever, an I, a capital I. But nevertheless, you get the context right. It starts with a V I A at the end. So capital I, capital double I doesn't make much sense. So you correct that. And that's just a dictionary. Typically, as you're coming from the regulated environment, Mattex, we cannot use dictionaries because there's a lot of different stuff like typing and screaming for blood types. You don't have typically, a dictionary for all of those different type and screen tests that can change the name as well. So you need to come up with something else. And that is actually where we came up with reverse ocr, where you can actually say, like, okay, we get the following text from ocr, but we were looking for this text, so what is the probability that this text that we got from the oci, which is wrong, it's still the right text that we expect on the screen? Because that is what you do as a human as well. I mean, you're abstracting from what you're seeing on the screen. You see pixels, and from that you. You create some contours. And based on those contours, you better match actually to what, you know, what you learned in the past, as always. And that is how you actually work. And that is what we resembled with digital testing.

All right, so you probably get the same resistance that AI products do as well about how do I know it's really doing what it's doing? So it's using visual methods, ways to interact with the application, and it's able to get past probably a lot of these things that like a misspelling or something. How do you know then that it's doing what you thought it's doing? I assume some sort of reporting is creating at the end that shows what it did and how it got around it.

Yeah, there are two parts to that. One is actually that we check. Every single interaction with the application is actually verified at every single step. And you can imagine, like, if I need to enter my email address somewhere and look at a confirmation code somewhere else, and I put in the wrong confirmation code, I will not be able to log in. So if I expect to be able to log in, it will already fail at that step. If I do the verification all the time. And the second one is induced failures. So most of people don't do that. And I only heard Bas talking about that, to be honest. It's like where you say, I do have a test case. And now I do have a runner that is executing the test case. And the runner can actually induce failures to the test case. So I keep everything that I do asserts or what I expect, like the verification points, I keep them the same. But if I do have an interaction, I just change up a character. So the expectation would be that the verification afterwards failed. And that's exactly what we do to prove that the test case actually work is like we let them fail. And if it doesn't fail back then, then your test case isn't. It's just not a valid test case. Right. It's. They're open.

Absolutely. All right, so I guess I'm just thinking of objections here. A lot of people probably have invested already in the testing infrastructure, like Selenium Grids or whatever, and they hear, oh, this is completely different. I can't use it against. I have to get. I have to buy in not only to this tool, but a whole other way to. To run the infrastructure to scale up. How does this work? How does it scale up? What do you need a different setup or how does that work?

Yeah, so I hear that every single time. Because typically people do have some automation, they are not happy with it. They are spending a lot of money on that. Most of the time they are not using it, but they have it. So you always have that objection. And the thing is, what we say is, yeah, Kiwi does this, but for sure you do have gaps in your automation, right? You do have PDF reports or you have some kind of dashboards, or you do have some native applications that you cannot automate. Start with that first. Try it out. If it works for you. If it doesn't work okay, it's the wrong tool for you, that's fine. But if it works out, you can do that step by step. And how does it scale? Yeah, you don't need Selenium Grid for that anymore. So what we do is there are two different ways of like scaling. One is because one of the problems in test automation is also test environments, right? I mean, you need to have an environment that you can test against. And most of the time that's not a single PC, but that is a network of different server systems and stuff like that. And you try to resample that actually so that you bring up a virtual environment, including multiple different systems, to actually resemble the reality and then work against that. And that is how it scales. So we can. I mean, cloud computing is cheap these days as well, and we had it actually, I guess it's a Misuse of cloud computing. How we use it, because we use it for testing. So we spin up 100 VMs for one environment, we test it like for 5 to 10 minutes and then we bring it down again. And if you are familiar with Microsoft Azure, for example, and that's just one example, it shouldn't be advertisement for them. They do have some boost VMs, they are pretty cheap and they are boosted for the first 15 to 20 minutes. So you get double the amount of CPU power and you pay, I guess, a quarter of the actual actual vm. So you can use that for tests because you only need it for 15 to 20 minutes for the test. Right. And then you kill them again. You just throw them away. And that's actually how we scale. Because for visual testing, I mean for Selenium, Selenium based testing or code based testing, more or less like you can use headless systems, you can just start them in your Docker container, have them in the repository somewhere and scale them to a million. For visual testing you actually need some kind of visual representation that you can test against.

All right, so once again I'm just trying to think of things that people might have experience with. I forgot what the protocol was Loadrunner that I used to use for decline applications. And if the resolution was exactly the same, it's going to fail. So if people have experience with that, how does test results handle that differently? Can you run it on different resolutions or do all the machines have to be exactly the same resolution in order for the test to be reliable?

No, that's typically a scenario. So you do have different resolutions. Right. And that is why it failed in the past. And I think that's exactly the answer to that. It's like, hey, that is in the past, like 15 years ago we didn't have enough CPU power to do that. I mean we did some edge detection and tried to overcome that with edge detection, but it didn't work out well. So these days we do have feature extraction. So we do have a lot of stuff that is, I mean look at that. Tesla is not a full self driving car right now. I guess we can all agree on that. But the perception of the environment that it has is brilliant. And you can do that in real time. It couldn't have done that like 20 years ago. And that's exactly what we do. I think in computer vision as well, there's a tremendous, tremendous different technology now than we had like 20 years ago. And that is actually what we are applying to testing more or less like it's it's just like we have so much better hardware just standing on our desk that nobody's using it. We are still parsing code and trying to get access to some HTML code. It's not required anymore. I understand that 20 years ago that was a requirement because we didn't have enough CPU power, Pretty simple as that. And that's gone. And if you compare today's like computer vision model to matching pixels like 25 years ago, you can do that, but the experience is different. So just give it a try and if it doesn't work, it's not for you, that's okay, but just open up your mind. And the thing is, what I, I mean there's one objection is, is a lot of people don't understand the approach. They are so used to finding a locator that gives them access to an element that they don't understand the approach that you don't need that if you tell them, hey, just, just work naturally, just select a text that you would actually use as identification for the entry field and that's it. Work like you would do it if you would just be an operator of the application. They are just extremely trained on. I need to get a locator somehow to that element.

Right. And that's like the only way it's going to work. So it's almost like get over that barrier. Because it's funny, you said over 20 years ago, that's what I was dealing with almost 25 years ago. That's scary. Need to learn, you need to evolve, you need to try new things, I guess as a tester, and that's the point.

Automation should change, but somehow it doesn't change. It's always the small steps. And now we do have Genai and we can generate a million of test cases. They are all just scratching the surface. Nobody cares. It's a lot of marketing and that is our biggest invention in testing really. We can generate a million test cases with Genai that are scratching the surface of the application. I mean, it cannot be like that.

Absolutely. I have a few other questions. Last other one is how about if someone takes the extreme, say, hey look, we don't do UI testing anymore, it's unreliable. We just do API testing. Is that something you ever hear? Or how would you say, hey, so.

Richard Bradshaw actually had a good talk, like in 2019, I guess, where he said, like, everybody's talking about testing pyramids, testing ice cones and stuff like that. And they are taking it as a strategy meanwhile that they want to do less, end to end. Testing, less UI testing and more API testing, component testing and stuff like that. And he had a good point in that talk back then already. I mean, that's five years ago where he said, like, hey, just adapt to the technology that you're using. So if you have a pure API, visual testing doesn't make much sense, right, because you have an API that you can test. But if you do have some kind of user interface, is it enough to test the API? No. Is it enough to test the ui? I would claim it can be. But in reality, the thing is, what you want to do is a combination of both, right? So you want to combine these UI tests together with API tests in the same testing scenario. So what I hate and what I see a lot out there is like people using Postman for API tests. They are sending some requests, they are checking them. If I get a 200, okay. Oh, everything is brilliant. That is amazing. That's not how an API is used as well. I mean, an API is used in a flow, so you get some data from the rest API, you go to the next call, you provide the data to the call and stuff like that. I mean, you know that, Joe. And that's the important part is like the whole software is always in the flow and somehow testing seems to be like static and it cannot handle dynamics. And that is exactly what you see. I mean, if the DOM is modified dynamic, I saw a webpage last time that is generating like the whole DOM dynamically all the time. It's pretty, pretty difficult to get a locator. And that's the case for all of the stuff is like, if you're sending API requests in Postman, you are doing a good job. That is actually brilliant. But the thing is, is that how the API will be used by a software developer? I'm pretty sure it's not. So you have an authentication stream in the beginning, then you have your bearer token. In the easiest case, you have your bearer token and then you continue with all of the different data that flows through the API to actually get the result that you want. And that is how we should test it. And that, I hope that answers the question. No, visual testing in the sense that we do. I mean, it's functional testing. It's not visual testing in the sense of screenshot comparison. Right? That's a given. But testing applications via the UI is not the holy gray. It's not the holy gray as well, but it's the bare minimum that you should do because your users will only have the UI and having the testing Pyramid and telling everybody like, hey, you shouldn't do end to end tests and you shouldn't do UI tests or you should actually minimize that to the bare minimum. Yeah, that is okay. If you cannot have stable UI tests. I mean, that's the root of that testing permit, right? I mean, in the end it's money and on the other end it's stability. That, that it's not a strategy to do it like this. It's like the result of the surroundings of what was given back then. And that is changing. I mean, we're here and not only me, all of the vendors and all of the testing committee is here to change that. But I see a lot of people just sitting there in their chair and yeah, I always use Selenium and I still use Selenium now. It's a fourth generation of selenium 20 years later. And a game changer. They have relative locators now. Nah, for God's sake, a relative locator makes it even more flaky. I mean, we have responsive web pages. If I say, hey, this needs to be to the right of the following element and I have a responsive webpage, it won't be to the right all the time. It might be below, it might be above for whatever reason, I don't know. But the thing is, we see a lot of changes in there that people call game changer, which doesn't make any sense. I mean, it might be in some corner cases. I give that to everyone. But as a testing or as an automation community, I would expect that we look at new technologies. I mean, what do game developers do? They have a lot of power in the, in the gpu. What do they do? What do they apply? What kind of algorithms do they apply actually to make us engage and how can we use that actually to analyze screens? That is where we need to go is like, hey, what should we do? And yeah, a lot of stuff should be unit tested. I honestly, I've never seen a project where people didn't introduce a single bug because of unit tests. But I understand that unit tests are a base requirement for modern software development. I get that. And so we do a lot of unit testing that is part of the automation. Right. But the thing is, I personally think that UI tests should be valued far more than they are today. I know the reasons they are brittle, they are flaky, they don't work, they are hard to maintain and all the stuff. But think about if they would be stable and if they wouldn't be hard to maintain, wouldn't they be the first choice to do because that is what your users will use.

Absolutely. And Tobias, earlier you mentioned, hey, people should try it. If it works for them, great. If not, don't. So I know there are a lot of skeptics out there, so I'm asking you to take the Tobias test Results IO challenge. You'll find a link for it down below and do what he says. Try it for yourself, see if it works for you. If it does, learn more. Because I think it's really gonna help a lot of people, especially at the enterprise level where I know there's a lot of people struggling. Okay, Tobias, before I go, is there one piece of actual advice you can give to someone to help them with their automation testing efforts? And what's the best way to find or contact you?

The best advice I can give, and it's changing in comparison to last time, is clear your mind and be open to new technologies. And I see that a lot of people and you know, all of those discussions, I mean, Jesus Christ, we discuss about is it checking or is it testing? You can do all of that, but it's just taking time. So open your mind and be open to new technologies. And don't learn how to code, don't learn how to refactor, learn to understand how algorithms actually work and learn how to apply them to testing. Or actually make developers aware of testing. Make them proud that they can support you in your testing endeavors. But get out of your chair and get the newest technologies to test.

Thanks again for your automation awesomeness. The links to everything in value we covered in this episode. Head on over to testguild.com a522 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. Joe, my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation testing and DevOps. If you haven't yet, join our vibrant community@testguild.com where you become part of our elite circle driving innovation and software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges. We're excited to collaborate. Visit test guild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

With lutes and lyres. The bards began their song A tune of knowledge a melody of code through the air it spread like wildfire through the land Guiding testers showing the secrets to behold.