Removing Pain Points in Mobile Test Automation with Daniel Knott

In a land of testers far and widely journey seeking answers, seeking skills, seeking a better way o the test Illinovation Testing Podcast Guiding testers with our nation awesomeness. From ancient realms to modern days, they lead the way. Oh, the testicle of nature. Testing Podcast.

If you're struggling with mobile testing, you're in for a treat because today I'm sharing with you a session from the last Automation Guild. At the last event we had some jaw dropping sessions, deep dives, unbeatable tips and automation tools. Galar. So today we're going to return to one of our most talked about sessions from 2024. Imagine cutting down the pain of mobile test automation with ease. Yes, it's possible. So join Daniel Knott as he demystifies the complexities and showcases a tool so intuitive that you'll wonder how you ever lived without it. And guess what? Registration for Automation Guild 2025 is now open. It's happening February 10th to the 14th all online, so get ready to experience more insightful talks, innovative solutions and connect with the best minds in the industry. Don't miss out. Register today@automationgill.com Register Hope to see you there before we get into it. If you're struggling with mobile testing, you're going to want to also hear from this week's episode sponsor Browser Stack App Live Check it out. Browser Stack App Live is a cloud based platform that allows developers and testers to interactively test their mobile applications on a wide range of real Android and iOS devices. With support for various OS versions and device manufacturers. App Live eliminates need to to maintain a personal cloud device, providing instant access to real devices for testing. Some key features are Real device Testing Test your apps on over 20,000 real devices without any setup or maintenance hassles. You also get multi device testing which allows you to run tests on two device simultaneously in one session, allowing for efficient testing of different scenarios. It also allows local testing, so test your apps that require access to internal servers or staging environments securely. Also, native device features allow you to replicate real users conditions by utilizing features like gps, network simulations, and more. It also gives you access to debugging tools for inspecting UI elements, monitoring network requests, and tracking app performance. BrowserStack app live offers a robust solution for mobile app testing that enhances productivity while ensuring high quality user experiences. By providing access to real devices and advanced testing features, it empowers developers to identify issues early in the development cycle, ultimately leading to faster releases and improved app performance. If you haven't already, give BrowserStack app live a try and see how it can streamline your testing process. Check it out in the links down below.

Hi everyone and welcome to my talk, Remove the Pain from Test Automation. My name is Daniel Knutt and I'm happy that you're here to join altogether the Automation Guild 2024 with us to talk about mobile testing automation, new trends in testing and what. Thanks Joe for having me today. And so let's jump right into today's talk. So here we are. Remove the Pain from Mobile Test Automation. What is this all about? Before we dive into the topic, quick introduction of myself for those who don't know me. My name is Daniel Knutt. Happy that you're here. I am 39 years old, living in Germany and right now I'm working as the head of product engineering in a company called Mabon Wolf in Germany. But for those who know me, I'm a mobile tester by heart. I started mobile testing in 2010 when I was allocated to the first mobile team. And back then there was literally no mobile testing tools available or no mobile testing knowledge around because apps just came around the corner and it was for me it was a green field. And over the course of the time I developed a lot of knowledge and I love to share my knowledge and I'm doing this on my blog, Adventures in qa. But also I wrote a book about mobile testing and also smartwatch app testing. And since two years I'm also having my own YouTube channel, talking all kinds of topics of software testing, testing, related topics, mobile testing of course as well. Lately a lot of stuff in AI. So if you haven't watched it yet, take a look at the video description and to find the link to my channel. So, but without no cut to the chase, no cheesy marketing anymore, let's go right into the topic removing the pain from test automation. And why is this so important to talk about? So this is what I would like to talk first. Current state of the mobile market though are some pretty amazing numbers I would say. Then I would like to briefly talk about the mobile testing challenges and then we go hands on in automation stuff and show you how easy it can be to set up from scratch. Today we do something from scratch in automation in a few seconds and then last but not least, quick summary resources and then I'm here for your questions. So, current state of mobile in 2023, more than 1 billion devices have been sold to people around the globe. And there's this it's just mind blowing number right? That so many devices are being sold. And also imagine that there are like a lot of devices still available and running and the devices or the pockets of people. So the sheer amount of devices is crazy. So there are a lot of stuff going to happen. In 2023, the mobile app revenue was roughly about US$935 billion. And it's not decreasing, it's still growing. So mobile app revenues is still a big thing. And because sometimes I hear people talking, oh, mobile, the mobile game is over, you know, there's no, no revolution anymore happening. Yes, that's true. There is no revolution happening. But at the same time they are like everything is happening on our mobile devices. Right? We have everything in our pockets. We use the devices on a daily, daily basis to do everything. Basically, yeah. So that's why revenue is still growing and that's why also company still investing a lot of time and effort in building and developing apps. 250 billion apps downloads per year, roughly in the mix of iOS and Android stores. There are some third party app stores available, but to be honest, they're like too small, at least for the, the western markets in terms of downloads and capabilities. We have more than 7 billion users worldwide usage of smartphone devices. That's also amazing. I mean, it's almost everyone except smaller kids using smartphones these days. And that's why it's really likely that a lot of people will use your app as well. You know, more than 4 million apps are available in iOS and Android store. And there's also a big number compared to the download numbers. It's of course rather small. But what I would like to show you with that number is like 4 million potential competitors. Of course you, you are not competing with all the apps out there. But if a mobile user is not happy with your performance with your app, with things that you are doing, it's really likely that that person is just like a single tap away to go somewhere else and then take something else to play around or to use it. And also a pretty amazing number, I would say, is that an average user, an average user is checking their smartphone roundabout 50 times a day or open 15, 50 apps open per day. That's also a pretty amazing number. So that people like really still going into apps, open them and use them for their, for their purposes and for their challenges, their tasks, whatever they would like to do with. So still the mobile market is huge compared to numbers. What are challenges? So for those of you who have seen videos from myself before and also some talks from conferences, nothing has changed from the challenges point of view. Right? I see the three main challenges when it comes to mobile development and also mobile testing, it's still fragmentation. Yes, it was a big topic in the beginning when, especially on the Android side when there were like thousands and thousands of hardware software combinations, potential customer coverage. Of course you don't have to test on all the devices, but still fragmentation is a topic depending on your target customers. You have to focus on the devices that you would like to test. Secondly, the hardware or the sensors is a big topic. Yeah, I mean I have seen companies struggling with mobile development and testing because they were relying just on emulators or simulators to test and develop the apps. And that's, that's a big mistake. The mobile smartphones are packed with so many sensors and things that the people can use and they have a huge impact on your testing and development activities. And that's why it's important to test against sensors and the specific hardware. And the third challenge that I brought for you today here is automation. That's why we are here today, right. Talking about automation is still test automation. I mean, as I said, I'm seeing a lot of companies and customers and also from my past experience, so many companies struggle with test automation because they just do it wrong. They think, oh, that's the newest, coolest tool on the, on the planet. Let's do, let's take it, let's, let's use it. That's wrong. You know, do your homework, do your evaluation, check what's, what is your tech stack, what is the architecture and then find the right test automation tool that fits your need on the specific layer of the tech stack. Right. It doesn't need to be like an, an end to end automation view tool. On the, from the UI level, it can be only unit testing or API or whatnot. So this really depends on, and that's why it's so important to invest a lot of time in automation. And yes, everything is hardware related. Right. So that's, that's the big topic. So it's, it's not, and it's not stationary. So people are really walking around, are on the move while they're using your product and that's why testing on real devices is so important. So that's what I was just wanted to say. Testing on real device is a must. You cannot rely on emulators or simulators. You have to go and test on real physical devices, period. So, and I think that's enough for today in terms of theoretical information in case you would like to know more about how to do mobile testing, testing activities. Check my Blog adventures in qa. You get lots of information as well. In case you have some questions, send me a message on social media, you will find me. I know. And yeah, I'm happy to help out. So now I would like to talk about remove the pain from mobile test automation for the remaining time and why. Why I would like to talk about it. And I've seen lots of tools not only from mobile, but also from the web space and other stuff. There's sometimes the tools are so complicated to set up, it needs hours if not days to have a solid setup up and running integrated in your pipeline and so forth. So therefore tools must be simple to use. That's it. We as developers or testers, we are just like users of that product of the automation tool. And that's why the tools must be simple to use. Otherwise I will go somewhere else. Yeah, tools must fit the needs. That's what I mentioned before. Do your homework and check what can what tool is best for your needs, otherwise you might pick the wrong one. You would like to cost saving, you would like to save costs with a tool. So that's also really important. Just take a look at what's the cost structure of the tool. Is it an open source tool, is it a paid tool, how does it integrate into your environment and so forth and can you save costs and time with the tool? And also last but not least, it should be easy to integrate in your pipeline. Yeah. And without that I would like to talk about today about Majesty. Yes, it's a new kid on the block in the automation space. If you haven't seen it before, let's take a look at together because I think it's a really, really great example how easy tool can be, how simple it can be to set up and how simple it can be to write automation from a user's point of view. Yeah. So user's point of view, user journeys, that's what we'd like to talk about today. So let's jump right into the hands on demonstration. So if we open up the Maestro page, which is maestro.mobile.dev you end up on that landing page here. Right. So it's a lot of information here like what is Maestro, how it can be used and whatnot. On the left hand side you have a really great view of things that are important to learn about. Myers. True. So everything is documented from examples from how to integrate with Android, with iOS, with UI, get Swift, your RigNet event and whatnot. It's really easy to follow up on. And also you have installation Instructions here, and that's the cool part. So if you would like to install, you can install it on a Mac, on a Windows and on a Linux system. I'm on a Mac today, so I would like to show you how easy it is to install Maestro and to do the very first things with it. Yeah, for you to mention, I already have an emulator, an Android emulator up and running for easier demonstration purposes to date. But you can also use Maestro on real devices as well. So what I'm doing right now, I'm copying the installation command curl, blah blah blah, and I will put that right now into my environment. As you can see here we are on the terminal. And for you to show, let's do the installation. So copy pasting it, hit the Enter button. So now we're downloading all the resources to the computer. Depending on your Internet connection, this might take longer or it's even faster. It's checking, extracting, done, Maestro installed, we can start automating it. That's it, Nothing more to do, no prerequisite, nothing. What we now do is type in Maestro and as we can see here, we have some information that's missing, but that's the sub command. So right now we can see what we can do with Maestro. So make this one a bit bigger here so you can see. So as you can see here, we can use Maestro already. Yeah, so if you check the documentation, what you can do here is one thing that is really cool about Maestro. It's coming with Maestro Studio. And if we do like this Maestro Studio, we are starting now, we already have the emulator up and running. So you can see the emulator we have now Maestro Studio up and running, it's opening up a console. And here you can see basically the emulator, right? So what you can see here, you get a direct view as well in your browser. So in case we navigate on the device and a few seconds later we have the representation here as well. And the cool thing is Maestro already provides you with some really simple flows that you can already use in order to interact with something. So for example, let's say we tap on Gmail. So we tap here can click Play. We just play that one. And we can see here we open up the Gmail account. And that's the cool thing about Maestro, you don't need to have access to the code base of the app that you would like to test. So just go to the App store, download any app that you would like to automate, and then go for it. Yeah, go for it. And automate it. And that's the really cool thing. So now we can say, okay, let's use this one. We can say, okay, we can copy it now we have copied it, we can put it into our flow, into our YAML flow package, so to say. But what I would like to show you here is though to actually automate the messaging feature of the hit the messaging app. That's what I wanted to say. Yeah. So what we can see here, we started the app basically. Oh no, I did something wrong. Give me a second. So that's the thing we would like to tag here. So what is this? We can double tap on the Maestro studio to get basically the command that we would like to use. So we can say here, tap on messages and we can now create. That's an example. Yes. So we can basically get rid of this one. I prepared this something here. So that's basically the structure. So you have to use YAML files in order to write Maestro tests. So you can use the identifier, the app ID that you would like to test. You provide this as well. You can say launch the app and say clear, stay true. So that the apps. The app that you would like to test is always in the same state as you would like to have it. Yeah, so that's what we have. And now we can just run it. Right. So we have a really simple, a simple script here that is starting. So we open up a new window. Okay, Maestro test first, test YAML and then we click that and we can see here we are just now starting the flow. We launched the app, we put it in a clear state and that's it basically. Right. So that's what we have done already. And so what else we can now do is now that you can see.

The view has changed, we got the latest view as well. So we can say, okay, I would like to have this one. And then you click on it and you get specific information from a yesterday again. So you can get the id, you can get the text that you would like to tap. So for now let's just tap on the button. So we go back to our. Here, make this one again a bit bigger. So we have this one as well. So we tapped it and now we can run it again. Let's see, just do the same thing. So we run it, as you can see here, it's now two steps. We open up the app, we have the clear state, we tap on, use messages without an account. Here we are. Next step. Test is passing. Now we can close this one as well, we say okay, we would like to see this one. We can tap it, but nothing is going to happen if we tap it right, that's the visual representation. We can say okay, assert. Then we can say okay, assert visible and check that the image hint is visible. We're copying it now, we can run it. We can see here we're putting the app in a clear state again, we tap the button, we log in without an account and we verify that the ID is visible. Yeah, so that's really cool. And as you can see it's fairly simple to basically write your automation script with Maestro Studio. And of course if you're more familiar with the whole syntax of it, you can just go ahead and do just check on the IDs and then use the commands that you can use in order to automate and that the commands is really cool. You have everything over here. So you have flows, of course you have the flows, you have the nested flows. You can also use JavaScript in order to enrich the automation scripts and so forth. And so forth. What I also wanted to show you here today is basically the samples. So in case you would like to see an advanced flow of Maestro, we have an advanced flow YAML file here as well. So as you can see here, that's example files from my yestro. So here we would like to test the Wikipedia. We have some text advanced and we have some subflows. So here we have a flow, a subflow defined. So run the subflow and if we go into that folder which is called subflows and then we have the Android onboarding, you can see here that we are launching the app, that we are tapping a few buttons and then we check on some search container, then we run a script and then we have some input text. So let's see what's the script all about. So we go into scripts, so we enter qwerty here or get this and then we have the input text and the assertion that we're also checking as well from that more advanced test case. It's a test case that you can do and that's basically the really quick hands on demonstration. I would like to show you how easy it is to set up Maestro just like a couple of seconds that is running and then you can start right away with Maestro Studio to actually automate, get information from the app that you would like to test, copy, paste it over and then make use of it. Yeah, and also if you are like checking Maestro ready, they're also working on more AI capabilities for you to make it even more, more reliable, more faster and yeah, basically to remove the pain from test automation. That's the most important thing. Because maybe you've used Appium before or other tools and it's pain in the ass sometimes to set them up, sorry for my words. To actually get it up and running. And that's not what should happen basically. Right? We are users of that tools and we would like to have good tools as well. And that's why I'm a big fan of my estro. So if you haven't seen it before, if you haven't tried it out, go ahead and check it out to learn all about myoistro today. And now let's jump back to my presentation and let's get to a quick summary because we're almost out of time. The summary. So the mobile world is still growing, that's still important. So if someone is telling you like, hey, mobile is not growing anymore, it's boring and outdated and whatnot, it's not true. It offers a lot of opportunities for you out there. So if you would like to know more about mobile, the mobile space, do your homework, do your statistic research. It's still growing. Still. Mobile testing poses many challenges. I mean this, what I've shown you today with automation is just like little tiny thing when it comes to mobile mobile testing, the more challenging part is the device handling and the testing in the wild, like going out and testing on a real physical device in your hands, with your fingers, that's important. As I said, the main challenge is device handling. Automation comes into place as well. So that's why make use of tools to support your testing activities. Maestro might only be like little tiny tool that you can use on top of automating, maybe 10 user journeys that you see in your app that are most important, most critical for you and for your product. And the remaining things might be tested on the API level or the backend and whatnot. So that's why do your homework, do your research in terms of which tools you would like to use and then, yes, invest the time to research the tool. I just told it. That's so, so, so important. Invest time in research in automation tools, otherwise you will fail the game of automation. You will notice really fast that the next hype tool might be not the right one for you. Maybe Maestro is not the right tool for you. That's totally fine. Maybe Appium is the one for you to go because maybe you can more customizing it. You can, sorry, you could build your own frameworks around it and whatnot, but really invest your time in the right tool for your setup. Yeah. And that's important for me to tell you today. Last but not least, I only can say thank you. Thanks for coming by today. So if you have any questions, there's my email address, there's my blog, there's my YouTube, there's my my ex, there's whatever, let me know in the in in in the Q and A session right now. And I'm happy to help you out in the mobile space. I'm happy to share my knowledge. Thanks for coming by today. Have a great one and see you soon.

Thanks again for your Automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com a524 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation testing and DevOps. If you haven't yet, join us, our vibrant community@testguild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges. We're excited to collaborate. Visit testguild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

With lutes and lyres, the bards began their song A tune of knowledge, a melody of code through the air it spread like wildfire through the land Guiding testers showing the secrets of to behold.