Top Free Desktop Test Automation Tools

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. And this is just a solo episode with no guests, just me sharing with you some knowledge. And I'd like to share with you a recent blog post I just re released on the top three automation tools for desktop applications for 2021 and beyond. If you do need sort of automation that is beyond a browser, you're going to need other tools other than Selenium. So I'm going to go over around 13 different options you have depending on what technology you need to test on a desktop to help you with your automation efforts. You don't want to miss this episode. Check it out.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs. Their cloud based test plat helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial. Visit testguild.com sourcelabs and click on the exclusive Sponsor section to try it for free for 14 days. Check it out.

All right, I think as a testing community I think we've been spoiled with a bunch of open source automation testing tools and for browser and API automation tools. But what about tools to help us with desktop application automation? I know I'm often asked the question do you know of any automation tools that could be used in automating desktop based applications? So if that's you, here are some of the top tools I found to help you with desktop automation. So I just want to make sure you understand this is not a list in order. So it's not best to worse or worst to best. It's just a list in a random order of certain tools that can help you with different kinds of automation. So the first one on the list is WinApp driver. And I actually was able to interview the developer of WinApp Driver, Yusuf Dor, who works at Microsoft on the Windows implementation of Appium to automate Windows applications. And so Windows Application Driver is a service to support Selenium like UI Test Automation. But the twist here is it's on Windows applications. So this service supports testing Universal window platforms, Windows Presentation foundation, otherwise known as WPF Window Forms, WinForms, and classic Windows W32 apps on Windows 10 PCs. I don't know about Windows 11, but we'll find out. And the Windows Application Driver compiles with the JSON Wire Protocol standard. It also has some of the same application management functionalities defined by Appian. So the pros are it takes advantage of the industry standard WebDriver protocol, which you're probably already familiar with. And a pro, I think it's a pro, it's a big company, it is developed by Microsoft. The con is. Here's the tricky piece now is that I just noticed on GitHub that that development has been put on hold for at least six months on WinApp driver and so it's been paused and based on the comments in GitHub it looks like it may not be a good option going forward. We'll see if it's been paused. It gives me paus to actually recommend it wholeheartedly, check it out and would think about it really hard before actually implementing this, before knowing 100% what the roadmap is for a WinApp driver going forward. All right, next on the list is a tool called winium. And Winium is a Selenium remote webdriver implementation for the automated testing of Windows applications based on winforms and WPF platforms in the why Winium? Section of their website and mentions that you have Selenium WebDriver for testing web apps and Appium for testing iOS and Android apps. And now you have a Selenium based tool for testing Windows apps as well. So what are some of the benefits? Well, one of them is because it uses WebDriver, you can use the same language bindings that you're probably already familiar with with Selenium. So you can write Windows based automation using Java, C Sharp, Python, Ruby, Clojure. Like I said, any of the language bindings that are currently supported by Selenium also would be some more supported by Winium. So that's one of the pros. One of the big cons I see if you go to GitHub and you look at the history over the past month or the past week or even the last update, it's been a while since it's been updated. So anytime I see an open source tool that has been updated, either it's super reliable and no update is ever needed, which I don't think is the case because you could see the actual defects that have been opened in it which have been kind of current, but it makes me feel like it's not being updated or there's not a lot of community support behind it. So just something to be aware of using this tool or any tool in general, always look to see how active the community is before actually going all in on a solution. Okay. Another solution on the list is Jubilee and So if you need to test desktop Java applications, Joobler might be the right solution for you. It's an open source tool for automating UI tests designed specifically for Java applications. It's also reasonably well known in the Eclipse community. It provides automated functionality GUI tests for various types of applications. It is aimed at teams who want their automated tests to be written by test experts from the user's perspective without requiring a lot of coding effort. So Jupiter Test incorporates best practices from software development to ensure the long term maintainability of the automated test. So pros of this tool is it's good for Java based applications automation and it takes a no code type of approach to writing tests. So it can help you save time improve the readability of your test cases. In general, some cons is is that it doesn't support running test in headless mode which some users have complained about. And it only supports Java, RCP, SWT, JavaFX and Swing applications. So if those technologies are not what you need to automate, then it's a it's not the right tool for you. Next we have Pywin Auto. So it's not really a tool, it's a library. But if you love using Python as your go to language for everything, which I do, and you also need to automate Windows desktop applications, you should definitely check out the library Pywin Auto. It describes itself as a GUI automation library written in pure Python and was developed for Windows GUI automation. And so at its simplest it allows you to send mouse and keyboard actions to dialogs and controls on both Windows and Linux. And it also has some complex text based actions supported only on Windows, not necessarily on Linux yet. And the latest release introduces MSUI Automation which includes WinForms, WPF, Qt browsers, Storap based test and more. So pros, since this is a Python library, it's great for testers already using Python and I love Python so I'm calling it a pro. That's a pro right there. A con is it uses Python. Believe it or not, there's some weirdos out there that don't like Python. So that could be a con for you. If you have to test Oracle applications, there's a tool for you as well. Next on the list is Oracle Application Testing Suite. So if your team needs to test any Oracle based software development that's going on in your organization, Oracle Application Test Suite, also known as Oats, is available. It contains a bunch of integrated products to help Oracle functional testing for regression testing of web applications and Oracle Load testing for performance testing of your Oracle Packaged solutions and Oracle explains that its Application Test Suite is comprehensive integrated testing solution that ensures the quality, scalability and availability of your web applications. Web Services Package, Oracle Applications and Oracle Databases this integrated full lifecycle solution enables you to define and manage your application testing process, validate application functionality and ensure that the application will perform under load. Also, with the Application Testing Suite, you can deploy your web application and web services in less time while maximizing the efficiency of your testing team. Pro is obviously if you need a tool to help test your regression and performance tests of Oracle based applications or Oracle databases, OATS is a must try tool. A con is that obviously it's not useful. It's not useful. You don't do any type of automated Oracle applications. Another on our list which was really popular a few years ago. I haven't heard as much about it recently. Not sure why is secoolie X Coolie X can help you automate anything you see on a screen of your desktop computer running either Windows, Mac or Linux Unix. So the reason why it's able to do that is sequelie X uses it uses image recognition powered by OpenCV to identify and control GUI component. So if you using a automation tool and you try to use a spy and it doesn't recognize any of the objects, it's like completely a black box to your tool, then the Coolie X may be a good solution for you. So Pro is it's a good option if you have an application that needs to be tested that has controls that aren't recognized by any other solution you've tried. A con is you know, obviously with image based automation it's always tricky and it's oftentimes unreliable because of pixel differences. I would use this approach only as a last resort. All right. If you're a dot net developer or tester, Fluid UI is next on a list. It's a dotnet library that helps you automate helps you with automated testing scripts for Windows user applications. So once again Win32, Winforms, WPF, Star Apps and Flui was developed because of one of the maintainers of the White framework, which we haven't mentioned. Had issues with White because the code base was really hard to update and support and it was a good move because I actually White has been deprecated. So if you're using White the White framework I would not use it going forward and I would try moving to Fluid to flaw. I don't even know how to say this. I think it's flaw flu flau to use Flaui instead, so obviously a pro it's a good option for automating Windows interfaces. Now some cons are it doesn't support multiple languages so it's only support C Sharp. And in corner to a few users in comments on GitHub they had issues creating a lot of extensions for it that are reliable so there's a ton of missing functionality they said and also doesn't have retry but mechanisms Another tool that's on the list that you probably know about if you've been doing automation for a while is autoit. It's a free basic like scripting language tool designed for automating the Windows GUI and for general scripting. When I originally wrote this post on Auto it, I said it was not the most robust of automation testing tools. I know many teams integrated AutoIT with selenium to work around non browser windows that appear in an automated test flow like a printer dialog or a Windows based dialog that popped up during a web based flow that they needed to interact with and auto it gives a way that you can actually do that using Java. But anyway when I wrote this I actually got a comment by Mohamed Marquis who told me that I probably didn't use it fully and that's why I said it wasn't very robust and he gave me a list of functionality that he actually said Auto IT can help automate and one of the things obviously was desktop application, but also Excel and Word, API testing and more. So maybe if it's something that you it just seems like there's a learning curve so if you really dive deep into it and come with a lot of automation but I haven't done enough with it. Some pros are it has easy to use syntax, it supports. Com. Some cons are it doesn't support Java. One tool on the list which is not open source but it is free is Zap Tests. It's one of the tools I've been hearing about for a while but never tried myself. I heard that it now offers a free edition. Zap Tests can be used to create automated tests against any GUI based application. It also supports any environment making a very flexible solution if you're looking for cross operating system support. So pro is if you're looking for a full blown desktop automation tool that also supports API testing, has a recorder, can run in parallel and integrates with Microsoft, alm, Rally or Jira. I would give the free version a try and see how it works for you. Some cons is it's not open source so that could be A hang up for you. All right, for game testers, you're not left out on this list. There are a few solutions. One is Aehr Test. Aehr Test is a open source test automation tool aimed at hard to automate applications like games. And the reason why it's focused on games is one of the largest gaming companies in the world. NetEase actually developed it. I think they developed it in conjunction with Google. So pretty sophisticated tool, has a lot of stars on GitHub, I think over 4,000. So pros are you write once, you can run it anywhere. And Aehr Test provides cross platform APIs. Some cons is it doesn't seem like a lot of worldwide adoption of it yet. So still on the fence of how strong the community is, but definitely a worthwhile solution to check if you're doing any type of game development for sure. All right. Speaking of games, I know one platform that's always very popular is Unity and I recently learned about a new solution, not a new solution. It's been there for a few years, but it's new to me. It's called Alt Unity Tester. It's UI test automation tool. So if you need to automate desktop Unity applications, you can use Unity not just for games, but also for Unity for applications, desktop applications as well. This is a tool you should definitely check out. It is an open source solution for UI test automation for Unity teams. You can use it to help you inspect and programmatically control objects of Unity based games and apps. You can also run your test on real devices or inside the Unity editor. So if you're doing anything with Unity, definitely give this tool a try.

All right, the next one on this list may be surprising to you and it's Appium. So I originally didn't have Appium on the list and then Jonathan Lips, who is the main contributor Appium, reminded me that hey, appium actually bundles WinApp driver, which we talked about earlier, as well as Appium for Mac which can be used on Mac desktop applications as well. So pros, most testers are already familiar with Appium. So cons, recently I need to double check with Jonathan as some functionality relies on Win App Driver. As I mentioned earlier, that particular project has been put on hold by Microsoft. It's not sure where this all fits now or if you should be concerned. I would be a little bit concerned because it has been paused. So definitely you may want to rethink this as a solution as well for the time being. All right, another one on the list is not necessarily a tool. It's a framework and it's robot framework. It's an all purpose open source automation framework and it could be used to test not only web applications but also has plugins for automating a bunch of different things like Kafka libraries for testing Java FX applications, has plugins for IBM mainframes, SSH, REST, SOAP, Java Web Style Applications libraries for testing SAP GUI clients, MQT libraries for testing Java applications with Swing GUI's libraries for automating Windows GUI. So it's an awesome framework that you can plug and play a bunch of different solutions and plugins to create an end to end solution that handles not only web application testing but all types of of what I call end to end automation, not necessarily just web apps. And I think for some reason the US it's not used enough. I know in Europe it's it's, it's, it's a bigger, it's more well known and I've spoken to a lot of people recently in the US that have used it and they swear that it's helped them really scale up their automation really easily. So I think it's a really good solution. It's been around for a while so some pros are it's easy to install and learn, it supports a lot of different technologies and has a very active community. I actually hosted in MC the online robocon this year and really great community, a lot of really enthusiasts so definitely check it out. Some cons are users Some users complain about the ability to refactor and debug and its debugging capabilities are kind of lacking since it doesn't support popular ides. So something to be aware of there. So if you're into keyword driven automation, the next on our list is a new tool I actually didn't know about it. I learned about it in the comments section of the blog. It's called nexeal NEX I A L and it allows you to automate Web desktop databases, APIs and much more using a keyword driven approach. So as I said I haven't tried it yet so it's definitely an option to definitely check out if you're into a keyword driven approach. It's a plus because it's easier to get up and running. So cons are, you know, I don't know how popular tool is yet, how much adoption there is and how the community is so be on the lookout for that. And last on our list but not least, like I said, this is not in any particular order. The creator of Karate actually told me that it actually supports Windows based automation, which I didn't know about. So Karate, if you don't know. I actually reviewed it really early back in 2017 when it was more of an API testing tool, but since then Peter Thomas has been really refining it over the years. It keeps adding more and more functionality to it. So it's much more than just API testing tool and now supports API testing, web automation, performance testing and much more, and also can now handle automating desktop applications as well. And as I mentioned, the creator Peter Thomas pointed out that it has a library called the Karate Robot. He pointed me to the Karate Robot Windows Install Guide which I'll have for you in the show Notes that shows you how to install Karate Robot and go step by step how to install it and how to interact with a Windows based desktop application. Also has a really good explanation how to install using Visual Studio code, which I know is becoming a more popular IDE with some folks using Karate. So I think it's a unique solution if you have to do end to end automation and you want to handle automating all the things including Windows desktop applications, Karate is definitely a solution you should check out and one that's often overlooked as well. All right, so those are my top automation testing tools for desktop applications. Let me know if I missed anything. I will add it to this blog post and give you credit for links to everything of value we covered in this episode. Head on over to testguild.com forward/a360 and while you're there, make sure to click on the Try for Free Today link under the Exclusive Sponsor section to learn all about Sauce Lab's awesome products and services. And if you haven't already, why not rate and review this podcast on whatever player you're listening to. Probably itunes ratings and reviews really help in the rankings of the show, and I read each and every review to help make the show even better. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full Stack Automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.