257: Oxygen Appium Selenium Made Simple with Nachum Dimer

Hit it.

Welcome to the Test Talks podcast, the place to go to geek out on software testing. And now your host whose mission is to help you succeed with test automation, Joe Colantonio.

Hey, it's Joe. And welcome to the Test Guild Test Automation Podcast. Oops, sorry, a little bit too early. If you haven't heard already. In the previous episode, I gave you a sneak insight into what's coming up in the next few months. I am working on totally rebranding my blog podcast into one site and then releasing other podcasts around niche topics like performance testing and security testing. So Test Talks is going to become the Test Guild Test Automation Podcast. And Security testing is going to be the Test Guild Security Testing podcast. And Performance Testing is going to be the Test skill Performance Testing podcasts. You get the idea, but just trying to prep you so that you're not caught off guard. So if you've been listening for a long time, Test Talks is going to become Test Guild Automation Podcast. Just want to give you a heads up and get you kind of used to the idea. Before I actually do it today though, we're going to talk about a new automation framework. So are you like me and you really like Cypress IO but feel like it may be too developer centric for your testing team? Or. Or do you still really want to use Selenium and Appium? You don't want to use a tool or solution that uses something else behind the scenes. You want to use what's the industry standard, but in a simpler, more straightforward way. If so, this episode is for you. We're going to be test talking all about the automation framework Oxygen with the co founder of cloudbeat, Nahum Dimmer. So listen up and learn about another open source tool that might be the perfect fit for your automation efforts. Check it out.

Test Talks is sponsored by the fantastic folks at Sauce Labs, the cloud based automated testing platform that eliminates the need to maintain your own Selenium grid and test infrastructure. Try it for free today. Visit testtalks.com and click on the sign up now link under the Homepage Sponsor section.

Hey now, welcome to Test Talks.

Hi Joe, thanks for inviting me.

That's great to have you here today. I'd like to talk all about automation in general, but specifically about your Oxygen framework. But before we get into it, can you just tell us a bit more about yourself?

Right, so I would start first with, you know, thanks for doing a great job with Test Talks. I think it's a great vehicle for QA engineers and developers to, you know, get know better the industry trends and get more people and leaders and so on. So thank you very much again for inviting me.

So I will start with a little bit background of myself. So I as my name probably hints I'm Israel in a home even so I was born in Russian Georgia, moved to Israel when I was 15 as many Israeli entrepreneurs. My initial background in software started in the Israeli army. So I started started with building one of first intranet platforms in Israeli Air Force. And from there I moved to become a full time software engineer in various.

Enterprises in Israel and international companies. And this was for my first six years or even more years of my career. So I haven't started as QA engineer as a tester, I started a software engineer. But then I felt I wanted to move on to do something more meaningful in terms of working with more with people and so on. So I moved to pre sale and this is how I get to qa because I became a pre sale manager company called Radview. So I was exposed to whole QA world and I was helping companies to use different QA tools whether in test automation field or performance testing field. And I find this very challenging and interesting because when you are a programmer you just write your code using the latest technologies and your challenge is pretty much the same every day. While when you are a tester your challenges are changing and actually you work on different technologies. Sometimes you testing very old fashioned outdated technologies, sometimes the latest one. And what I found also fascinating is that on one side software engineering is so much innovative field, on the other side, many QA techniques of practicing are coming from, you know, from practices that we used to do 20 years ago. And I think one of the biggest challenges today is how we adjust to, you know, much faster pace of release software release cycle. Anyway, so this is how I started in QA and I have my consultancy business for last 10 years, again mostly focused on performance and automation testing.

For the last four years I'm part of great initiative which we call Oxygen, which I will tell in a moment. And we also have a startup company called cloudbeat which produces tools to help QA engineers, testers and developers to get to the next level with the testing.

Awesome. So clearly you have a lot of experience, you've been consulting for 10 years. So just curious to know over that 10 year span, what's the state of automation you find yourself in today and how has it changed from the past?

Yeah, I can tell you when I started in 2005, so test automation state was completely different than it is now. You know if we take last 10 years, beginning of 10 years, which is roughly 2005, 2006 and 2007. Still I think market was dominant by mostly I call it old fashioned tools like qtp. And I think still most tests were manual with a little bit automation. Most automation I think was focused on ui. I don't remember any my clients at least having API test automation for example or automated integration testing, very little unit testing. But we still would go 10 years back. We're still talking about release cycles of year or twice a year now if we take a look now like last few years, I think release cycles have shortened to usually two week cycles. Right. Some companies release on daily basis or even a few times a day. Right. So I see much more automation testing happening on. I think the testing pyramid has flipped. So I think we see much more unit testing now, which I personally consider as part of automation testing and a lot of integration testing happening today, especially in CI CD oriented companies. I still think there is a lot of UI testing and I don't think it's going to change. I still think UI testing is probably one of best things to kind of see over how the platform works. But I still think UI testing is still the biggest challenge to the on market because I think if you take like API testing or integration testing or unit testing, it's pretty much straightforward. While I still think most companies are struggling with getting up to speed with UI testing and I think we can elaborate it more in our conversation today. But you know, there are so many frameworks today, so many tools, you know, some tools which already exist to market 10, 20 years, some newcomers. So on one side there's a lot of choice today and a lot of flexibility with almost industry standards like Selenium and Appium. On the other side I see de facto most companies still struggling to create a proper and stable test automation process, especially in UI field.

Absolutely. And like you mentioned, there are a lot of frameworks out there. I have a post, I think I have 48 or 50 of them listed. That's not even half of them. So I guess before we get into frameworks, I guess Oxygen is a framework. Just tell us a little bit more about Oxygen and then I'd like to break down why you thought there was a need for another framework.

Right, right. Just let me first to give categories to a framework because you're right, there are so many of them. But if we look at the framework type, usually we have classical unit testing frameworks like Junit, Testng, Nunit, Ms. Test and so on. And we have some frameworks more oriented on UI testing, automation testing, like Cypress. So it's interesting change which I think market evolves because I think like, you know, if we take 10 or 15 years back, I think we would typically find, you know, a classical unit testing frameworks. Now I think some frameworks are coming to solve purely test automation problem and not just coming for being developer oriented frameworks. So with this in mind, basically what we felt that in today modern world, I think QA engineers kind of stuck in between. So there are plenty of frameworks which are developer oriented As I mentioned, JNU, TESTNG and so on. There are probably 3, 4, even if not 10 frameworks per language. But those frameworks are usually very difficult for QA engineers to work with because they're oriented on unit testing, is oriented on developers. So trying to integrate with these frameworks, UI testing particularly and also API testing, sometimes it's quite challenging. Right. On the other side there are frameworks which are much more curated. I think Cypress is probably a good example. Even so I think they're very well also suit developers. But I think Cypress for example is a good piece of framework which helps also automation tests testers or manual testers to start working with automation. But we felt like it's more developers oriented. We felt there must be a better way to basically create a framework which on one side is simple, simple enough for even non programmer background people that like manual testers. On the other side it's powerful enough. So if you are more advanced tester or you are a programmer, then you can write more advanced tests, adding more layers to your automation, not just UI adding, API testing, database testing and even IoT testing. And so we felt so there is really either like frameworks which are just really simple and built for UI testing, but less suitable for all inclusive integration testing, or there are really developer oriented framework which are less suitable for UI and less suitable for classical QA engineering. So we saw there there is a gap and we wanted basically to help QA to move forward and adapt to CI CD processes to shift left. And so this is why we created Oxygen having in mind different scope of users, developers, QA engineers, DevOps and even it for.

Production monitoring. So this is what we had in mind when we created Oxygen.

Cool. So can you give an example of what you mean how you made it all inclusive? Is there a different view from someone you consider someone starting fresh or someone that's a developer?

Right, Good question. What we decided to do, we decided to divide Oxygen to two parts. We decided to Divide it to purely framework part which you can just take it as a framework, you can take your favorite ID.

And use it. And another part which we added on top of it is our own Oxygen ID which allows actually for less skilled or less technical people to be able to record tests out of box, to be able to debug test without need to write compilation based languages or needs to write a complex architecture like in case if you just use Java or C or any other compilation based languages. So we decided to split it to two parts and another decision we made is to use JavaScript because you know, we, as I said simplicity on one side. So simplicity, you don't really need a programming language. But then if you don't use, if you go purely codeless then you usually quite limited in certain level, especially if you have a large testing project which needs more than ui, you would be typically limited by capabilities of codeless framework. So we decided to combine both. We decided to combine some elements from codeless approach and some elements from Codelet. And we decided that the basis for our framework will be JavaScript because we believe this is one of the simplest programming language today and one of the most popular. So this was our initial approach to split into two parts, framework and ID on top of it and use JavaScript as a basis.

So I guess one of my other questions was how does this differ from like Selenium ide? I know Selenium, you can port it out to different languages. You just mentioned this just uses JavaScript. Are you able to export the script out into any language binding or is it just JavaScript?

No. So in contrast with selenium IDE which is kind of recorder and generator for other languages. So we build all in one fully integrated solution. So you use oxygen ID to record the test and generates very simple JavaScript which actually looks like KDT. So if you don't know JavaScript even then you can start using straight away our framework. But then you stay with it. So it's not an extension for just Recorder for Junit or Ms. Test, it's standalone framework which you would use. And again.

We think it's very much suitable for companies or for people who don't have any framework. It doesn't suppose to replace existing frameworks. Like you know, if you have Junit framework and you're happy with it, then that's fine. So it's not extension type of other frameworks, it's really just standalone framework which gives you all in what you need in Framework.

You didn't mention Junit but Junit I would think is more unit level. So does this also. Do developers actually use this for unit testing?

No, Oxygen at the moment.

Its purpose is not to be used for unit testing, but you can combine it alongside unit testing, classical unit testing frameworks like Jasmine and JavaScript or other frameworks as well. So you can combine them either if you are in JavaScript then obviously it's much easier for you to combine this framework with others. Or if you are not in JavaScript then you can combine it more on CI CD level. But this is not unit testing framework, at least at the moment.

Okay, so if I'm following along. So I would think Oxygen sounds to me more like you mentioned Cypress and that it's like an all in one solution. And it's not the same as Selenium ide, which is more like a helper tool where you can use it, but you can also export things and do more like this is an all in one solution, I guess is what I'm saying. All inclusive.

Yes, it's all inclusive. And what's important to say is that we. Because again, testing today is much more wider than unit test and UI test. So for us was very important to support more than just UI testing. So Oxygen, first of all, Oxygen is working, is fully compatible with Selenium and Appium. It's basically a wrapper framework. So we don't use any our proprietary engine.

Like Cypress do. So we just integrate with Selenium and Appium. But on top of it you can actually we have models different, we have I think eight or nine models. So you can also integrate in API testing, for example. So we have dedicated model for REST API for SOAP API testing, we have model for database testing, we have model for SMS testing, for example for email testing and even IoT testing. So you can even connect your IoT device with stereo port to Oxygen and actually create the integration testing where for example, some customers, they use Oxygen to simulate some actions on mobile device and then they go to their IoT device which operated by this mobile app and they verify that whatever they did actually happen on physical IoTP device. So I think the key of oxygen is UI and integration testing rather than unit testing. Wow.

So it sounds like then you could do like an end to end test almost. So you can do a setup and an API test, call a ui, do some manipulation there and then do something on the back end all in one test flow.

Exactly, exactly. The idea is exactly as you said, to allow fully end to end test, whatever it means, you know, if you involve SMSs databases, IoT, whatever you use.

Great. Now once again, I always just keep going back to Cypress. I know Cypress does not use Selenium under the covers. You are using Selenium and Appium under the covers. Is there anything you're doing to help enhance that experience? Like built in weights or anything that people struggle with? With Selenium out of the box because it's just an API, they need to write all the scaffolding around it because you have the scaffolding around it. Now, are there any other features you have baked in to help overcome some of the the issues people face with Selenium and Appium?

Very good point. So basically we call oxygen Selenium on pillows. And the reason why we call it this way is because, you know, working with pure selenium and Appium is quite tough. Right. So what we added on top of it we added different hooks which for example, as you mentioned, we have automated weights for different conditions. We handle for example what's called stale element in Selenium, for example, one element is covered by some invisible element or it's not visible at the moment. We have some automatic handling for these situations. We have some automatic handling for scrolling the page if your element is below the fold. For example in mobile space, in Appian we have some special handling for reading SMSs on device, for switching to flight mode, to switching on and off network, to doing OTP authentication and so on. So definitely there is quite a large or substantial amount of add ons in Oxygen to help you overcome typical issues with Selenium and Appium.

Very cool. And I just keep going back to that scenario you gave of integration testing. Why do you feel integration testing is so important? Why did you bake it in?

Because we saw again coming through with consultancy background and seeing tens if not hundreds of different projects over the last 15 years. One of big problems organizations have on top of just regular basic sanity and regression testing and unit testing is being able to test application with all its components in some kind of staging environment. I saw this one of main failure points of many organizations. So your application can perform brilliantly when you test it apart, but then when you put it together with some third party integration and so on, especially when today architecture of most of applications is quite complex, rely on many either internal or third party services and I'm even not mentioning microservices architecture. So there is a lot of failure on the integration part. And the thing is that many organizations approach integration like a kind of last moment testing. So it's really frustrating when you almost want to go live and then you figure out that there are so many problems in your integration environment and many organizations find it difficult actually to test it because there are so wide variety of different technologies involved in and because of most frameworks don't give you all this coverage. So you just need to write a lot, a lot of code to do it. So this is why we decided to improve this part and kind of focus on both UI testing and integration testing.

Cool. And I guess another reason why you may have been using going back to Selenium and Appium is I think I saw a quick video demo and you're able to interact with more than just Chrome where I think Cypress is just tied to Chrome right now. You're able to handle more than Chrome or do you just what browsers I guess do you support?

We support every browser that Selenium and Appium supports. And I think it's again very good point because you know, we need to separate I think today testing in pre and post merge, right. So just you know, to explain to our audience. So like pre merge test means developers committed some code and there's some quick testing that needs to be run to make sure that this commit makes sense. Right. From quality perspective, post merge means that developers code is already in kind of master branch or some kind of branch where it's together with a lot of code and now we can actually test it more towards integration, you know, adding more components and so on. Now the thing is that when again my, my feeling is that one of weakest points today in organization is actually post merging testing and in post merging testing you want to test in on widest possible variety of devices and browsers and infrastructure and so on. And this is where Selenium Napium doing a great work, a great job, right? Because Selenium app literally allows you to test.

Your application on any device. It's so flexible, right? So you can use your local infrastructure, so you can connect your local browsers or your local phones. You can use cloud providers as well and get your either virtual device or physical devices anywhere in the cloud with fraction of price, which would cost you just to buy all these devices. And I think all these capacities are coming with Selenium and Appium. And if you just take frameworks which are locked on particular device or operating system or particular browser, you don't have this flexibility. And I think in post merge testing this flexibility is very important. This is why we initially decided not to develop any of proprietary engine. So we have only our recording engine which is proprietary to Oxygen and but all playback is done fully on top of Selenium and Appium. So whatever you can connect to Selenium and Appium you can actually run tests on using Oxygen.

Yeah, I love this concept of pre and post. A lot of times people say choose this tool or the other. I think you use the right tool at the right time for the right team at the right point in your pipeline. So that's a great, great point. Yeah. So along those lines then, Selenium Appium are both open source. Is Oxygen open source? Is this a paid solution? Solution? How does this work?

No, it's all open source. Of course we felt that, you know, Selenium Appium contributed so much to testing world so we felt, you know, we almost must to make it open source as well. Because I think there is something great in having ability to adjust or contribute or extend tools and I think if you just release something which is proprietary makes it very hard for industry to adapt and very hard for industry to extend. So we are big believers in open source and Oxygen is fully open source solution with no.

Asterisks or small letters or anything like this.

So one thing I always worry about with any sort of tool, when it's open source, it's great that it's open source, but how popular is it, how much is it being developed? If I was to recommend this to someone, what's the likelihood that in a year or two it's still going to be around and you're going to be responsive to any bugs that they may find?

Yeah, good point, good point. So first of all, Oxygen is quite a mature project. It's already in development almost four years now and it's used over 400 companies worldwide. Unfortunately it still doesn't have a big contribution community because we actually we never invested in promoting it and making it publicly kind of talking about it. So we do it now for last half a year, you know, we were primarily focused on Israeli market because our operations, commercial operations were in Israeli market. But I think in the last half year we really got a boost for Oxygen also worldwide. And the good thing, I think it's a very, very good question, you know, what was the perspective for next few years because there are many frameworks which just disappeared over time. But again we are commercial organization. So cloudbit is a quite established startup which doing other things in Oxygen. So we have our test management platform which is a SaaS based and it's commercial. So we have our funds coming from our commercial activities to sponsor and fund Oxygen. As I said, I Would love to see more contributors of course, and not just because it can ease on us, but it can also I think bring the variety of perspectives and extend as a framework from technology perspective. But in terms of supporting it, I think it has a good backbone.

What we do in cloud bid. But again Oxygen is not premium product, it's fully open source product. There is no product which kind of Oxygen plus.

Yeah, so just to clarify because once again I'm going to the spec to the Cypress model they have like it's not really open source, it's freemium. But to run in parallel or do certain things that are critical, you need to pay for it. So is there anything gutches like that with Oxygen? Like it can't run in parallel unless you pay for it or you can't get reporting unless you pay for it?

No, actually no, we have both. We have built in parallel testing. Okay, so all you need, we have examples in our documentation is to configure a simple JSON file and you can run as many parameters test as you like. You already have built in integration with most of cloud providers. So if you want to run it on cloud infrastructure you are free to go ahead. I think we have if I'm saying four or five different reporting engines and you can extend it. Again it's open source. So you have HTML reporting, you have GI unit XML reporting, you have Excel reporting and so on. I think there are two or three more reporting engines built in. So there is really nothing which you can't do with Oxygen. The thing is that we, I mean the direction we are going because the cloud bits has the solutions to manage testing cloud. So what we going to do is to integrate Oxygen optionally for people who want to be able to store testing cloud. Okay, so basically today Oxygen is fully a local solution. Okay, so your scripts are just local files. You of course can commit it to your git or to whatever source control you use. You can run it either locally from developer or tester machine, you can run it from CI cd. So again there's no limitations at all. We do see that some of our user asking us to have some kind of more postman model where you can store your test in cloud and be able basically to cooperate with the teams. So you say you are team of 10 or 5 or 2 tests doesn't matter and you want to share the same code and on one side another side not, you know, with more straightforward models in git. So just you know to be able to change something without need to commit it and to you know, do kind of pull requests and so on. So I do believe later on this year Oxygen will be integrated with a free tier of cloud bid where you'll be able to store the tests. But again it's just integrations the same way I would integrate Oxygen with Jira or any other product.

So Oxygen itself, there are no limitations, no premium model. Everything you can do there you can do in any other tools we build.

Cool. So say someone loves Oxygen. They're like the company that made this are insanely cool. What else does cloud beat then have for them if they wanted to check out some other solutions that they might pay for?

Right. So basically cloudbeat test allows you to solve, I call it last mile problem. Right? So Oxygen helps you to solve first mile problem. What do I mean by first mile problem? I mean it allows you to create chests, right? So without creating tests you can't run a proper large scale test automation process. So this is the last mile problem. Other things we do in cloud is help you solve last mile problem. What's the last mile problem Is when you have a lot of tests, hundreds, thousands of tests and doesn't matter if you have unit test or automation test integration test, any kind of automation testing, okay. Not manual testing. Your last mile problem will be A how you know how you trace the scope of test, what you have. Right. And how you map them to your requirements. Okay. B how you manage your test lab. Okay. So how you run all this amount of tests on variety of devices, how you build your test suits, how you switch between different environments. So today if you take say you know programming based frameworks like as example Junit, it really doesn't matter. All frameworks in this sense are the same. Then you will end up doing a lot of configuration in files. So you would typically hard code user device IDs, you will hard code environment variables in files and so on. And then in CI CB environment many times you will fail just because somebody overtook your device or something has changed in the environment or you need to just change your test suit and suddenly Q Engineer needs an access to DevOps environment and so on. So we take this pain out with Cloud BTest. In Cloud BTest we allow to centralize all the testing part. So basically you upload all your tests and in this sense it doesn't matter if it's Oxygen test or Junit test. So cloud bid at the moment work with Java and JavaScript technologies and later on we'll work also with. NET and Python and other technologies. But the idea is that it's Centralizing it's kind of quality hub for your DevOps oriented organization. So you bring all your testing to cloud and then cloud boot allows you to set up your test environment. You can connect either different cloud providers or you can bring your local infrastructure. You can manage your test suite centrally and of course you can. Most important you can why your tests are failing. If it's due to flakiness or is it due to actual defects that you uncovered or maybe due to to environment problems. You can store all the results in one central place which is very powerful because you can see trends, you can do root cause analysis. And the idea is further down the road where we are going is to provide you also kind of business intelligence and artificial intelligence or what's the impact on your business of test failure. So this is kind of other solutions or other solutions we are working at and we have also at the moment.

Okay, now before we go, is there one piece of actionable advice you can give someone to help improve their automation testing efforts? Let us know the best way to find contact you or get a hold of Oxygen.

Yeah, I think well in terms of contacts is best just to go to oxygenhq.org and you can download Oxygen and Oxygen ID there. It works on any operating system and I do recommend you to also look at our documentation. Also if you're just starting your testing career I think there is a lot of information there to help you get started with Selenium and Appium and of course with Oxygen and Oxygen id. So this is on purely just how you get started with it. My general recommendation to testers is just open your eyes and be aware of testing more than just ui. Okay. I really encourage you to be more technical to look wider than just a scope of testing you typically give it because from my perspective look 10 years ahead. Then you know every tester will have to really understand well a technological background of the application he's testing. So I think my biggest recommendation is and I see a lot of testers have this concern is just don't be afraid to be technical. Go ahead explore different types of testing. If you want to start small, just take oxygen, take Postman, take Cyprus or take anything which is quite a smooth start for you and then just go wider, be more technical and help organization to test most difficult aspects which usually hidden behind the scenes in the kind of integration testing.

Thank you Nahum for your Oxygen automation awesomeness links to everything of value we covered in this episode. Head on over to test talks.com257 and while you're there. Make sure to click on that. Sign up for a free trial link under the Exclusive Sponsor section to learn all about Sauce Lab's awesome products and services. So that's it for this episode of Test Talks. I'm Joe. My mission is to help you succeed with creating automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Talks podcast. Head on over to www.testtalks.com for full show notes, amazing blog articles, and other automation awesomeness.