SauceLabs and TestProject Team Up with Kevin Dunne & Kristian Meier

Five, four, three, two, one. Drop that. Welcome to the Test Guild Automation Podcast. We're all getting together to learn more about software testing and automation with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today's episode is going to be about the recent announcement of the team up of Test Project and Sauce Labs to help to create and deliver a codeless test automation experience that you don't want to miss. You definitely want to check out this episode. We'll be speaking with Kevin Dunn, who is an SVP of Strategic Initiatives at Tricentis Test Project, as well as Christian Meyer, who is a Technical Director of Business Development at Sauce Labs. Also, if you're listening to this episode, On February, on February 2nd, you have one day, one day to register for the biggest online conference in the world. Dedicated 100% to helping you succeed with Test Automation Automation Guild. If you haven't registered yet, you don't want to miss this event. Register now. Head on over to automationguild.com and register today. And even if it's February 3rd, or the 4th, or the 5th, or even after the conference is over, you can still register. Everything will be recorded and available for you within the membership area. Just head on over to automationguild.com and purchase your ticket today. The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs. Their cloud based test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial, visit testguild.com sourcelabs and click on the exclusive sponsor section to try try it for free for 14 days. Check it out. Hey Kevin and Christian, welcome to the show.

Thanks for having us, Joe.

Hey, Joe.

Hey. So I'm really excited. Today we're going to talk about Test Project and Source Labs and I believe is a new announcement about integration between the two. And that's why I'd like to dive in today. But before we get into it, Kevin and Christian, can you just tell us a bit more about yourself? Kevin, can we start with you?

Sure, Yes, I would be happy to do that. So I've been at Tricentis for, for about six and a half years and recently the last year or so I've been focusing on helping Tricentis surround out its open source tool portfolio. So we did an exciting acquisition of Test Project which we converted Test Project into a free tool for creating Selenium and Appium tests. And I've been helping the team with launching that exciting new product and Getting that integrated to Tricentis and looking forward very much to this exciting integration.

Awesome. How about you, Christian? Yeah.

Hey guys, this is Christian Meyer. I've been with Sauce now almost five years. Started off on the pre sales side working with some of our larger customers, you know, helping them get, you know, move their Selenium Test and Appium tests over to the cloud. And that involves your work with Frameworks and kind of, you know, working with Sauce. The idea there is you can scale out to help with parallelization.

Within the last year, I've moved over to be part of our business development team, working with partners like Kevin and Test Project on integrations between the two. I'm also really excited about this partnership because it allows customers to easily create Selenium Appium test scripts and then run that at scale with the Sauce Labs infrastructure.

Awesome. Yes. I'd like to dive into that as a tester myself. I've been really looking at Test Project more lately and a lot of other testers I know have been looking at it as well. So it's really exciting to see that they're now partnering with Source Labs, which is the de facto enterprise wide solution that I think everyone should be using for their cloud based infrastructure. Automation for sure. So how did this come together, the integration itself then is it's going to allow Test Project to run in Sauce Labs and it becomes almost a seamless cloud based solution. Is that the vision?

Yes, that's the vision of the integration. So it was come to be from many conversations with customers, many of who have been asking us how they can run tests on different types of browsers, different types of mobile devices that they might not have access to in their local Test lab within their corporate environment. So we got a lot of requests for that. And as you mentioned, Sauce Labs is really the leader in this area, so there was a clear desire to partner with them. And we've had a lot of success integrating some of the other Tricentis offerings like Tosca with Sauce Labs and a lot of happy joint customers in that regard as well. So really it's meant to be a very simple sort of seamless integration. We're doing a webinar actually on the 23rd of January to discuss this in more detail, but really it will just be essentially taking a Sauce Labs username and API token and putting that into an area and test projects that facilitates the integration. And once that's done, all of the Sauce Labs browsers and devices in that account will show up just as any other local browser device would. And Allow for seamless execution and reporting and whatnot on those remote devices.

And Christian, is there any other extra features that folks get within Source Labs? Is it just the normal awesomeness that you get out of Source Labs using any other option or tool?

Yeah, I mean, so for those, it sounds like we're all familiar with what we provide, but for those who are listening, who are not. So we have 900 different OSM browser combinations that you can choose from so you don't have to spin up that infrastructure. We have Chrome on Mac, Chrome on Windows, Linux, Firefox, ie, Safari, all of that, as well as emulators and simulators you can run against and real devices. So the idea is that you, you don't have to own and maintain that infrastructure. You can point to our cloud and run your test at scale. Really the idea there is we have large clients that have corporate mandates to get every time they do a code change, pull request, etc. They want to run their entire test suite in under four minutes. You get instant developer feedback from that. Like I said, I've been at SaaS a long time and one of the pieces that we've always been missing and realized that we could help out our clients much better is the test creation part of the test. Because there are companies that have large team of SDETs that are able to go in, create elegant Selenium and Appium scripts, run them at scale on our infrastructure. But there's also teams within organizations that they don't have the people that can do it. Or if you're a smaller team and you're a developer and you just want to write code instead of writing test scripts. Test creation has always been the piece that we're missing. That's why I'm very excited to work with Kevin Tricentis, the Tosca product, and now the test project product. Because developers can quickly just write tests. We're using this test authorization tool and run them at scale on our SaaS infrastructure. And we also provide insights from analytics, how your tests are running, what's passing, what's failing, hoping to get it so that you can trust your test automation. We give you all of those tools, we give you screenshots, videos, log files of the test where it failed, so you can easily and quickly find out which tests are passing, failing, and fix it.

Awesome. So Christian, you bring up some good points there. I know. I've attended saucecon for the past three years and I'm part of a executive roundtable and they always talked about it's great your infrastructure is awesome. But is there a way that you can get more people involved with automation with lower overhead or with a lower skill set that they can get up and running quicker? And so, Kevin, it sounds like Test Project is kind of could be one of those vehicles. And once again, Christian brought up the point. Maybe a lot of people don't even know what, maybe they don't know what Sauce Labs is, or they don't know what Test Project is. So maybe what is Test Project for the folks that don't know? And what is that solution that then enables them to easily create code and then they can run in Sauce's infrastructure?

Yeah. So Test Project is a completely browser based user interface that allows you to record and store and manage tests against both web and mobile applications. There's also support for API testing built in there as well. It's also got an SDK. So we're not just catering to people who want to do sort of the traditional scriptless recording that Test Project has been known for. But we've also been developing and SDK for the past several years, which allows developers as well to take advantage of the benefits of Test Project from a reporting and a test management and a visibility perspective. But in terms of where we bring value to the Sauce Labs community, I think that's definitely in the recording aspect, both for web and mobile. I think what's really unique about something like Test Project versus a lot of other solutions out there on the market that do a similar job of sort of recording and exporting a test to something like Selenium or something like Appium is a couple things. The first one being the user interface I mentioned. It's completely cloud based, so it's all happening within a browser, which makes it really nice obviously for recording against web apps, because it's not like you're switching between Windows to add assertions, add validations, edit steps. The recorder actually lays directly on top of your web app, which makes it really easy and effective from a recording perspective. The second thing would be there's a number of add ons. So we have an add on store which is available, allows you to extend the capabilities of Test projects. So for example, if you need to interact with Siri, for example, as part of your test script, Jonathan Lips, who's one of the developers on the Appium project, he just released an add on in our store which you can easily use to interact with Siri and add that customization to your test script without having to write any code. I think that's a really powerful proposition. Then the third piece is we give you a lot of the infrastructure that these traditional record and playback tools wouldn't give you. That's common in most advanced testing frameworks. So things like an element repository or ability to parameterize data with CSV data sources, or ability to reuse steps and nest steps within other test cases, those are all things that we give you within test projects. A lot of record and playback approaches fall short and are very brittle and don't get to automation maturity because they lack a lot of those elements that are common in a framework. And test project is essentially bringing a lot of those elements along with the recorder to come up with a more robust and more mature automation solution.

Kevin, you brought up add ons, which is one of the biggest selling points of test project, which I definitely agree. I'm just curious to know there any add ons that give you functionality that may not run in Sauce Labs? Is that something someone would need to be aware of as they're creating test?

In most cases, no, because what we're using at the end of the day is pure Selenium and Pure Appium to actually drive and interact with the different applications. So our SDK really is a layer just built on top of those engines. So at the end of the day all the interaction with the app should be happening through those native drivers. So pretty much everything that works in Test project should be working in the remote devices and browsers on Sauce Labs as well.

Yeah, because at the end of the day we're SaaS is just a. You can think of us as a large Selenium Appium Espresso Xui created infrastructure that you just point to us and it just works with a bunch of other features on top of that. But if it interacts with native Selenium and Appium commands, it'll work and run on our grid.

So kind of the reverse question Christian, is is there anything you find that automation engineers do that create issues when they try running in a cloud based solution like Sauce Labs?

Yeah. So as I kind of mentioned earlier.

Within their frameworks, when you write tests locally and just run it on your local machine, you may have one version of Chrome on it, you test it, you run it. And of course there's no latency between the app that's running on your machine and the test script that's running on that same machine. When you move across the cl.

You know inherently there's going to be lately so what we did. Well part of my job is and really like about Sauce is we have a great team of pre Sales engineers as well as customer service engineers, where we have solution architects and these guys, you know, these people, they contribute to the open source community. We have people that we know write Appium. We have the guy that does a Ruby Water Webdriver Bindings, we have Christian Broman from Webdriver IO. So we have a lot of people on our team that are very active in the open source community and are working on these issues that when you move from running locally on your machines to running at scale, we do, I think it's 3 million tests a day on our infrastructure. We have clients that are running thousands of tests in parallel. Again, the whole idea there is to get your testing time down to under certain minutes so you get instant developer feedback. Because, you know, as I know, as I may have mentioned before, you know, everybody wants to work on the ui. There's not many people that want to work on testing. So we've been, you know, you know, Sauce has been working, you know, behind the scenes within the open source community to help make these, you know, Selenium Appium open source more robust and able to scale. And it really comes down to just tweaking the frameworks a little bit in order to run at that speed and at that scale.

So, Kevin, is there anything that would add overhead from the test project side that could cause it to run slower when it's running on a cloud based solution than it would locally?

We haven't seen anything of that.

Type of issue. We haven't seen that surface so far in testing the integration. We've had a number of customers that have tried out the integration and a beta phase and the execution has been very fast from what we can see. So it seems so far like we're not seeing any issues as far as a slower execution time when it's done remotely. But we'll certainly be monitoring that and we'll make tweaks in the future if needed. But right now we're very happy with the performance we've been getting with Sauce Labs. And that's definitely one of the big differentiators that we use to pick Sauce Labs as our go to partner for remote execution.

Yeah, and on top of that, so there used to be the old Selenium ide, which was kind of a record and playback feature. They didn't really add weights, explicit weights to find the elements. So it was very brittle and it broke. Where Test Project is really put thought through all of the scenarios that maybe you wouldn't know when writing tests how to implement weights correctly.

Particularly when you across The Internet and on a grid like Soft Lab. So we have data centers. We have multiple data centers here in the United States and in Europe as well. So wherever you are, whatever the closest data center is, you'll be able to get that speed. And we haven't seen any flakiness in those tests that are created through Test Project. That's why it's been great.

Kevin, I'm not sure. This hasn't been officially announced. I'm just looking under the Test Project's documentation. It has a section on test on Source Labs integration. Is this the integration that is going to be announced officially next week? So just a quick update here. I recorded this before this feature was actually released. It is now released. So this integration is now live?

Yes, that is. So we've gotten ready. As I mentioned, we have some customers using the feature in beta, so we do have the documentation out there public. But the feature won't be there for all customers until the initial announcement is made on the 23rd of January.

So that's a good point. One of the other benefits of Test Project is you don't have to really install anything. It's one install. You don't have to configure a lot of things. So it sounds like when this goes live, do users need to do anything or will it automatically appear as an option for them?

Yeah, pretty much. It'll appear for an option as an option for them in the integrations area. They'll be able to click on that Sauce Labs option in the menu, and then they'll need essentially two different pieces of information, the email that's connected to their Sauce Labs account, as well as the API token. Once they enter both of those things, that's pretty much all that needs to be done. It's just one person within that Test Project account that needs to do that. So, for example, if you were a company and you had 50 users in your Test Project account, you would just need a single person to go in, activate that Sauce Labs integration, and then from that point forward, that user, as well as all the other users in the account, can take advantage of remote execution on Sauce Labs in that particular Sauce Labs account, which was integrated.

Why do you think folks should be excited by this? Like, I know why I'm excited by it, but. Kevin, any thoughts on why you think this is a win win for folks?

Yeah, well, in my view, I think it's just taking even more maintenance and even more overhead out of the hands of the tester and allowing them to focus their effort on what's Important which is creating and managing great automation tests. So in my opinion, having a hundred percent cloud based solution that allows you to go from all the way at the stage of test planning through test execution on a number of different browsers and devices is a really winning proposition because testers can now focus on the things that really allow them to add value, allow organizations to get to market quickly and allow them ultimately to deliver the value to the customer. So for me that's the most exciting thing is basically that we're unlocking the power for testers to reach their full potential.

And I second that as well. For us as a tester, as a developer, if you're in charge of writing tests as well, you don't have to think about, you know, creating the test. You can simply use it through the test project interface. There's nothing to download, it's a web page that you go to. Same with, you know, running your test. Where are you going to run this? You know, you don't have to chase around a real device that's in somebody's desk drawer or you know, in a different building or on a cart that doesn't work. I don't have access to it. All this kind of stuff, you just point to using test project. You just point to whatever device it is, whatever device you want. We always have day of the real devices that come out, day of the operating systems that come out. I can also test on older legacy devices if I run into an issue that is causing. We've had customers. Macy's is a great example of this where they weren't testing on Safari on Mac and they once they were doing that, they realized a million dollars a month more in revenue because people are actually going through and able to complete their purchases. Because if you're sitting on a computer and even, even I work in the industry, if a website doesn't work, I'm just going to go to the competitor and get it from somewhere else because I don't trust, I lose trust with the customer. So for me you no longer have to think about the infrastructure, obtaining the different devices, looking for different operating systems, setting up servers or VMs or equipment. All I can simply do, go to test project, write my test and point to any sort of infrastructure you have. And again, along with the ability to scale. That's why I'm excited about it.

Yeah, I'll tell you I'm excited by just being an older tester. It's almost like magic. If I was to go back in time and all of a sudden be thrown here that you can download a free tool that is low code and then you can point it to a service that has thousands of different browsers and operating systems and device combinations that you don't have to deal with in your own lab and running against. It's pretty amazing what we've gone through in the past 20 years or so. Are there any things on the roadmap, I know this is just being announced that people would get more benefit from? Any other features that you could see happening?

Maybe, yeah. We're certainly going to be continuing to invest in this integration. I think that there's a lot that we can do to just enhance the ability for the two tools to talk to each other as far as passing data back and forth about different reports. There's a lot of great metrics in Sauce Labs that we could take advantage of and test projects.

There's going to be definitely continued innovation in this integration. We have most of the basics covered off around the execution piece in the initial release of this, but I think there'll be even more to come in terms of deeper linking of the test results between the two tools in future releases. So definitely stay tuned for that.

Yeah, we have it. So we have your screenshots, your videos, your log files, what we call test assets already integrated in with Test Project and Sauce is obviously we have a lot of data. We're using data scientists for AI ML learning of just testing how to improve test quality, test coverage and all that. And I think as those insights come out and analytics come out on the test or on the Sauce Lab side will be integrated more into Test Project. Joe, one thing is you also mentioned Sascon this year, which you know, for your listeners will be Monday, April 27 through Wednesday, April 29 this year in Austin. So if you guys want to come out and learn more about this integration as well as, you know, just kind of all things testing, I encourage you guys to come check us out.

Yeah, I'll definitely vouch for saucecon. I think it is pretty much the only conference I've been to on site the past few years. It's really, really a great conference. It's not just pitches by Source Labs, it's really a community. As Christian mentioned, a lot of the contributors to these open source tools are actually employees at Source Labs as well. So you're getting the best of the best of both worlds there from vendors in open source. So absolutely. Kevin is someone from Tricentis, I'm sure gonna be there as well, I guess, to talk about this, possibly as a session Absolutely.

Yep, we'll be there and we'll be definitely sponsoring the event and should have some demos in the sort of expo area during the course of the event to show off this integration. So we're definitely proud to be supporting that event. And we've also got our event Accelerate, Tricentis Accelerate, actually right after that, same week as saucecon, so we'll be making some announcements and presentations about this integration there as well. So definitely going to be an exciting and very busy week in April for us.

Very cool. Christian, as a teaser to saucecon, is there any other features that Sauce Labs is really excited about that they'll be featuring this year?

Yeah, I mean, there's stuff that's coming out that will be announced at sascon. Like you said, it's a cool event. This is the fourth year we've had it. We always have speakers from the testing industry.

Inside and outside of Sauce Labs. We have customers that come and share their experiences on how to really.

Improve test automation. Because like I mentioned before, it's easy to push code, it's easy to get false failures or false positives, and it's easy just to say, hey, we're shipping it anyways. And then there's consequences down the road. But it's just great because like I said, there's a lot of people from the industry, open source, as well as vendors that just share knowledge and it's really one of those places that everyone's really excited about testing.

All right, guys, before we go, what's the best way folks could find more about Tricentis and Sauce Labs and about this new integration? Kevin, let's start with you.

Yes, so we have a webinar coming up January 23rd, so that will also be available on demand. So I know that's short notice for people listening to the podcast, probably. So check out that webinar either live or on demand. And then, yeah, they can find out more about Testing Project at Test Project IO. They'll find the free account, sign up and can also find our documentation to show how to connect Test Project to their Sauce Labs account. Yeah.

And from the sauce side, it's sauce labs.com and we also have a free trial that you can sign up for. And you know, we, like I said, we have a great pre sales organization as well as our CSMs on the back end. Customer support managers happy to talk about the integration as well.

Thank you, Kevin and Christian for your automation awesomeness. Links to everything of value we covered in this episode. Head on over to testguild.com a like automation290. And while you're there, make sure to click on the Try it for Free Today link under the Exclusive Sponsors section to learn all about Source Lab's awesome products and services. And as I mentioned, if you haven't already, head on over to automationguild.com or guildconferences.com and register for Automation Guild today. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Thanks for listening to the Test Guild Automation Podcast.

Head on over to testguild.com for full.

Show notes, amazing blog articles, and online testing conferences. Make sure to subscribe to join the.

Guild and continue your testing journey. This has been a Joe Calantonio production.