TestProject OpenSDK with Kevin Dunne

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we'll be talking with Kevin Dunn all about Test Project and their new open SDK. Kevin Dunn is the General Manager of Test Project, ensuring Tricentis commits to innovation and delivering tools to create better software. He has a deep interest in the emerging trends in software development and testing. Kevin has been dedicated to collaborating with thought leaders in this space for a while. Kevin comes to Tricentis from Deloitte, where he managed testing at a large government and Fortune 500 engagements, delivering ERP implementations and custom software development. As one of the first employees at Tricentis, Kevin has seen many facets of the business, work and sales, customer support, marketing and product management. So he knows automation from all these different views. So you don't want to miss this episode. Check it out. The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs. Their cloud based test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial, visit testskill.com.

And.

Click on the exclusive sponsor section to try it for free for 14 days. Check it out.

Hey Kevin, welcome back to the Guild.

Hey Joe, thanks for having us back. Love to be on this podcast and look forward to chatting about these exciting new developments today.

Awesome. Thanks so much. Ken, before we get into it, is there anything I missed in your bio that you want the Guild to know more about?

Nope. I mean, I think you covered off on most of it, but I would say that the most exciting thing that I'm working on today is Test Project, which is what we're chatting about. And it's exciting to announce that it's just past one year of Test Project being part of the Tricentis family. So we're very excited to continue working on this exciting project and watching it continue to grow.

Very nice. And Kevin, for folks that haven't heard previous episodes on Test Project, could you just give a high level overview of what is Test Project before we dive into it a little bit more?

Yeah, absolutely. So Test Project is Tricentis community focused offering for test automation. So it is 100% free tool as you mentioned in the intro. And it's great for testing, web, mobile and even APIs. So it's built on top of Selenium and Appium. Gives you a lot of different options both if you're a non technical User. We've focused a lot of our first year of improvements on building a great recorder and enhancing a great recorder for those that don't know how to write code. But what we'll be talking about today is that we also support very much people that like to develop their test as scripts and their ides. And that's where the open SDK comes in and makes a huge difference.

Very cool. So great point. So before we get into that, Kevin, one other question I get asked all the time when people are concerned about free tools. A lot of times they say they like open source, but free they get worried about because something like Catalan Studio used to be free, right? And then they, they said they would stay free and then obviously they're no longer free. So I know you can't make commitments, things like that, but what's the vision for Test Project going forward as far as you know?

Yeah, that is a great question. And we'll talk a bit about for our SDK in particular, we have actually open sourced that because we're able to open source it and we want to do so where possible. Test Project is released as a free product, the platform itself separate of the SDK. And that's not because we have any desire to change that in the future or that we're scared to open source it. It's mostly because it is actually a cloud based platform. And so we deploy it and host it and it's a number, dozens of different microservices that run under the hood. So, you know, open sourcing it and making it available for customers to run just wouldn't be that beneficial. But we do understand that customers are a little bit wary of any tool that says that it's free. Now with what has happened with other tools that you mentioned like Catalan Studio. So we did actually make a pledge we call our Free Forever promise on our website that we will never make any of the core features, you know, that are available today. What you see in Test Project today will always be free. In the future we may release additional products that go around Test Project, you know, maybe things like a test lab or things like enhanced support for Test Project or things like the ability to have a privately hosted instance of Test Project, for example. That might cost extra, but we've made a pledge there that everything you see today in Test Project will continue to be free forever through Tricentis.

Awesome. What I also like about the tool as well as it uses best practice tools behind the scenes, so it leverages Selenium and Appium. So it's not like you're creating some sort of proprietary backend for a lot of the automation that people are familiar with as well.

Right, Yep. So it's a great thing because we allow you an easy way to get your tests into Test Project. So. So if you have an existing Selenium or Appium test, it's really just as simple as adding one line of code to sort of change from calling Selenium's drivers to calling test projects Open SDK drivers. But otherwise, because, as you mentioned, our SDK is built on Selenium and Appium, all of the commands that you're using today in Selenium and Appium are just available directly in our SDK. And with the new SDK, we've actually made a change which is quite exciting that it's actually reverse compatible as well. So should you decide, you know, hey, I've been writing tests in the Test project Open SDK for, you know, six months and I'm done with this. I want to go back to just pure Selenium and Appium. You can also change the test back and just go back to using only Selenium and Appium and not the Open SDK, just by changing that same line of code back from calling the test project drivers to calling the Selenium and Appium drivers. So you won't be locked in.

Yeah, that was actually my next question. So I'm glad you brought that up. So that's a great, great, great feature as well. So it's someone, like you said, no longer wants to use a test project. They could just revert back and they still have all their assets they created originally using Test Project, right?

Yep. And that was the whole idea, you know, is that we want people to be free to come and go as they please, but we think that as people start using the Test project Opensck, they'll see that there's a lot of benefits there that you get in the free package that we offer of Test Project. And we think that people will really come to enjoy using it and it should be very simple for them to pick up because, you know, all the syntax, all the structure of how you write a test would be the same. We're just filling in a lot of those gaps that Selenium and Appium don't solve on their own.

So, as you mentioned, this was acquired by Tricensis about a year ago, and I know you've been working on it for the full year, but I think this latest feature is a very big feature for you. So I know you've Always had the Tricentis SDK. Has it always been called the Open SDK? And if not, or if so, I guess what is the Tricentis open SDK?

Yeah, so we did, when we acquired Test Project, they did have an SDK and there were a number of customers using it already. It was a bit more limited than the Open SDK, which is why we've sort of announced this as a new product, because it's a sort of a wholesale change from the existing Test project SDK. So some of the major things that we added were the prior SDK only supported Java and C. The new SDK also supports Python. So this has been huge for us. We did our State of Open Source Testing survey, which we discussed with you on a previous podcast episode, and there we actually found that Python is more popular for automation engineers than C. So we wanted to get support for that. We've got that now and we also have JavaScript coming out soon. So that's one of the big enhancements. Another thing is we have completely Open sourced the SDK as I mentioned, so it's available on GitHub under an Apache 2.0 license, so people can go and actually look at what the SDK does. So it is truly open. Also mentioned to you the new ability to come and go as you please. We were trying to again create something that was open, gave people choices, didn't lock them in, and we think that that will help a lot with that. And I think the final thing that makes this SDK truly open is that we have distributed it on Maven and Nuget and also Pypi, so people can go get that off their favorite repository package manager and just download it directly into their ide. They don't need to mess with going into test projects, downloading an SDK, installing it, they can just really use it as they would use Selenium and Appium. So we think it will be much more kind of coherent and part of an open source strategy.

Nice. So that just brought up another question in my mind is you mentioned people could start using Test Project and then revert back to just straight up Appium, Selenium. How about people that have an existing framework, a large framework, already using Selenium and Appium, is it just a matter of adding the open SDK to the POM file and then they can just leverage Test Project from there?

Basically, yep. Once you have the SDK, wherever the test is going to be executed and you just switch the driver from calling to the Selenium or Appium driver to The SDK driver, then you're pretty much set, should still be able to use like if you've got BDD on top of that, or if you've got some sort of keyword driven testing framework or you've got, you know, some sort of other kind of custom framework that's driving Selenium and Appium should still all work because just really changing what's happening at the driver and sort of execution level, but shouldn't get in the way of what you're doing with the framework that sits above that.

Very cool. So I guess a lot of people may say, well, Slim's always supported Python, so why, I mean now Test project Open SDK supports Python, so what's the difference?

Yeah, it's a great question. So I think we're trying to not replace what Selenium has brought to Python, just only really enhance it with filling in some gaps that customers had mentioned to us in terms of just areas where they spend more time than they like, either having to customize what's there in Selenium or build out something new to fit their purposes. I think the big one that we hear a lot about is reporting. So without something on top of Selenium, the reporting is very minimal, I think, especially for people that are sort of in an enterprise context. So test project automatically creates reports and dashboards for you, gives you really easy way of either sharing those over email or sharing them over Slack and things like that with our out of the box integration. So that's been a really big one that I think people have been very excited about. You can filter the reports, customize them, et cetera. And I think one of the other major things that we offer, which is a big benefit as well, is, is just the whole agent concept that we have in test projects. So a lot of people have told us that they kind of struggle with managing the latest version of Selenium, you know, latest versions of any other dependencies that need to go along with Selenium on both their machine where they're creating the test as well as all of those remote machines potentially where they might be executing the test if they're doing parallel execution and things like that. And so the test projects, agents, I think people are finding it much easier to manage all of those different versions and keep their machines running and up to date because the agent is basically just a one click install that will package all those things up and put them on your machine, whether that's Mac, Windows, Linux Machine or even Docker. We have a Docker image now that you can use as well if you want to build a containerized test environment.

Now, that may seem trivial, but I always say this. I used to work for a large healthcare company. We had eight to ten Sprint teams of about eight people on each team. And inevitably what would happen is, you know, they'd get driver issues because I forgot to update the driver in the project itself. So I'd have to upload it and I have to tell them to merge down the latest and then it would work again. So it sounds like your agent takes care of all that for you automatically without you having to worry about it, right?

Yep, that's the idea. And so it's very simple. You basically can just go into Test Project the first time that you're getting set up and it will automatically detect what your OS operating system is and just say, hey, here's the right agent for you. You download it, you click and install it, just as you would install Microsoft Word, for example. And then you've got everything set up there. We've already set up all the configuration, so everything works nicely together. It's automatically hooked up to your Test Project instance as well. So it's automatically set up to send back all of the reports and things that will be coming from those tests that you'll be executing with that agent. So really very minimal configuration that you need to do to get that working.

Awesome. So, Kevin, this might be an odd question. I know sometimes core contributors to Selenium and Appium, they've spent a lot of their own time developing these tools, but sometimes they almost like an antagonistic type of approach between car contributors and then vendors that take that and then modify it. So, you know, I believe Jonathan Lips, who is the main contributor Appium, has actually worked on some Test Project things. Do you know who worked on the Test project Open SDK and how it's being received by some of the core users and contributors to Selenium and Appium?

Yeah, that's a great question. You know, we're always trying to give back to the community, which is a lot of the reason why we want to keep Test Project free. I think, to your point, when people are profiting off the work of other folks, you know, it's not always considered fair. So that's a lot of the reason why we don't want to charge for Test Project and kind of continue to distribute it as a free tool. We have worked a lot with Jonathan on the Appium side of things within Test Project, and he's been a great resource for us. We are Working with Baz Dykstra actually helped us to build the Python support for the new SDK. So I think, you know, many people know Baz. He's one of the most common kind of well known folks in the test automation space. And so having him on board has not only kind of helped the community relations, I would say, in terms of helping people understand what the SDK is here to do and, you know, the fact that it's for the community and not for profit and things like that. But also, you know, I think having him, he was able to understand a lot of the, you know, current challenges of Selenium for Python and sort of build the SDK in a way that we could overcome a lot of those challenges that we're now aware of with Selenium having been around for 20 odd years now. So I think it's been great working with him. And there's another number of other folks in the community that are working on training for not only the Python SDK, but the Java SDK and also C SDK. A lot of big names that we'll be excited to announce, so stay tuned for that as well.

So do you remember or recall what some of those challenges were that Baza recommended that you modify or include in the open SDK?

Well, I think a lot of it was just kind of making it a simpler experience to get set up. And so I think he did a lot, kind of the initial installation and configuration to make that easy. But also I think that he's done a lot as far as kind of advanced concepts to make it easier for things doing, for example, data driven testing. I know that he's focused a lot on that as well as keyword driven testing approaches and things like that, just making it easier for our SDK to plug into some of those frameworks. As you know, Baz is a big fan of bdd, so I think he came to it thinking a lot about how this SDK in the future could be used with other frameworks as well. So I would definitely recommend checking out. We just did a great webinar earlier this week with him where he spent about an hour and a half going through the SDK, how to get started with it, and kind of some of the finer technical nuances of that. So I'd definitely say if any of your listeners are interested to really do a deep dive and hear from him directly, that webinar is a great resource.

Awesome. And I believe he also, he's been a big proponent of rest testing for a while, using Rest Assured right Do you know if you built anything into the Python implementation that helps support API type testing?

We're working on that. We really started out with a focus on web testing, so we've really been focusing on kind of the Selenium part of the SDK for Python. The mobile part is coming out in the next couple of weeks and then there'll be API capabilities added in after that. So every SDK eventually will support web, mobile and API. Sort of had Baz focus on the web part first since that was sort of the largest number of requests that we had come in were for that. But yes, eventually we'll have support for that. So you can either write API tests through the SDK as sort of a coded test, or we also have the add ons for the recorder. So if you feel more comfortable kind of using a ui, you also have that option as well.

Great. I think a company builds a new feature, they usually have a target avatar in mind. So is there any optimal user you think would be the perfect person to use the open SDK? Who's the open SDK targeted for?

Yeah, that's a great question. I mean, we really think that this SDK is probably more focused at people who already have experience scripting with Selenium or Appium. We think that it will be really easy for those users to pick up and work with. I mean, I think if somebody doesn't have experience, they can obviously pick up test projects open SDK as kind of their first foray into scripting. But I think we're seeing that it's really best served as kind of a next step evolution from people who have at least tried working with Selenium and Appium, but maybe have struggled for reasons around not being able to integrate it well into their framework, or not having good reports and things like that, which the SDK would solve. So I think that's kind of our target audience is if you're using today Selenium or Appium for Java, C Sharp, Python or in the future JavaScript and you're not happy with it for some reason. We think that's kind of the core audience that we're going after with the SDK. And I think they'll get the most value because they can easily import their tests and they'll already know all of the syntax and ways of writing a test. Just be a very simple change to get up and get running with it by including the test project open SDK in their test.

So if those type of folks probably already know Selenium and Appium, they know the syntax really well and it sounds like it's just a matter of importing the open SDK and then starting to use the open SDK within your test. But how hard is it to become familiar with the open SDK? Does it really change the syntax that much for someone that is an experienced user?

It shouldn't really. All of the drivers should be the same driver commands. Sorry, I should say should be the same. So interacting with the app should be completely familiar experience. We've tried to really eliminate anything in the SDK which doesn't exist in Selenium or Appium natively from a driver command perspective, because we want to make sure that we can really have that reverse compatibility that we mentioned. And if we added in, you know, a bunch of other commands that don't exist in Selenium and Appium, then if you try to revert back to Selenium and Appium at some point, those tests are not going to work right. So should be very, very much aligned to the existing set of commands in Selenium and Appium so that A you can easily import your test, but B you can easily export your tests back into Selenium and Appium. Should you feel that you don't want to continue a test project for some reason?

Great. Another thing, I think testers, the developers in general, usually have an IDE that they enjoy using.

Yep.

Does this work with other IDEs like VS Code, Pycharm, IntelliJ, or are there any anything that doesn't support familiar ides that people may need to know about?

No, it should be compatible with all the same ides that people are using today in those kind of Java, C, Sharp and Python ecosystems. So shouldn't be any problem with that. Again, you can just download it from your package repo and get going using it. And because the syntax is the same as Selenium and Appium, it should work from a formatting perspective as well.

Are there any benefits of using Test project in the open SDK with the ide? Does it add any extra features or functionality?

As of today, we haven't really focused on that yet, but we are certainly looking at ways that we can enhance the developer experience. You know, I think getting this SDK out there first was the first major step, but then, yeah, in the future we definitely want to continue to develop the experience for the more technical users. And I certainly could foresee in the future some either IDE plugins or just advanced kind of features for the SDK that can be integrated at a deeper level with specific IDEs but part of the kind of process is we want to figure out what IDEs people are using with the test project open SDK so that we can really focus the attention of what we're going to build in terms of specific IDE support on the IDEs that our users are actually leveraging.

Great. So we touched on this a little bit. You know, Selenium and appium, they're just APIs. And a lot of times you need to build all the infrastructure and all the boilerplate code that goes along with that. But Test project does the heavy lifting for you and does that. I know you mentioned one feature that it helps is with reporting, so you don't have to develop your own reporting type solutions. Is there anything else in the open SDK that helps testers so they don't have to recreate the wheel?

Yeah, a lot of the other things that we see people struggling with are just from a maintenance perspective. Not that they can't be built, but just, you know, there's a lot of changes that often break them is integrations to third party tools. Things like Jenkins QTest, browser stack, salts, Labs, those types of things are often common integration points and a lot of people are sort of reinventing the wheel many times to build integrations to those tools. So we offer out of the box integrations from test project into those tools. So that is something that I think would help quite a bit as well. As we mentioned before, can send out reports through email and Slack as well. And those are other out of the box integrations we have. So I think those two things would be pretty major. The other DevOps tool chains integrations as well as the report integrations. And then would also mention that we have a REST API. So I know that people will want to build some custom integrations and one off workflows and things. And having a REST API that's stable, that's documented, I think that can help a lot as well. Rather than directly integrating tools at a code level, oftentimes will break when one or more tools ends up changing, when versions get updated and things like that.

So just to clarify, the open SDK is open source. So obviously just to clarify, that means anyone can contribute and look at what's in there right now. Correct. They downloaded it from Git?

Yep. Absolutely Everything in the SDK is open source. As I mentioned, the core test project platform, which is where like the reports are hosted or where we do user management and where you would do integrations into other tools, that's in the free product. But the actual SDK which you use to script the test that will be executed by Test Project. That's all open Source under Apache 2.0 license on GitHub.

Great. So it sounds like the biggest change was for me is the introduction of Python. So for the folks that may have missed it at the beginning, you support now Python, Java and C Shop. Is that correct?

Yes, that's correct. JavaScript will be coming soon, before the end of the year at this point is our plan. So we'll have that four languages. And I would mention as well, one thing that we didn't cover off on, which is a big change for many customers, is that now we don't require you to upload your test to Test Project either. As the central repository, a prior SDK, you would need to write a test and then upload it to Test Project, and then Test Project would then distribute it back out to agents for execution. We now support the storage and execution of tests locally without any upload to Test Project. If you want to just store the test project test directly in, let's say, your git repository, we now allow you to store them there without actually ever uploading them to Test Project, which should make versioning and collaboration in the SDK much more simplified and similar to how most teams, I assume, are working with Selenium and Appium today.

Did I help at all with performance? I don't know if there was ever a performance issue, but because it is all local, you can make it local. Does it help speed up at all?

Yeah, it does. And it also just helps a lot with some headaches that we had experienced with the first version of the SDK. People just having to package up the tests before uploading them and uploading them frequently, and having to figure out what to do when two people were working on tests at the same time and decided to upload them into Test Project and which version to accept and things like that. So it certainly solved a lot of those problems and just lowered the amount of storage as well that we have in the cloud. And I think it was more for security reasons than performance. A lot of users wanted to keep their tests local in Source code repository that's hosted behind their own firewall, potentially. And we can support that use case because if you download the agents and put it on your machine and your machine's behind the firewall, then you can go grab that test from your local git repository and execute it, maybe even against the local testing environment, which is a big win as well.

Cool. So if someone wanted to get started and Once again, the easiest way, like say if it's a C shop, they would just go to nuget.org and they would just type in test project open SDK. How would that work?

Yep, they can do that. Or we've really simplified it for you. If you sign up for a test project account and you go into our sort of developers section in the integrations tab, we basically have built a kind of interactive wizard that gets the SDK installed. So if you're looking at C Sharp, for example, it's kind of three steps. Basically you get a developer token from your test project instance or the test project ui, which we show you in this interactive tutorial. You copy that, you go to basically nuget and you download the package and we give you the exact command you need to run from your command line. If you want, prefer to just do it that way. And then once you start up the SDK, it'll ask you to enter your developer token. That's just what connects your local SDK with the test project instance that you've set up so that the reports flow into the cloud repository and can be seen in the UI and then you're ready to go. You can just start writing tests or importing existing tests and executing them and the report should just automatically show up into your test project instance.

Okay Kevin, before we go, is there one piece of actual advice you can give to someone to help them with their automation testing, test project efforts and what's the best way to find or contact you or get our hands on test project in general?

Yeah, that's a good question. So my advice would be don't hesitate to reach out to us and we're here to help. We actually have a number of folks that working our support chat around the clock and can help you hands on to get set up with this if you're interested in it. So that would be my advice would be don't be scared to reach out to us, just come to our site or log into the app and you'll see a chat there. And we've got a number of folks who are very knowledgeable about this SDK who would be happy to jump on and help you out. So take advantage of that and we hope that we can talk to many of you soon as you check out this new open SDK and give it a shot.

Thank you Kevin for your automation awesomeness. The links to everything value we covered in this episode Head on over to testguild.com a 3:17 and while you're there make sure to click on the Try them for Free Today link under the Exclusive Sponsor section to learn all about Source Labs awesome products and services. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack Automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild. Com for full show notes, amazing blog articles and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.