230: It’s back! Selenium IDE Reborn with Dave Haeffner

Hit it.

Welcome to the Test Talks podcast, the place to go to geek out on software testing. And now your host whose mission is to help you succeed with test automation, Joe Colantonio.

Hey, it's Joe. And welcome to another episode of Test Talks. If you're like me, you would have thought that Selenium IDE was long dead and buried and. And if I'm honest, I was glad it was. But in this episode, Dave Hefner will be test talking all about the rebirth of Selenium IDE and why it's a good thing. Discover why Selenium IDE was resurrected, how it can help you, and also some new improvements made over the older version that make it better than it ever was. You don't want to miss this episode, but before we get into it, I also just want to remind you that registration for the 2019 Old Automation Guild conference is now open. Automation Guild is the largest online conference in the world, dedicated 100% to helping you succeed with creating Automation awesomeness. You don't want to miss it. Make sure to check out our awesome speakers. Head on over to automationguild.com and for early birds that are listening as I release this episode, you want to register as soon as possible to get your hands on a free ticket for for Secure Guild? That's right. For the price of one conference, you get two conferences. So when you register for Automation Guild, you also get a free ticket to Secure Guild, a two day conference I'll be holding probably May 23rd of the 24th. Dedicated 100% to security testing. So head on over to automationguild.com, register today and pick up that extra ticket for free to Secure Guild. Head on over Automation Guild register today. Test Talks is sponsored by the fantastic folks at Sauce Labs, the cloud based automated testing platform that eliminates the need to maintain your own Selenium grid and test infrastructure. Try it for free today. Visit testalks.com and click on the Sign up now link under the homepage Sponsor section.

Hey Dave, welcome back to Test Talks.

Oh, thanks for having me.

It's great to have you back on the show. I guess. There's two main big announcements I want to talk to you about. First one is I believe you're now working for Applitools, so that's kind of cool. I like to hear what's going on with that. And the second one I think is you're working on a new version of Selenium ide. Not sure if that's a correct rumor or not, but that's what we'll find out. But so David, let's just first start off like it sounds like you did a transition recently to work full time for Applitools. I think you've been a consultant or free agent for a while, so you know, what's the deal? Why did you make that decision? What are you working on at Appli Tools?

Sure. So yeah, I've been self employed for about seven years I think and so a mixture of consulting, creating content, educating people around how to use Selenium and do test automation more successfully within their orgs and we more recently I switched gears and decided to go full time to work at applitools and the work that they've been doing on the Selenium IDE has been really tremendous. It's actually really great to see because I think that the Selenium IDE record and playback tools in general, depending on which group of people you hang out with in test automation, there's a stigma attached to record playback and I think that the stigma is rightfully earned in certain aspects because I think that when within the case of Selenium ide, I think that the tool wasn't modernized or built to keep up pace with some of the solutions to the known problems that we have in the industry. And and so the this was kind of punctuated when the old Selenium IDE reached end of life when I believe it was Firefox version 54 or so where Firefox departed from their prior experience extension platform to their new extension framework and as a result the old IDE just broke. But that was more of a coincidence because Apple Tools is already working on helping to breathe new life into ide. It was just, it turned into really a serious fire drill when it happens. But the tool is out, has been for a while and there's loads of people using it and it's actually becoming a really robust and great tool. So I was excited to work on it because for me it's kind of a full circle thing. If you flashback to.

Very early days for me in test automations like circa 2008, 2009, my first exposure to selenium was through the IDE and my first exposure with anyone in the Selenium community was in 2009 at the Agile Conference in Chicago. I got to meet Jason Huggins and Adam Goucher and at that time Adam Gacher was maintainer on Selenium ide. And I don't know it just from there I went and figured out how to export the code and then Kind of never gave much thought back to the ide. But then.

More recently, I think last year, Jason Huggins started talking an interesting game around advocating for simple tools. There was this anecdote attached with the IDE that was that everybody said, and I don't have the exact etymology of it, but I'm pretty sure Jason Huggins was the one that peddled this the most, if not created it. And it was that Selenium IDE is the training wheels. You use it to get started quickly and then you export to code and actually code it your tests to be more robust. And you get all the advantages of doing that. And you can get currency and you can add all these bells and whistles that you need for reporting and do all these things. That's kind of the path that most anyone takes that, that don't already use maybe a commercial tool that offers something similar. And at the end of last year, I want to say the fall SCP Con Software Test Professionals Conference, Jason Huggins keynoted and he had this whole topic around why he thought that that anecdote was wrong and he wanted to apologize for helping to spread it. He thinks that simple tools can be powerful and they can empower people to, to use tools in a way that basically demonstrates the solution that already exists to the known problem like these. We already have all these problems that people experience in test automation and they don't have to. There's people still running into the same problems over and over again, but there are known solutions for them. And I said the simple tools accounted for that. And so that's basically what he's advocating for. It's that we can have that we can have our cake and eat too. And he shared an experience of using.

I think perhaps the old version of Selenium IDE at the time, can't recall the details, but he used a simple record and playback solution to help test healthcare.gov to demonstrate a critical bug and was able to execute it in a platform grid in the cloud. And it was just like it took him minutes. It's either him or one of his colleagues together they did this and, and it really sunk in for him like, wow, this is actually really useful and super powerful. And I'm seeing kind of that, that kernel play out in the work that's happening within applitools. And I'm excited to join them because they have this long term vision and they're in the long haul to, to help make this tool what it needs to be for the industry. The fact that it's an opportunity for me to plug in in a different way within the slain project. Because I've been a member of the project for an official capacity for a few years, I stepped away the official capacity for a little while and now I'm back as a actual committer of code as opposed to helping on the administrative ends of the project. So for me it's just a really exciting opportunity.

So, David, there's a few things there, I guess. One is why would applitools spend time on this? What's the catch?

Yeah, so I don't think there's a secret here or a hidden agenda. I think it's pretty obvious that there's going to be integration for automated visual testing. But I think that the bigger goal here is that offer a really robust and useful tool that helps people who might not have the capacity or skill to write tests with code they want to be able to get up and running quickly, or there's people who already have an existing, existing, you know, chest full of, maybe that's not the right word, an existing gargantuan set of Selenium IDE scripts and they want to still use them. I think that we want to try to make sure that everybody has the ability to do automated testing and do it well and then if some of them happen to use automated visual testing, great. If not, they still have a really useful and powerful tool. And so, so much of the work that happens in test automation has to do with Selenium. And I think it's also a nice way to give back to the community.

Great. And like you mentioned, yeah. People that are pro and con against doing this, reviving this, what a lot of people think was really a bad idea, mostly because it's like any tool, people tend to abuse it. And so what is different going to be different about this implementation, if any, that will helpfully lead people to use it for the right things and not necessarily as a hammer for all their needs for automation.

Yeah, I mean, it's a great question. So I think that there's always the balance of trying to. You could either have an opinionated tool or you can have a flexible tool that enables people to do what they want or intend to use with the tool. We're not, I think, in the business of creating a tool that's super opinionated or limiting because at the end of the day we want people who are using the old Selenium IDE to be able to use the new one and make sure that backwards compatibility exists within reason. But we also want to offer Opportunities for people to realize that if they either made a couple of changes or even didn't have to make any changes, maybe they just by virtue of using the new tool, get some nice benefits that they didn't before. And so one example of something we're doing is creating a command line runner that will take the tests out of the IDE and give you parallel execution that is guaranteed to be thread safe. Basically the turnkey, you can plug it into your CI and ta da, you now have a concurrent test execution, which for a lot of people is like a really substantial milestone. And there's loads of ways people can do that in code and screw it up and, and paint themselves into a corner and, and what we're trying to do is offer at least that and actually ideally something better. So having hoping to get to a point where there is better, you know, guarantees a short word, I think try to offer consistent execution that's on par with what's available in code.

Awesome. So you know what, all the features are you thinking of are going to be implemented here and is this available now or is it still in development?

So these, the, there's, there's a lot of things in, in flight right now we are working on the runner and it will be shipped sometime later this year. I believe that there's a beta implementation right now that offers just one mode of parallel execution. We're working on different degrees of parallel execution depending on what your needs are. And we've also shipped what's in beta right now is an extensions framework. Basically people can author plugins for the tool to extend the functionality that's there to either add things that aren't on our roadmap that they would like to have exist in the tooling or for commercial platforms to offer integrations able to execute their tests against a third party service. The list goes on. And so that's just, that's in beta as in we're still adding features, but it's available now.

And then we're doing some things behind the scenes which will help I think dramatically with how robust and reliable the tool is. And that's how the, when you record a test and when it plays back is that playback right now is reliant on some very old legacy code that has to do with the Selenium RC implementation. And so we're actually moving away from that into a execution that's going to use a grid hub. So it will be as if it'll be the same playback execution as when you run your tests and code and then we're also looking to find ways to improve the selector accuracy, which.

To my mind is probably this One of the single biggest things that people cite when they think of why record and playback tools are terrible is locator accuracy. I've been doing quite a lot of work on locator accuracy, in particular over the last year or two. If you've actually followed anyone else, any of your listeners have followed any of my talks around ways that you can find better quality locators, and I think it's a fascinating topic. I think that we can apply some of those practices in the tool.

That sounds like an awesome idea. So Dave, that brings me to the next topic that you're probably sick of hearing, as is everyone else, AI and machine learning. So in trying to identify better locators, is there going to be any type of machine learning that you're going to bake in there? Is that like too early though to.

I think it's hard to say what the exact implementation is, but I could foresee potentially a some kind of machine learning implementation. I think in order to get to the next step where if you record a test and you have, you have locators that are available and reused and then for some reason they no longer work, that there'd be a way for the tests to be able to inspect the page and determine what your intent was, find a better locator. And I think that I could foresee this being something put into the tool as well. But right now we don't have any, any hard concrete plans on the, on the roadmap, but I think that I could see that happening.

Cool. I know you speak at a lot of conferences, you speak with a lot of thought leaders in this space. My perception is before you know, Selenium is the de facto automation API that people use and with the W3C standard coming out, it's going to be probably it's be supported by all the browsers. But what I've been seeing is more and more people moving towards once again vendor tools or more tools like Cypress IO which is open source part of it, and then other tools that aren't like Testim or Mable, so that some of these don't even use Selenium under the covers. Any thoughts on is Selenium going to be like QTP was and eventually it'll be replaced by these tools or is this something that you see as an ongoing evolution that Selenium will always be around and people shouldn't invest time in it?

It's hard to say what the future holds, but my opinion is to be patient and wait to see what actually happens. I think that there's always been a narrative of alternative tools to Selenium, and Selenium is terrible and all of these things. And I think that I see it as someone who's part of the project. I'm frustrated that people don't want to actually contribute to making the project better to solve their needs. Instead, they just want to go and create an entirely new thing. But in terms of will, like something like Cypress Surpass Selenium, I doubt it. I think that it probably serves a specific customer for a specific, more specific use case than what Selenium might offer. And I may be wrong. It may attract ire from your listeners. And I think Cypress, I mean, from what I've seen, I think it's just one example. I mean, even Testim, I think they're all great tools. I think that they help people. I think there's plenty of people who use them. And I don't think it's a matter of will Selenium go away or not. I think the fact that it's a standard and it is being implemented in all the browser, the major browsers is a testament to the fact that it's going to be around for at least as long as we're in the industry. But until there's a phase shift in technology to something away from browsers to whatever comes next, then sure, maybe it goes away. But for right now, I think if people have invested a bunch in Selenium or they're thinking about getting into browser automation and they want to step through the app as if their user was stepping through it, then I think that Selenium is their best bet. I think that having a better tool for people that are less proficient with code or don't have the time or develop resources to invest into writing effective test automation code, then they should look at Selenium ide. I think that we're making it into a tool that is on par with what people could accomplish in code.

So I definitely agree. I'm completely tool agnostic. I believe also each team should determine what tool works best for them. And it may be different based on the industry and what you're trying to do. So not one's better than the other, it's what works for you. And so once again, that brings us back to Selenium id. What kind of team I think you touched on a little bit do you think would benefit then from using Selenium id And what do you see As I know you don't want to be descriptive, but, you know, what are some of the uses you think would be a really good use of Selenium id?

Well, I think that the end goal would be anybody that wants to just record a bunch of, you know, tests for use cases as, you know, all the customer facing, business facing tests of the app, the same use case of what you would use Selenium for. And I think that if people would rather write them in code, then Selenium IDE is not for them. But it used to just be that people who are less technical would be handed the IDE and say, okay, record your tests using this, save them and rerun them. And then that never ended well. But the end game here is that that use case will end well. And it's also not just for less technical people. It could be for the same people who use Cypress. I mean, they're super technical front end developers who use Cypress. And it's supposed to be like, you know, it's more approachable and it just, you know, it works. It's simple. These kinds of things, like we're trying to make it more approachable not just to less technical people, but to more technical people and actually get it to a point where there's the choice that they can make between do we do this in code or we do this in the ide. And they would say, you know what, let's do it in the ide. It gives us all the bells and whistles for free and we don't have to build anything. The same reason people would pick up a framework, because it would just have have these things baked in.

Awesome. I definitely agree. And I get some bad reviews that I've been talking about guild conferences. Too much, but too bad.

So in my previous Guild conference, Testing Guild, we had some sessions on exploratory testing utilizing automated tools to help you. So not necessarily to automate everything, but just leveraging them to give you some quick feedback. So a lot of things around using like Postmodern to help you do some exploratory testing with your API. And I'm almost wondering if that also would be like a user case, because I love the idea of creating like maybe a quick and dirty script and just replacing some values just to see how your application behaves or to actually look at how the elements are being generated, but not necessarily hardcore all the time. So is that a use case? You would think?

Yeah, I think there's a couple other use cases. Sure. For getting the app to a certain state that it just requires a bunch of steps and they're always the same. Using it for the case you described, which is similar but maybe a little bit different depending on your needs. And then there's also you found a bug and you want to show a reproducible case. I think that capturing a test that demonstrates the failure condition that you can then attach to a bug report is a phenomenal use case. And I think that, you know, that's way better than to me, a defect tracker that doesn't have anything to other than like steps listed out and just pure rows.

So once again, I love that that idea. So is there going to be any functionality around that where you're able to repro a bug and send it and someone could just replay it or the reporting what kind of features are built around that that would help people share that particular bug and get people to look at it?

Yeah, I mean the general way that people have done it is they just record an individual test case and then they save it. And then that's what people can run is you just have to have the IDE on their machine, they load it up and then they can run it. I think that anything that is kind of expected normative standards and other tools, I could reasonably see something like that making its way into the IDE either as an extension, if not part of the core functionality. But I mean other. I think some tools offer kind of file this issue in Jira or attach it to Jira for example for people who actually like Jira and or you know, attaches to a Trello board, you know, whatever it is, I think there's ways to enable people to have, have less friction, to have, have an automated test case that demonstrates the defect and have it be plugged in somewhere else. I think that that's something that could reasonably be done too, especially through the extensions framework.

I guess I'm thinking back to questions I used to get asked. People used to ask me about Selenium ide. One of the biggest one was does it only work against Firefox? And the answer was yes. So is the newer version also completely just tied to Firefox?

Nope, it's on Firefox and Chrome. You get this, you can get it from the Chrome store or from the Mozilla Add Ons store. And that's what we're doing right now. We tried to keep, keep consistency with what was there before and try to enable it in more than just the one browser.

That's awesome news. And I guess another question people always ask me is can you use a check in Selenium ID to your version control so you could do code reviews and keep track of any changes.

Yeah, the export of the IDE is actually one file now, as opposed to individual HTML files for each test case. And so that file could be checked in and any change to that file would obviously be tracked in Version Control. And I think that the code review, I guess would be trying to think the correct word or not. But yeah, you can store it in Version Control, you could review it. I'm not sure if that, if that would be the best way or floating it up to the IDE and looking at it there would be more appropriate or not. But you could definitely check in all of your changes to Version Control.

Great. And I would speak up to Simon, the original creator of WebDriver and he mentioned something about how Chrome has these new performance metrics and this other data that you get out of the debugger and that he doesn't see right now baking that into Selenium. And that's why sometimes people use maybe a tool like Cypress because they're able to tap into it. Any thoughts on maybe having Selenium IDE or would that be a good plugin that someone can write to be able to when using Chrome to grab those particular metrics?

I think that that sounds like a great candidate for a plugin. I see the plugins are plugin, framework, platform, whatever we want to call it, the right word there, the extension architecture that we have. I think that anything that is on a roadmap that we haven't implemented yet or that's not in the roadmap but the community thinks would be useful. I think that something like that would be a great candidate for the extensions because based on everything else that we have going on, I don't foresee us doing something like that anytime soon.

I know you probably can't tell me or even if it's official yet, but you did mention that you may have some features where, or maybe you do already, that you could do some sort of visual validation which I think would be a huge win. Are there any like. I know that's a paid service and it should be a paid service, it's a great service. But is there any thoughts about like a threshold that people can use like a visual check, like at a minimum to get them kind of used to what visual validation is and then if they need more that they could go and get the full blown version of applitools?

You know, I actually haven't been part of the conversations around the product integration. For that I've been focus mostly on just the core features to keep parody with what is in the old IDE that's not there or fix, you know, helping fix bugs and building out some new things that the new features that haven't existed before in the core feature set of the ide. So I think that's a good. Stay tuned to see what the marketing message is.

I guess. So Dave, is this your, your sole focus now is just Selenium IDE or you're an R and D and this is the only thing you're working on? And if so, what's going to happen to your awesome resources like your, your books and your training and your, you know, going out and speaking? Hopefully that's still part of the deal. Any thoughts around that?

Yeah, so I am fully focused on the ADE and a mixture between occasional things like this, interviews, perhaps some talks, conferences. I think that's still TBD in terms of what the appetite is for the community to have have talks like this at the conference. And there's more than just me. I mean it is an open source project and there are other people who are working on it. Like my, my co worker Tomer is phenomenal and he I think is going to be a name that's going to be synonymous with test automation very soon in terms of speaker credentials and rightfully so. And so I'm happy to let other people run with it, but I'm focused solely on a mixture of the development efforts for the project and anything else I can do to help move the project forward in terms of documentation, the PR aspects of it. And so maybe I'll be at a conference talking about it or maybe I'll just be at a conference just to still be tapped into the community. As far as my existing stuff, I mean that all kind of keeps going, going. The newsletter still runs, the book is still in existence. My training business is no like I'm not going to go and do I think all day workshops or anything. I'm not going to do private trainings or any more consulting while I'm working full time at Apple Tools. But yeah, I'm just dedicated to working on the ide.

So Dave, I'm always struck by when I go to a slam a conference it's always the same contributors and I'm always wondering why that is. And I always worry that one of these folks leave that you know, we're going to have problems. So like what would you do to help people? Encourage them to contribute to like selenium ide. If someone wanted to help with the Selenium ide, how do you think it would be the best way for them to get started?

Well, I think that the best place for anyone to get started is try out the tool and see that it's, that we're serious about it. And, and then from there I think if people, you know, people like it, give us some attaboys on, you know, the Chrome store and the Mozilla add ons and then if there is issues to hop over to the GitHub project and let us know because we, we want to hear that stuff. We want to know if you're importing old tests, why didn't it work when you tried to upgrade? If you're trying to step through a use case that's right for you, you know, it didn't work. We want to know what the use case was so we can make sure that we can track that stuff and ideally push a fix for it. And if people actually want to. So let me just start by saying back up a second and say filing an issue on an open source project is an open source contribution. Like you took the time to submit something to the project and we have issue templates to try to make it very straightforward for what, what we, what we need in an issue to, to make it useful and easy for us to track because we have so many issues that are open at one time and then beyond that there's always room for like plugging holes in our documentation. That's always a very straightforward place for people to start if they use the tool and they realize like this, this thing was confusing and I figured out how to do it. But hey, there's no documentation. They could contribute some simple documentation and that'd be great. If people want to pick up an issue and send a pull request for a bug fix. Like if they're, if they're more inclined to contribute code, that's great too. I think that trying to up on the IRC or Slack Selenium chat and ask questions and like get to know people like, that's another way to plug in. I think there's loads of ways just try to so basically use a tool, let us know if you have problems and try and get in touch with us so we can create an open dialogue to make sure we understand how people are trying to use the tool and if it falls short, why, and go from there.

Right. Dave, before we go, is there one piece of actionable advice you can give someone to help them with their Selenium IDE test automation efforts and let us know the best way to find contact you or to learn more about getting this new Selenium IDE on our systems.

So the best place to get plugged into the project is you can actually just go to the Selenium HQ GitHub organization and under the organization there's Selenium IDE repo and then that should have everything that you really need to know in terms of what's available. Or you can obviously find it on the Chrome or Firefox Mozilla stores and then in terms of people can find me on Twitter, I'm ordodave and if people are interested in just using the officially supported client findings for Selenium and that's their jam or you're already doing that and you want to know how to structure your code in a way that's going to lead you in a direction towards success, then do check out my content elementalselenium.com and seleniumguidebook.com those would be some good resources for people who are still inclined to do it work in code. Otherwise I think the IDE is going places.

Thank you Dave for your Selenium IDE Automation awesomeness. For links to everything of value we covered today, head on over to test talks.com230 and while you're there, make sure to click on that Sign up for a free trial link under the Exclusive Sponsor section to learn all about Sauce Labs awesome products and services. And as I mentioned at the beginning of the show, click on the banner for Automation Guild 2019 register today and you'll also get a free ticket to Secure Guild for Early Birds. So you want to do that as soon as possible. So that's it for this episode of Test Talks. I'm Joe. My mission is to help you succeed with creating automation awesomeness. As always, test everything and keep the good Cheers.

Thanks for listening to the Test Talks podcast. Head on over to www.testtalks.com for full show notes, amazing blog articles and other automation awesomeness.