SeleniumBase Recorder Mode with Michael Mintz

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Really excited about today because I have a special treat for you. We have Michael Mintz from Selenium Beasts talking all about some new features of Selenium Base. He's been working on this for a few years, always comes out with new features and I think you're really going to be excited about this new feature. So you want to stay tuned to hear what this new feature is and I think you're really going to find a lot of value for it from it. If you don't know, Michael is a awesome experienced web automation engineer, open source wizard, and as I said, creator of Selenium Based, which is a python based automation solution that I think everyone needs to know about. If you don't know, you don't want to miss this episode, check it out.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Source Labs, the cloud based test platform that helps ensure you can develop with confidence at every step from code to deployment for Every framework, browser, OS, mobile device and API. Get a free trial. Visit testgill.comSourceLabs and click on the exclusive sponsor section to try it for free today. Check it out.

Hey Michael, welcome back to the Guild.

Hey Joe, how are you doing?

Awesome, awesome.

Happy new year. Happy 2022.

Yeah, happy new year. Happy 2022. And I guess as a kick off the New year, you offered up a new feature that I think we're going to dive into with Selenium Based. But before we do, Michael, it's been a while, so anything that I missed in your bio that you want the Guild to know more about?

Well, let's see. Ever since the pandemic began, I've been working from home, which saves me probably three hours a day in commuting because normally I'd be traveling into Boston on a regular basis. Rush hour traffic is really heavy, it could take time to get in. But thanks to the pandemic and everyone gets to work from home, which means I have all this extra time that I can spend maybe improving the slending based automation framework so that it can do more for me and everyone else that uses it.

Very nice. So we talked, I think in like March of 2022 Episode 296 and so I'm just curious to know now that you have this extra free time. I know you're very busy to begin with, but some new features I'm sure you've been working on. One of them that I just saw that caught my eye is something called Recorder Mod. So I thought we'd start with that. So I guess at a high level, before we even get into that, for the folks that may have missed episode 296 or don't even know what Selenium Base is, let's start off with what is Selenium Base?

So Selenium Base is a Python test automation framework that essentially combines the unit testing capabilities of Pytest with the browser automation capabilities of Selenium WebDriver. And Pytest on its own, has been serving people in unit testing for a very long time. People are able to run their Python unit tests in a very organized way. There's all these additional plugins that are already part of the pytest ecosystem, such as ordering tests with markers or running tests multi threaded, etc. Combining all these abilities with Selenium and adding on additional command line options such as setting the browser you want to use, or additional features, you can do some really powerful web automation that's more reliable than just raw Selenium imported on its own. Because Selenium Base adds on to the additional Selenium libraries so that every action, such as a click or typing text into a text field, is done more reliably than before. Because Selenium Base will wait for the ready state of the page to be complete and for web elements to be fully loaded and interactable before performing actions within them.

So I think it's a great point. I think a lot of people, before they use a tool, they want to know like does it use industry standards? And this sits on top of Selenium, which is basically the industry standard for automation. So people shouldn't be worried about that. I believe this is open source as well, so it's a win win.

Yep. All the code is on GitHub.

Nice. And I'm just curious to know, you know, Selenium 4 recently came out and I know it has a lot of new features. It's able to tap into a lot of the Google. I forgot what they call it, the CPC or something. So it's able to get in the API and do a lot of cool things. So is are you now able to do more than what you were probably before, maybe with.

Yes. So selenium 4 is for Python versions 3.7 and above. The older versions of Python are unfortunately stuck on using Selenium 3.141, which is the latest 3 version. However, with the Selenium 4 Python library, 3.7, Python users and above get to take advantage of all those new features, such as being able to fully support Shadow DOM in the web browser, as well as take advantage of the Chrome Dev Tools protocol, which essentially lets you do a little more than you could before. So selenium 4 just also adds a little more reliability and a lot of other features that were included within that. And it was also Simon Stewart's final real Selenium thing.

All right, cool. So funny you mentioned Simon Stewart. I know, I talked to him and he did an awesome contribution. He was basically the person that invented Slender webdriver, the second part of it, not the original, which was Jason Huggins. But anyway, he's been consumed for a while and like you mentioned, he's no longer leading the chop. But I know they have a lot of great folks that are starting it. And so he left us Selenium 4 and he gave a lot of shout outs to Selenium Base. If I remember correctly, last time I talked to him saying he got a lot of inspiration from it. So I wonder how much of your input or how much input you give to Selenium or how much maybe was maybe made into selenium 4 based on maybe some cool things you've done with Selenium based that they saw. I said, hey, why don't we just bake this into selenium 4 as well?

Yeah, I had the honor and privilege of speaking with Simon a few times on the Selenium Technical Leadership Committee meetings, which were basically biweekly meetings for the Selenium organization, where there'd be a video call and it would be Simon, me and several other members of the Selenium community and we'd get to talk and just bounce ideas off of each other. And Simon was very encouraging about, you know, seeing some of the features that Selenium Base had to offer in order to help the Selenium ecosystem with its own standalone Python framework. So it was great, you know, being able to speak with Simon and just hear the ideas that he had and being able to learn from that. He's a really smart guy and he pretty much, you know, started the whole thing with Jason Huggins many years ago. And Simon's now, you know, letting others, you know, he's passing the baton on to others to carry the legacy of Selenium for him. And now there's plenty of others that are there to help out with that.

So I have to ask you this. We're going to get into this new feature, but I just, my mind is getting with questions like Simon busted his hump for free on Selenium, like why contribute to open source? Why do this to yourself is why? I always wondered. Because obviously it's, you're doing it for free. It's a lot of your time. You get a lot of flack from people for some reason when they want a feature that's not in there, if they find a bug. I'm just curious, what's your motivation for even doing an open source project like this?

Well, I just saw that, you know, several years ago when I started the Selenium based project that like test automation could be improved on from what it was at its existing state. You know, people always complain, tests are flaky, there's no structure. So with Seleniumbase I want to help solve some of those problems just to make test automation better in the world. And with Selenium Base, I think I was able to achieve that in the Python world by just giving the tools needed to people so that they can run their tests reliably and with all the advanced features that they could want to help them generate test quickly. Which brings us to this latest new feature of Selenium Base Recorder mode, which allows anyone to essentially manually perform actions in a web browser that will get recorded via a Chrome extension. And then Selenium Base will take the data from that Chrome extension, which was essentially saved into the session storage of the web pages that actions were performed on. And then SeleniumBase will run some Python magic, you know, some scripts that basically take those actions and convert it into Selenium based code so that you can run it as a regular Selenium based script.

Nice. So I think a lot of people may be more familiar with something like Selenium IDE back in the day.

Exactly.

Or older tools like windrunner. So is this what the recorder mode is?

Yeah, it's essentially I wanted to, like, I tried Selenium IDE before and I just wanted something that was a little more powerful and that generated cleaner scripts and that would map directly onto like a very simple Python Selenium based script. And the thing with Selenium IDE is that you had to like first download the Chrome extension and then, you know, set things up and then record things, and then you'd have to go like into export and then export that into the programming language that you wanted the language bindings for, and then you'd have to modify the script and then add in assertions. After that. I decided to simplify all that down by just being able to quickly generate the script that you wanted while also being able to say add text assertions or element assertions right as you're doing the recording.

I assume because it's Selenium based, it's optimized, optimized for Python. So would the code being generated be more geared towards, not better or you know, just like a Python specific maybe kind of ways of doing things better? Practices are more common in the Python community.

So for Python, like the Selenium based Recorder mode should give you the simplest way of basically performing actions in a web browser and generating a test directly off of that, while also being able to add in your test or element assertions directly during the recording. And it just generates a very clean script. I don't know if you've taken a look at some of the examples from the Selenium based skit page or the Seleniumbase IO page on Recorder mode, but essentially it cleanly generates a script that you can run right away. And you can also add additional command line options such as demo mode, which will highlight the actions that are being performed, which is very nice. So that you can actually see what's going on more easily than just it quickly running through without you seeing what happened because it happens so fast if you just run it in the regular way.

Absolutely. I know if people are familiar with windrunner back in the day, but in winrunner you could do that and it would highlight the fields and I believe this does something similar in debug mode of a highlight. So you can actually watch and see, okay, it's going to the right thing and you're able to look really quickly to make sure it is doing the right thing. Is that right? It does the highlighting that adds to it.

Exactly. So when you play back the script, at least with the Recorder desktop app, that is a new feature, you can essentially run the script in demo mode, which will highlight all the actions being done as well as the assertions. So if you asserted that an element appears on a page or a certain text appears on a certain element, it will highlight that and even put a little box in the bottom right corner of the page saying asserting this element here or asserting this text here, and it'll give you a green highlighted box around the specific element that you were asserting. And if you're clicking on an element or typing text somewhere, it'll also like bl the element a few times so that you can see what exactly is happening more easily.

Nice. So speaking of assertions, how do you add them does it add it automatically or what it thinks is the best assertion? How does that work?

So if you press shift 6, it'll switch out of regular Recorder mode into Assert element mode. So if you start clicking on anything, it's going to add that as like assert element. And if you do shift 7, it will switch you into Assert text mode. And then if you start clicking on that, any text that is found within an element, it's going to say assert that this text appears in this element when it reaches that line and it'll generate a row for that. So basically there are three parts of recorder mode. There's just the regular recorder mode, where it will record any click, any text that you Type in somewhere, etc. Then there's the assert element mode with shift 6, which then lets you click on something and it'll just assert that that element is there on the page. And then finally there's the Assert text mode, which will assert text within an element. If you do shift +7, to switch into that and to get back into regular Recorder mode, you'd hit Escape once. And if you hit Escape twice, it will stop the recorder completely.

Nice. I just want folks that are listening to know that Michael create a really great demo video on this. Actually, you can see it. So I will have this in the show notes, so don't worry if you're trying to visualize and you can't, you will be able to after you hear this podcast by going in the show notes. So don't worry about that. So, Michael, a lot of things people might be thinking is, oh boy, you know, back in the day this was considered a bad practice. How is it different now in 2022 as it was from when I started in like 1996? How is this different than those old record and playback? They used to call them snake oil tools.

I guess back in the old days, like the selectors that were generated weren't very reliable. And I basically worked on like some complex algorithms in order to give you the most simplistic selector that you can. So if an element has an id, it'll use like the ID to locate the ele. Or if the element has a, like a name, it can basically say name equals that with a element tag to basically cleanly give you the element. I think most tools that I've seen give you a really long X path that isn't very reliable because if just one thing changes, if you like say move something inside a new div, that would break very quickly in an automation script. But instead of using like really messy XPath, I just use cleaner CSS selectors. And if you look at some of the examples that are generated by the Selenium base recorder, you'll see that it generates it in a very clean way. And because when Seleniumbase runs a test, it has all its extra reliability built in, you're pretty much guaranteed to make sure that the element has fully loaded before you interact with it, so that you can click it successfully without running into the many common Selenium exceptions that people might be familiar with, such as element not interactable except exception, or one of the several others that are just found with people who just use raw Selenium without a wrapper that checks for all those things.

Absolutely. So what are some use cases for this? I know back in the day I used to like to use a recorder, just get a feel for an application, maybe play around to see how it's picking things up, and then I tighten up the code myself. But I use that basically as the bare bones of the structure of what I was going to use and just cleaning up and adding my own things by programming in it directly. So what are some use cases you think this would be good for?

So if you're a new automated engineer and you're not familiar with like you know, all the code, or you're still new on figuring out how to properly figure out the optimal CSS selector to use when you're trying to perform an action on an element, the Slenderbase Recorder can help you simplify that by just you run the recorder, you start recording your actions on a website, and then it'll generate the script for you. And it's an easy way to quickly get started with automation, because you don't necessarily have to learn the full language first. You can just see what scripts are generated after you perform actions on a website, and then from those examples, you can learn how to write your own automation. Even better if you need to customize the script or just write your own script from scratch. The Selenium based recorder is great for anyone who just wants to quickly get an automation script out for a test. And you'll be able to add in your assertions and then play back the script immediately after you've recorded it. There's no like copying or exporting. And then, you know, put that in a new file. You can immediately just see the Python file that was generated with all the code from the recording and then play it back.

Nice. So we went over when you should use it, how you should use it. Now a lot of Times there's unintended consequences of cool features like this. So how should people not use this? How can you envision people using this? And you're like, I did not create it for that reason. Please don't do that.

Let's see. That's an interesting question. Not to use it. Well, I suppose if you're sure that the webpage is going to have like a lot of major changes, you know, coming up, then you might find yourself, you know, they're gonna totally rewrite a page, for instance. Then you might find yourself having to re record the script. So it could be that or you know, that certain selectors are gonna change, like they're gonna change all the IDs, but they're gonna keep the names. Like for instance, developers say, we don't like the ID naming system, we're just gonna switch to. We're gonna keep the names that are there, but we're gonna give them all new IDs. For instance, then it could be that the default version of going to create a selector with an ID instead of a name might not be a thing because we basically have a prioritized list of say if there is an id, use the ID first. If there is a name, use the name. And then there's like different types of CSS attributes and values with combinations of that that might be prioritized over others. So ID would be like number one, priority name two and then class would be way at the bottom because there could be multiple like classes within element. So you could have like segmented out into multiple classes in the same element. So a class wouldn't be very reliable because that's likely to change and sometimes there's auto generated data. So it could be that the based on if you know that a certain type of SEL is going to be used and it's not usually the primary one like an id, then maybe you want to set the selectors yourselves and then manually write the script. Or if it could be that, okay, you know that the page isn't likely to change and you want to quickly get out as much automation as you can, then the recorder can quickly help you generate those scripts for you.

Nice. Just random thought popped in my head. I know one of the features of webdriver IO that a lot of people like is it works with Shadow Doms pretty well. I think maybe even Selenium 4 has new features. Does this work with the. What kind of applications does this work? Obviously it's browser, but is there any browser type of functionality that this would not Work with.

Yes, that is a good point. So Shadow DOM was recently changed completely in Selenium 4.1, because the latest version of Chrome, I think it was like 96 or something, basically changed how shadow dom works. And now there's like a brand new method for how to interact with Shadow dom. So for now, if you're trying to record something in Shadow dom, it's not gonna work. That is something that you have specifically click on a Shadow DOM element. That's not going to work for the recorder right now. However, if you need to click on an iframe, there are ways to make that work, but it requires an extra step. You just need to switch into the frame first and then you can start clicking on things within the frame and it should work. But there is an additional step because you need to specify that you're now in this iframe, not in the default content of the regular webpage. Because an iframe is essentially a website inside a website. And it could be that there's an element in the iframe that has the exact selector of an element on the regular page. For instance, the regular page will have like a head tag and a body tag. And now if you also have an iframe in that page, there will also be another head and another body. So if you're trying to say, grab all the text from the body tag, is it going to grab all the text from the body tag of the regular page or the body tag of the element in an iframe? So due to this complexity of having websites inside of websites, you just have to specify, hey, switch into this iframe first. Now grab all the text within the body tag, because I just want that text or that webpage, not the entire web page.

That's a great tip. I guess another tip I'm curious to know is how do you avoid code duplication with this approach? Rather than someone go, okay, time for another script, and they start scripting it from the start to end. Same code as maybe like 80% that's already been written. Is there a way to get around that?

Yes. So that is another reason why, if you're new to automation, start with the recorder. Because you could be doing a lot of duplication. Like if you need to perform like actions A, B and C. And then on another test, A, B and then xyz, you just performed A and B twice, which means that you probably could take that code and put it into a reusable method. So that you could say run AB and then, you know, run C Here for this test, and then on the other test you call the method that runs ab, then you run X, Y and Z after that. So whenever you have duplicated code, that's the perfect time to start using reusable methods so that you can minimize the amount of code you have because you don't want to have duplicated code. And if you start using the recorder to record over the same things again and again where there's a lot of overlap, then you could still use the recorder, but then you'll have to update the script after, afterwards, so that, okay, I'm going to copy this section out or cut and paste into this other method. And this method is now going to be used by all these other tests. So it's a way to help you get started. But when you're dealing with duplicated steps where there is code reuse, you want to break that out into a separate method so you can have clean code and not have duplication everywhere, because duplicated code makes things a lot harder to maintain.

You know, it'd be cool. I don't know if it's possible if you had like a dropdown and you're like, okay, I'm on a login page, I know I have code for that drop down login, puts it in, then you start the recording from wherever is new on the page. That would be kind of sweet. But yeah, I don't know.

Actually there is another way of using Recorder mode. I might have added this into the Recorder docs, but instead of starting the recorder from the very beginning of your test, you can drop a line, self activaterecorder, a Python command that basically will start the recorder in the middle of the test so that it does start recording the actions that you've already done and then you can just grab that part of the recording.

Oh, that's cool.

There is a way to do that that should be mentioned somewhere. I believe that's already in the docs. It's a long, I don't know if you've seen the page for it, but it's a very long set of instructions for all the various things and ways you can use the recorder.

Nice. Now, for people that aren't familiar with Python, you know, it's 2022. I always look at the top programming languages for developers and it's always Python. The past few years it's been py. And then when I look at job requirements for a lot of automation engineers, one of the top jobs is Python. So if someone doesn't know Python is selenium Based almost like a good entryway to them, not only to learn automation, but also get their hands on actually stand and write some Python code.

Yes, there's a lot of documentation for people who are new to Python. Like, it starts you off. If you've never done anything with Python before, it says go to python.org download python and then here's how you'd PIP install. You know your first package you do PIP install Selenium Base or PIP install any Python package, which is how you get started with grabbing a package. It also shows you another way of installing it by cloning a repo from GitHub with the git clone command. And then if you're using a cloned version of SeleniumBase, you'd use pip install e. Which basically will install that in development Mode or just pip install.to pip install python package in regular mode. And it also explains just various steps that might be important to know if you're using Python, such as a Python virtual environment. And I have a link to say this is what a virtual environment is and why it's important. Like if you have Python Project A and Python Project B and they have conflicting dependencies, you need to basically put those in a separate virtual environment so that you could have like version one of the this Python package here and version two of this Python package here, if they require different versions of a dependency. So virtual environments is a very important thing to learn if you're dealing with multiple Python projects. If the only time you'll ever need to use Python is for web automation and you're ready using, say JavaScript, etc. For like creating a web app, then you might not have to worry about using virtual environments. But if you're an active Python developer and you have multiple Python projects, that's when virtual environments are going to be very key in order to keeping things working smoothly.

So another trend is AI and machine learning. You said you worked on a lot of algorithms to make this a little more tighter in this recorder mode feature than maybe what's in Selenium ide, just because it's still Python specific. So I'm just curious to know another reason why I think Python's popular nowadays. It has a lot of modules that allow you to do AI machine learning, type things. Is there anything built into Selenium Base that has AI machine learning that can help people with their automation yet?

I wouldn't call it, I don't know, I guess if you can call an algorithm AI, then it's like a technicality, like yes, it's using algorithms to calculate, okay, I'm going to have the ID first, then the name attribute, then like a data QA tag, and then after that like alt, title, you know, heading, translate, aria, label, ng, model, href, etc. There's a whole list of, say, attributes that I prioritize over others. So I do have an advanced algorithm that determines like, what selectors to use over others and that. And for generating selectors, that's important because you can't just use a random selector combination, you need to keep it consistent. So I could say, okay, if there is an ID or a name or like a data QA tag or attribute, I mean, and then there's an alt title, heading, etc. These are various common things that you'll find within the CSS selectors of elements. So there's an algorithm that determines the priority of that and other things when building that out, such as, okay, H1 tag, H2 tag, H3, then there's like a center. If I'm using say the contain selector, which is unique to Selenium base, which also exists in jQuery, which allows you to basically create an element based on the text that is visible within the element. There's something for that in XPath. But CSS by its own does not have the colon contains selector. So that was something that I added into SeleniumBase to make it easier to select elements that have text in them, but don't have like an ID or a name or some other unique identifier that allows you to select, say, a button on a page that you want to interact with. So there is a fancy algorithm that is used in order to, say, work with the recorder to generate all the selectors and all the code. But calling it AI, that's a technicality. But from what I've seen in the real world, if you have an algorithm, you have AI, you have, you know, that's what they call it. Machine learning is totally different. Machine learning would mean that what you do would change every time you run it because you look at past results and maybe like try to optimize it. But if you're running like the same version of Selenium base and you perform the exact same actions on the same web page, you should get the same results each time because it's using the algorithm, which AI, if you want to call it that. But it's certainly not machine learning, which basically goes through data. And then if you give machine learning bad data, it'll give you bad results. So the double edged sword of machine learning is that if it's fed bad data, you get bad results. But if you've got an algorithm, an algorithm, you've got like a determined definite path. And if you want to call that AI, great. That's up to whatever people want to call algorithm algorithms these days. But yeah, some people just call that AI, great.

Okay, Michael, before we go, is there one piece of actual advice you can give to someone to help them with their Selenium based automation testing efforts? What's the best way to find contact you or get our hands on Selenium Base?

Yes. So go to the selenium base GitHub page. There's going to be lots of ways to. You can either open an issue or ask a question in the discussion section. GitHub basically has this, this new area where you can add discussion on something. So if you have a question you just want to ask and not necessarily a ticket you want to open, you can go to GitHub discussions. It'll be a tab that shows up at the top of the GitHub page and you'll be able to go there and just open a question instead of having to open an issue. Or there's also various ways of reaching me via chat, such as the Gitter chat, which there is a link to from the Selenium based page on GitHub. So there are several different ways of reaching out. People want to have a conversation or ask questions, or if they have an issue, lots of ways to ask. Just check out sundingbase GitHub page or seleniumbase IO and they will find their answers there.

Any actionable advice, Michael, for anyone for automation or Python in general?

Well, if you're a Python developer and you're new to automation, just start with the basics, go through the tutorials, see some of the videos based on the over 100 examples that are in the Slenderbase examples folder. Anything that has probably been done out there, like any type of action such as hover clicking, drag, drop, etc. There's probably an example for that in selenium based shadow, DOM, etc. Dealing with iFrames. Basically go through the examples for whatever you're trying to do and you might find the code that you're looking for so that it can help you in generating your script. And if you want to try out the recorder, there's just a single command line option, Selenium Base Recorder, which will open the Recorder desktop app if you've already installed Selenium Base and then you can try recording a script and then playing it back and then if you like the results. There you go. You've generated your script very quickly.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com ed.

And while you're there, make sure to click on the Try for Free Today link under the Exclusive Sponsor section to learn all about Sauce Lab's awesome products and services. And if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.