260: Python Test Automation with Andrew Knight

Hit it.

Welcome to the Test Talks podcast, the place to go to geek out on software testing. And now your host whose mission is to help you succeed with test automation, Joe Colantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation podcast for formerly known as Test Talks. Well, not yet, but more on that in an upcoming episode. But today we're going to be talking all about Python Test Automation with the automation pander himself, Andrew Knight. Andrew shares the reasons why he loves Python and tips on how you can get started using it for all your test automation efforts. You don't want to miss this episode. Check it out.

Test Talks is sponsored by the fantastic folks at Sauce Labs, the cloud based automated testing platform that eliminates the need to maintain your own selenium grid and test infrastructure. Try it for free today. Visit testtalks.com and click on the Sign up now link under the homepage Sponsor section.

Hey Andrew, welcome to Test Talk.

Thanks, I'm glad to be here.

Awesome. It's great to have you on the show. A few folks have requested that you join us. Python is a hot topic, especially in test automation. I thought that's what we talk about all about today. But before we get into it, Andrew, can you just tell us a little bit more about yourself?

Sure. Thank you. I'm glad to be here. My name is Andrew Knight. I live in Raleigh, North Carolina. I went to Rochester Institute of Technology class of 2010. I'm all about testing automation. I do love Python. I've been to a number of different Python conferences and that's all I got to say for now.

So just curious question, you probably get asked this all the time. Why did you name your blog Automation Panda?

That's a great question. And yeah, people do ask me that frequently. When I started blogging, I did it as kind of like a personal challenge. I wanted to kind of take the next step in my career. I felt like I had learned a whole lot about doing testing and automation. I felt like I had something worth saying and I was like, let me blog. Well, if I'm going to make a blog, I need a catchy name for it. So people kind of know who I am, they recognize it, all that good stuff. And that name just came to me. I knew I wanted something with testing or automation. I love pandas. In fact, my wife is from China, so we have a bit of connection there. And the name just came to me and I stuck with it and I ran with it and now I'm the Automation Panda.

That's awesome. It's very memorable. So I think it was a good name.

Oh, thank you.

So, Andrew, I know you have experience a lot of different languages. I think on your blog you talk about Java, C Sharp and JavaScript. So before we get into the automation piece, why do you have a preference for Python or why do you seem to let. You seem to lean towards Python? From what I've seen, looking outside, looking.

At, yes, two main reasons why I love Python. First reason is that I love the language. Python, for anyone who's used it, is very concise, it's very readable, you can do a lot with very little verbiage. There seems to be a package to import for everything if it's not already in the standard library. And I first started doing Python actually back in high school as part of a survey of computer programming languages course. I didn't touch it again until about 2015. By that time the language had very much matured, but I just, I fell in love with it when I was using it. I thought to myself, this is what brings me the joy of coding, being able to write nice concise logic that works and works well. The second reason I love Python is for its community.

Python is very, very much a community driven language. It's not necessarily sponsored or backed by any one big company. It's not trying to seek a profit from anything. It's really, you know, it started with Guido Van Rossum who wanted to make a cool language and, and everybody just jumped on board to the tune of now, like Pycon every year, has thousands of people attending and it's now arguably either their first or second most popular language according to all sorts of leaderboards and stack overflow and such. So if you ever get involved in Python community, whether that's a local meetup group or you go to a Python conference, you'll find that the people are just great. Everybody is super motivating, everybody's super encouraging, excited for you to be there. Everybody wants to learn whether you're a newbie or a pro. And I just, I love that. And so I, I very much am a Pythonista now.

Very cool. And I guess it could be a pro or a con, but I can take Python with limited knowledge and get it to work for me. And I guess that could be bad or good because I'm a hack number one. And number two is I wonder if that leaks into automation. Like, did you see a difference when someone uses Python compared to, say, Java, how they create code? Is there, is it in the Python community? It's known that because it's so easy to get started with that people write a lot of bad code with it.

Not necessarily. I mean, you're going to have people writing bad code no matter what language it's in. I don't think just because it's easier to get started that necessarily it's going to result in bad code. In fact, it could be the opposite. Right. Because in Python there is.

That the language is opinionated that you ought to do something one way that is the right way. A lot of times, even if that way is simple, it tends to be the best or Pythonic way and that lends to good code. But I will say.

It is easier for people who aren't as versed with programming to get started in Python. And in terms of testing and automation, especially when I look at say people who historically have been manual testers and they don't know programming and they need to learn programming for the first time. Right. They may come into language like Python and be able to get started much more quickly. And yeah, their code's not going to be that great as say a more experienced programmer. But I would say that's not so much the language's fault as it is more just the nature of their experience level.

Absolutely. Like you said, it has a lot of patches, so I use it almost like duct tape.

I know. Oh man. Yeah. Last year at pycon there was a talk called Sloppy Python and it showed how you don't need to write the best code ever just to get your job done. Sometimes all you need is a quick write it and trash it script to munch some data or to do some transforms or scrape some text. It's like, yeah, why not use Python for that? It's easier than shell, at least in my opinion.

Oh yeah, no doubt. And like you said, not everything needs to enterprise level code. Sometimes you do want, like you said, a throwaway code and that's okay, as long as it saves you time and gets the job done, I think is a good idea.

Bam.

So Python though, for automation, as we mentioned, there's a lot of libraries. Are there any specific libraries you would use in order to say someone wants to use Python, they want to use it for automation. What libraries would you say that like is the core that makes up what, what you use for your day to day automation efforts?

Sure. First off the bat, PyTest. PyTest being the core framework, I think is one of the best test automation frameworks in any language. I absolutely love pytest. You have some other options in Python for core frameworks like Unit Test, or there's the BDD variants and all. But I think pytest is great first and foremost because it is.

So concise and focused on the tests, right?

In pytest you'll write test functions, not test classes, test functions. And if you want to do setup and cleanup, you write other functions that are tagged as fixtures and you can declare them in a dependency injection way where the fixtures will be automatically called based on their scope and whatever they return gets in injected into your test functions, which makes setup and cleanup much, much more scalable and much, much more reusable. So Pytest first and foremost, I think it's a fantastic framework. Other packages really come down to what you plan to be testing with your automation. If you're doing web UI testing, you're probably going to use the Selenium webdriver package. There's plenty of wrappers around that too. Like there's one called Splinter, that one. I don't use that personally, just because since I do a lot of cross language stuff, I like to keep it to Selenium API. If you're doing REST API testing using something like the requests library, which is I believe still the most popular Python library, by sheer number of downloads, that one's indispensable. There's also a. See there, like a framework or extension, I forget which, called Tavern, which kind of makes your REST API testing much more declarative. That can be helpful. Basic things like reading command line args like argparse, or if you need to do JSON or YAML, there's packages for those. The standard logging library is also very useful. You know, just plug and play with that, no need to change anything. So those are typically the ones that I gravitate towards. Oh, another one that's out. It's not necessarily new, but it's getting a lot of traction right now is hypothesis for property based testing where you can define, hey, I want to test integers of this range and hypothesis will just crank through all sorts of variations for you automatically. That one's really cool.

Cool. So hypothesis testing, do you use it now? Have you started implementing it? And if so, like, could you give a more detailed example how you are using it to do some sort of automation you weren't able to do before?

Sure, sure. So, disclaimer. I have not used hypothesis in any sort of industrial level project. I've only done a tutorial on it at pycon and then I've played around with it a little bit on my own. So I, I don't know if I have a good like case study of it, but the little bit I have done, it's, it is really nifty because it's a new way of thinking about tests, right. Previously whenever we would think about writing different test cases, we would think about inputs in terms of equivalence classes, right? For example, if I'm doing a validation test on some sort of input field for numbers, right? Your basic equivalence classes would be things like positive, negative zero, fractional number, decimal, maybe some E to the power of. Right. But with.

Hypothesis you would basically just say, hey, I want to test any number, any real number in this field. And so you don't need to spill out all these different parameters or variations of your test. And it just, and it's really not that slow. I mean it takes a little bit more time to crank through it all, but it can just generate all sorts of different kinds of data. If you have multiple parameters, it can do permutations very easily. So you can kind of just crank through a whole bunch of stuff at once because, you know, like if you start doing combos of things, it can just explode. Right? So that's a little bit more about hypothesis.

Very cool. And I believe also you mentioned, I've watched a bunch of different talks on YouTube. I believe you use BDD as well. So do you see as a good user case for Python, is it a good strong BDD framework?

It does, it does. In fact, Python has multiple BDD frameworks. What I find amusing with Python is that the major BDD framework out there, behavior driven development framework. For folks who don't know bdd, the big buzzword project is cucumber, right? Cucumber in Ruby, Cucumber in Java, SpecFlow and C. Python actually doesn't have an official cucumber framework, but what they have are a bunch of cucumber esque knockoffs, so to speak. I don't want to call them knockoffs, that's kind of degrading. But like there are Python versions of it but not officially sanctioned. The most popular one, based on what I can get a pulse on, is called Behave. It's almost identical to Cucumber, but it's in. Python has the same thing. You have your gherkin feature files, you have functions for each of the different gherkin steps given when then you have an environment PY file that you can do some setup and cleanup stuff in. Like hey, let me put my webdriver initialization in there. Based on my conversations with people in the Python community who do Behavior driven development, Behave seems to be the most popular choice. It's got the most support, it's got the Most users on GitHub, I think PyCharm, the major Python IDE from JetBrains, they have support for Behave, which you have to pay for that license to get that support, but it's still there, so it's pretty good. However, I personally now prefer pytest bdd, which is a plugin for pytest, the framework I mentioned before. Very similar. I mean you're still doing gherkin feature files and the steps are still functions. So it functions just like a cucumber framework, but with PyTestBDD, because it is a plugin for PyTest, you get all the other goodness of PyTest with it. For example, if you want to do generate code coverage reports, that's another Pytest plugin. Boom. If you want HTML reports, if you want to do parallel test execution, which, which is something. For the longest time Behave has struggled with anything you can do with pytest you can do with pytest bdd, which in my opinion makes it much more powerful because Behave is a standalone framework, which means you need somebody deliberately adding it to Behave to extend it.

Very cool. So with all these packages, for someone that's never used Python, is it all open source or do you have to pay for certain packages or is it freely available?

All the ones I've mentioned are free and open source.

Nice.

That is a hallmark of the Python community. It's for the community, by the community. Very, very strong presence of supporting open source software, free software and also resources to help you get started. Like, you should not have to pay a dime to start learning Python and.

To start using Python in your day to day job. Like, are you using Python? And if so, like does your team use Python or do you use something else?

I wish.

So here's where the emperor has no clothes. As much of a Pythonista as I am, I actually don't use Python in my day to day job. I know that's probably shocking and even scandalous to some people, but I mean, I don't hide that fact. I'm actually a C NET developer by day. At my current day job.

I will say I do. My company does do Python. In fact, our whole data division is pretty much now a Python shop. We all went to Pycon together, which is really cool. On my own, I do lots of Python. My wife runs small businesses. I've developed web apps for her using Django. Anytime I need to do Something quick and dirty for myself, I'm using Python.

And who knows, maybe in the future I'll get another opportunity where I do Python with my day to day.

Very cool. So I'll be honest, this might be scandalous. I like C better than Java, so I think the Visual Studio, the IDE is pretty good. It's a nice IDE to develop.

I don't think that's scandalous. In fact, more and more over time, I'm starting to be in your camp. I don't advertise that because it's shocking. I don't think Java is bad, but there are definitely advantages in the C language and tooling.

If someone is using C, isn't it like iron Python or something? An implementation you could use if you.

Iron Python was around for a long time and supported for a long time, but I think it's a dead project now. I don't know if there's been any updates recently, but yeah, the idea was that iron Python was supposed to be Python on Net, so the same language stuff under the hood.

Gotcha. In one of your other videos you were talking about making Python thread safe. I guess you were getting to the point of how to make pull Python automated test run in parallel. Could you just talk a little bit about what you meant by like how to make a script thread safe in order so you can run it in parallel?

Sure, sure. So when you have.

Above unit tests, black box tests, Things like REST APIs or especially web UI testing, or any sort of end to end test, right? You're testing a real live product as it's doing its thing.

And parallelization is not as simple as putting it passing in a command line option, right? You can run any number of tests at the same time and if they're all hitting the same thing under test, you could very easily end up colliding the tests, right? Where let's say you log into a web site and you set one field and at the same time another test logs into the website and unsets that field. Well now your test assertions later are going to explode because you just messed up each other's state. That's what I call a collision. And collisions can happen, you know, in the product, but they can also happen in the code. So and this, this applies for any language, not just Python. Like if you, if you're using global variables, not global constants but global variables, or you're using a singleton that is not immutable, stuff like that. All of a sudden now your non thread safe code could be Colliding and changing values and messing up tests. So, like with Python, if you're using the pytest framework, there is a plugin called Pytest XDIST for like, distribution. Like, if you're going to distribute your tests over multiple test clients, or if you want to just simply run multiple tests in parallel on one test machine. Right. You can ease. It's just dash n and then a number. Like, if you want three threads, dash N3. Right. You want 30 threads, dash N30. You can do that. There's nothing to stop you. The question you'll need to ask yourself as a test automation engineer is, is my solution thread safe? Meaning are all my test cases truly going to be independent of each other if they run in the same address space? Right. Am I making sure that I'm using things like dependency injection instead of global variables or a singleton? Am I making sure to avoid static variables like the plague? Am I making sure that when it comes to interacting with the product and features under test, that either every single test instance is interacting with a different instance of the product, or if they're sharing an instance of the product, that. That there's no stickiness to my data, that could have interoperability problems? Right. That's kind of what I meant by making sure your tests are thread safe.

Cool. So along those same lines, I guess, when you're running in parallel, another issue comes up with test data. So, any tips specific to Python and managing test data?

Yeah, test data is a very abused phrase. What does it mean? Are we talking about the, like, the test case variations? Like if I'm doing a Google search test, you know, searching for Panda versus searching for Python versus searching for elephant? Or am I talking about maybe some.

Configuration data that I need to input into my automation system, like usernames and passwords or app URLs. Because maybe I have a development environment, a staging environment, production environment. It could be stuff that needs to be already set up in the app. Like, like you need to have a user account set up and that could be considered test data. So all those kinds of things, I mean, we could talk all day about test data.

Never, never commit, like config data to your repo. Of course, that's language universal. You don't want to put a password into git. Don't do that.

Python, though, makes it a little bit easier to handle test data, I think. I mean, every language there's some sort of files. But, like.

One thing I typically do, especially if I need to set up a little tiny test Solution real quick. Or if I'm doing an example to demo how you can handle test data or configuration data, meaning, let's say configuration data inputs into a system that define a configuration.

I like to use just plain old text files, usually like JSON or YAML. And with Python, the JSON library in Python is part of the standard library. You don't need to pip, install anything, you don't need to go download another package. You have JSON out of the box. And so a lot of times I'll just be like, hey, let's define all the different fields we need to read in. Maybe it's app, URL, username, password. Easy, right? We can define a very simple JSON file. And in like one or two lines of Python, I can read in that JSON file and have it parsed as a dictionary, and then I can inject that into my tests as needed, right? Read it once, inject it into each test, and voila, you have a config data solution. Nice, neat, tight, very cool.

So are there any tips for arranging your tests? It just seems to me in Java with packages, maybe because I'm a novice, with Python, it seems easy to arrange a framework in a way that makes sense to folks. Is Python the same way?

So if we're looking, just if we're opening the can and looking inside a Python test project directory, it's going to be fairly similar. In Python.

A project is going to be organized into packages. You can easily tell a package in Python is is because it's a folder, and inside that folder you have a file. Underscore, underscore. Initially you can tell there's a Python package and the package is how you can name space to import things.

Almost always in a Python project, right next to all the packages at the top level, you'll also have a tests directory. And in there, that's where you put all of your test code. Inside there, you may or may not have packages for tests, because let's say maybe you have a Web UI test solution and you're putting your page objects in there. Maybe there's a package for that in your tests directory.

Beyond that, all of your test cases, assuming you're using something like PyTest or unit test, are all going to be just inside of regular Python modules. So something something py. And then you'll have either if you're using unit test, you'll have test classes, or if you're using pytest, test functions and you can really organize them however you want. I like to group them by functional area. Right, Whatever makes sense, right? If you're talking about like, okay, this, this particular part of my app or this particular set of APIs, kind of group them together, maybe put all those tests in one module, or maybe have those modules in a package that are appropriately named, much like you would in, say, a Java Maven project or a C project. Not much different at all.

I know with pytest, and I'm pretty sure with Unit Test I'd have to double check, but I know for a fact with pytest you can easily filter when you choose to run tests. You can filter by a specific test directory, you can filter by test function names, you can filter by markings on those tests which act like tags. And so, yeah, I mean, you basically define the structure however you want. I like to do it by clustering with functional areas and maybe have a small little hierarchy going with that for whatever makes sense. And you just run and go, awesome.

Now you did mention Maven, and that's a good way to manage dependencies within Java. Is there something similar in Python? If you go on a brand new machine, you go, hey, here's the framework, go run it.

Yes, yes. Virtual environments. Python packaging can be a little bit sticky. It has a storied history, it has a lot of frustration, a lot of people hate it. But to its credit, you know, Python does all of its Python packaging works, and I think it works pretty well, so. And a lot of people have put in a lot of hard effort to get it working way better than it has been in the past. So kudos to them. But with, with Python packaging, the best practice of a workflow, anytime you start a new Python project, what you want to do is you want to create what's called a virtual environment.

You'll create your directory for your new project, right? Make dear my happy project. Boom. CD into my happy project. Whatever. Excuse me. Then what you want to do to create the virtual environment, the most common tool for this is called Virtual Env. There's another tool out there that's becoming popular, called pipenv. But what you do is you invoke one of these commands. You say, hey, I want to create an environment like virtualenv. I can't even remember the command off the top of my head. You easily Google it. If you Google Python Virtual Env, you'll find it. You'll create the virtual environment. And what that does is it creates a local directory for your project that's going to contain all of your packages for that project. So that way it doesn't matter. Whatever Python is globally installed on your machine, right? Your project is going to be dependent upon the packages in your virtual environment. So that way it's safeguarded against other versions that might be running wild. Whether or not your machine has a package or not, you'll have it here locally so you know your project is safe. And yeah, it'll even put the Python interpreter in there for you. And so you can just, at that point just run your tests or run your code and it'll pick up those packages instead of the global ones. And that's in a nutshell, how you handle packaging in Python. What's nice about that is then you can freeze the requirements in a requirements txt file so that if someone else needs to come by and pick it up, they just say, hey, you know, I want to install. Use pip Python installer for packages.

They can say pip install requirements txt and it'll grab all of the packages for that particular project.

Or if they're using pipnv, they can do like a pipnv install and that uses a slightly different files called pip file and pip file lock. But it's, it's the same basic concept, just a little more technical.

Cool. So earlier you mentioned there's a lot of resources around Python. So if someone wants to get started with Python. Once again, one of your videos, I think you mentioned three books and I don't remember what they were. One was, I think Brian Arkin, any resources? I guess my question for someone that wants to get started with Python or learn more about Python.

So if you want to get started with Python, just not necessarily about test automation, but the Python language.

My best recommendations are. Okay, first of all, are you willing to spend money or not? If you want purely free resources, that's totally cool. If you just Google learn Python, Python.org has links to tutorials. There's like a. I forget the different websites and all, there's like Pythontutorial.com or LearnPython.com WikiBooks, sister side of Wikipedia. Wikibooks has a decent Python book.

All just great resources for like, oh, how do I create a variable, how do I create a list? What's a for loop basic kind of stuff. There are other good books out there for learning Python. I know nostarch Press has a really good book on it. There's even ones targeted precisely for kids, not necessarily adults. You can go to your public library and probably find any sort of books on Python. Check them out for free. And then also different courses from websites like Codecademy or Udemy. Not trying to like say one is better than the other. I mean I haven't taken any of those courses but I just know that Intro to Python is one that there it seems like someone always their freebies that they're saying hey, you know, try this, right? And if you stack up enough courses you're probably going to get a pretty good.

Learning experience out of it if you stick to it.

If, if you're looking for.

Trying to practice Python and hone your skills, there's a, there are different websites I can recommend, one of which is called Pybytes P Y B I T E S. What they do is they have Ninja code challenges, right, so you can, they'll give you a little tiny bite sized exercise and they'll say, hey, here's this problem, solve it with Python and you code up your solution, you give it a try, pass, fail. If you, you pass, you win. It's a great way to kind of stay sharp with your skills. Another one very similar to that is Python Morsels. I think that's like a weekly challenge. Trey Hunter is a guy who runs it. He sends it out like you know, every so often. I think it's like on a weekly cadence he'll send out another problem and you try to tackle it. And the idea, it's a bite sized problem to keep your skills sharp. So those are excellent resources for learning Python and practicing Python in general and of course conferences go to conferences, everybody go to Python Conf. If you're looking for learning Python for test automation specifically, the two best books I can recommend are Python Testing with Pytest by Brian Ocken and Pytest Quick Start Guide by Bruno Oliveira. Both of those books are excellent resources. They're a little bit different in how they approach the problem. Bruno's book is more friendly for I'm new to all of this. Just let me learn from the basics, how I get Pytest up and going and getting some tests in it. Whereas Brian's book is, and his is like the power manual. Like it still has that tutorial sense of walking through and learning from, from the beginning but it's densely packed with amazing information. So both are excellent resources. I recommend them both. I have both, I've read both, I endorse both. They're great.

Okay Andrew, before we go, is there one piece of actionable advice you can give someone to help improve their Python automation testing efforts? And let us know the best way to find or contact you.

I mean one, one quick thing off the fly would be if you want to be good at Python test automation, I would say above any framework, use Pytest, learn Pytest, and specifically learn Pytest fixtures very well, and I've already given the resources for that. I think Python is a tremendously awesome language for test automation, but ultimately you've got to know how to write good tests, right? You can write good tests in any language, and so if the focus is on testing and automation test, definitely make sure you can write good tests before you worry about writing good tests in Python. So if you want to contact me, you can reach out to me on Twitter. My handle is automationpanda and you can also reach out to me on my blog@automationpanda.com thank you Andrew for your Python Automation awesomeness.

For links to everything of value we covered in this episode, head on over to testtalks.com 260 and while you're there, make sure to click on that Sign up for a free trial link under the exclusive Sponsor section to learn all about Sauce Labs awesome products and services. So that's it for this episode of Test Talks. I'm Joe. My mission is to help you succeed with creating automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Talks podcast. Head on over to www.testtalks.com for full show notes, amazing blog articles and other automation awesomeness.