Smashtest Rapidly Describe and Deploy Tests with Peter Tesler

Five, four, three, two, one.

Drop that.

Welcome to the Test Guild Automation Podcast. We're all getting together to learn more about software testing and automation with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we're going to be talking with Peter Tesler, founder at Smash Test, all about Smash Test, which is a language for rapidly describing and deploying test cases. So listen up and discover a way to greatly speed up your automated testing.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Source Labs. Their cloud based test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial. Visit testguild.com sourcelabs and click on the exclusive sponsor section to try it for free for 14 days. Check it out.

Hey Peter, welcome to the Guild.

Hey Joe, how's it going?

Great, great. I'm so glad you, you agreed to be interviewed. Someone that I was speaking to the other day, I posted something I think on Selenite and they said, oh, do you know about Smash Test? So I've never heard of it. So I'm really excited to learn more about it. But before we get into it, Peter, could you just tell us a little bit more about yourself?

I've been a software engineer for pretty much all of my life. I'd say I've been a tester for a good part of it.

Did some work at Microsoft, did some work at Disney. I'm a serial entrepreneur at heart. So I've started numerous startups, small scale, just seed funding. Learned a lot from that experience, from those experiences. And now Smash Test is the thing I'm working on.

Very cool. So Peter, what other startups were they? Were they all testing focus type startups?

No, actually I had startups and pretty much everything. We had a website for debating, for logically organizing debates.

There was a dating site that I built many, many years back. What else did I work on? Yeah, those are the two big ones. I did some AI stuff early as well, but that never took off.

Now Smash Test and then under Smash Test we've tried a number of products and the language that you alluded to, that's the latest thing we've been working on.

Cool. So it sounds like you're always looking for opportunities. Why start up a business then in the testing arena?

Because that's the thing I'm passionate about. I like breaking stuff, I guess as a kid, I know it's a bit of a cliche nowadays, but as a kid I was always the one who would knock down the tower blocks after building it. And so that's the thing I like. I like solving puzzles, I like figuring out how to compromise a system. And so I decided, since that's the thing I've been working on professionally for such a long time, I said, you know what might be a great opportunity for a startup, especially with just how much effort and how much capital companies are putting into testing nowadays.

Big time. Yeah. It seems like a lot of people are rushing into the space to throw cash around. Especially if you said Smash Test was a AI and machine learning enabled, you'd probably get funded right away.

So, Peter, I thought Smash Test was the language. So it sounds like Smash Test is the company and then one of the products is the language. Can you explain that?

Yeah, smashtest is the language. So.

The organization is called Smash Test, but so is the language. Right.

Right now it's an open source project. So we have a number of contributors. Myself being full time.

Yeah, that's where it's going right now.

You did mention you worked for Microsoft and Disney. Did you learn anything from your experience there that helped you start this company? Especially in the testing phase, dealing with.

People, I guess, working with large teams, working cross team.

Testing. Testing something that's currently in development. Right. Testing something that changes very rapidly. That's the big one. A lot of things that companies, a lot of things that companies work on are always changing around and you may think you're testing it one way, but they change it up on you and you have to switch course and you know, course. Correct.

So absolutely, it sounds like speed is essential for testing and especially if things are changing around. So is that one of the reasons why you created Smash Tests?

Yep, absolutely. Speed is the number one factor because in my experience at least you build a test, stuff changes and you can't blame stuff for changing. Right. Especially in the software arena. Software is meant to change really, really quickly, especially while it's being developed. So the thing with test is that you want to be able to write them as quickly and effortlessly as possible. You need to be able to build them, update them, maintain them and tear them down inevitably with as little effort as possible. And that's been a problem, especially in the field of automation. Test automation, because it takes a while to build and test with today's tools.

So Smash Test, we alluded to it that it's a language. Is it really a language or does it use JavaScript and you're using some sort of keyword driven approach to create your Test script?

Yeah, it's a language, it's a tool.

It'S everything. That's the name I use, pretty much everything to encompass this whole effort. It is a language. It. It comes with a interpreter, right? So when you download SMASH Test off of npm, it's an interpreter written on top of Node js. In theory, you could extend it to any programming language, so everybody has their favorite ones. There'll be a version for that. Right now everything is Node js. Basically, you type out a test in plain English and each of the.

Phrases you type out map to some code that either you implement or is pre implemented.

The code itself is structured as a tree, which is the.

Selling point of SMASH Test. It is what differentiates itself from the rest of the pack. Whenever you think as a tester, you're thinking in terms of permutations. SMASH Test allows you to express a test as the permutations that come out of each point in your test, as opposed to a linear set of instructions that traditional tests follow.

So is this different than cucumber or bdd? Is it that type of syntax or even gauge has some sort of like.

Their own version of a syntax that then is implemented in the background in code?

Yeah. No, in terms of syntax, you're right on the money.

It strives to be as close to human readable as possible. So written in English or whatever language you choose to write it in, it comes with a bunch of built in steps. Before that, I should probably talk about the packages it comes with. Right now it comes with two packages. By that I mean pre installed functionality. It can test web UI with Selenium WebDriver and it could test HTTP APIs. So right now it does those two things.

At any time, anybody can come and write their own package and use that. So you could test whatever. Right. You could test mobile apps if you want. Anything that runs in Node JS that can be used to use within SMASH Test. The major thing that separates SMASH Test from a typical TDD or BDD framework is the fact that you write test as a treat. What I mean by that is whenever you're building out a test, you always think of the permutations that emanate from any given point. As an example, let's say you're a web developer and you're testing a web page. Suppose there is a text box on a webpage. People say that testers try to think like the user, try to emulate the user, which is true to a certain extent, but at the same time the tester thinks breadth first, right? Whereas the user thinks depth first. So going to that webpage example, right, you have a text box, let's say for a username, a user comes in, types in their username, bye bye, doesn't think about that text box anymore, and they move on to further steps. Password, enter, whatever it is. Right? Logged in a tester, when they see a text box, they think of all the different permutations that can emanate from there. So I could put in a valid username, invalid username, I could put in an empty string, I could put in something that's too long ago, put in something that has very long UTF 8 characters. You name it. You tend to think of all that stuff at once.

When I was building SMASH Test, I was saying, let's build a language where what goes on in your head maps to what goes into your editor, right? So when you're looking at that text box and you're thinking of all the permutations, all those permutations go down underneath in one place, right? And all the permutations that can, let's say, emanate from you typing in a wrong password that goes underneath the wrong username, that goes underneath the wrong username. Leaf in the tree. So it's hard to explain verbally. It's much better to show people, and it's easier to explain it that way. But the whole gist of it is that how you think is how you document, type out, automate a test, and that speeds things up much much.

So why would it speed it up? Is it because you already have the setup and then you're just doing the different permutations under that so it's less that you have to write?

Yep. So that's one of the things definitely.

When you're writing a traditional test, right? You get to that text box, you choose a permutation, and then you continue on your linear test. All right, Here's a username, here's a password. Then you might hit enter, and then you think of a bunch of steps, right? Then you have to go back to the very beginning, and you have to go all the way down again. Okay, we're back at the username box. Let's see, which permutation did I think of? Which ones did I not think? Which ones did I cover yet? Which ones that I didn't cover yet? And then go down the list of steps with SMASH Test, when you're at the text box, you type it, Type out all the permutations at once. And then from that point forward, you don't have to return in that place ever again. You just choose a permutation you already jotted down and from that point you come up with more permutations, like a branching tree.

So there's less to type out, there's less unnatural thinking. You don't have to go back to square one and start thinking of what you did before. You think you type out in the same manner, in the same order that you think in.

Yeah. So it seems like it's more tester centric, where.

Absolutely.

Okay. It maps to way A lot of knocks on automation is it only validates one thing or it doesn't validate five other or 10 other things you want to know. So it sounds like this is something that can help you get that in a style that's easy to read and a flow that a tester would be very familiar with.

Absolutely. And we cut back on some of the stuff we found to be excess in automation. Like nowadays, everybody names their test, right? Everybody. You have to type in the test name, but.

What'S in a name? Right. Do you get much value out of naming your tests?

Turns out most tests, and we were talking about, you know, various kinds of experiences, you learn as a tester what. Most names of tests tend to be very unhelpful. They're like car test number 15 or something. Right. They don't really convey much information. And a lot of tests are very close to other tests in terms of steps. They only vary in one thing. Let's say maybe you had a checkbox checked in one, you don't have it checked in the other.

So really the best way to describe a test is with the steps themselves. And there's no real.

Naming a test only slows you down is what I'm getting at. So in smash tests, you wouldn't even name your test, although you can if you want to. You just type out your steps and type out how functionality branches from each point in time.

Is there a way to annotate it, though? Because I'm just thinking of regulations. When you're working in healthcare, a lot of times they're like, oh, you have a patient criteria here. Show me the test that maps to that requirement. Are you able to do any sort of annotation on the test? You could do mapping to a requirement if you had to.

Yeah. There's quite a few ways of going around that you could put in what is called a text step with the name of the annotation you want to put in. You could put in a group identifier with A hashtag. And there's always the good old fashioned comment. You could put that in. And then you just do a search and you find the line with the comment with the ID of whatever regulation or whatever requirement you're trying to find.

Is there any concepts of a code reuse or a step reuse so that you're not constantly saying the same thing over and over again?

Yep, absolutely. So we have a concept of functions here. And you do something very often you make a function refactor. You have, let's say a function called login and you have all the steps to log in.

And then anywhere in your test you just say login and it calls that function and then the report that it spews out at the end, it'll contain. You can un collapse the login part and you see all the steps that were generated from there that fall under login and help troubleshoot that way if something goes awry.

Another great thing is that within functions you can branch as well.

Let's say there's two different ways of doing something. Let's say.

When you log in, you can either hit enter or you can click a login button. Right? So within your login function you can have a branch, one branch that.

Clicks the button and the other branch it hits enter.

And then whenever you use login anywhere in your tests, it'll automatically create two tests. It'll branch, it'll double, and it'll do login the one way. And then another test, it'll do login the other way. And if there's a problem with one, then usually, usually it's a problem with one. Right. It'll find that smash test is a great way to express many different permutations in as little typing as possible, as little coding as possible.

Nice. Besides this type of functionality, you have other things like a live report. What is a live report?

Yeah. So while your tests are running, you open up your report and you see in real time which step is currently running. By default we do parallel. So you run your test, they usually run in parallel. You see which which tests are running. You can click on a step, you could see which. Well, you could see which steps have completed which ones have not completed yet which ones pass, which ones didn't click on a step immediately, see the screenshot before, the screenshot after of that step. And yeah, it's a good way to monitor, especially when you're running your tests as headless. You can't see the browsers, which may might be a good thing. They don't get in your way, but at the same time you're wondering, well, did something freeze? Did something? How many of my tests are left? Which ones failed? I want to start get a little head start on my debugging. Which ones failed already? So open a report and see that.

People that do have long running test suites within their CI systems, sometimes it takes an hour or two. So it sounds like this is cool because if, if something's failing, someone could start triaging before the whole test suite is done.

Absolutely.

What other things or benefits you get in the reporting? Are there screenshots or good error messages that let you know where exactly the error may have occurred?

Yep. So you get error messages, you get screenshots, pretty much everything you said. It's a pretty nice looking report, at least in my opinion. It's dark background, so at least it's easy on your eyes, like your editor probably is. One of my favorite features of the report is that it consolidates errors that are similar. So let's say.

You have a particular test, right? And it does something in Chrome, and then you have another test that does the same thing in Firefox, right? Let's say that test fails in each of those browsers. So it'll consolidate all those failures into one error. And so you can open up that failing test and see, oh look, it failed in every browser. So I guess the browser is not the culprit. If it only fails in one browser, then you say, oh look, it only failed in Firefox, but it passed in every other browser. So it must be a problem with Firefox specifically. And that's great when troubleshooting because it'll easily tell you, because you could easily permute on every step. You can isolate problems much quicker that way, but just by seeing where it fails everywhere versus where it fails at a particular point.

Cool, Peter. So can you give a quick example?

Let's see. I like the browser example, right? But it doesn't have to just be browsers going back to the login example, right? Suppose you have a test that tests multiple users. So like on the homepage we have an example username is Joe, Bob and Mary, right? So we test that you log in as Joe or Bob or Mary. In each of those cases, you verify success, right? And verify success is, let's say, a function of verify certain things on the page at the end. So you run your test. It fails when you're Joe, it fails when you're Bob, it fails when you're Mary. In the report, it consolidates all those failures together and you look at it and you say, huh, looks like it's not a problem that's specific to a particular username. It's a problem that lies elsewhere because the test failed for both Joe, Bob and Mary. Just like with the browsers, right? Tests can fail in all the browsers you try, or could fail in a specific one. When it's a specific one, you know that you have to dive into that like the culprit, the cause of the bug is probably there. So with usernames, if it only fails under Bob, then that means there's something up with Bob, right? Maybe Bob is a specific type of user and there's a bug related to that specific type of user. The good thing that what the report does is it consolidates in a way that's really easy to read, so you don't have to look for, hey, what are all the other similar tests that failed? They're all in one place.

I believe it also helps you run tests in parallel. That's another thing teams struggle with. Just a little bit more about running in parallel. Are there any benefits out of the box that you get with it?

Yeah, it runs everything in parallel by default.

I found that testers, or at least automation testers, find parallelism to be challenging, both setting it up and then actually running in a way where each of the parallel instances doesn't interfere with each other. Right. So I'm testing a. I don't know, an airline booking website, right? And each of my tests log in with the same username and they start doing stuff. And one puts a flight into the cart, the other one takes a flight out of the cart, and they're all messing with each other because they're all messing with the same piece of state. The SMASH test comes with a solution to that, which is a exclamation point operator. You put that in and tells you everything underneath all the branches underneath this line. Don't run them in parallel, run them sequentially. You could run other, other branches in parallel to them, but these, the ones that I marked, run them all separately because they interfere with each other. But absent that operator, it runs everything in parallel by default. You don't have to think about it. And that's great because with Smash Test, again, it's so easy to create permutations. You just add a line, you can easily double the amount of cases you've got covered.

You could generate a lot of different cases really, really quickly. And like you said, you don't want to be sitting around for hours and hours and hours. Waiting for that suite to finish. So parallelism by default is the best situation in my opinion.

Very cool. So obviously it also integrates with cicd. Is this because it works on a command line like Maven, you can pass the commands of how you want it to run and things like that?

Yep. And it all comes out of the box, so there's no. Whenever you're building a suite from scratch, you take Maven, you have to take all these other tools, you have to smash them together and hook them up, make sure they all work. Smash test, you just create a smash file text file, put your test in, you run smash test file name, hit enter, and you're good to go, obviously, with the stuff. So if it's a web test with Selenium pre install, obviously, but that's usually fast.

Peter, how long has this been out for?

Oh, it's been out for a whopping two months now.

We had some good press initially.

Some Reddit, Reddit liked us, which I don't know how often. Reddit likes something very rare.

Yep.

So we had some good press there. A couple of other blogs picked us up, some. Some news organizations picked us up, so we got some. Some good traffic there. Got to build the community a little bit. Got some. Some. Some really good people there.

I think in the first few days, somebody translated our documentation to Chinese. Somebody wrote a VS code extension. That was a big one. Somebody wrote a VS code extension for SMASH Test, because we already had an extension for Adam. But obviously everybody's got their own favorite editor. Everybody's so passionate about their favorite. And, you know, VS code is pretty popular nowadays, so we got that going thanks to the community. A lot of bug fixes. It's been pretty stable as of late, so it's still in beta officially, but that might be coming off sometime soon.

I think this is something you need to try because it seems very unique to anything I've heard before. So any feedback on your approach so far on the direction you've taken this?

I have to think about it because everybody in the area is opinionated. It's been overwhelmingly positive. Usually the feedback is, oh, hey, can you. Can. Does it work with, you know, this CI provider? Does it work with this cloud service? Does it make French toast? You know, you know, it's like. It's like, does it do my thing? We had some people, you know, some people are passionate about languages. So it's a node js. But then they ask about, what about Python, what about this, what about that? Easily portable. It's all coming, coming Up.

What else did they say?

Stuff like Appium is coming. It's very easy to write a package for Smash Test and have it work with your favorite X, Y and Z.

What else does it do? It does. A lot of. It does mocking really well. It does things like you could throttle network. You could test under different network conditions just by typing in, hey, the Internet's down. And there's. That's not the actual line, but, you know, when the Internet's down, then do this, do that. I think what's coming up next is I'm going to release some training for Smash Test because a lot of the questions, a lot of the feedback is stuff that's in the documentation. But I think some people prefer like an audio or a visual version of it. So I think that's what's on my radar right now. That's usually the feedback.

Okay, Peter, before we go, is there one piece of actionable advice you can give someone to help them with their automated testing efforts or even getting started with Smash Test and let us know the best way to find our contactor?

I would say test automation is a tool, right? So it's a tool to help you test. And testing is all about trying to figure out what the requirements and user expectations are and making sure a system follows or conforms to those expectations. So part of what makes testing an art, in my opinion, is that you sit there and you try to figure out what all the different permutations are. How can you hack this particular system? So I would say if you're going into test automation, don't forget that.

Forget the art of testing, right? The art of trying to understand how system works on the inside, figuring out what the risk points are, where components are being held together with gum and sticky tape and concentrate your efforts on those points.

And yeah, that's what I would say in terms of testing, Smash Tests. I would say check out the site Smash Test IO and yeah, there's a nice tutorial on there. I'll be coming out with some YouTube videos shortly that should be. That should be some good place to start. And there's some really good blogs that are out there, some really good tutorials that some other folks have been playing around with Smash Test have put forward. So Google that contact. Yeah, smashtest IO is a good place to start. Or you can email me directly at PeterMarsh smash test IO and I'll be happy to answer any of your questions.

Thank you, Peter, for your Smash Test Automation awesomeness. For links to everything of value we covered in this episode, head on over to testguild.com alike automation a272 and while you're there, make sure to click on that. Sign up for a free trial link under the Exclusive Sponsor section to learn all about Source Labs awesome products and services. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe and my mission is to help you succeed with creating Automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles, and online testing conferences. Make sure to subscribe to join the.

Guild and continue your testing journey. This has been a Joe Calantonio production.