240: How to Listen to Your Tests with Alex Schladebeck

Hit it.

Welcome to the Test Talks podcast, the place to go to geek out on software testing. And now your host whose mission is to help you succeed with test automation, Joe Colantonio.

Hey, it's Joe. And welcome to another episode of Test Talks. Today we're going to be speaking with Alex Schlatterbeck. I recently got a tweet from Angie Jones telling me I needed to get Alex on Test Talks. And my rule is anytime Angie Jones tells me to do something, I would do it. So I was very grateful that I did. Alex has a wealth of information on testing. We cover a lot of different topics about linguistics and how that relates to testing, how to stop hating your UI tests, and a whole other array of topics. You don't want to miss this episode. Check it out.

Test Talks is sponsored by the fantastic folks at Sauce Labs, the cloud based automated testing platform that eliminates the need to maintain your own selenium grid and test infrastructure. Try it for free today. Visit testalks.com and click on the sign up now link under the homepage sponsor section.

Hey, Alex. Welcome to Test Talks.

Hey, it's great to be here.

Awesome. Today I'd like to talk about all your experience around testing. I've been researching a bunch of different things. You have a lot of topics in a lot of different areas, so we're probably going to cover a whole bunch of different things. But before we get into it, can you just tell us a little bit more about yourself?

Sure.

You've kind of put it together quite well. I'm interested in a lot of things and I do a lot of things. My official title is that I'm head of Software Quality and Test Consulting at Braidex. That's a great title. I love it. And it means I can do a lot of things. So I coach testers, I go to customers and do workshops. I go to development teams and help them to maybe see where they're having some problems and do workshops with them as well. I get to go and talk at conferences. Of course. The great thing about talking at conferences is that I actually end up learning a great deal as well, which I bring back to my team. I help to define the company's strategy when it comes to quality and testing and also some of the software stuff that we do. Yeah, it's a big mix of things. That's the work side. And outside of work, I'm also a little bit of a whirlwind. I like to play music and do lots and lots of sports and I'm a bit of an Adrenaline junkie. So roller coasters and obstacle races and things like that.

Wow. Very cool. So, yeah, the first thing I noticed was your title Head of Testing. And it sounds like such a big title. Like, man, I don't know if I would want that responsibility. So how does that work in your company? Did I go to the Head of testing for everything testing related?

It's, it's a bit crazy. I always said I actually wanted the title Master of Disaster, but no one's ever actually put that on a. On a business card for me yet.

Anyway. So I think the nice thing is that I kind of. I very much grew into the role. I've been at the company for a long time. I was this, I think, Yeah, I was the first person, I think, to join to be in development teams that didn't have. Didn't have a background in development. And of course that was not going to say people were mean to me. But of course it's difficult to work with those people or to know how to work with those people at the beginning. And over the years, my team has grown. A lot of people have come to the team. I've always been there. So they've always seen me as someone, okay, well, at least even if it's only two weeks, you know, she knows two weeks more than I do, which is sometimes what you need. And then when it came, I think it was maybe about four years ago, they said, okay, you know a lot about testing. We want you to be responsible for this as a domain. And then over the last two years, they said, hey, you know a lot about testing and you know a lot about people as well now, so strategy and line management and things like that. So fortunately, it's. I think it's just always been the case that people have gone, oh, yeah, Alex, she's the girl that does testing. That's cool. So I've never really had any problems with it. I. To be honest with you, I don't know what they call me when I'm not listening.

And I also noticed you have a master's degree in phonetics and penology. Peonology. I've never heard of this before. I guess it has to do with sound or language.

Yeah.

I like talking and I did linguistics and languages at university, which is how I learned German and French, how I eventually ended up coming to Germany. And linguistics, I just love it. It's the science of language. And the really funny thing is if you work in it, people say, oh, you did linguistics. That means you did semantics. Right? And I was I was the worst at semantics, like computer linguistics and semantics. They were not. They were not my best topics. Phonetics and phonology is. You're right, it's the study of sound. That's phonetics. So how we pronounce things and the very, very fine detail about what that is. And then phonology is what meaning is behind that. So some different sounds that we make mean the same things and some different sounds that we make mean different things. And then that gets really exciting that some tiny, tiny changes that we have, they don't mean anything different, but in social contexts like conversation, they lead us to do different things. And that's conversation analysis. Absolutely fascinating stuff. I could go on about that for a long time as well. I'm a little bit of a geek about it, but that's where I come from. Yeah.

Awesome. The reason why I ask is I imagine there's certain patterns you might hear, I assume, in certain sounds across different languages. Are there any patterns you see in testing that you can. That because you're a background, you're able to maybe get better insight. Not better insight, but different insights than someone else that maybe doesn't have this type of background?

I honestly think that's a really good question. That's just made me realize that I think maybe I do have that. One of the things that we learn in linguistics is looking at detail and then abstracting to models and getting back from models to. To look into detail. And the other thing that we have when you look at historical linguistics and sound shifts, which is I did less of that, but you're aware of them, is that sometimes things stay quite the same for a long time with minimal changes here and there. And then, and then there is a. There's a shift. And. And you look back at those two versions and you go, oh, wow, they, they don't sound like each other at all. Or it's hard to see where one came from the other. And I. Those are things that I noticed within the industry. I'm fond of saying that test data management is the new continuous integration. When I started out, continuous integration was painful. Nobody wanted to do it. Developers were often like, if the testers really want it, then they're going to have to do it themselves. And testers were often saying, well, we need your support and you to do this as well. Now continuous integration is something that most people are like, yeah, we should totally be doing this. This is everyone's responsibility. And then I look at the next thing where I see testers going, please, will you help us to do this, it's relevant for. And that one of those things is test data management, doing that sensibly. And I see the same conversations happening there. So we've got kind of change that starts change that's painful and then you look back and say, well, of course we all have to work on this together. So yeah, I tend to pattern match. I like analogies. I like looking at the world in terms of what that like or what does it remind me of? And I think a lot of that comes from, comes from doing that kind of studies.

Talking about test data management, like I said, you have so much experience. I'm just going to freeform here in Prov. It is a big topic and it's one that I've been hearing more and more of and a lot of. And it's something that a lot of companies I speak to struggle with. So what patterns have you seen in test data management? Or like, do you have any suggestions or advice for someone that is struggling with test data management? That's the question.

Yeah. I think that the first step is just even being aware that it's a thing that you're going to have to deal with because I think so many people, it's one of those aspects, especially when you're talking about automation. Obviously it doesn't just have to be with respect to automation, but I think you start as noting the pain of it when you've got, let's say you've got some kind of test suite that can run again and again and again and then suddenly you notice that your test data isn't set up to be able to do that. You know, you're not even allowed to delete from the database or it takes ages or you don't have any test data that you can really play with. And I think even step one, smallest possible increments is just being aware that that's a thing and knowing that that's actually going to cost you effort and cost you money and it will mean that people are going to have to do some work and do some upfront work. Currently that's one of the, it's one of the biggest hurdles, just getting people to be aware that this is something that they need to be looking at. I think that there are so, so many people that still kind of say, hey, yeah, we're going to automate stuff. That means that we need to buy one tool and have one person specialized in it. And it's just. And it's going to be dead quick and dead cheap. And I like saying now, that testing in any kind of testing, any kind of automation, it costs money. It's worth that money if you do it correctly. So why are we still complaining about the fact that it costs money?

It's important. You should do it, I think, more concretely, really, working together with, first of all, finding out from customers, what are your examples? Right. Give me some concrete examples of data I can use for this, working with developers to say, okay, how are we going to make sure that this data or these examples that I can use them again and again, and if I'm playing with them, it's not going to bother the tests if they're running, if you're playing them, it's not going to bother me. And then also, I've seen an example of this recently, also going one step further and not just using the example data. I was doing exploratory testing with projects and their customer a few weeks ago and they said, hey, you know, here's the Excel sheet you gave us as an example for something that we can import and it works and everyone's very happy. Cool. And of course, the customer had brought their own Excel table. And I'll give you three guesses.

It did not work because there were other things in it that they hadn't thought about. So it's a big topic. It's about examples. It's about.

Having ways of being able to automate your setup, automate your cleanup. And to be perfectly honest, I think sometimes for some tests, it would make more sense to first of all ensure that I've got good data in an automated way that I can build up, set up, get rid of than having any kind of test automated. That's probably where a lot of the effort goes. But of course, we like seeing things run and we like green ticks. So that's where we tend to go for.

Absolutely. And this is something my team struggles with, is do we spend time automating the input of the test data before each run and then tear it down, or do we have an environment that we populate as the golden copy and every time we run our test, we bring that up and that has all the data? Because we have like nine Sprint teams, they all test in the development environments and it's just really hard to get that good mix?

Yeah, I mean, as a general rule, I definitely disprove using the test to build up the test data. I know that the, you know, if you're using UI or anything, the robot doesn't care how much it does, but it makes your tests harder to Write harder to maintain. It means that they don't just have one purpose. It means if some, if you need to analyze something, it takes a lot longer to analyze it because you first have to figure out, okay, did the actual, did my test question actually fail or did I fail because of data setup or something? Anything that we can reduce from that is, is definitely a bonus. The problem is, and I've been in that situation myself, you can be in a project as a tester, know that you're not making headway on your begs, please, or whatever to get this. And of course you, you have the opportunity, then you use the tool to do it and then it's there. But then you run into those problems and then of course the feedback that comes is all the tests are flaky and you're like, yeah, but if you'd have, if you'd have helped, then they wouldn't be, you know, I understand how those things happen. But yeah, it's, it's, it's a work in progress. It's got better. But yeah, this. Back to the patterns. I realize I'm getting older and I'm like, I've seen this before.

I'm there, I'm right there with you. Speaking of patterns, I guess another on YouTube, you have a good session that you did, I think at agile test days or something like that. On music and the parallels between music and testing, I was just curious to know, can we touch on a few of those topics? I think you had some bullet characteristics and music and testing how they make kind of patterns between the two.

Yeah.

This was a great idea that Hype and I came up with when we both realized that we play instruments and the things that we've learned from playing instruments have helped us in our careers as testers as well. And we had things like there are heuristics. So you have four chord songs. So so many pop songs are written using the exact same pattern of chords. That doesn't mean that all of them are and it doesn't mean that you can just instantly take those four chords and make the song that will definitely sound good. But you know, rule of thumb, heuristic, there's a fantastic. I think it's from. Oh, I forget the band. Ah, Axis of Awesome. First of all, they, they have a song that puts together loads and loads and loads of four chord songs. And then you realize, oh my goodness, this is actually the same thing over and over again. And they have another parody which is them writing a song about how easy it is to write a song with four chords or a love song or something. And those are, that's great. I mean, humans, humans are heuristic, right? I mean, if we had to, if we had to make every decision using, you know, the system to the deepest part of our brains, it would be exhausting. So it's obvious that we're going to do those kind of things. But as a musician, when I've been writing songs and I'm like, oh, this is really good, then I know about that heuristic and I know that that can turn into a bias that I'm likely to go to it and then I can check my song. I can go, oh, wa. Alex, you've just written a four chord song. Is this what you really wanted? Because if not, this is a way of checking it. And we have that same thing in testing the heuristics that we use. Something I'm also incredibly interested in at the moment of how we decide to take the next step in exploratory testing. Because those are fascinating processes going on in our heads. Those are shared within the community. You notice that you do similar things to other testers. They're also based on experience and they can also sometimes lead us down the wrong road if we're not careful. So that's, that's just one of the things. And one of the other things that was, I mean, it sounds obvious, but practice, right? My violin teacher always told me to do more practice and she was right. If I practice, I do get better.

And it's the same with testing. And we have to keep, we have to keep practicing. And there were. Yeah, the video, I think the video is worth watching. It was, it was a really, really fun, fun talk to do, to really pull parallels between two things that I love so much.

Awesome. Yeah, it's a great video. I'll have it in the show notes so people can check it out there. What I, what came to me as I was watching was improvisation. So if you're in a band and you're improving, a lot of times you have a framework, but you just, you have certain structures that you follow. But it could always be different. And if you listen to one another, if someone modulates, you're able to go with that. But if you stop listening, the train goes off the rails. And it's almost like agile to me. You know, there's no meetings required when you're improving, if you're listening to one another. And I wonder the same thing with agile. If you have to have a meeting, it's maybe, are we not listening as we're testing and developing to one another, that we could automatically adjust.

You know.

I like talking about. That's something that's quite dear to my heart as well. I understand this idea of having the ritual. The ritual is important to make sure that it's the framework, right, to have something is in place to make sure that this happens. And then for me, in brackets, if it's not happening already.

And I get this. One of the things I notice with people that I line manage now, I have monthly meetings with them to basically check on how they're doing. I don't, of course, I care about their project status, but I care about them as humans. Right. How are you doing? And people come to those meetings and we chat, but the fact that we have that ritual means that they know that they can come and talk to me about those things. And then sometimes we'll, you know, someone will come and say, oh yeah, we actually just talked yesterday, you know, because I came to you then and I don't know, I wonder sometimes whether those kind of meetings and rituals are training wheels until you get it right and then you can manage without. But of course we know that the context is always so dynamic and things change and any tiny little change can have huge effects on a project. So we're almost never at the stage when, when we could get rid of the training wheels completely. So it's good to have them there, but it's. But just doing them for the sake of it, that's not going to help us either. You have to understand why you had.

Another one on Stop hating UI tests. And this is kind of controversial to me because it's usually the opposite. People just hate UI tests. And I probably have been one of those people that actually advocated hating UI tests. So what are some misconceptions about UI tests that I'm missing or other folks might be missing?

Yeah, I think so. The two main things are it's fine to hate the ones that are written badly or written to do the wrong things. And the problem is that so, so many UI tests fall into those categories. And again, I understand how you get there. You have a customer that's interested in automation the most people are interested in things they can see and they can understand. And somebody from business is going to be more excited by watching their software being tested automatically than a Junit bar going green.

Right.

That's understandable. So you use it as kind of the gateway drug to get customers interested and invested and then you get people from all sides, testers, developers, customers getting Very, very excited about the fact that you can automate lots of things. I was like this as well. Some of the stuff that I used to automate, just because I was so excited that I could. Those are not good tests. Right.

And for me, it's a question of what is the best level to find the information that I need. And then, of course, should it be automated? Or can it be automated or not? But assuming it should, there is one of the things I say. I mean, if you. If you do look at something like the pyramid and, okay, then there are a few UI tests, few still doesn't mean none. And for me, for me, that then becomes the question of if they're done correctly to show us that really everything is working together for the customer, as the customer will see it and as the customer will use it. Obviously, if you're using Selenium or something. I saw a great Twitter thread about this a while ago. I forget who wrote it. No, Selenium doesn't use the application exactly as a customer would. Of course it can't. That's not the aim of it. For me, the aim is to show that there is at least one way, via the ui, of doing the thing that the person has paid me for the value that they need. Because at the end of the day, all of the unit tests having run, all of the integration or API tests having run, if I can't actually do the thing via the ui, that might be a problem. The thing is, if you've done enough testing at the other levels, that might be a minimal risk and you might decide you need very few UI tests, or maybe you decide you can get away without them completely. But I maintain that there are cases where it makes sense to do that, and then if it's done properly, they're not the big burden that that has. It's often turned out that they are. If you do it wrong.

Absolutely. I think, like you said, sometimes it's a bad product. Like with language, you can hear things, so you get feedback. With this, it's almost. People get hypnotized by a UI test because they can see it. So therefore, everything must be a UI test where I can see it, rather than a unit test or an API test.

Yeah. I mean, and even, to be honest with you, even if it turns out that you end up having that, you end up writing some UI tests in your product or for your product or whatever to keep your customer interested in quality and aware of quality, for me, that's a good reason to do it as well. But that still doesn't mean you should do all of them. Right.

Yeah, it's a.

And context is always important. You know, if my team isn't the one that's even able to work on the backend and the only way I have is to check whether things are still working is to do some stuff via the ui, then maybe that's what I'm going to do. It's maybe not ideal, but I don't think I've ever actually been in an ideal project. Right.

I hear that they don't exist.

It's interesting what you just mentioned. Someone saying Selenium doesn't exercise the UI like a real user does. I actually usually hear the opposite. Is that that's why they don't want to do API tests or unit tests because that's now our customers use the application. And what would you say to someone that says something like that as an objection?

Yeah, so I like to use, I like to use the metaphor that if you, if you do do a UI test, then it's like taking a stick and prodding it into a deep dark pond. Right. The UI test does help you go through all of the layers. You will find various things with that. You know it by or some people say end to end. Right. It's going to go through everything it needs to go through to do the thing you want it to do. And that is an advantage of it. So however some of those things you might analyzing a UI test that's gone wrong for some crazy reason all the way right down at the bottom is harder than analyzing the unit test that found it. Right. And so no, your customers aren't using it directly like that, but your software is calling itself like that. And each, each one of those layers is worth finding out if it's doing its thing correctly. I think another way that you, especially with websites, another way that you could approach that kind of argument these days as well as to say, okay, your customers might not be, but hackers are.

You know, you always have the how could somebody exploit this? And even if it's, even if it's then not just to do security testing, being aware of, yeah this is actually working how we want it to and it deals well and gracefully. Although also third party systems. Right. As soon as, as soon as anyone that's not my user, it's another system then that is their interface. So I think I'd find some reasons.

All right, so people that know me know probably that bad practice are minted automation tools, all kinds of open source tools. And that's Just my thing. And I noticed you're the product owner of Joobler, but I've never heard of Joobler. What is Jubilee?

Jooble is an open source tool for automating UI tests for Java applications.

That's something so weak. Goodness. I mean that was one of the first things I got into at the company and it's, it's still, it's still there, it still exists. It's mostly, I'm not going to say it's done, but at some point products get to the stage where the things they can do, they can and the things they can't do. That's not gonna, you know, we're not gonna take it and change it and make it be able to test iOS right? It does. Java. There are plenty of Java projects in the world. We work on quite a lot of them and other people find it useful as well, so I don't. It's interesting. I think in the Eclipse community it's reasonably well known and some other people don't, which is understandable because I think, yeah, web is big and yet for some customers there are still quite a lot of Java applications going around. So, yep, it's UI automation for Java, which explains how I know so much about UI automation and how I care about it so much.

And yet.

That is the way that I've learned that as much as I love UI automation, it can't be the only thing that you're doing.

No doubt. So, Alex, I know we talked about UI testing, unit testing, a lot of things that gets overlooked, but to me seems to have the most benefit of actually finding real bugs is exploratory testing. But people like, oh, we need to ship software faster, we don't have time for that. What are your thoughts around exploratory testing?

Exploratory testing is one of my favorite things in the world.

For me.

It'S become almost. I know that we always say there are no actual best practices or I say there are no actual best practices. And yet for almost every problem or every issue, every risk I see in a project, one of my answers is almost always then, wait, maybe we should explore this.

I have had a great conversation just this morning actually with one of my testers said, you know, the first time you explained exploratory testing to me, I was like, yeah, why do we need this? And now I've seen how much information we find and how that's done in a lightweight way and how quickly and how the developers get involved with this as well. Now I really understand why this is so important. And that's what I love about it. It's a thing that for me, always has to happen, because we can automate as much as we like, but we can only automate risks that we've seen. And there are always going to be risks that we haven't seen. My current favorite example of this is that I went running in a park in Essen when I was visiting a customer. And I went running very early in the morning. And so obviously I have various risks that I can think about. I might injure myself, I might get lost, I might get attacked. Those are things that, that I thought about as it turned out, in the park, they have a pelican. I'd never seen a pelican before, and to me a pelican looks slightly like a dinosaur. So it was kind of interesting and scary. And I approached this pelican to get a photo of it, because we are, you know, the Instagram generation. And the pelican kind of made a threatening move towards me and I ran away terrified. So on the list of risks that I had in that park, in that morning, being attacked by wildlife was not one of them. And that for me, is one of my best explanations about why we need exploratory testing at the moment, based on real life facts. And I've been doing a lot more of it in the past year as well, talking about it not just how important it is, but working on how are we teaching other people to do it? Because a lot of the time practitioners of exploratory testing, or we explain it, as.

People say, it's a lot to do with intuition and experience. And I agree, that is, those are big, big parts of it. But that can't be all of it, because how am I supposed to train a tester? Do I just have to say, come back in 10 years when you've got intuition and experience? Surely not. And one of the things that I've been working on, I started working on this with Hype. I've done a little bit of work on it with Marit. And this year at Agile Testing Days, I actually did a live mystery exploratory testing session, which was for an application that I hadn't seen before. And the idea was, I'm going to test this, not necessarily to find bugs, but to analyze how I'm testing it. So it gets very, very meta. What kind of things am I doing when I'm exploring? There's a few heuristics that I've identified that I've tested on a few people and they say, yeah, I do that too. And that's going to be one of the things I'm talk about in a few weeks at European Testing Conference. I really want to get behind our brain patterns as testers. How did I know that was going to happen or how did I decide to do that? And that's all really exciting stuff for me at the moment.

Okay, Alex, before we go, is there one piece of actionable advice you can give someone to help improve their testing efforts and let us know the best way to find or contact you.

Okay. So to improve their testing efforts. I think for me, asking questions has always been the way forward. I love learning. I always have done. I was that kid at school that sat at the front with their arm up asking questions and trying to show my knowledge. So not being afraid to not know something, finding out about it, joining in the next conversation, realizing, okay, I don't know this, but. And then asking again.

I do like the people who say that QA means question asker. And I think that is one of the biggest skills that we have because tools change, processes change, projects change, people change in the project. We have to be able to adapt to lots of new situations. And asking questions is one of the best ways to do that. In terms of contacting me, I'm on Twitter, I'm Alex.

People can follow me. I think I'm on LinkedIn, I am on LinkedIn, and if there are specific questions to me, then you can get.

Me@Alexbradex.De thank you, Alex, for your testing awesomeness. For links to everything of value we covered in this episode, head on over to testtalks.com 240 and while you're there, make sure to click on that sign up for our free trial link under the exclusive sponsor section to learn all about Sauce Lab's awesome products and services. So that's it for this episode of Test Talks. I'm Joe and my mission is to help you succeed with creating automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Talks podcast. Head on over to www.testtalks.com for full show notes, amazing blog articles and other automation awesomeness.