Testing Beyond Bugs to Unlock Growth with Gojko Adzic

Then testers are selling themselves kind of short or cheaply. Testers can provide a lot more value to the organization by figuring out these things rather than just reporting bugs or evaluating whether the product is ready for a release or providing kind of feedback on that quality.

I think in a land of testers far and widely journey, seeking answers, seeking skills, seeking a better way. Oh, the Test Kill Animation Testing Podcast Guiding testers with our nation awesomeness. From ancient realms to modern days, they lead the way. Oh, the Tested Automation Testing Podcast.

Hey, if you're ignoring test result outliers or weird random bugs, you're missing out. Here's why. Today we'll be talking with Goyko Adzick all about his new book Lizard Optimization. Unlock the growth by engaging Long Tail users. You don't want to miss it. Check it out. Hey, are you tired of managing your test cases using multiple tools and spreadsheets? Introducing BrowserStat Test Management. It's an AI driven solution I think will really help your QA teams. Imagine a world where test case authoring is a breeze, automated coverage is tracked effortlessly, and your entire test process is streamlined in one modern, intuitive platform. That's the power of browserstacks Test Management and with all their AI powered features, you can generate test cases based on JIRA stories, get smart recommendations and identify the most relevant tests to run. Plus, their automation first approach means seamless integration with your existing frameworks and CI CD tools. So, ready to transform your testing process? Learn more about BrowserStack Test management and support the show by heading over to TestGuild Me BrowserStack and see the difference for yourself. Hey Glyko, welcome back to the Guild.

Hey Joe, thanks for inviting me again.

Great to have you. Maybe a little background why you wrote about Lizard Optimization, then we'll talk about what that even means.

Last couple of years I've been building my own software and operating it and it's been tremendous joy to do that. But again, building anything is a bit of a struggle and you need to do good testing, you need to do good product management, you need to do I, as a self funded founder, developer, tester, support person, do everything. At some point you just start kind of experiencing crazy stuff from people. And the product had 9 million active users last year. And at that scale, people do crazy things. And you know, some of these crazy things are just crazy things that you can't really figure out why somebody would ever do something like that. Some of these crazy things are actually really good ideas. Some people struggle using the software because the software is bad and you know, it might pass all the tests and be functionally the way where you want it to be as a product owner, but it's not where the users want it to be. And then people do crazy things with it. And kind of. I wrote the book because I was really inspired by what had happened. And in a sense the product kind of started growing nicely. And then one of my competitors stole all my traffic. They got good funding and invested a lot more in marketing. And the product started kind of dying slowly, although it looked like, you know, it was on, on its way to success. And I was trying to do lots of things to focus where I thought I could make the biggest impact, which is kind of the mainstream, the main group of users, the main features and fixed helping people do more of that. And nothing really helped. And I was going to actually just put the product on ice and abandon it. And in preparation to do that, because I was also doing support, I didn't want to disappoint existing customers. So I tried to kind of optimize all the sharp edges around it everywhere where people were kind of getting stuck or getting hurt or, you know, not experiencing good UX and having to contact me about that. And really, surprisingly, kind of doing these things started to unlock a lot of growth. And, and a couple of months later, the product started growing like mad. And from November 2021 to November 2022 actually the usage increased 500 times, which is insane. And you know, seeing something grow 500 times in 12 months is crazy. I mean, that means that things that would normally happen once every two years start happening every day.

Wow, that is incredible growth. So I guess can you talk a little bit more about the hidden power of edge cases that you were seeing them with all that traffic, if you.

Have a crazy edge case that somebody would say reports a bug and says, well, I tried to press this button, the thing exploded. And then you do some analysis on it and a developer concludes, well, that's such a crazy edge case, it would happen once every two years. It's not worth fixing it. Fast forward happens twice a day. Now you need to fix these things because there's so much more people doing it. And it was just a stream of crazy things that was happening. For example, the product started as a way to create videos from PowerPoint presentations very easily. And some people kept building blank videos, which made no sense at all. And they were paying me to build blank videos and digging deeper into that, I figured that I made text to speech conversion so easy that kind of they were doing Blank videos just to extract an audio track and use it as voicemail messages or use it as something else. They didn't really need the video, they needed an audio file. And there was like a, you know, less than 1% usage. But it was one of these crazy things that I couldn't really kind of figure out what was going on. And I think, you know, the book is called Lizard optimization, because when you find something like that, it's like it's not been done by humans. It's like somebody's following their own lizard logic or something like that. You can't really understand it. But if you try to figure out what these people are doing, they don't look crazy anymore. It's not lunatic just creating blank videos. It's a person that really likes the text to speech conversion, just does not need the video. And then there's no reason to put them through the hoops of building a blank video, creating a PowerPoint, removing the of visuals to extract an audio track. And I made a very simple screen for people to just use the text to speech function that was there. And then that's kind of part of what really unlocked growth so much because at the moment, more than 99% of the usage of the product is people creating audio files, not video, which was like, you know, a discovery of this. What are the crazy people doing?

So with my experience as a tester, how do you report something like this without it being shot down, saying, oh, a real user wouldn't do that, or that's unrealistic.

Discovering something like that is often a struggle. And then when they do discover it, it's an uphill battle with the product management, because product management wants to keep true to the vision. At least. You know, when I was working as a consultant with companies, you have this very strong vision of a product and then you don't want to kind of get sidetracked. And then when people actually do something like this, it's often considered a stroke of luck or serendipity or, you know, something like that. And I think actually my experience with this is that it's a genuine product growth strategy. It's not something that happens as a stroke of luck. It's not something that we should fight against. And I started researching this and I found that, you know, lots and lots of other people do something like that. It's just that they don't really think about it as a proper strategy for doing things. And it's done haphazardly, it's done kind of again, with a lot of struggle. One of the best examples I found was PayPal where basically at the start they wanted to create mechanism for transferring money across PalmPilot devices. That was in 1998, 99. And when they built the PalmPilot app, because the web was emerging, they built marketing website to promote the app on the web. And the idea was that people would use the website to learn about the PalmPilot app and then download it to their PalmPilot and use it and things like that. And in order for people to experience some value, they built a way to transfer money across the website. And then a bit later they had a million and a half active users on the website and 12,000 active people on the PayPal. Sorry, not on the Palpilot app. And the product management was fighting against it all the time. They were trying to shut down people who were using the website on ebay and fight against them and prevent them from using it until they realized that actually nobody has palm pilots but everybody's using the website. And they restructured the whole thing and now they're one of the biggest payment processes in the world because they were able to stop thinking about people who are using it on a website as crazy edge cases and fighting against it and embrace that as a way of discovery. And I think if you look at science that a lot of scientific discoveries kind of helped by chance or helped by experimentation and looking at something that's maybe not that kind of obvious or something that happens as a fluke. I remember, you know, when I was still teaching people testing, when you do an exploratory test and you test for some purpose, but you see something else and you say, oh, that's interesting. That's usually one of the kind of most interesting discoveries you can have. There's lots of stuff hidden behind something that happens that's interesting, but it's a side effect and there's lots of knowledge that we can discover there. And I think this whole experience of doing exploratory testing has given me a different perspective at product management. So I think lizard optimization is kind of a combination of exploratory testing and product management that really helped me unlock exponential growth.

So I know a lot of times people here probably know about pivoting, but I think you actually have like a four step method to help people with the pivot, which seems a little too like, like you said it just random almost. So you're looking at outliers, it seems like. So what are the four step process then to the lizard optimization? I think the first one Is how did you learn how people are misusing your product without going down a rabbit hole?

So I think there's this kind of four steps and for people to remember it easy. The initial letters are kind of lzrd, so lizard. And it's kind of learn about how people are misusing your product, then zero in on one behavior change or one area you want to really improve, then remove obstacles to user success and then double check that no unintended or second order effects happened. And there's lots and lots of ways you can discover how people are misusing your product. For example, for Air Kit, I have lots of monitoring around what people are doing with the site and especially when they get into an error condition. That's really interesting. So for example, we have a dialogue where people can upload different types of documents and then the product imports it. And occasionally people upload crazy things. I mean.

On a text to speech conversion screen, people keep uploading Android package files. Makes no sense at all. I can't explain it. My best explanation is that it's one of these, you know, hacking techniques where you leave a USB outside of an office and you hope that somebody's going to load it. But you know, some people upload things that kind of make sense. We just don't support that. And we had a bunch of people uploading subtitle files from videos and this is like, okay, I can kind of see the logic in it. It was a fluke, really rare edge case. But I can, I can see the logic in it. You upload the subtitle file, you want to do text to speech conversion and you know that there was enough of these people that, you know, let's implement it, let's support that, let's see what happens. And that was kind of, I said, a direct result of us capturing these workflow errors. So you upload the not supported file, you get an error saying this file is not supported. But I also get some monitoring and I get some aggregate statistics what people are uploading so we can track what crazy things people are doing. And then when we kind of, I built this kind of subtitle conversion to audio with timestamps and making sure that's aligned. That turned out to be one of the most profitable things ever in the product because people from a large E learning department in enterprise organizations, they do these training videos all the time. They need to translate into different languages. You need to kind of, you want to have a voiceover track in a different language. And even with the text to speech engine, that kind of speeds up how quickly you generate the audio. You're still left with hours and hours and hours of editing the video to align everything nicely and make sure it's done. But if you upload the subtitle file, you get back a synchronized version. It saves you hours of time. And not a lot of people needed that. But those of our users that needed it used it for quite a lot. And a couple of our biggest spending customers came to the product because we had this kind of subtitle to audio conversion that nobody else had at the time. And again, that was a direct result of discovering what crazy people were doing on the site. And I think.

Balance between research with your users and what I think is common sense now in the product community is that you have to do a lot of research with your users. You can't represent users, but there's a balance of what you can do with research upfront. And at some point you need to stop and build the product. And whether you've involved the right people in the research, whether kind of the audience changes, whether the audience preference change over time becomes a really interesting thing. Because I think when I originally built the product, subtitle importing wasn't really kind of, you know, nobody needed it. But then it turned out to kind of have a simpler text to speech function, not a video function. Then people tried to build lots of these text to speech things from weird formats. And you know, somehow we came to this subtitle importifa I just focused on the initial research. We would have kind of completely missed it because we were not researching that, we were researching a different audience. And I think kind of this optimization where we look at unexpected things that people are doing with our products, how people are hacking the products, how people are misusing the products, it can complement this initial research and it can point to our blind spots, it can point to changing preferences or changing audiences. You do your product audience is one thing this year, then you have a lot of growth. It's a different audience, their preferences are different, their needs are different. Technology moves very quickly. You know, the research that you've done two years ago might no longer be valid. And by looking at what unexpected things happen with our software or usage of the software is a really interesting thing. There was a lovely video I found researching this topic a bit more from a lady called Rachel Newman in Australia and she was at Eventbrite, I think looking at bridging the gap between product management and product support. And her idea was that you have this kind of product support function or Customer success function where people phone and report problems and complain about bugs and all this other stuff. And that usually then goes to testers, testers do some testing, then it goes to developers for fixing whatnot. And she said, kind of these people are, you know, speaking to hundreds of thousands of your users every day or every year or every month, depending on your product kind of volumes. It's such a rich data set that, you know, needs to be mined for product ideas, needs to be addressed. And lots of organizations have this big disconnect between product management and support, which I think is very weird in a sense because you can get all these ideas to complement your research for free effectively. You know, not every customer call about a bug is a bug. Not every bug is important. Not every unimportant bug is that unimportant. Maybe there's some hidden gem there that you need to know. And I think bridging this gap is easy for me because I said I'm, you know, everything in this product. But larger organizations I really think should look at this disconnect. And I remember, well, maybe even 10 years ago or something like that, we were working with an insurance company as consultants here in England to help them rebuild their testing function. Lots of testers left very quickly and they were left without internal testing knowledge. And they tried to kind of hire new people to take over and they wanted to make sure that everything gets started quickly. My colleague helped them hire this kind of head of testing. And this lady turned out to be such a wonderful cross functional expert in everything that she became the head of product at the end because she learned so much about kind of the whole product, approaching it from a testing perspective. And she had this mentality that, you know, you need to evaluate everything, you need to figure out how to do things. And I think this, you know, from, you know, like most of your audience are going to be testers in this podcast. I think, you know, if you look at something like that, then testers are selling themselves kind of short or cheaply. You know, testers can provide a lot more value to the organization by figuring out these things rather than just, you know, reporting bugs or evaluating whether the product is ready for a release or providing kind of feedback on that quality. I think, you know, I'm thinking more about this in a sense where when there's a problem with the product, it might be, you know, kind of as a consultant, I think about quadrants, you know, then and it might be that the product is doing something that user expected or didn't expect. And it Might be that the product is doing something that the product management expected or didn't expect. Like, did we want the product to do this, yes or no? And, you know, do the users expect that to happen, yes or no. So you have this thing where.

If the product does something that you didn't expect it to do as a tester or as an internal person, developer, product owner, and the users didn't expect that, that's probably a bug. That's. That's a strange problem. You know, you have these things where the product does something that you wanted it to do, but the users expected something else. Is that a bug? Is that a UX issue? Is that an unexplored potential for growth? Is that something that, you know, is very, very interesting feedback that says.

We don't know something about our audience, we've missed something there, you know, and then you have these kind of strokes of luck where the product does something the users expect, but it's not what you wanted, which is sometimes a security issue, sometimes you have people figure out how to exploit you. Sometimes it's again, an interesting area of growth because it might be that you think it's a bug and it's actually not a bug. That's how people expect it to work. And then I think, so there's a lot more here than just, is it a bug, is it not a bug? And I love kind of. I read a UX book called Mismatch by Kat Holmes while researching this, and it's a book about accessibility in software. And they have this concept of a mismatch that's kind of a much more neutral world. It's not a bug, it's not a problem, it's not an issue. It's a mismatch between what the product does and what the users want it to do or expected to do, or what the user's capabilities are. Now, that mismatch can come from lots of different sources. You can have a mismatch of somebody's cognitive ability, you give them too much text, they can't read it quickly enough. Or it might be a mismatch of UX where the button is in the wrong place, or it might be a mismatch of functionality. And I think kind of thinking about these mismatches is really, really interesting because then you can figure out whether you want to address it, whether it's worth addressing, whether it's something that we do differently, maybe, and things like that. And one thing I really love about this whole Mismatch concept is they talk about one of the four principles of accessible design from Microsoft, where the author worked. And one of those principles is solve for one, expand to many. So when you solve a problem like that, you don't solve it just for that particular group, but you try to improve the product so that the product genuinely improves for everybody. And I think that's kind of, you know, thinking back at some of the best kind of bugs I found working on other products and some of the kind of most interesting experiences improving that. That's actually it. You find a weird thing, you find a weird bug, but then, you know, you don't fix just that bug. You fix a whole class of different things. You improve it in a way that makes the product genuinely better. And I think that compounds growth. And that's really, really interesting. They have some statistics. And again, the book is focused a lot on accessibility and people's capabilities and things like that. But I think there's, you know, lots of things we can take from there and lots of parallels we can take from there. And they talk about kind of three types of disabilities that cause three types of mismatches there. And one is permanent. Say, you know, if you're looking at accessibility and disabled disability and things like people there's. I think they said there's 16,000 people in the US who have a permanent disability of one of their arms. You know, the arm was amputated or some kind of genetic disorder. They can't use an arm. And then you suddenly have somebody who cannot use two hands with your software. That's not something that people often test, like, can you use your software with one hand? But in the book, they talk about how if you just look at that population, then it's kind of often not economically justifiable to fix it. It's one of these things where you say, well, you know, it's not a use case you want to cover, but actually solve for one, expand to many. Then you look at two different, two more types of mismatches and disabilities. They say you have this kind of temporary. So it's not just somebody who is completely, you know, without one arm, but some like a mother holding a baby in her arm, and she does not have two free arms. So you have somebody who's, you know, carrying a bag from a shop with one arm. He can only use your software with one arm, with one hand. And then you have this kind of situation where somebody's profession prevents them from using both arms, like they have to call something. And then the talk about how in total there's like 30 million of these people in the US at the moment. So by optimizing the software to be usable with one hand, you're not making it better for 16,000 people, you're making it better for 30 million people. And that's a totally different economic proposition. And I think, and looking at that, that's kind of, I think, drawing parallels with lizard optimization. If you find somebody who's doing something crazy like trying to upload subtitle files or trying to create blank videos, and you make the software better for these people, you don't just make the software better for these people, you make it better for lots of other use cases. And that's why I think doing that is a wonderful way of thinking about product development experiments, figuring out how people are misusing the software. And then you can say, well, out of all these misuses we caught, because there's going to be lots and lots of noise, you need to figure out the signal in the noise. You can't, of course, fix everything. You can then kind of, from product management perspective, decide, well, this is the one worth pursuing, and let's then design some experiments around that. And I think that becomes a really interesting aspect of the whole process. We collect lots of noise, and then from the noise we need to kind of draw some signals. And then when you do that, I think one of the kind of the way my thinking really evolved around this is that when you have a mismatch like that, people often think about, oh, I don't want to build more features, or we don't want to invest in kind of, you know, just this because it's adding more complexity to the software. But if you think about it as well, it's not adding complexity to the software. It's about removing obstacles that the product is putting in front of users. Unintentionally, my product was getting forcing people to create a blank PowerPoint, to upload it, then to download an MPEG4 file, to extract an audio. Those are obstacles in front of a user's workflow. They're not additional features that should be something completely different. And then kind of removing obstacles to user success is really, I think, the kind of key there, because then we can look at what the user success looks like, how do we.

Make them more successful? And this is where kind of growth really comes from. Because when you get people to be more successful, they will stay longer with the product, they will bring their friends, they will recommend it. And then the last step is double checking whether we've caused any additional unintended impact. And this is really, really important because the whole premise is that people are doing something we don't understand or we didn't expect. That means that our research, our internal knowledge, our data, our mental models about our users are wrong. And then we try to do something else, but they still might be wrong. There might be some other kind of type of crazy and things like that. And one really good example of that is I was trying to optimize payment success rates, especially kind of for people in Europe. Companies in Europe need a bit more bureaucracy than in the US in general. And they need to have their tax numbers on the invoices, they need to have a proper invoice and things like that. I'm using an American payment processor. They don't really understand Europe. That's kind of, you know, and they have this field where you can put in a tax ID but they don't validate it correctly. Lots of people were dropping off because they couldn't input their number correctly. A non trivial number of people were kind of selecting Russia as their country just because the payment processor doesn't validate Russian tax numbers. And then I would have to go and say, well, this is a French company, they've selected Russia as a country for tax. This is insane. And I thought, well, what about just removing the number there and adding it later in the step on a webpage icontrol and saying, go through the process, you can put in a tax number later, I'll generate invoice. And that way I was able to configure the payment processor not to prevent these people from succeeding. But then actually the results were that fewer people were paying because they were expecting the tax fields and their mental model was to send me an email saying where do I put the tax field when I removed it? So it's something again where I thought this was a good idea, actually it wasn't. And.

Double checking that it was good and measuring and proving what the effects are and if they are second order unintended effects is also really interesting because then it helps us kind of again improve the loop and get some insights about what we want to do.

Nice. So I love the four step process. I love the idea of mismatch review process with the lizard optimization. That's awesome stuff. And how do you ignore people using the Russian field? How do you ignore the default thinking, oh, our users are dumb and just glossing over it rather than saying this is probably actually an issue rather than just saying oh, users are dumb, don't worry about it.

Well, I mean I can, you Know, this is kind of. It's very, very easy to say users are dumb, isn't it? And that's kind of especially if you are in our industry kind of, you need a decent IQ to work in the things we do. It's very easy to say uses a dumb users are stupid uses. But I think kind of one of the things that really I want to do with the book is help people understand that people are not dumb. They might be just following a different logic. And that's why I like kind of this term lizard, you know, because it's not human logic, it's lizard logic. But then you kind of dig in a bit deeper there and then we can think about. Well, you know, if you think about something's not done by humans, it's done by lizards. Let's go and understand what Elizabeth doing what the logic is. And I think part again of that is thinking about that we can't represent our users. Marty Kagan in Inspired wrote that one of the biggest mistakes product managers make is think that they can speak for the user. We can do lots of research, we can do monitoring, we can do lots of interesting stuff, but we do not speak for the user. The user speaks for the user. Figuring out what the users do or want to do is a really important part of our job. If the users are stupid, if the users are dumb, then it's a mismatch between the user's cognitive ability and the difficulty of the user interface. Then maybe the user interface should be simpler if the users are actually dumb. But maybe the users are not dumb. Maybe the users are just doing something different. One really interesting example from that is one of my products is Mind Map. It's a mind mapping tool. We originally built Mind Map expecting that professionals will use it, but we built it to scratch on each. The idea was that people will create mind maps to brainstorm during testing, exploration during product management and things like that.

But we built the Google Drive integration at a very interesting time. When Google was promoting their Chromebooks in schools and their services in schools, and they were giving hardware away, I think, for free to get people hooked on their services in schools. And all of a sudden, the usage in schools started to kind of increase quite a lot. And the software was designed for adults. The software was designed for professional users who are generally comfortable using a keyboard and things like that. And all of a sudden, you know, we started getting all these weird metrics that would suggest the users are dumb, but the users are not dumb. The users are kids. That's a totally different use case. It's, you know, they're not, they want to put images rather than words. And we actually kind of, as the usage in schools started to grow, we removed a lot of the complexity from the software. We made the software significantly simpler. We did a major redesign to make it not that smart anymore, you know, and as a result of that product grew significantly. And I think we made the genuinely better product because if you make something easy for kids to use, then the adults are going to use it easily as well. And so yes, you know, sometimes you do have people who just start playing dumb, of course, but I think in a large majority of cases that's a cop out. That's basically saying a UX is bad. That's saying this feature is too complex to use. That's saying, you know, we've messed up, but it's a lot more comfortable to say users are dumb than we've messed up.

Yeah, and I love this book and I know it was written mainly for software product managers, but as you mentioned multiple times, I think testers play a significant role in implementing this to help really expand usage of the product for features. Especially at the beginning of a. Maybe planning for a Sprint maybe. Hey, here's an idea we found from production. Let's do an experiment type deal. Is that something you see a tester being involved in as well or mocking something rather than as a bug like you said, mark it as a mismatch and maybe at the end you have a database of all these mismatches. You can go over and see there are any commonalities and maybe that aren't really bugs, but things you can maybe help make the product more usable or better features for your customers.

I think the unfortunate part of the industry is that testers are kind of at the bottom of the food chain. And I always thought that was crazy. But I think people with skills that are critical thinking skills that are kind of skills to evaluate a product from lots of different perspectives, to kind of put in a different hat. Michael Bolton talks about kind of a builder's hat and a critic hat. If you can put on a critic hat and evaluate or criticize the product like a food critic or a movie critic, then you have really interesting skills that can be used to drive a lot of product innovation. Now, whether kind of testers should be doing it, absolutely. Whether the organizations are going to let them do that, that's a totally different question. And I think that's kind of part of what people have to kind of fight for for their role. And I Think, you know, I came into testing from kind of the development side of things. I came into testing because I was running a small team and we, when we built stuff, when it goes out, it wasn't supposed to come back. So I had to learn how to test the software, not just build it, because it was my money if it came back and we had to fix it. And I think for me it was kind of slightly easier than to argue about lots of different perspectives. But I think it is possible for testers to kind of assume more responsibility and get more responsibility. And in particular, as you said, running experiments, figuring out how to evaluate things, taking the responsibility of that. That's really interesting. I think more and more we will be seeing product experimentation as an important part of development. I quite like how that area is growing and there's lots of kind of interesting voices in that community becoming more and more popular. And I think product managers are learning that they need to do that. But testers can provide quite an interesting, valuable service that's very complementary to their skills if they learn how to do product experimentation.

Absolutely. What I also love about the book, AI is all the rage. Yet this book is very human in the loop type thing where I think only a human can evaluate what is a mismatch or what's not. Do you agree with that? Or you think maybe AI could step in and try to find these anomalies and flag it as a potential experiment?

Oh, that's interesting. From my perspective, AI so far has only generated support for me. I regularly have people who try to use the API in a way that doesn't even compile or doesn't work with my API. And then when I ask them, have you even read the documentation? Because no, I tried to use ChatGPT to generate an API call and then it uses a different type of authentication that's not supported. It's kind of talking to non existent URLs and things like that. So I think lots of people who are inherently Lazy are using ChatGPT to be even lazier. I've not really seen a big benefit of these generative AI things, but at the same time my product is based on text to speech, which is kind of AI, not generative AI, but old school AI. I think figuring out the signal in the noise is a big challenge. It's a big challenge for me, especially when I look at all the data I'm collecting. Because when you start collecting data about how people are misusing your product and you have millions of people using your product, then there's going to be a lot of weird misuse. We're talking about things that are very, very long tail in the usage curve. And I think there probably is a good use case for AI to.

Figure out the signal in that noise and propose some, you know, trends or propose some patterns that you might not see. Obviously, I think, you know, for me the challenge is always keeping up with everything that is going on and something like that might be a good use case for AI. I think the other stuff is probably more human, at least for now.

Absolutely. Okay, Coyco, before we go, is there one piece of actual advice you can give to someone to help them get started with Blizzard optimization and what's the best way to find or contact you and get our hands on your new book?

So the best way to get hands on the book is basically going to be to wait until the 1st of September until the book comes out and then get it on Amazon or kind of a bookstore near you. The lizardoptimization.org website is going live tonight in preparation for that. That's going to be a good way to do that. I think like the number one you asked one one piece of actual advice is if you currently don't have monitoring in your software to tell you when people are getting stuck on doing something unexpected, put that kind of monitoring in and you will be surprised how many interesting ideas come as a result of that. That I think is the biggest lesson for me is when I started putting analytics to tell me where people are getting stuck and where people are getting problems, all sorts of weird ideas started kind of materializing and I realized how much I've missed in research. And I think partially.

Lots of companies now do have product analytics, but product analytics are mostly looking at things you expect to happen.

You're looking at how many payments do we have. You're looking at your user funnel as you expect it to be. You're looking at are people using this feature or that feature putting in analytics around what's happening that we don't really expect is really, really interesting. And where are our blind spots? There are some interesting ideas in that market or that segment from all the observability folks. And I think that's kind of the again, some overlap where testers can help is figuring out from testing in production, from observing in production what we missed in the original product. And again.

That was my initial approach is from a testing perspective. If you monitor the software in production, you can look at things you've missed during testing before the release. Those are kind of problems that went there. But sometimes they said they're not problems as in the product is not doing what it was supposed to do. Sometimes there are problems as in the product is not doing what the users expected it to do. And then you wouldn't catch that with testing at all. You can only catch that in monitoring in production. So the observability guys folks have developed lots of interesting tools to catch technical exceptions, but these are kind of user exceptions, business exceptions, workflow exceptions, and capturing these things, analyzing those things is I think a really, really good first action of step for people to do.

Thanks again for your automation awesomeness. For links to everything of value we covered in this episode, head on over to testguild.com a509 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation testing and DevOps. If you haven't yet, join our vibrant community@testguild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate visit test guild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

With lutes and lyres, the bards began their song A tune of knowledge, a melody of code through the air it spread like wildfire through the land Guiding testers, showing us secrets to behold.