Confidently Deliver Quality Software with Joel Montvelisky

Get ready to discover the most actionable end to end automation advice for some of the smartest testers on the planet. Hey, I'm Joe Calantonio, host of the Test Guild Automation Podcast and my goal is to help you succeed with creating automation awesomeness.

Hey it's Joe and welcome to another episode of the Test Guild Automation Podcast. Today we'll be talking with Joel all about confidently delivering quality software. As you know, in this day and age a lot of people just focus on maybe automation testing and scripts and really miss out on the whole thing or reason why we're doing testing and that is to deliver quality software. So really excited to have Joel join us. He is a Practitest co founder and Chief solution architect. He's been in QA since 1997. Almost as long as me working as a tester, QA manager, director and a consultant for companies in Israel, US and Europe. Joel is also a Forbes Technology Council member, a public speaker, a blogger, a chair of Online Test Conf, and he is the co founder of the State of Testing Report that. I hope to dive in a little bit here today as well. And if you want to learn more about test management and Practitest and see a free trial for yourself after listening to this podcast, be sure to check out practitest.com you don't want to miss this episode. Check it out.

This episode of the Test Guild Automation Podcast is sponsored by the Test Guild. Test Guild offers amazing partnership plans that cater to your brand awareness, lead generation and thought leadership goals. Get your products and services in front of your ideal target audience. Our satisfied clients rave about the results they've seen from partnering with us, from boosted event attendance to impressive roi. Visit our website and let's talk about how Test Guild can take your brand to the next level. Head over to testguild.info and let's talk.

Hey Joel, welcome to the Guild.

Hey Joe, thanks for having me. Glad to be here.

Great to have you. Really excited about this Joe. I guess before we jump into it, I'm always curious to know when I speak to founders and co founders, why the testing space? Like what got you to say, hey you know what? I want to start a company and start a company around like creating a test management platform.

Well, for me it was, I think really trivial because that was my background years and years even before we founded practices. I think that for my partners in the company it was maybe a little bit different, even though they also came from development backgrounds. But I think that whenever you're Starting a company and it was not unlike in practice. We look for a place where we thought there was a need and there was an unanswered need. And we're talking here about 16 years ago, 15 years ago, when there used to be like two, three test management solutions and there was really nothing for the rest of the world. And either you had a lot of money and you had Quality center, test director, Quality center back in the day, or you were working with spreadsheets and it just didn't cut it. So it was trivial because I've been there for a while.

Absolutely. I used to be an old time Mercury user, so test director, everything was integrated with one another. And then as I got older and open source came on the scene and CICD and all that, it seems like there was a lot of open source solutions but not necessarily one place for them to go and share the results and have more of a platform where you can manage all this data. So over the years, since you've been around for 16 years, have you seen a change as well? Like you had to adapt to like. Okay, I had this assumption when I started, but now it's completely different.

Oh for sure. Meaning I think that there's a lot of, let's say internal changes as well as external changes. And even within those operations internal changes might be methodologies. So we're talking here. Back in the day we're working on large waterfall projects and maybe just had a little bit of XP and agile. Right now you were talking about the state of testing survey. So we have been surveying that for 10 years and right now the norm is that everyone is doing agile. Some of them are also doing some like dinosaur projects that are still working on waterfall. But that was like an internal change. I think that external changes and some people might call it internal. It's something that I like to call the automation tool explosion in the sense that you were talking about Mercury and by the way, I used to work in Mercury. I used to be the QA manager for Test Director Quality center and then I became the QA manager for QTP and Windrunner and back in the day it was really like Windrunner, QTP and Silk. Yes, there were other players but. But on the functional area it was like that. And then all of a sudden we started seeing more and more and then the Selenium and Open Source came along. But I think in the last, not sure if it's going to be six, five or four years. Don't take my on the specific word, but we have a lot of New tools. And when you have so many tools and you have A cheaper tools so more people can use them, B, better tools, you're actually able to work with them without having a PhD on automation. And so within practice, as we've also seen that paradigm change back in the day, it was everyone had manual testing and maybe 5% had some sort of basic automation. Right now we're seeing the other way around. We're seeing that everyone has automation. 90% of customers that work with us have automation. And we're starting to see those people who really have, quote unquote, 100% automation, meaning all of their basic coverages is automated and they still do some exploratory testing on the site. So as a test management tool vendor, we had to adapt from being a mostly managed manual testing to a kind of mixed approach. And I think that's also an opportunity for us.

Love it. So have you also seen the user change a lot of times? It used to be QA was siloed and we had our own tools, our own group, and everyone else was like, okay, quality owns it. And they didn't really get involved in testing. To me, it seems all these other tools, a lot of times it empowers developers to do testing as well. I don't know just from my end, I love speaking to vendors because you speak a lot of companies. Do you see that as a shift as well? Do you see more than just testers now testing? That makes sense.

It does. And the answer is yes and no. I think that we see more developer using the tools that used to be essentially for testers, the functional testing tools. But what we're seeing is that those testers are actually part of the development team. So the moment that we started working in this complex or composed agile teams and the multidisciplinary agile teams that even we work internally in them, then on the one hand, you start seeing more testers working side by side with developers and then like in a very natural way, you start seeing developers starting to use our tools in the same way that we're starting to use their tools. So it's kind of a mixture here. Now, having said that, it's a little bit idyllic what I just said. I do believe that 80% of the users for the functional automation tools, even for the API testing tools, are still testers, just as 90% of people using Junit and Nunit are going to be developers. Okay. And we're seeing like kind of the lines being blurred, but there's still like a separation of tasks Absolutely.

So as I mentioned, I started off with the Mercury toolset when I was a Mercury maven and then I went to a company and eventually they moved all open source. And it seemed like even though a lot of times having a vendor is tough because you have the payment and everything and it's hard to change and integrate with other things, we lost that insight the test management gave us and it seemed like we had all these open source tools now and we had nowhere to see this data and then we had to create our own dashboarding and things like that. Is that where like a solution like Practitest would come into place now where it can not only work with like its own specific functional test tool, but it actually you can integrate it with all you basically your CICD pipeline and all the things that developers and testers use nowadays?

For sure. I think that. And again, going back into what we're seeing in the field, it used to be the case that. And again back in my Mercury days, you would go to someone who had two tools or you would be as a test management tool, you would be interacting with one or two tools. What we're seeing right now, when we get to a customers, we start talking about anywhere between 5, 7, 8, 9 different tools and we have first of all more tools because as I said, capital explosion. So for every technology you will have a separate tool. Okay. But then if you have multiple teams, people are being less strict, less rigid on what tools you can have, so the same organization will have multiple tools. And on the other hand, we're starting to see more and more testers interacting with other type of tools. So back in the day we would never have thought about compilers. Okay. Right now testers work with CI systems because it's only natural. Most places where we go, the automation framework is based on their CI system. Okay. I don't care if it's going to be Jenkins or Circle or Actions right now that a lot of people are working with, but the amount of tools it's going up now. Why is that? That's a challenge for people because when we get to most of our customers, they have like siloed tool reporting, meaning anything that we're doing with Selenium, it's in Selenium. And if they have testim IO then everything will be in Testimio. Anything obviously if it comes from development, they have their junit reports and if they have some API testing, so everything's siloed. And you see QA managers and test managers when they need to give that information over to stakeholders looking at all the different reports and trying to figure out how do we put it together. Okay, so that became a big issue. We have a lot of different tools, but everyone is dispersed and no one generates a centralized and organized execution and or reporting engine. And that is one of those places where we are seeing a lot of traction in practite test.

So I guess that brings us to a term I've been hearing a lot more of, and that is test orchestration, meaning how do you run all these different silos now of tests that are all over the place, but actually all serve hopefully to show whether or not your software is good enough to ship to your customers. Can you talk a little bit maybe? Are you seeing this test orchestration idea really starting to take off or try to address this issue? How do you handle these all these different toolings? Now in cicd, we are in a.

Place where people feel the pain and they don't know how to articulate it. They don't know how to give it a name. And we are working on the name and a lot of meaning. We started talking test orchestration and we see more people starting to talk about it. And I think that that's exactly the point. The point is, how can we stop seeing each one of our testing inputs, separated factors, and how do we put it together in an organized fashion? Because if you think about it, our stakeholders, our CXOs, or even our VP R&D or whoever has to make the decisions, they don't care. Where did you take your testing results? So if you tell them, oh, our selenium is saying that our manual test is saying that on our. I don't know, whatever Jane is saying that they don't really care. They just want to know, can we release it or not. So what orchestration basically means is let's put everything together and let's give a normalized picture. One of the biggest challenges that we see with people is that if you have unit tests, so you might have thousands of results and then you have some integration tests on. So you have hundreds of results. And then on your functional suite, you made it like very, very wide, so you have tens of results. Now, is it one failure on the functional equivalent to 10 failures on the unit? Not necessarily. And by the way, As I mentioned, 80% of organizations, if not 90, still have a heavy investment on manual. How do you put it together? So orchestration starts by saying, okay, let's make sure that everything.

Basically goes down into one funnel and we're able to provide one solution and that's really like the first step. The second step becomes a more interesting step when you start not only providing that solution, but starting to show efficiencies and inefficiencies. One of the things that I love is when we start doing this orchestration approach and we start seeing, hey, wait a second, we have on this area, we have coverage on the unit test, integration test, API test and functional test. These areas we almost are not covering at all with automation. So do we have places we have over coverage, under coverage? How can we make it more efficient in the amount of tests we're actually putting it? And by the way, then using manual testing in a smarter way to say, okay, these areas are basically uncovered because it's very hard to automate. Let's focus only there and not on the other places that have already been automated and done.

That's a great point. Once. I don't mean to harp him back on the past, but you know, when we had a centralized tool, we were able to map requirements to a test. So when a test failed or when we had multiple tests, we could say, okay, we have these amount of tests for these requirements got harder as we went open source. You could do tagging and things like that. So now that you have like unit tests, like you said, API test, functional test, and you have them running in one place with this test orchestration idea, are you able to map then like you said, you're able to actually see then the bigger picture. Say, okay, we have all these unit tests, we have all these functional tests, but there's this part of this application has no tests whatsoever. And maybe you have some duplication you can delete because you have too much coverage. I don't know if that's a thing. But do you see that type of insight then?

We see that. And again, remember, you're not executing on the same place because that has been a very, very hard place. You might be running everything on Jenkins if you want, but everyone will be working apart. And you want to maintain that because basically running the same framework is different. But yes, the moment that you're able to trace it back to user stories, to requirements, we're seeing a lot of people modeling, meaning not necessarily to user stories or requirements, but to application areas, components, modules, services. And so you start to see places where you have a lot of investment. So it's not like, okay, just drop it. But if I'm going to be right now writing usually the unit tests, if you're working BDD or tdd, you're going to be Writing them upfront. But when you're making that decision, what do you automate on the functional level, on the UI level? That's when you have more information to based on, oh, here's where I'm going to be investing my test cases. Here's where I'm going to be putting the effort. So the moment that you're able to model to something, and again, for some people, it's going to be user stores or requirements. I personally like to model based on application areas, topologies of the application more than anything else. Then you start seeing those places that are overcrowded and those places that are sparsely covered at all. And you can basically correct that in there. And by the way, people don't think about it because usually you say, okay, so this is my sanity, I'm going to make my sanity. And you write your sanity. And unit tests, you expect everything to be covered with unit test. But it's not like that. Meaning that's only on notebooks and the university. Maybe when push comes to shove, there's not enough time to do anything. The first thing that actually starts sacrificing is testing coverage. So here is a more intelligent way of doing that.

All right, so speaking of a more intelligent way, AI has been the rage. Getting people to pay attention then and know, like, where do we stand with this particular build or this particular code check in that we're about to release? Does practice test have anything? I believe you have something called like some sort of test value score. I saw on the. Can you talk a little bit more about that and how maybe it can help us with our tests, especially with risk analysis, which I think is the biggest bang for the buck that a lot of companies may miss out on and how effective their testing strategy is.

So that's a great point. I think that a lot of people have been talking for a number of years about AI in testing, but the bulk of that talk has been around how do we make our automation more sturdy, less flaky, how do we recognize objects? And it becomes less fragile because if you modify one of the, I don't know, identifiers of the object, it doesn't break that much. And again, we've done a lot of advancing that and that's great. But we're not using AI in order to help managers and architects to make their decisions easier and faster. And this is where actually we're already doing that. In practitest, you mentioned about the value score and value score, it's actually pretty neat idea if you have a thousand Test cases in your database right now. Some of them are going to be manual, some of them automatic. The automation, you will have your multiple flavors and types. You never have time to run everything. How do you know which of your test cases are the most interesting? Quote, unquote. Now, what's an interesting test? It depends. It depends on your project, it depends on how do you test. So for some projects, an interesting test is a test that is finding a lot of bugs that are getting fixed, whereas some test that doesn't find a lot of bugs is less interesting. And for some places, an interesting test is going to be a test that is modified all the time versus test that hasn't been touched. Or an interesting test might be a test, and this is something nice. A test that is passing and failing based on patterns, not on flakiness. So it's passing. All of a sudden it fails, which means that you're testing on an area that you found a bug, and after a couple of builds it starts passing again. So some will fix the bug. So if you have a test on that pattern, that means that it's more interesting than one that is constantly passing and not finding anything or constantly failing. No one cares if it fails. So what we did, we were able to look into millions and millions of runs in our system. We're a hosted Solution, we're a SaaS system, so we have access to millions of runs. We run a little bit of machine learning algorithms on that. We found we came up with at the beginning it was much 30 different parameters. We lowered that to about 10 different parameters. And we're using AI in order to understand in each project which test case are more interesting. Now, the idea at this point is to tell customers, okay, if you have 10 test cases and you can only run five, these are the most interesting, the value score, the most valuable test case. Now, what happened was that we released that feature and immediately something that I didn't foresee was that people started seeing, oh, this test has a value score of 76 and this test has a value score of 43. How can I increase the value score of the 43? Like, oh, how can I make my test more valuable? So we started giving suggestions. A suggestion might be, you need to run this testing in more test sets or you need to modify the test so to start finding bugs. So stuff like that is basically trying to help people out. So this is like our first step in AI. We already have additional steps that we're planning. One of them is, as you mentioned, we want to take a look into different areas of the product and not necessarily tell you, oh, this release is going to be more problematic. But in this area you usually have a lot of issues or usually have a lot of anything from bugs that you find to bugs that are escaping and trying to tell you, watch out here, or this. If you can only test 10 hours in here, you should invest more than in here. How much? Not sure. That still is going to be the decision of the test manager, but at least to give you some indications based on.

What you have in your project, the way that you're working. So there's a lot of AI into that.

I love this, I want to unpack this a little bit. I love the gamification in a good way aspect. Usually people try to gamify things to make it look better, but this actually seems like it's actually assisted AI to help you make your test better. Is number one that I came out of that. The second one you said something was really interesting. Like you said, it seems like a lot of people or vendors come up with solution. It's all around functional testing, like locators and things like that. And I always thought it was kind of odd because things like performance testing and test management has a lot of data where I think machine learning is like a great fit for that. And so I love to see that you're actually investing in something I think has real potential and real value because it really, like you said, can help with the risk analysis and let you know, I used to work for a healthcare company and we had a release and like, okay, what test do we run? I don't know, let's run all of them and then it's all noise because they're failing. But if we had like a risk analysis or like a value score, we'd say, okay, let's run a highest score test against this and that maybe would get us better coverage type. Things like that I see as a big value.

And when you start mixing, for example, value score with modeling, okay, because not every build is the same. So this build touched this and this in these areas. What should I run there? Not only that, but where do you have test cases that are covering these two areas? And so they're even more valuable to be run over there. So you get a lot of stuff in there. Now one of the things that you're saying about people not using the data points, I think that when you look at one project, one project will have, even if you have automation, a couple thousands of runs. I'm going to go very, very deep and you have a lot of unit tests. You have 100,000 runs in there, maybe a million runs, but that's not enough. The part that actually made it worth it for us is given the fact that we have thousands of users and we have tens of thousands of projects, then we obviously have millions and millions of runs. So we were able to infer more about what makes stuff interesting there. And to tell you the truth, it didn't start as a feature. It basically started. We were looking for a way of helping our customer success team. How can we tell our, our customers how to make better tests? So we started giving a value core to projects based on the value score to test cases. And then we saw, oh, wait a second, we have a great feature and let's actually provide that to our end users. So again, I think that as many things in life, it came out of an unforeseen activity, but people are actually liking it. So we're happy with that as well.

I love it because it was almost like we were running blind before. And having technology like this really can bubble up insights to say, let's be a little more strategic about it rather than just run everything against all the things and dealing with all that noise really being more laser focused. Also, it seems like, you know, with cicd, everyone's thinking about faster, faster, faster, and people are like, okay, I run faster. So we have to run all our tests then and just throw them up against containers and run all over the place. This might seem like a better solution than when you're talking about CIC to run faster. I have a check in of new code. What tests do I need to run and what tests can I ignore, which then makes it less maintenance and faster in that way as well. Do you see this as some way that can help us run tests faster than. Because we're only running the core of what we actually need?

I think so. I think that people should still have basically kind of catch all. If you have your nightly build and you can basically waste 10 hours in testing and parallelizing stuff like that, it's great. But let's say that you're towards the end of your sprint and you really want to get these rolling and you know that you want to test this in, I'm not going to say five minutes, but let's say 15 minutes, which are the test cases that I need to run in here. Mostly also because a lot of the time of the architect and the manager goes around deciphering the data, okay, stuff will fail. Do I care about it? I don't care about it. So you want to limit the amount of data that as a QA manager, test manager, the stock detect, as any type of analyst, you will need to analyze in order to make a solution, a decision in there. Okay, So I think that that is going to be very valuable. Now a lot of people keep asking, oh, so wait a second, that means that in five years we will not need testers? Of course we'll need testers. Why? Because you still analyze, you still make decisions, you still have to make gut calls and take risks. And I don't know an algorithm that's going to be good at that. I think that the moment that we have AI doing code, we will have AI doing testing. Until we get to that point, we will still have humans doing both of them. But that's my take.

Yeah, I agree 100%. Also what I love about this approach is it's not just about reporting. Like here's a bunch of reports like, okay, it's almost like it helps you analyze it, but also benchmarking. It gives you like a benchmarking statistics and KPIs you can work with to see if you're getting better or not. So it's almost changing it from just, just a reporting tool to almost like benchmarking assistant to make you more efficient.

It is. And, but we need to be careful with benchmarking. I've seen it taken to very, I'm not going to say dangerous levels because no one died, but the S2 levels where people were benchmarking test cases from different technologies or test cases even from different projects and trying to say, oh, why do you have here higher value test cases than this? Does it mean that you are a better QA manager than that? Not necessarily. Okay, you need to remember, for example, in this case it's basically a 1 to 100 test resulting that we're not trying to give up grades. I think that again you go to the same issues that you have with metrics. A metric is only good in the context that is being used. If you don't understand the context, don't use that metric. Okay. The same goes for value score, the same goes for everything. But I think that again, AI, it's only a tool and in this case we're using that tool in order to basically try to tell you which test cases are more valuable. And if you want to, right now we're telling you how to fix or increase the value of your test cases. Hopefully later on we will be using the same AI as we said, in order to point to places where we have more risk. Ideally, we will be able to use that same AI in order to analyze data happening in production and then trace it back to requirements. As you were saying, that is the beauty of our approach to test management is more of quality management because testing is basically the middle part of it. But we are seeing a lot of shifting left. For me, shifting left means that QA team is more involved in what's happening in unit testing and CI and even requirements. Obviously, we're starting to see the shift right. A lot of people starting to. Not enough people starting to monitor exceptions and escaping defects from production environments and basically bringing that back as feedback into their testing processes. So we see the job of the test management system to link everything together and to make sense of it all. Okay. And again, the biggest wasted time in automation, in my mind, is the time that it's wasted trying to put together or to analyze reports separately and trying to paint together a picture. Because we're back to the days where the QA manager walked between the cubicles of their his testers or her testers asking, what have you tested? What did you test? What did you find? What did you find? And then you went to the VP of R& D and told them, oh, we're good to go. We're in the same place with automation right now and that's what we're trying to solve.

Definitely. So you mentioned something about everyone saying AI is going to take my job and things like that. I was just looking at your report for the 2023 State of Testing. I don't know if this means anything, but it seemed like a majority of people, 40%, have been testers for over 10 years. So to me, that would show that testing is not going away per se, like people have been painting it. Because I would think there'd be less people with that much experience in testing if it was such a bad profession, people will be abandoning it. Did you feel the same way or was there any other insights from that report you saw that went, wow, I never saw that. I never thought about that.

Things have changed. First of all, there are plenty of results in there. So on the one hand you start seeing that people are working more and more years. And I think that it was Matt Heiser who said it some years ago. I remember him saying that, yes, maybe one organization will need less testers, but we have more and more and more organizations who need testing. And you know what? It's true. We see it as well, organization needing test management, that digital transformation. I always thought that quote unquote, digital transformation was a term for marketing or to sell more something. But no, it's happening. Every single place that we're working with are using more software. If we're talking about, obviously banks, but hospitals and retail chains and transportation systems, you have a lot of different systems being used there. All of them will require testers. So maybe the average number of testers per company has gone down, but the amount of companies requiring testers has blown up. And so you have more testers now. I also thought that I would see less testers in teams, as I was just saying, but there are still many large teams out there, meaning we asked all the time, how many testers do you have in your organization? And I thought that the number will go down. But we still have a large percentage of people saying, no, we have 50 testers in my team, I have 100 testers in our team. It's not like we only have three or five testers here. So that is something that was a little bit counterintuitive. But the report goes even beyond that. We're talking about automation. We're talking about shifting left and right. We're seeing evidence in the report that testers are shifting left. And for me, shifting left, as I mentioned, it doesn't always mean, oh, we're going to work more with user source requirement. There's a closer left. And that is, are we interacting on the testing phases of our development team? So we've been asking all the times, testers, how aware are you of the CI in your company? It sounds like a trivial question, but you know what? We've been seeing that the amount of testers who don't know about CI has decreased. So, for example, 2022, 21% of people didn't know about their CI systems. 2023, only 14%, a third less. So we have again, a lot of people knowing more and interacting more with their CI system. How many of the testers are involved, literally involved in the CI? Do you participate in defining it and analyzing it? We saw an increase from 55% last year to 59% this year. So that shift left is actually happening. And I think that it's important because it's giving us more tools in order to come to our development team. Mine, at least part of them is actually sitting right here and say, as a tester, I want to look into a unit test. I have what to say in here. You said that you're working tdd. Show it to me. Okay? And then we can have a higher participation in there. If you find the bug earlier, it's going to be better. And I think that the job of the tester doesn't start when they get the first working build. It can start a lot earlier. And we're actually making that difference.

Well, that's great. And like I said, you've been doing this report for over 10 years, so that's a great trend, I think. So the testers listening in here, you know, back when I started, I may have thought I had these skills I need to know about, but nowadays it seems like it's a whole different skill set. Are there any skills that you think testers nowadays need to know more about? Or especially you keep talking about this four to seven tools that every team has nowadays. Like, are there certain tools or techniques you think if as a tester in 2023 and beyond, you definitely probably need to know more about? If you're not knowing, knowing CI cd, that sounds like one of them. Are there any other ones?

That's a tough question. I think that if we look at the year to year increments, the increment is not that different. But if you look at a dinosaur like you or me, back when I started, I was afraid of looking into code. Okay. I remember the first time that I sat with a developer. I thought that I was going to see zeros and ones in the screen. And then I saw, I think it was Delphi or something like that. And it was like, oh, that looks like English. But we see a lot of testers who need to become more technical. Even if you don't have a technical background, a lot of things is pseudo English, so it's okay. But going back to your original question, I think that people need to start getting more involved in CI systems. Not necessarily configuring it, but yes, feeling comfortable with that, feeling comfortable with reports. I think that a lot of people need to start looking into monitoring tools and understanding how they work. More and more systems are working. SAS and I have a shocking discovery that testing in production is not new and it's not testing. It's basically you're running the code in production. Just look into your logs and try to understand if you have exceptions or not. That's like half of testing in production can be done that way. So it's a lot about, on the technical aspect, trying to work more on that side of the place. How do we analyze the data that we already have? And Brent Jensen from the A B Test Team podcast competition. But, you know, it's good for the industry. Brent became a data scientist, and he was a tester and he was always talking about the fact that that transition from testing to data science, it's a trivial one because as a tester, you have tons and tons of data that you could actually be tapping into in order to make better decisions. So I think that that is a big step. You don't have to become a data scientist, but you need to feel comfortable working with a lot of data and you need to start working on that other stuff that you need to feel comfortable with. AWS or Azure or anything like that. If you're a tester today and you're not involved in your environment, and again, everyone is working with a hosted environment of one sort or another, need to feel comfortable with that. That's also a big change. I'm not even talking about tweaking different languages and stuff that's trivial. But yes, you need to be more on the infrastructure side of the house, and I think that's a difference. Having said that, you know what's interesting? We have been asking testers for 10 years now, what's your most critical skill? What's the most important skill in your virtual toolbox? Every single year, without an exception, it has been communication skills. The most important skill of a tester is to know how to communicate. And when we started looking deeper into that, not only how to express yourself better, but how to listen better and how to catch clues from separate areas and how to understand what's going on without being told what's going on. Okay. And I think that that's an important aspect, not a technical aspect, but a really important one still.

No, I mean, I always have one last question which I'm about to ask is give me one piece of actual advice over the years, and I've been doing it since 2014. It's funny when you break it down, a lot of them have been more like communication skills, soft skills, rather than technical skills. So it's, it's funny you bring that up. I've been seeing that trend as well, for sure.

But having said that, I think that the most critical skill that a tester has to have, it's basically technical curiosity. And for me, technical curiosity means not being afraid of tweaking stuff and asking questions and playing with things and breaking stuff down and installing all sorts of tools in your computer and taking a couple hours a day just to learn something new, even if you're not going to use it, it's going to make your brain adapt faster and catch more stuff. And I think that it's trivial For a lot of testers to go into communication factor, critical thinking factor, analyzing, talking to customers, that's important. I'm not underestimating that part, but I think that testers also need to put more and more efforts on their technical skills. Maybe not all their efforts, but definitely more than they're doing right now.

So, Joel, you may have just answered this, but before we go, is there one piece of actual advice you can give to someone to help them with their automation management testing efforts? And what's the best way to find or contact you or learn more about practitest?

So the first question is what do you need to do in order to improve your automation management effort? Is to start asking the question of how can I improve my automation management effort? How do I before I start writing the code or before I start looking to automation, what information I'm looking to provide, how am I going to be serving that to my stakeholders? Okay, because it's not trivial. It will dictate a lot of how to report stuff. How do you write your assertions, Even what test cases do you write and how do you structure where. So don't only think about how cool is it to automate and maybe what is the best flow from my customer perspective, think about how you're going to be serving that information along to your stakeholders. A lot of people think about that too late in the game. So that's going to be an important point. How to reach me? That's a great question. Meaning practices. Www.practice.com that's a great way of doing that. We have a blog post in there that I try to write once in a while. I'm out from COVID We're starting to do a lot of more conference circles so we're going to be attending more places. There's the online test conf that is coming up in November. We've been doing that even before COVID made Zoom popular, so it's also a good place to be. And again, all around I think that the testing community, it's a small circle so we're always around.

Absolutely. Thanks again Joel. And for everyone that's listening to learn more about test management and Practitest and to see a free trial for yourself, be sure to check out, as Joel mentioned, practitestractitest.com, which will also be in the show notes for this episode. Thanks again for your automation awesomeness. The links to evident value we covered in this episode. Head on over to testguild.com a 4.6x6 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world, we'd love to have you join the fam@testguild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the Guild community, I'd love to hear from you. Head ON over to testguild.info and let's make it happen.