Modern Test Management Systems with Izhar Livny

Foreign.

Get ready to discover the most actionable end to end automation advice for some of the smartest testers on the planet. Hey, I'm Joe Calantonio, host of the Test Guild Automation podcast and my goal is to help you succeed with creating automation awesomeness.

Hey, it's Joe and welcome to another episode of the Test Guild Automation podcast. And today we'll be speaking with Izar all about test management systems, a really cool technology I just learned about. I think you're really going to get a lot of value from. Before we get into it though, if you don't know Isar, he basically considers quality assurance to be his forte. He has a lot of experience in recent years working with QA teams around the globe, helping them spot obstacles and put the QA team's work before everyone else notices. So he really focuses on the QA teams and over the years he's accompanied QA DevOps teams on their journey to select the right tools and the methods of work of their automation manual and mobile testing as well. So he's really great at looking at tools and teams to see the good fits for what works for everyone. And he also has a broader outlook on the key elements of every QAIT stakeholder and makes sure that when they're onboarded that they know which tools probably work best for their needs. So he really is, he provides them a lot with must have criteria determining what tools and we're really going to dive into this as well, especially when selecting a test management system. Don't want to miss this episode. Check it out. Hey, Izar, welcome.

Hey, thank you very much. My pleasure.

Before we get into it, is there any bio that you want the guild to know more about?

I guess you said it all. Come on, you said it all. I'm speechless.

Perfect. So we met a few weeks ago and you show me this cool technology around test management. So I thought we'd talk a little bit about test management systems and then maybe dive a little bit more into your solution. So, you know, if someone's just listening, you know, it's 20, 23. How would you describe what a test management system is?

You know, as we spoke of course, as you mentioned a couple of weeks ago and you know, we both followed the market and there's a lot of vendors out there. I think if I'm looking at it, around 22 different vendors. There's at the end of the day, aim to sell the same thing, but each one of them has, I guess, its own unique selling point and I can share with you that besides selling feature which you know, at the end of the day everybody sells software and software is a feature that it's bits and bytes and it should be plugged in and display. But I can share with you that in today's market days and in the modern QA teams and the modern or the agile companies, they're looking for a solution not for a software and the TMS is going through some changes, you know, if I can dare to say, and I would say they're stepping out of the legacy systems, but they're looking at something different besides selling features that combine automation and manual and regression and unit testing, exploratory testing, which all important, they're looking to engage the technology with the human asset. That's what we see or at least that's what we hear from VP of Engineering, VP of R& D since all the medium sized and up companies in the world are scattered and there's no longer one headquarter, one team that develop all the different DevOps dev team and DevOps units and they're all based themselves on sometimes different tool chains, sometimes different CICD, pipeline, whatever. So in today's modern world, 2023, let's say the stakehold or the one that take decision need a better outlook, a better view of what is going on down there and they're looking for a solution that combined besides the feature more they look for more precise or more granular outlook about their people that holds and carry this capacity or this sow of the OQA activity. That's what we see, that's what we hear and test gear, you know, shift itself into this path and of course I believe we will speak about it. That's why we developed our new version that deals with that.

So you mentioned the human asset. A lot of companies may not necessarily focus in on this and I assume since a lot of companies, a lot of people now work in remote, this is probably even more of a big deal over the past few years. So can you talk a little bit more about how people can get that human asset, get the insight like you mentioned, the granularity they'll look about the engineers and the, the teams and what they're working on to really pop that out of what the human asset is actually maybe struggling with that you may not get out of traditional tooling.

As we said, the teams are being scattered all over again, medium sized companies and T1 you need at least 5 to 10 different locations worldwide and it's very easy to work on silos, you know, each one of them Works like that. But at the end of the day, like in all business, everything should be consolidated and both in marketing and sales and hr, everything is being consolidated. It should work the same. So just imagine a VP of engineering, even a regional one that based on the North America and the other one in Asia and the Pacific, the other one in Central Europe. They need such an outlook not only about their land, their autonomy, they need something cross functional across, across the board. And this kind of outlook, when you set the right engines within the tms, you'll be able to control and gain much more than working on the same method that all the other tms or the usual typical legacy tms. I don't know what would be the right term to use, but you probably understand what I mean. So once the right engines are being installed within the tms, you can gain much more from the TMS itself and you can take a leverage on the human asset. Because at the end of the day, with all the respect to the technology that we provide the market with and each one of us is dealing with the end of the day behind every keyboard, every screen, there is a human being. And human being need to know exactly what he contribute. He need to know all the, you know, all the flow it starts, you know, shift left from left to right from there to go live he needs to understand exactly what the STLC means. And at the end of the day when the feature go live, it's sometimes not only about his unit, it's sometimes a combination of different units from different geo from different, you name it. So it's a combination, it's a huge mix of teams, technologies that should sing the same song. And without the full transparency and clarity about the process, which means a better outlook from top it won't happen. And when it won't happen, no quality at best the apps that the companies are providing the customer with, probably with bugs or defects or poor. And at the end of the day, less transaction than expected, less transaction, less profit, less, less, less, less. And you know the story, what it means. So it sounds, you know, it sounds very easy, very simple. But none of the tms are putting their efforts or putting their mind into the human asset. They're more dealing with the feature, feature, feature, which again it's important but behind every feature there's a human being that need to understand what is wrong, where it starts and where it goes from the shift left to the right, where is the right, what is at the end, what is out there. So with the I can share with you if I could Share. Of course, in test gear we have the gamification mode. It's not only about motivation, it's not only about incentive, which is. That's the aim of gamification. But it's much more than that. It's an engine, super sophisticated, I would say even an algorithm that runs behind the scene and derive the team into excellence. So besides motivate them and beside, let's say set the like or dislike, for God's sake, if it happens, but let's say besides encourage them, it derives them this kind of, let's say, competition, inner competition derives the team into excellence and derive the whole business the same path into the same goal, which is provide better business, better application. Better application, better service, better service, Much more money to the company, much more money. Everybody's enjoy at the end of the day. So this is what we see. And the demand is emerging and people are more, much more speak about it. You know, the early days, let's say, of the COVID and stuff like that, people only were dealing with how I can do my job hybrid in the most efficient way. Now there's a kind of focus about the employee itself after the COVID because people are starting to meet and people are starting, you know, the world is being opened again. But the human asset, as we see, is a major and crucial part of this success. That's what we see.

Absolutely. So can you talk a little bit more about the gamification piece? So you have a test in the system. What are you tracking? What are some things that you're monitoring? I know you can probably track and monitor a whole bunch of things, but do you find that maybe like certain group of people checking code on Fridays and they probably, they break the build and they probably need more training. Like what exactly are you tracking and monitoring? I guess the question, sure.

You know the beauty of it? You know, when you say gamification, people start to get, you know, to get chills because they say, oh my God, the big brother, he sits on top of me and he's going to see everything. The beauty of it, it's not there. Everybody can see everything. So it's. There is no hidden, there's no secret, there's no hidden part of it. And besides that, the leader, whatever he is, director of qa, QA lead or vp, whatever, everything is so transparent because again, the aim is to derive, to unite as we unite, a manual and automation into the same workspace. It's to unite all the people into the same goal. And once people can see what their peer is Doing and what his assignment for today, you can see the assignment for today and for the week ahead is himself. He can, you know, he can compare and understand what he needs to do. He has his goals. So we monitor every single aspect of the QA activity. You know, like, like a team, you're a manager, you have a team, you have goals, you have action items that need to be taken. So every little thing is being monitored. That's the beauty of it. And, and everybody can see everything. Teamwork, as simple as that. Like an NBA team of the NBA is five people on the court. But let's say a football when the 11 on the court. So let's say the typical, I don't know, the qa team is 11 people, so. And the goalkeeper. So everyone can see everything. Everyone can see everything. The goal and the pass are so clear. They have their set of action items that need to be taken. And you know, within the system there are some elements that can motivate them and can kick their ass if they, if needed. But the whole aspect is that the human being is in the middle, in the focus and for the good side, not for the best side. It's not a big brother that's gonna punch you if you're gonna miss something if something went wrong. There is a kind of method of analyzing and you know, kind of trying to understand what is going wrong and why it's going wrong, but not in terms of I'm going to give you a punishment night right now because you didn't accomplish your assignment. So you know, you need to see it in live in order to understand what I mean. But the whole concept is take the whole bunch of people of yours and unite them, combine them, let them know where they should go, they know where they start, they know exactly where they should end and what each one of them contribute to these assignments. And once everybody knows exactly what is wrong, I guess I believe, you know, like in every single thing that we do outside of that tech life, in our personal life. Once we know exactly what we are expected and once we know what we need to contribute and we know our value. Listen, it sounds psychological. It sits on the psychological aspect of all human being. Once you know what is your value, what you bring to the table, it's much more easier to you to collaborate, to give yourself, to dedicate yourself, to devote yourself to something. And it works on the same psychological aspect and you will be surprised. It works. It completely works.

So in order to collaborate, it almost sounds like what I've seen a lot of times. Is teams have different systems, development has a different system, QA has a different system. You know, SREs have a different system. It sounds like in order for this to work, everyone needs to be on the same system. So does, does your solution then allow developers and everyone to work all in one location? Is that one of the benefits of then you can then use this collaborative approach to get these insights to kind of gamification insights that you're getting?

Yeah, if I understood it correctly, we are kind of agnostic, what kind of system are being used down there, which means what the environment contains. As I said, different teams, sometimes different pipe, different tooling, different CI, CD elements, whatever. Some of them works on Azure, DevOps, others on AWS, some of them work with Jira, other with whatever, I don't know. So some use backtrackers, the other zone. So we don't care. I would say we are agnostic to it as long as we can something with API, sometimes with not we can connect to any, any single element, any single environment. We actually at the end of the day brings to the Table 1 workspace, which means one unified view of all these silos. So once, let's say many silos become one landscape and this landscape is being monitored by test gear. You know, as I said, like from bird eye to the most granular part of it. And yes, to refer to your question, we are agnostic. It doesn't matter for us which tools do you use, we can do the magic.

Great. It sounds like you have an API that you expose and then you can integrate all these different apps into this one central location. So who mainly uses the central location? Is it the whole team? Do you have like a point person that monitors these insights and then reaches out to the different teams or does everyone work on it? Like you said, almost like a social media platform, like a Facebook for software engineers or something like that?

Yeah, it's a good question because you know, I was under the impression that there will be kind of split between managers and if I can call it managers and engineers. But at the end of the day, every single person that deals with QA has a license. You know, at least in test gear, when we sell our solution, we aim to the qa and it doesn't matter for us if it's a junior qa, senior lead or or vp. But at the end of the day, the motivation or the will of every tech lead within R D teams is to enable all the team to use it. Because as I said, every single human being has its contribution, has its value. And once you give them all the access into the system, you can gain much more. You know, the, the sum is bigger than its parts. So this is exactly what works here. And I can share with you that the recent gfc, I would say the last two years, I don't know how much you, you are, if you notice that, but I will start it differently. Typically the way QA teams, Dev teams and DevOps team, we've noticed that now the QA themselves are being, I would say swallowed because in order to hit the bug as sooner as possible, every developer needed to debug itself from the very first line of code that you write. So we found ourselves dealing with Full Stack developer as well as Q engineers. We, because they need to know exactly what's going on in their land before they release their code into the pipe and get rid of it, I would say. So it's not only Q engineers, the full stack developers, even the web developer and the devils that controls the environment needs a license too. So when, you know, when we started test gear, we thought it will be, let's say, for a selected group of users within every company. But at the end, till now, we found ourselves selling kind of enterprise licenses because the motivation of the managers and even every single person, he would like to know exactly what's going on. So he needs a license. And of course he would like to make his life much easier. When you have a better outlook about what you're doing, it makes your life easier. So yeah, so it's weird. Let's say it's a contradicted what we thought in the beginning. We thought that testview will be for a very selective group of people within every dev team. But we found ourselves selling much more licenses than we thought.

That's awesome. So have you gotten any feedback from these developers? Why? Is it because the communication, the dashboarding, because they're able to find issues earlier before it goes into production. Like what's the feedback you're getting? Why that is, you said it right.

Communication at the end of the day, communication because you know, to set your test and to set your code and to set everything that will go into the pipe, let's say it's the easy part. But to communicate with your peers and to be alive with what they're doing and with their contribution to the same apps that everybody's now working on, the communication part is the most challenging way. This is exactly what the social media brought into the world of the techie people. And everybody looks for feedbacks and everybody looks for the like they would like to, to get, you know, to get rewarded. No, as I told you, it works on many psychological aspects that I was amazed to see. People are looking to see that they are rewarded, they're appreciated. So this is exactly what our system is doing and provide them with kind of appreciation for the work. Of course we are not, you know, we are not providing everybody with his appreciation if it's not, if he didn't accomplish his work accordingly or by the set of action items that he should take. But the platform enables you to get a better communication between the team to engage, engage, engage people with one, with their peers in order to make the work much efficient, cheaper in terms of maintenance and at the end of the day, better communication, better knowledge transfer, better knowledge at all and better delivery. That's what we're seeing.

So how people appreciate it though, is it like, do you have a leaderboard? And it's like Joe Calantonio is doing awesome. He did all these different activities before the release and no issues. So that's how he's been. And it goes to everyone like Joe or like, how does the appreciation work? I guess what I'm trying to get.

Have you used teams of Microsoft? Have you ever used the Instagram or Facebook? Do you know this heart, this red heart, the like so they can appreciate the works of their peers by like they can even set a comment in the right place. So really a communicative dashboard or platform, you know, it's not a chat in order to send gossip from destination A to destination B, but it's a place where people can appreciate, can share valuable information about their work. So yeah, the red heart is one of the best icons in our system. Yeah.

Yeah. So I just curious to know, you know we talked about seamless integration, we talked about gamification, test management. I know it also I believe handles manual testing and test automation. So I thought maybe we could talk a little bit more about how test scare helps with minimal testing and test automation.

Yeah, of course. Test gear take the put a lot of efforts in order to take a leverage on your automation part. And yeah, you know, I'm not a techie guy, I'm a sales guy but I can see very clearly that once users are using test gear, it doesn't matter what kind of automation tool in, in their shop. I'm amazed by the leverage. I would say test gear take a leverage on these tools and the work become much more massive and aggressive even I will say. And yes, automation become a major part. And we have, you know, we have within Testgear we have some unique features on top of the automation tooling that makes the automation tools themselves even much more powerful. So we ride on top of them and we enhance them and we boost them into. I would say it's an automation on steroids. So yeah, absolutely Testgear, absolutely. Accelerate the usage of automation.

Do you have any numbers? I know you're large in certain areas of the world, a big, big user base. So any numbers like company X implemented this and they got I know X amount of savings or whatever, any type of those numbers or case studies.

So we found out that if you use test gear you accelerate testing cycles by three times and reduce testing cost by 50%. It's a lot. You decreased defects by 25% and tested 30% faster. Of course, while we support every dev methodology, hybrid, agile, whatever, it is just a part of the dashboards. We have dashboards and predefined reporting views. So built on demand real time dashboard for stakeholder review. By that, by the pre built that we have, we reduce reporting time by 40%. Listen, the numbers are amazing. And now for the most nice parts. With TestGear you spend three timeless efforts due to its codeless interface and it's user friendly. I know that I market now I market and sell the product. But again it's only clicks, clicks, clicks, no code, nothing like codes or script is being involved while using the platform. We found out, we did this research in among 100 customers that has more than 5k employee nice, you know, nice companies, it's not small companies. And we found out that the product get released up to 35% faster due to some smart engines. Some of them I mentioned, some of them not. But all of them relates to the unified manual auto testing. By the way, this is a kind of what I meant that we take a leverage or put the the automation part on steroids by that they can release 35% faster. It's you know, in numbers and when you put some and you put or translate it into money, it's a lot of money. And again the last important thing, it's not numbers but we found out that with test gear better communication teams are more efficient and much more engaged. So these are the numbers that I can share with you. Just published recently after some benchmarks that we've done. You know, the numbers are significant, you know, to cut 50% of efforts or to get 35% faster release. Listen, it's if you translate it into time, money, people, you know, labor, it's a lot of money at the end of the day.

How are people able to achieve this? Do you have like an implementation roadmap when someone purchased your product or do you just give them the software and they're. They're up and running? How would, I mean, it's, it sounds like once they have implemented it correctly, they're getting great results. But how do they know how to even implement it in a way to get them those type of results that all your other customers are getting. So I don't know if that makes sense.

Yeah, first of all, you know, I told the platform is codeless and user friendly. When I say user friendly, you need to. It's a SaaS product. You don't need to install anything. You just need to open your path towards servers and then you can display it. Let's say. I'm sure you're going to laugh, but if I would say that it would take you four to five days to get the understanding what is going on behind the scene. You are way over than what we expected. Because usually because, you know, all the customers that we are selling to, they're not newborn with regards to how to work with automation or manual. They are mature within their QA environment. They use either, I don't know, legacy systems of automation like Cypress, Catalon, you know, all this stuff, or the more advanced one, or they use, or they, they write the script or proprietary code in the manual one. So they are well equipped with the knowledge of qa. So, so we provide only this layer, you know, a management layer on top. And this management layer not involves any code, it sits on top, it gets its connection automatically. As I said, some of them are native connectivity, some of others with API. And once it happens, everything start to work. You know, we have a manual guide and we have training and we have everything. You know, we are putting a lot of efforts in order to make the life of the engineer much easier. But, but we found out that people are. We have a support team and we have a professional services team, but I don't know if I should be happy or not. But they're not unused. So people are doing that by themselves. Customers are doing that by themselves and by three to four days they're up and running with that. Nothing more is requested.

So I think a lot of things we talked about, people actually really need to see it themselves. So is there a way people do offer like a free demo or a trial version, people can try to actually see it for themselves. Some of these things that we talked about today.

Yeah, absolutely. We encourage all the engineers, all The QA in the dev side, the full stack developer to go into our website to download the trial version for 14 days. Play however and whenever you want. We are on top of every download that has been made and we can be, we are attentive and we are there in order to help if needed. But as I said since it's codeless, since it's so easy to use, people usually try and we have a self service of purchase. So we found ourselves like a slot machine. We hear in the website the deals at lending because sometimes people try it, they see how easy it is, they connect it to every environment that they wish to and purchase, you know, credit card, the company credit card and it works. But yeah, of course this mainly happens on the small size of companies but in the medium and the large one we are getting more involved because sometimes we need to explain, you know, you know the process of selling for enterprise. It's not that smooth as a small one but it's the same product, the same values, the same assets. But the process a bit different but it goes quite quickly.

I usually don't ask people about pricing but I know on your site itself you actually, which is a lot of people don't do this. You actually say exactly how much pricing is. There's like a calculator I think where you put the number of people and you can slide it for licenses, annual, monthly and it tells you how much it is. So it seems like pretty affordable based on enterprise solutions.

I've seen super affordable and I told you I'm very proud about the solution we sell. And I think once you're proud and you expose everything and you disclose everything, especially about your technology. So with regard to the pricing, what should I need to hide? Why should. It's not hide and seek. Everything's so transparent. So that's a price. We know our value, we know it's affordable. And you know what? I tell you a secret. There's a profit behind it. Even in these numbers there's a profit behind it. So. So if it's profitable and affordable so everybody are having fun out of it, everybody get their benefits. So it's easy. I think that's, you know, I believe in openness and direct and, and put everything on the table, no hidden cards. I don't need to pull out my joker card. At the end of every sale process I put everything in advance. This is a software that its values, this is the thing that we are focused on. We bring test technologies and human into the same table and that's what it costs affordable, I know. Even for small, medium. Of course for large enterprise, it's, you know, it's peanuts. To be honest, it's peanuts. But it is what it is. It still, it brings a lot of value and you know, numbers speak for themselves. We know what we're getting and we see the large amount of customers that are being gathering every month, every quarter to our install base. We are happy, they are happy. At the end of the day, it's mutual. So we did our best.

Awesome. Before we go, what's the best way to find, contact you or learn more.

About Testgear in order to reach out to us? First of all, there's a website, TestGear IO. You can reach out via TestGear IO. You can reach out into our LinkedIn page. You can reach out to me in LinkedIn if you would like my email, my corporate email just told you no. No hidden parts. The pricing, the technology, even my email is shareable. So if you need my email to ask me question, go ahead. I'm willing to share it with all the audience. No worries.

Awesome. And all this will be in the show notes. And just be careful when you type in Testgear. It's Test Gear IO that'll take you directly there, but there will be a link in the show notes for it. Thanks again for your automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com a 4, 3, 2. And if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of tools, 27,000 of the smartest testers, DevOps and automation professionals in the world, we'd love to have you join the fam@testguild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the guild community. I'd love to hear from you. Head on over to test guild.info and let's make it happen.