QA Agents of Change with Lisette Zounon

Get ready to discover the most actionable end to end automation advice for some of the smartest testers on the planet. Hey, I'm Joe Calantonio, host of the Test Guild Automation Podcast and my goal is to help you succeed with creating automation awesomeness.

Hey, it's Joe and welcome to another episode of the Test Guild Automation Podcast. Today we'll talk with Melissa all about QA automation, probably robotics AI. Who knows. She has a lot of experience so we're going to be diving deep. If you don't know, Lissette is a passionate quality engineer leader. She has over 18 years of experience helping people and companies improving the quality of their applications with solid tools. She has a simple process she follows and she also knows how to create a really smart team around her. She's also responsible for leading and managing high performing quality testing teams throughout all phases of the software development testing cycle, ensuring that all the information systems, products and services meet or exceed organizations and industry quality standards as well as end user requirements, which we know is so important nowadays. This includes establishing and maintaining the QA strategy, process platforms and resources needed to deliver 24. 7 Operation Critical Solutions for many of the world's largest companies. I met her at Style West. She told me all about the awesome things she's doing there. She's also a proven leader that thrives in a high quality, high technical software development environment. Really excited to have her on the show today. You don't want to miss it. Check it out.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Source Labs, the cloud based test platform that helps ensure you can develop with confidence at every step from code to deployment. For Every framework, browser, OS, mobile device and API. Get a free trial. Visit test guild.comSource Labs and click on the exclusive sponsor section to try it for free today. Check it out.

Hey Lisette. Welcome to the Guild.

Welcome. Thank you for having me. So excited to be here. I've been a listener. I was excited to see you as Star wars and I'm happy that we made it happen today in 2023.

Absolutely. Yeah. Really excited. After I met you, I'm like, I have to get you on the show because it sounds like you're really doing a lot of cool things there so that maybe that's. We'll start with that. It sounds like you're working in a robotics industry so maybe you could talk maybe a little bit like what your current testing needs are and maybe some challenges that you're dealing with that maybe other people aren't. Especially when it Comes to maybe robotics.

Yeah, challenge, definitely. And I love challenge and that's why I took on the role. I've been at Fort Robotics for about six months now and what we doing is this what attract me to this role. And the challenge that we have is really serving our robotic and autonomous machine customer. So we have a set of tools that are really a combination of hardware, firmware and also software and cloud based application and eventually in the future mobile application. So you can see that we're hitting all this spot and my team is responsible of managing and leading all the QA testing and quality effort in that organization.

Very cool. So you're known for process implementing processes. So just curious to know like you're dealing with firmware, it seems like you're dealing with a lot of moving parts there. So how do you implement a QA automation strategy or process within this company?

Yes. So that's the challenge and I was attracted to it because in the past I've worked in hardware company, just dealing with hardware alone, cloud based company and then software. So this is the first time that I'm in a startup where all of these are a good challenge to have. Right. And the way I'm approaching it is really from one area to another while I'm also keeping a consistent process across the organization. For me it's really about simplicity, simplicity, having simple process, understanding each team needs and trying to meet them where they are so that we can solve the challenge. Let it be in the hardware team, let it be on our firmware, but also be the one that allow each team to cross collaborate. So one of my favorite words of this year is like cross pollination so that every team is collaborating. Because I feel like the more we collaborate in our test strategy and through our automation, we are able to find those edge case bug that really kind of like drive us nuts toward the end of our implementation. And that's exactly where we are right now in our challenge.

How do you do that cross pollination then? Because I know to try to get everyone together on the same page is difficult. How do you get people to actually contribute and say, oh, get kind of excited about this?

Yeah, that is a challenge totally. And we love to do that. But what I've implemented, I've implemented this in the past. I usually like to call QA folks. I see QA folks, people that wear that hat, you know, as agent, agent of change. That's really what they're doing on a daily basis. And I even like to call them FBI agent of change sometime. Right. Because if you are the FBI, you have a little bit of the investigation trying to really crawl and find out what's the issue. So I'm using my team to really read at the center of that between Dev, DevOps, Automation Engineer and other other needs to be able to really find what is the issue. And then how do we add that to our really test suit so that, you know, not only we thinking about the problem of today, but how do we resolve it so that we get ahead of the issue? Because as a process person, I'm always trying to like sit and understand the challenge, the pain point that occur now and how do we add that into the loop so that we get ahead of it early in the cycle next time? Because for a startup there's a really fast change happening all the time. Right. So you really have to be cognizant of that. But also make sure that when you start a new process you can work. And that means working with product team, working with a developer which has various type of developer. You know, we have a cloud developer, we have a firmware developer, embedded system developer, you know, so each of these people have a specialty, right? And qa, cross cut, cut across all of them. So making sure that, you know, you really go and understand each of them. Challenge, resolve that, but make sure that it's consistent across so that what we learn from one team we can also apply in another team. Because everybody does not have the same challenge at the same time. Some and every team at a different maturity level as well. Because that's another thing that is cognizant for me. So when we learn one challenge, I try to document it, I try to keep track of it so that as long as another team has it, we can solve it for them either in the same way or in different ways.

Yeah, we think challenges around firmware are much different than the cloud team. So.

Yes.

How much automation do you put into place and how do you know like automation can be applied to firmware? Because it's not like you're using selenium on a gets a browser application. So how do you know? Like, all right, I know automation is helpful and it has to be able to be helpful in firmware, but maybe some teams don't know how to do that. How do you know what's automatable, I guess or how you can help teams?

Yeah, that's a good question. One is to really understand what they're doing, you know, like at a very granular level. And in my mind I just had that conversation today with my team is you know, I have this funnel in my head. Right. It's like anything that is close to the code is really easy to automate. Right. So from a firmware perspective, you know, unit testing we've this year we've invested our last year we've invested a lot in our unit testing because those were like the low hanging fruit. And we've seen effort being around that really helping us a lot. So now the next stage of that is what kind of integration not far from the firmware that we can do, you know, and sometimes the hardware can become like a super black box to us. So we try and is really trying to like chip away at it from the middle layer as much as possible. If you think of the hardware being lower the next middle layer and, and it's the same thing also on the cloud team, you know, trying to chip away at it from the API layer as much as possible and see how many tests can we focusing on. And then the other area that are not. That could be a challenge or that are much adapt to flux because hardware can be a huge problem. Right. Like we have to make change in our hardware sometimes frequently, sometimes because of a bug that happened in manufacturing. So we leave that alone. But the next layer is where our focus is and we can run repeatable tests so we can easily identify issue. So those are kind of like the work that I have to lean in with each team and understand how we can help them. Yep.

So that's interesting also that's probably another challenge dealing testing against hardware. So it's not like you can go against a browser and just spin them up. And how do you handle that then? Do you have bottlenecks where you can only do so much testing because you're tied to maybe an environment that has the hardware you need and you can't just easily replicate it?

Yeah, so we, that's a good question. We use simulator some places. You know, a couple of our team use simulator. So but even we use simulator that's just in doing testing sometimes we use that as a early testing methodology just to get a sense of like where we are just to even understand how our test is solid, you know. But then at the end test in the true value of test and just not to throw another wrench. Another challenge that my team is focusing on is really in our organization is safety and security, you know, so we have a lot of safety standard that we have to follow. So our final test to meet our safety certification and all that also have to run, you know, on a. In the real hardware and the latest version is possible before we can certify ourselves as well. So there's, there's a really granular level that we have to go through. But we've used some simulator as well as early tests to just get a sense of how our tests are. Just testing our own test pretty much with the simulator and then we run it on the real hardware as well. And based on the hardware revision, we gotta keep pace of that too because if our hardware team make a change in a hardware revision, we gotta, you know, make sure that we, we also run our latest test on those. A lot of moving part.

Yeah. And then you add security on top of it. So, you know, you say your QA is like QA agents have changed and they go across, all across the different verticals, I guess. Are they responsible for everything? Like, do you expect your QA person to actually know security or do you have people embedded that experts in it? Like, how does that work? Because it sounds like you're trying to cover a lot of different things.

Yes, yes, yes. So when we hire, I mean, that's a good question because that's a challenge that I was facing last year when I was hiring first. Right. Because for us, security and safety is our product. That's part of our product. So we have some security expert, developer and architect that are really architecting how security should be building into our hardware, in our firmware and also in our cloud application throughout the whole product pipeline. So from a QA perspective, you don't have to know security, but if you know security is a good thing. So we look at it as part of the product. So you just go, you need to understand the requirement and you testing security, you know, and it's a little bit more granular for us because it's like secure boot, you know, our hardware need to be secure boot, we need to have secure updates. So those are like functionality that we are selling out there. And these are part of our functionality. So we need to understand that. So we are building a team that will be, you know, world class on that, understand cyber security and also safety as well.

So how do you get, how do you know you're doing the right things then? Do you have any metrics you use to know? Okay, with security? We're doing this. I have a, like, how do you keep track of all this stuff?

Yes. So I'm a huge fan of dashboard, you know, and we are definitely work in progress. And dashboard is like an area of my responsibility actually, because I like a dashboard, I like to know what is the KPIs, you know, are we meeting the KPI? So each of our product has some KPI that they need to meet. We also have KPI for the software team. You know, we have KPI for quality team and we have KPI for our hardware team. You know, even in terms of like how a customer, I mean manufacturer, they produce the hardware that come into our organization or even that goes to our customer. So we need KPI across all that. So that's how we, right now we are defining the KPI. And then next we'll try to like make sure that we have a baseline and then try to improve upon those data that we collect from each release.

All right, so it sounds like you have a team also that is distributed. So is everyone in house. Are they all over the world? How do you deal with not only all this technology, but it sounds like, I think you have resources all over the world as well.

Yes, it's not a huge organization, but it's very an efficient team. And we are definitely all over the world. It's a mixed bag of in house. Most of our hardware folks in house in Philadelphia, you know, in the lab on hands and then we have teams that are across in the United States and then we first come in the lab as needed. So it's also that mixed bag that we have. Totally.

So how did you set that up? Because believe you built this organization, it was a startup, you helped build it from the ground up or you have experience doing that. How do you know when it makes sense to have someone in house as opposed to. All right, this makes sense to even to tell management, look, this person's great. They're not going to be, I know we, you know, that they're distributed, but they could still work remotely and get the job done.

Yeah, I love that question because that was something that I had to like finally clearly define, you know. And although I've done this in multiple organization in the past, I've been working with distributed team for the past 10 to 11 years, actually 12 years since I was at Yahoo. So that is not. I've used to work with team that are like geographically diverse. India, California, Texas, and also in New Zealand, you know, so, so I had like all those time zones, so I'm kind of used to that. But it just require a little bit more of like flexibility and agility among the team and a lot of collaboration. But to your point, what I have found out and have decided and learned on, you know, as we try, is the most experienced team member can be remote Meaning people that have like a lot of experience, you know, they're more independent, you know, they're more flexible, they're more agile. Those are the senior role, senior senior engineer. They couldn't be remote. And it has worked so far because they are more responsible, for lack of better word, they have a lot of accountability. And then they travel as needed to a headquarter in Philadelphia. The folks that are a little bit more, I would say junior or with lacks experience, even with the tool and technology and even our hardware, we love for them to be local because then they get to learn a lot more about hardware and play with it, you know, because we all like little kids that love to play with those hardware, right. So if they're more local or really close to the headquarter, they can go in office as often as possible because you learn by doing a lot more and then you can collaborate with other folks in the lab. So that's the combination that have worked for me so far.

How do you get people on board? Do you have a training system in place to help them? Like I said, it doesn't seem like a standard web browser type situation. So how do you know, like how's your onboarding then that you're planning?

Yes. That was also another area that I had to tackle last year because I mean hiring it take time, you know, you have to go through all that process finding the right talent, which was not easy, you know, because you can see all the combination that you need. Right. You know, if they have security. Yes. If they have safety, that's even great. They need to have a embedded. So that was a good challenge that we, well, we had some great folks that joined the team. And then the next plan for me was let's plan the a great onboarding for them. Right. And for me I feel like onboarding is probably never ending. But the great thing that we have at for robotic, we have an excellent team that is really great at collaboration. For me it start with documenting our onboarding. Right. So the first, I would say the first month is really intensive. Meeting with various posts, you know, and just having detailed conversation about what they do so the team understand how they feed in the whole landscape from product all the way to our integration team, meaning like our operation, you know, production team. So they have to meet with all those folks on a regular basis. We have a roadmap conversation so they know what product they're working on, what are the feature of those products and then get into like tools and technology that they need to learn, you know, so there's really different layers for that. So that is like a long month and then that that doesn't stop. Right. I do a continuous conversation with them. We create, we have a customer team are fabulous. They have great customer training, you know, because you have to learn about. Another part of our tool is it's used in a variety of organization. So we have agriculture, we have a telecom communication meaning like autonomous driving, we have manufacturing, we have construction, you know, we have like a wide range of industry and each industry have different use case as well. So our sales and customer team has done a great job like creating various training. And when you go through those training you really give you a great sense of like what is our product and how they use our product out there. Because I feel like keyword folks also need to have that lens, right. To understand how. Because sometimes you can get too much in the weed of like your daily testing and you lose sight of actually the customer part of it. So that training also happened. It's part of the onboarding and that is a long process as well. And we're still refining our onboarding but so far you know, we're getting feedback from these folks so we can refine it so that when we have future of hire we can also, you know, make sure it work well for them. But onboarding for me it's super, is critical.

So these questions just coming into my head, how about tooling? Because once again it seems like are you creating your own tools, are you purchasing tools, are they open source? How does that work?

It's a mixed bag and I have to tell you that tools is something that we're going to focus more on this year because we're really trying to get our product complete and ready in the market for general availability. Because most of our product has been used by early adopter, early user to give us feedback. So we, we are really focusing on having our general availability product out there. You talking about internal tool for us, right? We have a couple of tools but they're not consistent across team as we want to. We use Pytest, you know, for some of automation we've using Cypress. I just had talked to my team today that we might dabble into a little bit of selenium and even play. Right. So we really trying to. I'm personally somebody that is super tool agnostic because you know I've used so many tools in my almost two decade career that you know, I don't get really married to a tool much. I have like a whole speech that I do about you Know, picking your tool is almost like dating. So I'm really open and tool agnostic. I actually let the team select the tool that work for them. Right. And just tell me what. So this year. But we're going to get a little bit more clean up and align on our tool across the organization because that's also important for us as we growing and expanding and scaling. We want to really get that consistent as well.

I love that approach. Don't fall in love with tools. Use the tool that's right for you and your team. If it works for you, great. If it doesn't, move on. Yes, for sure. So no at style. Wesley, I think you were on a panel on AI and testing with Tyrique. So does AI come into play at all in your current job? Like do you actually use AI testing? Where does the AI piece come in? Or is it just something you got from another company?

Another company? Yes. So not yet at Photo Robotic, but I would like to bring it. But I wasn't. Yes, you actually recall I was on the panel to talk about AI because in since 2016 I have been involved, you know, I was working another organization where we had a lot of testing to be done with a very small team again. Right. So I was really trying to find the right solution. And, and again me being somebody that is a super tool agnostic, you know, I went on the marketplace and I was an early adopter of an AI based tool that my team used successfully. You know. So I have that case study that I really shared during the conference about how we had like almost thousand test cases and the process that we went through using an AI based tool with a team that was not automation engineer from the Strat. Right. They learn as the tool allowed them to become automation engineer in the span of less than a six month. And how we were able to like reduce our efficiency, I mean actually increase our efficiency, reducing the time we were spending, you know, going from like three hours running our regression, I mean, no three days running regression to like three hours, you know, so that was a huge efficiency. So from that moment I got in, I was in love with AI. I've been involved with AI from graduate school way back then, you know. And I've always been a fan of understanding how we can use AI based test to help, you know, key way. So that is something that I'm passionate about. I'm actually starting a course in few weeks in February, in two weeks actually just teaching quality and then artificial intelligence because I feel like there's a good correlation. I just Have a lot of challenge right now in my current organization before we can introduce that because we need to get a strong baseline, you know, into how we automating things before we get to that next layer.

I need to get you back on the show dedicated just to that. So I guess at a high level then what should someone as a tester know, need to know about AI? Is it just they need to know, like, do they really need again the guts of it? Like, like high level. If there was a syllabus, like what would they need to know about AI in order to be successful with it in testing, if anything?

Yeah, for me, I feel like depending on the level. Right, because you asked the right question. How deep do you want to get into it? The team that I was and I do talk about that in my talk. Right. It's all depend. You don't want to get deep like me or deep like my friend Tariq King, you know, level. But you know, some of us are just geeking out about it a lot more. But depending on your interest, just I. My talk is usually about be aware that it can help you, you know. And I have refined my talk from 2019 to even now 2023, because back in 20, I'll say 19, people were still a little bit reluctant. When you say AI and then you say qa, right, Because Q is such a role where it's already fragile. And we know that there's a lot of layoff happening, you know, in 2023 and QA is already a role that is fragile. So when you bring AI into a Q organization, people are wondering like, are you trying to reduce the cost? Are you trying to reduce people? And so my talk was always around, use it as something that's going to help you, you know. And now 2023, we have chat, the GPD out there, right? And I see you talking about it. So it's happening. We've been telling you guys it's gonna happen since maybe 2018, 2023. It happened. It has happened. So in 2023, how can you leverage AI in any of your role? Not even QA anymore. Everybody needs to look at how they can make their job better and it's not replacing you. We've already had that conversation so many times now is. And I actually saw something funny last night. I think 5 Serve Company, they wrote an open letter to AI and say, let's work together from a human perspective and the AI, right? So that's the conversation. Let's see how we can use AI and the three low Hanging fruit that I knew that it was giving you. There's so many tools. Back in 2016 there was only one or two vendors that was like barely starting with it. Right now when you go to like a Star west conference, everybody maybe last fall or even two falls, they go now have like AI AI in their tool now. What level are they giving you? We don't know. You have to do your homework. In my presentation, I tell them what they need to ask based on what they need, you know. But for me the visual testing is great. You know, it can do visual testing for you. My favorite one is self healing because I've suffered when I was a QA automation engineer coming back in the middle of the night or the next day just to fix my code to figure out why the test failed. Spend the whole afternoon trying to figure that out. So the self healing is something that I personally love. And the next one that I really want to even understand a little bit more about it is the test creation part. You know, because we pushing the boundary. Let's see how far we can go in our strategy, right? So if we can create the test for us now and we can refine it with our human eyes or with our human brain based on the requirement, why not Anything we can do to just be efficient so that as a tester, as a QA person, you can actually do the job that you were hired to do, which is, you know, bring quality to the organization so you can spend time in those meeting and, and cross pollinate, you know, and do those integration that's do the boundary test or do the test that nobody think about. You know, those are the what AI really help you for.

I love it. And so I assume you've been using ChatGPT a little bit.

I'm, I'm, I was skeptical. I'm like yeah, I'm a skeptical person. So yeah, a little bit, A little bit. I'm making my daughter try just play with it. You know, for me I look at it as a bigger Google really. And there's a book that I read long, long time ago. I forgot the title but it's a sci fi book. I was not a fan of the book, but the book really mentioned a lot about this. So for me I'm like we are there and was in College about 25 years ago and we were talking about everything that AI can do and that what hooked me into AI, right. And I feel like now, 25 years later we are finally there. You know, as of January 2023, we really close you know, so I'm really looking forward to the opportunity. I just don't want. I feel like people hype it a little bit. Way too much, you know, and some of us that are really close to we, we can look at it and see what it can help us do. But I think we are. The marketing is a little bit louder than the actual, you know, what it can do. That's my personal opinion right now. I know you have like I saw and I did sign up for it. I was like, I want to hit hear what this guy has to say because I have not spent time to actually do the due diligence and see how people can use it. But I'm curious.

Yeah, it's pretty wild. I mean from what I've seen, it's not even, it's only early version. I heard they're coming up with even a more powerful version coming out in a few months. I was pretty impressed what it can do. And like you said, it's almost like a better Google.

Yes.

And it could do all the things like you mentioned. It could write like a selenium test for you, it could do all these things for you. But like you said, it's not going to replace you because you still need to be able to look at it and say, you know, review it, review it, modify it. Right. So it's just going to help you, I think, do your job better and.

Quicker but be more efficient at your job.

Yes, exactly. Yes. I love how you saying don't be afraid of it. You need to embrace it, learn it. Because it is coming. It is here.

It is here.

It is here. Chat GPT. Right. So it's like, I don't know, I was really blown away. I know a lot of people are like, oh, it's nothing, it's a toy. But I just really see it as really powerful. Getting even more powerful. For sure.

Yeah. Even where I've been paying more attention interesting. Has been marketing. You know, for some reason I have a couple of people in marketing in my feed and they've been like talking about it really aggressively. You know, I'm like, okay, I want to know and everybody should do that exercise. Because that's what got me interested in AI and qa. Because I knew automation was something that we have to. It's a problem to solve. Right. In Kiwi, it has been a problem for the last decade for most team that I've been involved in. So for me it was in the quest of finding solution. That's how AI was one of the solution in My toolbox. If there's another one, I'll use it. If it's chat gpd, I'll use it. You know, anything that I can do to solve for the automation challenge that my team is facing, I'm all for it.

I love it. I love your attitude. Awesome. Before we go, is there one piece of actual advice you can give to someone to help them with their testing efforts and what's the best way to find or contact you?

Oh, good advice. I think you just stay curious. That's what I always say, right? Stay curious. And that start with like when you go into requirement meeting and you listening to them, don't just think, oh, I'm just a QA person so I don't care about the requirement. Yes you do. You know, be curious and ask all the right questions because the best QA folks are actually SME for the product and I've been one in the past. Right? You are like the go to person. I've been in team where there's a new person join and they send it to me and then I train the person based on my test cases because I'm close to everything back to that cross pollination. I'm talking to everybody in the organization, all the different components. So it's the curiosity that drive you to get to that level of detail and understanding. So stay curious. When you're talking to the developers, stay curious as well. Ask them about the code, why are they making this decision? Because when you testing, you need to understand why those decisions are being made in the algorithm or the way they implement it. So stay curious for me is like a golden rule, right? And that's actually the reason why I love my job. Because the curiosity make you want to continue to learn and where to contact me. I think LinkedIn is the best way. I am not a Twitter person. I start to stay away from Twitter. But I think LinkedIn is the best way.

Thanks again for your automation awesomeness. For links to everything of value we covered in this episode, head on over to testguild.com a436 and while you're there, make sure to click on the try for free today link under the exclusive sponsor section to learn all about Sauce Lab's awesome products and services. And if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers.

Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the the world, we'd love to have you join the fam@test guild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the guild community, I'd love to hear from you. Head on over to test guild.info and let's make it happen.