How to Build a Culture of Comprehensive Quality with Matt Gilbert

In a land of testers far and widely journey seeking answers, seeking skills, seeking a better way O the Test Illumination Testing Podcast Guiding testers with our nation awesomeness from ancient realms to modern days, they lead the way. Oh, the Testicle Automation Testing Podcast.

Hey, I'm hearing a lot from different testers wondering how do they get real analysis and clear insights around AI and testing, security, performance, accessibility, all the things. And if that's you, you're in for a special treat because we have Matt Gilbert joining us. You don't know. Matt is a versatile professional who's been an impact across many different industries. I think he's worked like in insurance and startups and SaaS and healthcare. He's also the founder of Clear Insight, a web testing platform focused on usability and performance and beyond building tools. Matt is a prolific blogger which has insights on everything from testing fundamentals to leadership, team dynamics and the role of AI in testing. He's passionate about turning complex testing ideas into practical, actionable steps that anyone can apply. That you'll hear already on this show as well. You want to miss it, check it out. Hey, Matt, welcome to the Guild.

Hey Joe, thanks for having me. It's great to be here.

Awesome to have you. So you have quite a background. I'm always curious to know, especially with founders. I guess the first thing is how did you get into testing?

Yeah, so very early on in my career I knew. Well, I thought at the time that I wanted to be a developer. I enjoyed building things, seeing how things worked, and I still do every day. But just out of college, I heard about software testing and I felt like it was a good stepping stone to get into software development eventually. And so I took a job. It was very early on, my first professional job, I would say. And that's where I kind of got my feet wet with software testing and learned what it was and maybe picked up a few bad habits along the way. But I'm here now and better for it, so love every bit of it.

Nice. So why do you think it's so hard to get these tools to work with one another? Is it because they're open source? Is it because a lot of companies just don't have the expertise to implement a lot of these different other types of testing that you probably don't even think about?

I think it's a little bit of both. For accessibility testing especially. That's not something that's very commonplace nowadays, and it's very concerning considering that, you know, that's a legal requirement in some aspects. So during like healthcare, like I am. Our, our space is very regulated and we have to ensure that we're making our websites and our tooling accessible for a lot of different people of different varying backgrounds. And performance is one of those things that you don't really notice until somebody brings it up to you is typically at that, at that point it's user. It's users. Right?

So.

So people like me who are using the site and giving you feedback and saying, hey, this is slow, or, you know, why is this happening? And at that point it's kind of too late. So.

Absolutely. Amit, you've worked in many different industries. Do you think a lot of people just forget about security and accessibility and performance because maybe it's not a risk area for them or maybe they don't know it's a risk area for them.

Yeah, I think people probably don't realize how impactful it is to users. And honestly, their business security being, obviously if you're hacked, like that's really bad, you're going to lose customers, you're going to lose money, are you going to lose data, you're going to have potentially have some fines associated with that. And with performance and accessibility, it's just not something you think about every day as you're visiting a website. As an end user yourself, you might think about it, but as you're building an application or a website or anything like that, it's not the first thing that comes to mind. The first thing that comes to mind for a lot of people is does this do what I want it to do? Not does it perform well and is it secure and is it accessible and is it usable and all those things.

So why is that? So I guess even if there is a solution, how does someone know that that's a problem, if that makes sense. So if someone's listening, like, they may not even know, like, oh, I need to worry about security, accessibility, how do you get over that hump of not only is an issue, but like, here's some tools.

Yeah, there's really just education. And so with clear insight, that's kind of the path that I took for that is what does performance look like one for your website or your webpage and what do these values mean? So like you and I know that, you know, fcp, lcp, tls, all those values are impactful to users, but really getting granular and saying, hey, education and really deciphering those values and letting business users know that X value is bad because Y. Right. So it's really about education for some reason.

I get a lot of slack for some gurus out there. When I say a tester should be able to do security testing, some high level performance testing, like, oh, no, God forbid, I'm a performance engineering professional and therefore on the I shall do performance. I don't know if you get any pushback like that. Do you think, like, it's something I say, oh, you need to stay in your lane. You're a functional tester.

I've been fortunate enough in my career aspects that, you know, my manager and director, especially at my, my current job is they're very appreciative and they want us to continue learning and growing. And if that's, you know, learning about security testing and helping augment the team that we already have, that does that as well as teaching others how to do that. That's something that everyone's always encouraged me to do. But I have heard that being a case at other companies or other individuals who have been kind of stifled into a particular role or forced to, you know, do manual testing all the time, you have to expose yourself to different areas and learn to grow. Otherwise you're going to be getting pretty stagnant in your career.

Yeah, a hundred percent agree. I guess another, I think maybe, maybe I'm. Or maybe another hurdle some testers go over is a lot of the information probably return is really technical. And so how do they wrap up that technical feedback to make it so that management can make good judgments on it or make better outcomes on the information? Well, so again, from the system, from test, they may be like, ah, I have no idea what this is, number one. And number two, once I understand it, how do I explain it to my manager?

So the information that the testers are providing to business or the information that they're getting from maybe the system under test. Okay, gotcha. Yeah, it's going to come from experience and just doing it. When you're learning about, when you're testing a system and you're getting data back from it, that's where you have to kind of dig into it a little bit and do a little bit of your own research to understand what the data is and where it's coming from and why it's coming from that place and all the interdependencies that go along with that. And then when it comes to playing that to the business, a few things that I found helpful are leveraging AI for that. So I know you've had a few people on your podcast talking about AI in the past, you know, month or few months or year. Everybody has a different perspective on it. But my thoughts are it's going to help augment and already has helped augment a lot of different roles. I don't think it's to the point where it's going to replace anybody, but it is very good at basically text generation. So if you give it a clump of text and say, hey, analyze this or explain this in a, in a way that makes sense to X, Y and Z users who might not be technical, I think that would be a good starting point from there and then yourself as a testing professional can then look at that output and if you agree with it, you know, continue. If not, then you can tweak it a little bit. So I think it's a, it's a good tool in that aspect.

Yeah, no, I like how you mentioned it's not like all knowing puts output out and, but it's up to you to look at it and put in your expertise and work with it.

Almost exactly. Yeah.

So I guess you are in healthcare, do you get any pushback like how you can't use AI because or regulated or anything like that?

Yeah, there's a few areas that obviously we're not supposed to use AI for and a lot of the AI tooling that we're using now, all of that is protected. It's all not. None of it is being used to train any public models. So any of the AIs that are in place now are. I don't know the correct term for it, but none of, none of the data that we're passing it is being used to train any public models. And then going a little bit further, there are guardrails put in place. So, you know, if patient data were to get in there, it's. There are guardrails you can set up to not allow that to happen. That way you're only passing in data to the model that you know is not going to get you fined, basically. Oh, that would be the infrastructure team usually. Yeah.

Who put in those guardrails. Just curious. All right, so I know once again, going back to performance, sometimes when testers get involved, functional testers, it's not too late but like the architecture is already in place. Decisions that are even new to make it as performant, it's going to be. How do you make it so it's more shift up, let's say.

Yeah. So you want to provide that feedback as early as possible. Ideally, you also want to educate your developers on what good performance is and why it matters. I've been fortunate enough to work hand in hand and collaboratively with a lot of developers who, you know, want to do the right thing the first time rather than just, you know, throw something over the wall and hopefully I won't complain about it or send it back or whatever. So a lot of those things, it's just going to come with education, working with developers and that's where a relationship and being able to talk their language is super important. So if they respect you and you have that mutual respect, then they're going to be more receptive to actually taking your feedback and applying it.

Seems like your developers are pretty open minded already, but do you know any ways maybe to encourage leaders to create a culture that's more open maybe to quality?

Yeah, it really starts with the people at the top, but then it also is up to people like me and the people that are on my team and if I can, you know, explain in a way that is useful to business users like we were talking about earlier, I'm gonna, I'm gonna restart that. Can you repeat the question? I'm sorry.

Sure. Eric Edit. Eric's me editor. Eric Edit. Edit, Edit. Okay, go. So being, you know, being used to talking to leaders and having developers are pretty open minded, there's anything you can do or you suggest how people can create more of a culture being more open minded to testing, not just functional testing, but all types of quality.

Yeah, it really starts at the top, but then it also starts with people like me and showcasing the value that we provide every single day. So that's going to be the collaboration that we have with product comes to requirements analysis. It's going to be collaborating with developers, making sure that they have the tooling they need to perform their testing well. So you talked about shifting test left earlier. That's kind of what my role now is. A lot of days that's what I'm doing is building tooling out for developers to actually help them perform testing and doing it well before it even gets to me. And then it's really, it comes down to education. So the things that I'm doing every single day to, you know, improve, help improve requirements, help improve user stories, all of that trickles down to the end result being, you know, a releasable product that users want to use and performs well. And I think, you know, hopefully business users see that side of it. It's really developer focused right now. So a lot of pre existing tools that are less UI based to help get data into the system, data creation and then some UI based tools as well just to Visualize certain aspects of the system and query and search for things that you're looking for without jumping through 10 to 12 different hoops to get there.

So I guess another thing I see testers getting caught up on is they have like, almost like God rails on that they put on for themselves. Like, I can't do this, I can't do that. It seems like you have your hand in everything. I assume when you get hired they're like, oh, you, you're going to be able to write tools for your developers and do this and that. I assume you just started doing it. Like, how did you be able to do cool stuff and not be so pigeonholed? I don't know if that makes sense.

No, like I said earlier, I've been really lucky at my current job to have that bandwidth to actually go and do those cool things. Like I said earlier, I enjoy building things, I enjoy coding. It's something I like doing and I think you just have to do it. There's something we do at my current job where it's called Passion Project Fridays and we do that once a month. And it just started out, you know, as a small project. Every Friday, once a month, I would start iterating on it and you know, eventually it grew beyond just like a simple poc. This particular project was grabbing data from our repositories for our user stories that we use Azure DevOps for that. Do we store our user stories in there, our acceptance criteria, all the descriptions, all of the tasks. This particular tool was basically a way for testers to get that data into a model. And then that model would then generate some test scenarios for that and then spit it out into an Excel file with those scenarios outlined in it. And then from there we could then, you know, tweak those, modify those, delete those, add to those scenarios and saves us 8 to 12 hours a week on generating those. And those are test charters. I don't know if you're familiar with those, but it's less like test scripts, you know, the do this and then click this button. It's more high level, like, okay, as a, you know, X user doing why thing, what happens? And so that's kind of where it originated. And then from there it's just talking to the right people and showing them that, you know, that's something that you actually are passionate about and you care about and does add value in the end. And that's kind of where I'm at today.

Nice. So I know a lot of people just doing functional testing, flaky tests takes up a lot of that time. I could imagine if you're sad and like security, scanners, accessibility, performance that you have all these results, like how do you know? Like how do you prioritize that? Is that something you could see? Like, okay, it's great now you've been doing all the things, but now you have to deal with all the things.

Right. I'm sure, you know, there's no limit to the number of tests that we could run. Testing is endless. It's up to us to kind of decide what tests we are going to run and things that we're going to check. It's context based, right? So depending on the system, depending on the timeline, depending on how much time I have, that's going to all be dependent on the scenarios that I hit. It's risk based testing, it's going to be performance. Maybe if I have the bandwidth. Unless it's a, you know, customer facing web application that, you know, has SLAs that need to be met, then obviously performance is going to be up there. But again that's contextual. So it really comes down to the tester needing to understand all of those details and apply it towards their testing flow.

For a lot of people it's a lot to ask to get them to understand how to do this. Getting up and running, I assume. Like why Clari Insight then? It seems like you have a kind of unique model where you don't have to invest billions of dollars just to get a result to see if it's worthwhile. Maybe you could tell us a bit more about that.

Yeah, I was hoping to do a live demo today and then I was playing with it beforehand and it didn't go so well. So yeah, can actually show what it's doing.

When you do get them running, you can do a quick video and send it my way and I'll add it into the podcast. It will say, all right, like check out the video. So let's check out a quick demo.

Do you want me to show the accessibility or performance? Let me see if I can share actually.

Oh, I mean that'd be even better because I think most people know the functional testing.

So I know we were talking earlier about, you know, potentially making this different subscriptions and allowing people to log in and run their own tests. The reason I didn't do that was this is all running locally as you can see. I have my own APIs that are set up and I don't have to pay for huge AWS bills or being able to host all this infrastructure, it's just running in a Docker container. So pretty simple, I can go out here. I already have myself selected as a client. Ideally I would have more out there, but you know, times are tough. So go out here, choose my category, I'm going to do accessibility and then I'm only going to run it on one page. Normally, if I don't have this selected or if I have this selected, it's going to go through a bunch of different subpages and perform the checks on those as well. So right now it's spinning up a browser in Playwright, it's navigating to the web page and it's running the validations on that web page. So completed, get HTML reports and let me actually reshare. So this is the report it just generated. July 16, 3:29pm gives you a high level view of the past evaluations and failed evaluations. The raw analysis is really cool to look at. You can see like what type of coverage. So this is using Axe, if you're familiar with that. So on the web page it looked at 367 elements and it found one violation. So yay me, I can. My accessibility is good, at least on this page. Right.

So it's a fixed guide. I like that. Sort of. Some of me now understand it.

Where the top, the very bottom. Oh, down here, yeah. So this links you just to the AXE documentation for the particular rule that was failing. So we have the different rules here. So this is the region rule and then we have the different category types. So again, just going back to the education aspect of it. Hopefully this is useful for end users. The performance and security is set up similarly. And then the functional test generation, which I'm not able to demo right now, there were some AWS errors I was getting, but those are going to be basically all packaged up into a zip file and it'll give you a readme along with a package JSON. So a user will be able to download that, do an install and then be able to kick off the test in a matter of, you know, a minute.

How often would someone use this, need this, do you think? Is it every release? Is it every order? Is it every build?

Yeah. So for accessibility and security and honestly, performance testing, it's probably not something you're going to want to be doing every week. I would say it's going to be. It's again, contextual as often as you need it. Right. Running these accessibility checks, it gives a good baseline on the state of your application as it is today. And then if you want to iterate on that, check it again a few months down the road just to make sure you haven't regressed any. I would say that's reasonable to do, but it's not something you're going to want to be doing every month. Maybe even every other month.

Nice. Okay, Matt, before you go, is there one piece of actual advice you can give to someone to help them with their testing efforts? And what's the best way to find contact you and learn more about Clear Insight?

Yeah, I would say if you're just getting started out there testing, or if you've been testing for a while like me, just explore and create and experiment. Try different tools and see what you like and don't give up. And then as far as getting in touch with me, I'm on LinkedIn blogging on substack and you can visit clearinsight IO and that's I N S I.

T E. Write links to all this awesomeness in the comments down below. Thanks again for your automation awesomeness, the links to everything of value we covered in this episode. Head on over to testguild.com a555 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation testing and DevOps. If you haven't yet, join our vibrant community at Testing, where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges. We're excited to collaborate visit testguild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we.

Can achieve O the TESCAL Automation Testing Podcast with lutes and lyres the bards began their song A tune of knowledge, a melody of code through the air it spread like wildfire through the land Guiding testers showing us secrets to behold.