https://www.youtube.com/watch?v=-IRSb4FRlio

Hey, I'm hearing a lot from different testers wondering how do they get real analysis and clear insights around AI and testing, security, performance, accessibility, all the things. And if that's you, you're in for a special treat because we have Matt Gilbert joining us. You don't know. Matt is a versatile professional who's been an impact across many different industries. I think he's worked like, in insurance and startups and SaaS and healthcare. He's also the founder of Clear Insight, a web testing platform focused on usability and performance and beyond building tools. Matt is a prolific blogger which has insights on everything from testing fundamentals to leadership, team dynamics and the role of AI in testing. He's passionate about turning complex testing ideas into practical, actionable steps that anyone can apply, that you'll hear right here on this show as well. You want to miss it? Check it out. Hey, Matlock to the killed.

Hey, Joe, thanks for having me. It's great to be here.

Awesome to have you. So you have quite a background. I'm always curious to know, especially with founders. I guess the first thing is how did you get into testing?

Yeah, so very early on in my career I knew. Well, I thought at the time that I wanted to be a developer. I enjoyed building things, seeing how things worked, and I still do every day. But just out of college, I heard about software testing and I felt like it was a good stepping stone to get into software development eventually. And so I took a job. It was very early on, my first professional job, I would say. And that's where I kind of got my feet wet with software testing and learned what it was and maybe, maybe picked up a few bad habits along the way. But I'm. I'm here now and better for it, so loving every, every bit of it.

Cool. Cool. So speaking up, talking about picking up bad habits, what made you think, why to start a company? I know you started Clear Insights. What is Clear Insights? Why did you start it? What was your, your reason for that?

Yeah, so my motivation for starting Clear Insight was really just seeing that there was all these tools out there that were kind of disjointed and not really easily accessible for people. You had to know about them to be able to use them. And some of them are a little bit more difficult to implement into your, like, pipelines and stuff like that. The output can be kind of complex, like the JSON structures it's providing you. You kind of have to know what you're looking for and then translating those results into actionable insights for you or your team to look at and implement on your end to resolve some of those issues that you're saying with like accessibility and performance and security I felt was another area that was a little bit lacking. So really just taking tools and combining them together into a cohesive platform that allows non technical people to understand the data behind the validations that are happening and then if they want to take action on those, they can. If they want to pass off to somebody else and decipher them, they can as well.

Nice. So why do you think it's so hard to get these tools to work with one another? Is it because they're all open source? Is it because a lot of companies just don't have the expertise to implement a lot of these different other types of testing that we probably don't even think about?

I think it's a little bit of both for accessibility testing especially. That's not something that's very commonplace nowadays and it's very concerning considering that, you know, that's a legal requirement in some aspects. So if, if you're in like healthcare, like I am, our, our space is very regulated and we have to ensure that we're making our websites and our tooling accessible for a lot of different people of different varying backgrounds. And performance is one of those things that you don't really notice until somebody brings it up to you. Typically at that, at that point it's users, right? So people like me who are using the site and giving you feedback and saying, hey, this is slow or you know, why is this happening? And at that point it's kind of too late. So.

Absolutely. You've worked in many different industries. Do you think a lot of people just forget about security and accessible and performance because maybe it's not risk area for them or maybe they don't know it's a risk area for them?

Yeah, I think people probably don't realize how impactful it is to users. And honestly, their business security being, obviously if you're hacked, like that's really bad, you're going to lose customers, you're going to lose money, are you going to lose data, you're going to have potentially have some fines associated with that. And with performance and accessibility, it's just not something you think about every day. You know, as you're visiting a website, as an end user yourself, you might think about it. But as you're building an application or a website or anything like that, it's not the first thing that comes to mind. The first thing that comes to mind for a lot of People is, does this do what I want it to do? Not does it perform well and is it secure and is it accessible and is it usable and all those things.

So why is that? So I guess even if there is a solution, how does someone know that that's a problem, if that makes sense. So if someone's listening, like they may not even know, like, oh, I need to worry about security, accessibility, how do you get over that hump of not only is an issue, but like, here's some tools or here's a solution that.

Can help you with it.

Yeah, it's, it's really just education. Um, and so with clear insight, that's kind of the path that I took for that is what does performance look like one for your website or your webpage and what do these values mean? So like you and I know that, you know, fcp, lcp, tls, all those values are impactful to users, but really getting granular and saying, hey, um, ed education and really deciphering those values and letting business users know that X value is bad because Y. Right. So it's, it's really about education.

For some reason. I get a lot of slack for some gurus out there when I say a tester should be able to do security testing, some high level performance testing, like, oh no, God forbid, I'm a performance engineering professional and therefore only I shall do performance. I don't know if you get any pushback like that. Do you think like people say, oh, you need to stay in your lane, you're a functional tester focused on that. Don't worry about all these other aspects.

Yeah, it's something I have heard from other people. I've been fortunate enough in my career that, you know, my manager and director, especially at my current job, job is they're very appreciative and they want us to continue learning and growing. And if that's, you know, learning about security testing and helping augment the team that we already have, that does that as well as, you know, teaching others how to do that, that's something that everyone's always encouraged me to do. But I have heard that being a case at other companies or other individuals who have been kind of stifled into a particular role or forced to, you know, do manual testing all the time. And it's like you have, you have to expose yourself to different areas and learn to grow, otherwise you're going to be getting pretty stagnant in your career.

100% agree. I guess another, I think maybe, maybe I'm wrong. Maybe another hurdle. Some testers go over is a lot of the information probably return is really technical. And so how do they wrap up that technical feedback to make it so that management can make good judgments on or make better outcomes on the information?

So the information that the testers are providing to business or the information that they're getting from maybe the system under test.

Well, so they're getting from the system, from test, they may be like, ah, I have no idea what this is, number one. And if it's so once I understand it, how do I explain it to my manager?

Gotcha. Yeah, it's going to come from experience and just doing it. When you're learning about, when you're testing a system and you're getting data back from it, that's where you have to kind of dig into it a little bit and do a little bit of your own research to understand what the data is and where it's coming from and why it's coming from that place and all the interdependencies that go along with that. And then when it comes to explaining that to the business, a few things that I found helpful are leveraging AI for that. So I know you've had a few people on your podcast talking about AI in the past month or few months or year. Everybody has a different perspective on it. But my thoughts are it's going to help augment and already has helped augment a lot of different roles. I don't think it's to the point where it's going to replace anybody, but it is very good at basically text generation. So if you, if you give it a clump of text and say, hey, analyze this or explain this in a, in a way that makes sense to X, Y and Z users who might not be technical. I think that would be a good starting point for from there. And then yourself as a testing professional can then look at that output and if you agree with it, you know, continue. If not, then you can tweak it a little bit. So I think it's a, it's a good tool in that aspect.

Yeah, no, I like how you mentioned it's not like all knowing it puts output out, but it's up to you to look at it and put in your expertise and marketplace.

Exactly. Yeah.

So I guess, you know, you are in healthcare, do you get any pushback like how you can't use AI because we're regulated or anything like that?

Yeah, there's a few areas that obviously we're not supposed to use AI for. And a lot of the AI tooling that we're using now, all that is protected, it's all, not, none of it is being used to train any public models. So any of the AIs that are in place now are, I don't know the correct term for it, but none of, none of the data that we're passing it is being used to train any public models. And then going a little bit further, there are guardrails put in place. So you know, if, if patient data were to get in there, it's, there are guardrails you can set up to not allow that to happen. That way you're only passing in data to the model that you know is not going to get you fined basically.

All right, so I know once again, going back to performance, sometimes when testers get involved, functional testers, it's not too late but like the architecture is already in place. Decisions that have been made to make it as performant, it's going to be how do you make it so it's more shift left, let's say.

Yeah, so you want to provide that feedback as early as possible. Ideally, you also want to educate your developers on what good performance is and why it matters. I've been fortunate enough to work hand in hand and collaborative collaboratively with a lot of developers who, you know, want to do the right thing the first time rather than just, you know, throw something over the wall. And hopefully I won't, you know, complain about it or send it back or whatever. So a lot of those things, it's just going to come with education, working with developers and that's where, you know, a relationship and being able to talk their language is super important. So if they respect you and you have that mutual respect, then you know, they're going to be more receptive to actually, you know, taking your feedback and applying it.

So being, you know, being used to talking to leaders and having developers are pretty open minded. There's anything you can do or you can suggest how people can create more of a culture being more open minded to testing, just not just functional testing but all types of quality.

Yeah, you're, I mean it really starts at the top, but then it also starts with people like me and showcasing the value that we provide every single day. So that's going to be the collaboration that we have with product when it comes to requirements analysis. It's going to be collaborating with developers, making sure that they have the tooling they need to perform their testing well. So you talked about shifting, shifting test left earlier and that's kind of what my role now is a lot of days that's what I'm doing is building tooling out for developers to actually help them perform testing and doing it well before it even gets to me. And then it's just, it's really, it comes down to education. So the things that I'm doing every single day to you know, improve, help improve requirements, help improve user stories, all of that trickles down to the end result being, you know, a releasable product that users want to use and performs well. And I think, you know, hopefully business users see that side of it. It's, it's really developer focused right now. So a lot of pre existing tools that are less UI based to help get data into the system, data creation and then some, some UI based tools as well just to visualize certain aspects of the system and query and search for things that you're looking for without jumping through 10 to 12 different hoops to get there.

So Matt, I guess another thing I see test is getting caught up on is they have like almost like guardrails on that they put on for themselves. Like I can't do this, I can't do that. It seems like you have your hand in everything and I assume when you get hired they're like, oh, you're going to be able to write tools for your developers and do this and that. I assume you just started doing it. Like how did you be able to do cool stuff and not be so pigeonholed? I don't know if that makes sense.

Right, yeah, no, like I said earlier, I've been, I've been really lucky at my current job to have that bandwidth to actually go and do those cool things. Like I said earlier, I enjoy building things, I enjoy coding. It's something I like doing and I think you just have to do it. There's something we do at my current job where it's called Passion Project Fridays and we do that once a month. And it just started out, you know, as a small project. Every Friday, once a month I would start iterating on it and you know, eventually it grew beyond just like a simple poc. This particular project was grabbing data from our repositories for our user stories. So we use Azure DevOps for that. We store our user stories in there, our acceptance criteria, all the descriptions, all of the tasks. This particular tool was basically a way for testers to get that data into a model and then that model would then generate some test scenarios for that and then spit it out into an Excel file with those scenarios outlined in it. And then from there we could then, you know, tweak those, modify those, delete those, add to those scenarios and saves us, you know, 8 to 12 hours a week on generating those. And those are test charters. I don't know if you're familiar with those, but it's, it's less like test scripts, you know, the do this and then click this button. It's more high level, like okay, as a, you know, X user doing Y thing, what happens? And so that's kind of where it originated. And then from there it's just talking to the right people and showing them that, you know, that's something that you actually are passionate about and you care about and you know, it does add value in the end. And that's kind of, you know, where I'm at today.

Nice. So I know a lot of people just doing functional testing, flaky tests, takes up a lot of their time. I could imagine if you're sad, you know, like security scanners, accessibility, performance, you have all these results, like how do you know? Like how do you prioritize? Is that something you could see? Like, okay, it's great now you've been doing all the things, but now you have to deal with all the things. Right?

Right. Well, I'm sure, you know, there's, there's no limit to the number of tests that we could run. Testing is endless. It's up to us to kind of decide what tests we are going to run and the things that we're going to check. It's context based. Right. So depending on the system, depending on the timeline, depending on how much time I have, that's going to all be dependent on, you know, the, the scenarios that I hit. It's going to be risk based testing. It's going to be, you know, performance maybe if I have, if I have the bandwidth. Unless it's a, you know, customer facing web application that, you know, has SLAs that need to be met, then obviously performance is going to be up there. But again that's contextual. Right. So it really comes down to the tester needing to understand all of those details and apply it towards their, the testing flow.

So you know, for a lot of people it's a lot to ask to get them to understand how to do this. Getting up and running, I assume, like why clear insight then it seems like you have a kind of unique model where you don't have to invest billions of dollars just to get a result to see if it's worthwhile. Maybe you could tell us a bit.

More about that I can show. So the only thing I couldn't get working was the functional test generation. Do you want me to show the accessibility?

Oh, I mean that'd be even better because I think most people know the functional testing.

All right, can you see my screen? Okay, so I know we were talking earlier about, you know, potentially making this different subscriptions and allowing people to log in and run their own tests. The reason I didn't do that was this is all running locally. As you can see, I have my own APIs that are set up and I don't have to pay for huge AWS bills for being able to host all this infrastructure. It's just running in a Docker container. So pretty simple. I can go out here. I already have my myself selected as a client. Ideally I would have more out there, but you know, times are tough. So go out here, choose my category. I'm going to do accessibility and then I'm only going to run it on one page. Normally if I don't have this selected or if I have this selected, it's going to go through a bunch of different sub pages and perform the checks on those as well. It should be done soon. So right now it's spinning up a browser and playwright, it's navigating to the web page and it's running the validations on that web page. So completed, get HTML reports and let me actually reshare. So this is the report it just generated. July 16, 3:29pm gives you a high level view of the past evaluations and failed evaluations. The raw analysis is really cool to look at. You can see like what type of coverage. So this is using axe, if you're familiar with that. So on on the web page it looked at 367 elements and it found one violation. So yay me. I can say I'm an accessibility. My accessibility is good, at least on this page. Right, so where the top. Oh, down here. Yeah. So this links you just to the AXE documentation for the particular rule that was failing. So we have the different rules here. So this is the region rule and then we have the different category types. Yeah. So again, just going back to the education aspect of it, hopefully this is useful for end users. The performance and security is set up similarly and then the functional test generation, which I'm not able to demo right now, there were some AWS errors I was getting, but those are going to be basically all packaged up into a zip file and it'll give you a readme along with the package JSON. So A user will be able to download that, run, do an install and then be able to kick off the test in a matter of, you know, a minute. So.

How, how often would someone use this, need this, do you think? Is it every release? Is it every quarter? Is it every build?

Yeah. So for accessibility and security and honestly, performance testing, it's probably not something you're going to want to be doing every week. I would say it's going to be, it's again, contextual. So it's as often as you need it. Right. Running these accessibility checks at least gives you a good baseline on the state of your application as it is today. And then if you want to iterate on that, um, and then check it again, you know, a few months down the road just to make sure you haven't regressed any. I would say that's, that's reasonable to do, but it's not something that you're going to want to be doing every month, um, maybe even every other month.

Okay, Matt, before we go, there's a one piece of actual advice you can give to someone to help them with their testing efforts. And what's the best way to find contact you and learn more about Clear Insight.

Yeah, I would say if you're just getting started out there testing, or if you've been testing for a while, like me, just explore and create and experiment. Try different tools and see what you like and don't give up. And then as far as getting in touch with me, I'm on LinkedIn, I'm blogging on Substack and you can visit Clear Insight and that's I N S I T E Read links to all.

This awesomeness in the comments down below.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com a555 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them.

So that's it for this episode of.

The Test Guild Automation Podcast. I'm Joe. My mission is help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good.

Cheers.

Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation testing and DevOps. If you haven't yet, join our vibrant community@testguild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit test guild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve. With hoots and lyres, the bards began their song A tune of knowledge, a melody of code through the air it's spread like wildfire through the land Guiding tester showing the secrets to behold.