The Importance of Trust in Software Testing and AI with Damian Synadinos

In a land of testers far and widely journey, seeking answers, seeking skills, seeking a better way O the Test Skill Automation Testing Podcast Guiding testers with our nation awesomeness. From ancient realms to modern days, they lead the way. Oh, the Testicularation Testing Podcast.

What does trust have to do with software testing and AI? Well, that's what we're going to be talking about all today in this episode with Damien. If you don't know, Damian has over 30 years of experience. He does all the things he does with testing and does a lot of speaking, a lot of conferences, a lot of keynotes, a lot of talks, a lot of training. You probably know him, so I'm really excited to have him. Also, he's really helped in the past organize QA or the highway, which has become one of my favorite testing conferences in the real world at qa on the highway.com you definitely should check out and he, like I says, well, I don't know if you know this, he actually does a lot of improv and comedy. He's also an author and illustrator. He does all the things. So really excited to have him back on the show. If you want to know more about trust, what the heck is it? What does it have to do with AI and testing? Like I said, you don't want to miss this episode. Check it out. You probably heard the old saying seeing is believing. And that's why I think this goes a long way towards building trust that we talk about in this episode. And that's why I'm excited about this week's sponsor. Look Visual Bugs Bugs don't just look bad, they kill retention, trust and revenue. 88% of users won't return after a bad digital experience and 75% judge your brand by a UI. And 35% of your revenue is lost due to poor UX manual. Visual testing is tedious and often prone to errors. Open source tools are outdated and require high maintenance. So what should you do? That's where Browser Stack Percy comes in. It makes visual regression testing for websites and complex workflows effortless even at scale. Think of it, hundreds of screenshots being compared at once. Also, it requires minimal code changes to set up. It offers seamless indirect integrations with Your favorite tools over 50 + included Figma and Storybook. It also has this really cool feature called Visual Scanner that helps you monitor thousands of pages across real devices and browsers in under a minute. What's more interesting is their Visual Review Agent, which is powered by AI that helps cut through the noise and false positives Reducing manual review times by 3x with Percy. Teams like LinkedIn, IBM, Adobe, etc. Are already shipping faster with full UI confidence. But as I said at the beginning, seeing is believing. So try it out for yourself using that special link down below. Hey, Damien, welcome back to the Guild.

Hey Joe, thanks for having me. It's me, it's you. Together. All right. Good to see you.

That's finally again. It's been nine years since last time we talked.

Too long.

Too long on the podcast. Yeah, for sure. So what's changed in nine years in the testing software kind of world?

Boy, nothing and everything.

The basic tenants are still there. Those are done changing and the technology is changing at an ever increasing pace, so.

Absolutely. So one thing I know is obviously a hot topic since last time we spoke is AI. Before we get there, we're going to try to go high level and then like actually see explain to people what what it means and why it pertains to software testing. And that is you want to talk about trust and like trust, what the heck is that about? So why did you want to talk about trust at it like a high level?

Yeah. Oh, that's a great question. And I get that a lot. So the kind of the origin story for this talk was it actually began to be honest in the last US Election. If anyone anywhere in the world is unaware, we had an election last year. And watching the news, talking with family members, the topic of trust came up over and over and over. Do you trust this candidate? Do you trust this party? And I'd hear it on the news, I'd hear it with family members, and I started to think about whether I do or distrust certain candidates, certain parties, and then why do I trust these things? And then as I dug deeper and began to read and learn more, I realized that trust was incredibly important. Not just with regards to elections, but in regards to my life, every aspect of it. As I dug deeper, I realized that this was an important topic and I really wanted to get the word out simultaneously. As a full time public speaker, I was trying to get into conferences and events and typically because of my 30 year testing career, I would speak at a lot of technical and testing events and getting soft skills stuff, which is my bread and butter. That's the topics I talk about. Epistemology, communication, semantics, linguistics, empathy. That stuff is tough to get accepted at tech conferences. So I was trying to think, how can I get the topic of trust accepted at a test conference? And I thought, well, trust invades every part of my life, including my career. As a tester. And what other things does trust relate to? It relates to AI and testing. So I thought, well, I can create a talk that's ostensibly about AI but truly about trust. And that's kind of the origin story of the talk.

Awesome. And I think you could chuck me off the ledge here. Some breaking news. A few days ago, Google just released Veo 3. Not sure if you've seen it. It creates really, really realistic video and talking about politics. I hate politics. I think it's ridiculous. I think politicians actually create a fake thing to rile their different segments up to vote against another. Like that's all it is, both parties. That's all I'm saying anyway. And the reason why I'm saying that is performative. What's scary is this video I could see, doesn't matter who, what party using this to say this person said this or this person said that to get their little segments upset and riled up. And then do you see that as. How is that going to impact trust? I guess let's boil that all down to this new. I don't know if you saw the new video released by google flow veo3, but it looks really realistic and I see that's being really. It's only going to get scarier as we go along. Any thoughts on that as it applies to trust?

Yeah, I've seen what you're talking about. This new release is something else. It's incredible technology. I think in the modern parlance, we're cooked is what they say. So I've seen some demonstrations of it on various feeds and it is unbelievable. So how does trust relate to that? It makes trust even more important. Questioning, asking, challenging the things that you trust or distrust. And why, why is it so? Why do you trust this thing? Why do you distrust that thing? And digging deep to find those answers, you may realize that your sense of trust or distrust is misplaced. And if so, you can reevaluate your decisions, your thoughts and actions and make better thoughts and actions, make better decisions. So I think trust is increasingly important with the advent of AI and new technologies like this video.

So as a tester, how do we, what mindset do we need to know? Like not just make assumptions like oh, this is real or you know, this requirement is correct and then based on that trust of the requirement without maybe seeing if it's a real requirement or not testing it. I don't know if that makes sense. But how do we know what to trust when we should trust it as a tester?

Well, I think the trust is infused by a lot of different things. Being a tester, I think good testers, anyone can test children test toys, they poke and prod and learn from their actions and respond accordingly. So everyone tests from childhood on. But being a good tester, doing it with agency, with intent, for a particular purpose is different. So being a good tester, I think is all around curiosity. You can't be curious unless you question things and wonder about things. And questioning and wondering is about questioning and wondering your assumptions. So you're given a requirement, whether it's explicit or implicit or someone tells you or it's written down and you might just take it at face value and go, this is the way it is. Or as a tester and you're curious, you go, hmm, why is this? So I wonder, where did this come from? Is it true? And those are the types of questions that make you challenge and question your sense of trust and dig deeper and find out, is this requirement something that that is valid, something's true, something to be trusted, or something that I should dig deeper and find out? Alternate viewpoints, perhaps? Alternate requirements, conflicting requirements.

How do you convey that to your organization? Like, I know this is what you said the customers want, but maybe we should validate it. Like, how do you not just go, I'm just going to do what they say, I want to keep my job, you know, I trust they know what they did, it's requirements or something.

Well, if, if keeping your job is your number one priority, then you just do what they say.

For each thing is a time and purpose. Yeah, but if you want to be a good tester and you want to truly challenge these things and, and test as effectively as possible. I think one way is focusing on these fundamental concepts, the things that I speak about. I talked about communication and empathy and epistemology. How do you know what you know and trust? These are very, very fundamental concepts related to people. I mentored a tester one time, somebody that was in customer service and they wanted to get into testing and they said, you know, there must be a better way. I answered the same questions over and over. Wouldn't it be better if we solved the problem at the root rather than just putting band aids on it each time a customer calls in with a problem? So I sat down with them. We began talking about things like epistemology, how do you know what you know? And communication. How do you express the ideas that you have in your head and get them into someone else's head and transfer and exchange ideas? And we talked about perspective and relativism, how things might. A bug is not just a bug as a bug, it's relative to someone at some time. And these are very fundamental concepts. Communication, epistemology, relativism, causality. What causes things going forward and backwards? What are the effects and causes? And after several weeks and months of talking about these topics, my mentee said, hey, we haven't talked about bugs, we haven't talked about requirements, we haven't talked about testing. And I thought, or have we? In fact, we've been talking about the fundamental concepts that flow upwards into all the testing activities that you do on a daily basis. Exploring, poking, prodding, experimenting with the product, communicating your findings to others, talking about risk and epistemology. How do you know that something's a risk and communicating that to others. All of these things can help you write better bug reports, test more effectively. So to answer your question, how do you communicate this to an organization? Rather than starting at a high level and saying, oh, with requirements, do you trust them or not trust them, I would start at the low level and teach them the importance of trust. Which is again what my talk Trust but Verify is trying to do. It's ostensibly about AI and testing, but it really is applicable for architects and plumbers and bakers and anybody, because trust invades all parts of our lives. So if I was giving, if I was trying to convince a baker, whether they trust a recipe or an architect, to trust the blueprint, I wouldn't start with talking about recipes and blueprints. I would start with the talking about the importance of trust and the importance of epistemology, the importance of empathy and communication. And once they understand those core concepts, then I can say, now let's abstract those to a higher level and talk about recipes and blueprints and requirements and bugs and those types of things. So to approach an organization, I would start at the lowest level, which is exactly what I try to do with my talks.

Absolutely. And I'll have a link for that talk down below. But you actually had something that talked that I thought was interesting, being an old guy. Trust is not binary. I like black and white. I like truth. And nowadays people like it's my truth. Is it? Anyway, so what do you mean by trust is not binary?

Well, binary is easy. Black and white is easy. True, false, yes, no is easy, but it's not always right. Trust. You can have a high degree trust in something or a high degree of distrust in something. And trust can be infused by many different things. If you think trust as a definition is based on Faith, it's based on belief. If you think about the most famous trust exercise, what can you think of when people are trying to exemplify the importance of trust? What do they do?

So I, I just. Because I saw your speech, right? It was the. When you fall back on a partner and they try to catch you.

All right, you cheated. But that's right, the trust flaw. Yes.

They actually did that at ge. You used to have a school they'd send people to for these things. And obviously I'm like, this is a cliche, but anyway, yeah, absolutely.

Yeah.

So the trust fall is the, the golden, the golden example for exemplifying trust. Now, the reason it works is because you're falling backwards into someone. You're not falling forwards. If you were falling forwards, you would have sensory input. You could see them, that they're going to catch you or not catch you. And therefore you're informed, you have knowledge whether the person is there to catch you. But if you're falling backwards, you have no sensory input, you have no knowledge that they're there. So you have to rely on faith and belief and those are the core elements of trust. So that's. Trust is very much based on faith and belief, but that's not all it is. Trust could also be infused by other things. For example, authority. Someone might say, I trust, I trust someone because someone else trusts them and I trust that person. So by the transitive property A, trust B, B, trust C, I trust C. So the transitive property because of authority. Or maybe you have an intuition or an emotion, you know, a gut feeling that you can trust this. Or maybe it's cultural. You, you trust something because it's been handed down generation after generation. But another way that you can trust something is through personal experience. Maybe you trust something more because you can see its internal workings, or you might trust something less because it's invisible, it's opaque and you can't see how it works, so you distrust it. Or maybe you trust something because you've had a previous experience with it and it went fine. You've ridden elevators before, driven across bridges and they were fine. Therefore you trust this new elevator and bridge will be fine. So evidence and experience, those are types of evidence and evidence leads towards knowledge. So you can trust something or distrust something. You can have high degrees of trust with just a little bit of information and knowledge informing it, or you can have less trust and more knowledge based beliefs. Does that make sense?

It does. I'm wondering if what people think about faith based trust, you know, I Went to Providence College. I was taught by Dominican monks. They taught something called secondary causes. I see smoke, I don't see a fire, but I can make assumptions that it most likely is a fire causing that smoke. Is that what you mean by faith based kind of trust?

Inductive deductive reasoning? Those are ways that can influence your, your sense of trust. Absolutely. So when there's smoke, there's fire. Maybe, maybe there's not a fire.

Yeah, right, right, right.

That's that you're trusting based on previous experience that when there's smoke, there's probably a fire, not necessarily is a fire. So I would rephrase that old. Where there's smoke, there might be a fire or there probably is a fire. I like to deconstruct a lot of old idioms to make them more, more correct and accurate and applicable. That's one of them.

All right, so I know a lot of times you go to management, they're like, should we release yes or no? They just want a yes and no. How do you reconcile maybe a faith based belief with the binary type of answer that they want?

Well, I think a lot of it is due to risk. Low risk situations are situations when trust may be just fine. It may be appropriate and reasonable to rely on faith on beliefs, not on knowledge. Because if something goes wrong, it's no big deal. Or if something goes wrong, it's a big deal, but it's an outside chance. In that case, maybe trust and faith based beliefs are just fine and you can answer to your boss or to management with a yes or no answer because the consequences are very low. But in high risk situations, it may be more reasonable to explain to management, this is a high risk situation. I can give you a yes, no answer, but my confidence in my answer is going to be very low because this is a high risk situation. We should rely more on facts and evidence and knowledge and information to make more informed decisions and rely less on trust and faith and belief because if something goes wrong, it could affect the bottom line, it could affect our standing with the, our customers, that type of thing, our brand. So I think a lot of a way of convincing management is talking in terms that they understand bottom line money and expressing it in degrees of risk. Low risk situations, faith is fine. High risk situations, you want to rely more on knowledge and evidence.

Absolutely. You also mentioned something about healthy skepticism and I'm just curious to know, does this relate to improv or acting? I forgot what the saying is. When someone's improving, you say yes. And rather than cutting them Off. So it almost keeps you open to areas you may not have thought of. Even like maybe you were so focused on one area and rather than just say no to someone's suggestion, you go, oh yeah, let me try that. I don't know if that makes sense. Is that connecting?

Yeah, I like that correlation. Thanks for bringing improv into it. I love improv.

I don't know if I've made that connection, but I can definitely see how it relates. Improv is about going with the flow, responding. Yes. And is about agreeing with whatever was said, whether it's reasonable or unreasonable, and then adding onto it. And it's a sense of open mindedness and absolutely. Skepticism is about being uncertain. It's about having an open mind, whether to trust or distrust something, whether to believe or not believe something. Skepticism often gets a bad rap. It's seen synonymous with doubts or suspicion or even being jaded or cynical. But in fact, skepticism by definition is about being uncertain. It's being not sure. And so same thing as improv on stage is having an open mind that whatever said you can go with the flow. So I like that correlation. I hadn't thought about it myself. Well done.

Awesome. Awesome. Now you can add it and just give me credit when you mention on stage. Awesome.

Yeah.

So, Damian, I guess this leads me to AI. You know, you mentioned Michael Bolton. We had a really, really great conversation around LLMs. And the thing that came up is it's very persuasive to make you think like it's telling the truth and it's not thinking, but it's like it makes it like it's so certain of what it gives you as output. How do we not get lulled into being thinking that this is someone that's actually thinking and giving me results?

I'll draw a correlation again. This talk is ostensibly about AI. I'll talk back. I hate politics as well. But that was the origin story for the talk. And if you think about politicians, as you said earlier, their entire game is crafting a very carefully crafting a message that is consumable and appealing to the masses. Right?

Yeah.

So they may say something that's absolutely not true, but say it in a way that really feels true.

Yeah.

Stephen Colbert coined the term truthiness a few years ago. So they have a lot of truthiness in their words.

Yes.

And outside of politicians, you may have relatives or coworkers that are soothsayers that can really tell you something and you trust them. You boy, man. The way that they said that is absolutely. It seems true. To me, I don't why question that another source of seemingly true information is AI? It can seem really, really fact based. It can seem really convincing, really persuasive. And so I really don't see AI in this sense as that much different than a smooth talking politician or a suave uncle or a charismatic coworker or something like that that can deliver a message in such a way that it allays your doubts. So that's why I say, rather than just accepting things based on faith, if you're in a high risk situation, it might be smart to apply skepticism, be not sure and start doing critical thinking, challenging assumptions, asking questions, pausing for reflection, those types of things.

Love it. And I also, I think in your presentation you may have mentioned something about prompt engineering being the modern version of asking better questions. So how does better quality questions influence maybe the trustworthiness of AI? Because I think it goes both ways. You can ask really terrible questions, you're gonna get terrible responses like anything, garbage in, garbage out. So how can we work on better questioning then?

Well, there's an entire field around it called prompt engineering as you mentioned. So essentially that's the modern term with regards to AI for asking better questions. How do you ask better questions? It's an art and a science. Several of my talks around communication, semantics, linguistics focus on this, about the art of asking better questions. Whether they're binary, closed ended questions, whether the open ended questions that allow for exposition and being more verbose and more information. Polls, questionnaires, surveys, likert scales, ranking and rating. There's all sorts of ways that you can question people or AI to get better responses. If your question, like you said, garbage in, garbage out. So if you're asking AI poor questions and you're getting results out, your confidence might be low because you say, well, was my question sufficient to begin with? So I encourage people to learn if they're using AI to learn to ask better questions outside of the context of AI. I encourage people to think about the question that they're asking their uncle, their co worker, the politician or whoever they're talking to and say, is this a reasonable question? Does it properly embody all of the things I want to know? If not, then, then your question is part of the problem in the bad answer that you get back.

Absolutely. So you know, is it part? Is it, I guess trust goes at the heart of everything. It's not just testing, but it's all things in life, but especially when it comes to a job. Do you think it's our responsibility to question things Better to our management because, you know, once again, Microsoft just made an announcement, I think two weeks ago, they're laying off 6% of their company for. They said it's going to be replaced by AI. How do we trust these people that are supposed to be trustworthy? You know, the leader of OpenAI, the leader of Microsoft, all saying AI could do all these wonderful things and it could replace you. Is it up to us to question them and say, look, this is what AI can do and what it can't do to try to maybe level the playing field?

That's a tough one. That's a big, big, big question. Let's see. I would say that there's a time and a place. I'm very context driven tester. I'm a very context driven person in my life and my work, in my play. So I let context drive and part of context is purpose. What am I trying to achieve? We mentioned earlier, if you're trying to keep your job, maybe just do what they say. If that's your number one priority. Maybe convincing someone that they're wrong or incorrect isn't a prudent thing if you're trying to keep your job. But if that's not the top of your priority stack, if there's other things, ethics and morality and those types of things are important and doing good testing and producing quality work out of yourself, then I believe that it is prudent and advisable to challenge people and question them. Now this is not popular. You said earlier, black and white. I like it, you like it, it's easy, that's why we like it. Easy things are fun and easy, but really sometimes they're not the best for long term outcomes. Sometimes easy, fast things have good short term outcomes but but can have catastrophic or negative or unwanted long term outcomes. So think about the purpose, what you're trying to achieve is the short term gain. Does it outweigh the long term consequences? Are the long term consequences bad enough that you say boy, in this case ethics and morals and doing good work is the top of my priority stack. Therefore I need to talk to management and help them understand these things, why what they're saying is incorrect or an alternate viewpoint. You know, this is all about tact and how you communicate your message. But if that's the top of your priority stack, I absolutely think it makes sense.

Absolutely. I guess it's all how persuasive you are rather than me seen as a gatekeeper. I think it goes back to your point about risk. If we could say, look, we're not gatekeepers. Here's what you want to do. Here's. We see this as risky. 1, 2, 3. But it's up to you as a decision maker whether or not it's released. But I'm giving you the information of what's risky or not and then it's on you. I don't know if that makes sense, but it's more persuasive, I think, rather than saying, nope, I'm qa, you're not releasing type deal.

Yes. Persuasion partly is about providing information with. I think that testers are information brokers. They, they deal with information. We're not decision makers. We can offer opinions based as a proxy for the user or the customer or based on our own self, but those are not the decisions really. We offer information to others so that they can make more informed decisions. So part of that might be persuasive. Offering information in such a way that if you think it's important and it's ethically important and keeping your job is not the priority stack at top of your priority stack, then maybe you can offer that information in such a way that it's more impactful through. By being persuasive.

Absolutely. Okay, Damien, before we go, is there one piece of actual advice you can give to someone to help them more with their testing, kind of risk based, kind of skepticism, kind of, you know, trust but verify type approach. And what's the best way to find a contact too?

Great, great question. The advice I usually give is simple and incredibly difficult at the same time. It's asking. Asking is costs nothing. It's easy to do except for it's incredibly difficult to do because of social norms and not wanting to appear as unknowledgeable or uninformed. So there's all sorts of reasons that people avoid asking questions. That's a whole topic in itself. But I would say my number one piece of advice is ask. And if you think that someone else doesn't understand what you're trying to convey and communicate to them, ask if they understand. Ask if they can paraphrase and repeat it back to you to your satisfaction. So my piece of advice, and this goes back to the curiosity, which is the main attribute of a tester, I believe, is ask questions and how to find me. I'm@ineffable-solutions.com that's my professional website for speaking, training and consulting. I also draw strangers. That's at idrawstrangers.com and my book is at hankinstellabbooks.com I'm all over the place doing a lot of different things and.

You can find links to all this awesomeness down below. Thanks again for your automation awesomeness. For links to everything of value we covered in this episode, head on over to test guild.com a547 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. Joe, my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation testing and DevOps. If you haven't yet, join our vibrant community@test guild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit test guild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

Oh the Tescel Automation Testing Podcast with hoots and lyres, the bards began their song. A tune of knowledge, a melody of code through the air. It spread like wildfire through the land, Guiding testers, showing us secrets to behold.