Scaling Quality Engineering: How to Deliver Faster Across Global Teams

Hey, welcome to another episode of the TestGuild Test Talks Automation Podcast. I'm Joe, and I got a question for you. Why do so many test automation and quality transformation efforts fail, especially at scale? And more importantly, what actually works? Well, you're in luck because today we're going to break it all down with our expert Sunita McCoy, who's joining me. She's a global engineering leader focusing on platform quality, SRE, and observability at Fortune 100 companies. So she really knows her stuff. She's been in the trenches leading large-scale transformations, and today she's sharing real-world insights on scaling quality, adopting AI, and building teams that actually deliver. You don't want to miss it. Check it out. Hey, Sunita, welcome to the Guild.

Thank you, JoeBot. Good afternoon. Hope you're having a good day.

Absolutely. Really excited. Uh, I think the test— the folks from TestMew hooked me up with you. Said I have to have you on the show. And I saw your background, I'm like, oh yeah, absolutely. I know you're kind of like a leader with leading transformations, and I know we're going through a major transformation with, you know, testing automation and now AI in the mix. But before we get into maybe a little bit more about your background so people know a little bit more about your awesomeness.

Yeah, sure. So if I told you I always dreamed to be a transformation leader, you wouldn't believe it, would you? I, you know, I started my career by virtue of, you know, going to school for software engineering. And as I got out, one of the first things was to be an app developer. I was in the trenches. I started out with writing code, building solutions, design, and, you know, really enjoying what at that time was integration platforms. So, that's kind of where I was, you know, studied more of that object-oriented programming, building modular software, but then got to work and was thrown into what was called a TIBCO, which was an integration platform to be able to bring the old and the new together through messaging queues and things like that. So, that's kind of where I started my journey.

Nice. So, start off pretty geeky. How did you get into more of the management leadership stuff then?

Great question. I think I've always been a leader at heart. Just, you know, growing up, I've enjoyed being in the forefront, taking a bullet or two. And as I was at my first few years in software engineering, I was young in my early 20s. And I'm like, gosh, don't tell me this is how I'm going to wrap life. I saw the opportunity to be a consultant. I said, this is amazing. Let's pack our bags, live out of a suitcase. Every week you're in a new place, new customers, new people. And I enjoyed— I think I did it for about 5 years. I enjoyed every bit of it.

Really?

Because it was—

sounds like a nightmare to me.

No, something new every day. Something new every day. You know, you make plans to come home and party hard with your friends, and then you're like, gosh, can't make it back home. Because I have this, you know, release to work through or a production issue or a deployment issue. And so, you know, like I said, young 20s, very restless, you know, didn't have to think and worry about anything much. So, it worked out fairly at that time.

Nice. Can anyone be a leader? Or do you think you're born a leader?

Anybody can grow into being a leader if that's what they would like to do. And I truly say that. I believe there's nature versus nurture. And if you are passionate about leading and can actually focus on not just yourself, but actually the enterprise around you and think holistically and drive for outcomes, you definitely can be a large-scale enterprise leader. I also believe we all are leaders in our own self as we are navigating the day-to-day. You know, you are constantly problem-solving, you are showing up with initiative, you are stepping into trying something new, sharing thoughts, sharing your knowledge. So I think, you know, it's the scale, scale is what differs, but everybody is a leader in my opinion.

All right, I know I'm all over the place, but now that we see ourselves in the age of AI, I want to dive into AI a little bit later, but do you see us as testers or people that are really technical needing to kind of go up a level to being like leaders, almost leaders of AI and quality within our organizations?

To be a leader in the AI era, if I'm understanding your question correctly. Yep. It'd be important to try what it is. What is this whole thing called AI? Figure it out in your own way. So I'll share personally how I started out with it. Because in our organization, we were squirmy about leveraging AI initially, just because of the security information breach, you know, don't put your SSN and your tax documents into Copilot, right? So there's definitely a lot of squirminess around around it. But then, if you have the right governance, education, and curiosity, it can be such an accelerator. So, I truly just started out with, hey, build me a strategy, 2-year strategy about AI adoption. And, you know, just using word prompt and getting a lot of information and then customizing it to be my own and building presentations. I use some you know, Copilot to coach me for an interview or, you know, a conversation, exec discussions. You know, what should I be thinking about? What gaps do I have? What should I be addressing to mitigate risks? And then I'll share, you know, most recently I've been dabbling in Cloud Code to actually build my own professional website. And I'm like, In 90 seconds, really? 30 years ago when I did this, it took me days and weeks to, you know, have a good HTML page. So, yeah, that's— I think you definitely, you know, have to touch and feel it so you can actually relate better as the conversations are ensuing.

Absolutely. So, I guess another fear I think people have when they're initiating AI especially with testing, is the quality. So as a leader, how do you own quality and really drive that home for maybe large teams across the world? Like, hey, this is what AI does, but here's some guardrails you might want to put into it to make the quality better.

That's such a great question. You know, if you think about the bell curve, What is it? About 2 to 3% of us are innovators, about 13% early adopters, then, you know, 16% major adopters, 16% another late adopters, and then you got laggards. So, if you think about that curve, every organization seems to be going through that journey at their own pace. We've been talking about AI for a very long time. I think it is becoming more of a reality with the number of growing platforms that we have. And as I'm navigating a strategy for AI adoption, I'm met with cultural nuances. I'm met with friction because we haven't built capacity for learning something new. And upskilling our team members while they're focused on delivery. Like, hey, you gave me 5 user stories and I need to get those out in 4 days. There is no time built here to do anything new. And so the conversation for me, for the strategy to pan out, really starts up top. Like, do I have sponsorship? Is leadership really truly vested and behind us to ensure we can navigate that cultural shift. And then working with product owners to say, hey, we got to build capacity within our backlog so we can realize these new superpowers that are out there. You want to think about quality across the lifecycle. You want to shift left. You want to build it into you know, your pipelines, think about quality as a product, right? Versus quality as a phase. And to realize that and navigate that cultural nuance, team members need to feel psychological safety. They have to feel that there's trust. They can come to the table, have real conversations. And that we're creating the space for them to truly learn and grow. And as we champion that strategy and allow the grassroots momentum to take place, I've had team members come back and tell me, "Hey, I was squirmy because I didn't have a technical background. I've been at this company for decades." And, you know, you got to remember the number of generations that we're navigating at any given point in time. And now they're saying, gosh, I'm able to use this platform. And I can actually write my automated scripts with no-code platforms in minutes. And that's an efficiency gain for me. I don't know how long it would have taken me to really do this by myself, learning the platform and writing from scratch, but also a confidence boost that they're able to meet edge cases that typically are left behind because we're not thinking about it. So the feedback is, I see AI platforms being more of a peer reviewer, where I have a higher level of confidence about the tests that I'm delivering and the software that we're putting into production. So win-win in my opinion from that perspective.

I love how you mentioned, you know, you need to give your team members time to upskill, but a lot of times you have initiatives top-down saying, thou shalt use AI, go at it. And then you have people down going, what? And I don't have time. Like, how do you make that connection from top-down and bottom-up so that you're in harmony though? You're speaking the same language. Does that make sense?

Yeah. Yeah, it is not gonna be a one-man show. I don't think if I stood on the top of the mountain and I screamed, "AI, let's do this right now," that it would work. I will admit, you know, at Foot Locker, we actually have a very, very strong AI adoption culture and mantra momentum underway. And we talk about AI at office hours recurring. And promoting team members to try something new and rewarding and recognizing them as they are actually trying new things and experiencing new superpowers. So, my recommendation for anybody on this journey would be that you're doing this at scale and there's transparency and that you're talking about the squishy squirminess, right? So, we actually have our security leaders come in and talk about, don't do this, don't do that. Here's some cautionary tales. Here's what we're seeing in the news. Here's the type of breach. And some of these breaches are happening at some of the top name brand branded organizations. So, you know, it's not like some XYZ, you know, startup is going through struggles, but happening in organizations where you would least expect it. So it's a lot of awareness bringing to the table and, you know, enjoying the journey, celebrating the journey, I think, really calls for, you know, recognizing your failures, sharing them openly, and also celebrating those wins, because failures are only the stepping stone to success. So I think I've definitely seen a lot of good strides, you know, navigating and in this manner.

Nice. So you did mention Foot Locker is very AI forward. How do you separate the hype from reality? I know like a lot of news coming out recently is like Block laid off 40% because of AI. And I, I just think that's BS. I think it's more to cover them. They're not making money and they're just saying AI is, is replacing it. Maybe I'm wrong, but like how do you know what's hype and what's not? Like, does that make sense? Like how do you know what's real and why people shouldn't be afraid to embrace it? Cuz when you hear news like this, you're like, I'm going to be replaced. Why, why embrace it?

It's a great question because, you know, definitely that's, that's like the running sentiment. It doesn't matter which job function you're in. There's consistent fear and concern that my position will be eliminated. But if you think about it, you're really getting efficiency gain. You're allowing your subject matter experts to pass on the mundane, exhaustive job to an AI platform, while you can still be a human in the loop focused on the context, having empathy, making sure, you know, there is meaning and purpose around what we're doing. I like to say we are in the business of humans and dealing with humans because we're probably the most complex it can ever get. I don't think any AI will ever be able to, you know, replace us or be as complex as we are. And so that context, I don't see AI being able to do that. Even when, you know, I give a command prompt or a word prompt, whether building a website or writing a test I can't take it at face value. I spend a lot of time reviewing and ensuring it's meaningful and it's relevant because sometimes things that spit out are not— hallucination is a real thing. And so, you know, you gotta account for those things. And I strongly encourage team members to embrace change. If you think about the tech journey, every few years, there's a constant evolution. And we become irrelevant if we're not humming along with that change and embracing that change. Be brave. You picked technology for a reason. You know, it's not for the faint of heart. So, be courageous, embrace it, you know, tell your story. Share your story, right, along the way. So I think that'll be the best way to stay relevant and in the, in the mix of it all.

Absolutely. I love the human-in-the-loop aspect of it. I know you have a lot of experience, other things like SRE, observability, quality, AI. So here's what I'm thinking. Do you see the role of tester maybe changing where we all become software engineers? And therefore software engineers are responsible for code, which is quality, which is built for production. SRE, observability. Do you see like a transitioning, or is that, is that hype that I'm buying into at some point? Will there always be like a delineation between you are a tester, you are a developer? Do you see everything just merging? You are a software engineer, go at it.

I, I think there'll always be a delineation. I've thought about it in that aspect, even before AI. And the reason being, developers like to build. Testers like to problem solve and, you know, find issues and defects. And so it's a niche skill on either side. And Even with AI, you have several flavors of AI, right? You have the no-code, you have the low-code, and then you can deep dive and get as crazy as you want to with your AI/ML. And when you think about the true makeup of any organization right now, you have several different generations. 30, 40 years ago, there wasn't even object-oriented programming, right? We were doing assembly and DOS prompts. And so when you think about it from that perspective, you have to recognize that we all have to stay together in the space at any given point in time, and we all have different superpowers. And how do you actually maximize those superpowers versus saying everybody needs to be an AI engineer? It's, it's just, it just doesn't seem feasible and practical. And everybody is not going to have the same interest. But for any organization to be successful, you need a mix of the old and the new, the subject matter expertise that is deep-rooted within a team member, If someone put in 30 years, 40 years of their life, gosh, I can't ever imagine replacing that knowledge or earning that knowledge, right? And so you can do as much documentation as you wish to, but this is actually true hands-on keys expertise that was built over several decades. And then you have new perspectives coming in, maybe younger generations, AI-native engineers, and they're deep-rooted in AI/ML, they're able to really create these platforms. If there can be a happy medium where both generations are able to come together and figure out how best to work with one another's superpowers and win together, that's where success is going to be. That's my perspective as I'm seeing these navigate and the experiences that I've had within the last, 2 decades, most organizations, you know, 50, 100 years old, some of us still have mainframe and are not quite, you know, cloud native. And a lot of solutions that are coming out these days are cloud native solutions. So what about the mainframe stack? Right? And so it's just, you know, you got to account for your tech stack while you're going through that transformation of legacy to new gen. You still have to coexist together until you really can get to that point.

That's a great point. Every time I hear like a use case of, uh, AI automation replaced everyone in my organization, I look at their app and it's like a, it's like a game app or something. It's not like an enterprise app. I used to work for an insurance company that still had mainframes, a healthcare company that did all kinds of crazy old school stuff. So that's a good point. to get over the hype for sure.

Yeah.

So you mentioned generations a lot. Do you see like Gen X different than other, like I'm from Gen X, so do you see them more skeptical? Like, uh, you know, we, we say low-code now and low-code, I think with AI actually, um, is helpful. But back when I started, it was more like, this is just snake oil. Like, how do you then get the skepticism of one generation to embrace the enthusiasm of another? Is that something you have to actually be as a leader to kind of negotiate?

Wow, that's such a great question. I'll share just from experience. And this is true of whether it is AI or a new process, applying lean methodology, trimming fat, adopting metrics or dashboards, et cetera. 30, 40 years ago, metrics, we didn't really operate with a lot of metrics. A lot of it was emotion-based. And, you know, if you bring in a leader like me who's very metrics-intrigued, my first question is, I understand your emotion. Can you back it up with any data for me? And then, I'm typically met with, here's 15 darn reasons why we don't have metrics. It takes away time. You're asking us to document stuff. You want us to have traceability. Then you want to have a centralized tool or a dashboard. We don't even have the expertise or the time to build something like that, right? And so, much like, you know, what we tell our kids, you got to crawl, walk, run, right? You kind of have to allow them to try something be okay with them falling, getting up. But it has to be their idea. They have to feel that this is truly a superpower, not just a waste of time. There's probably some front-loading you have to account for, right? But then once you get to the other end, I've had team members come back and tell me, gosh, you said metrics, they were so painful. But we see the value right now. Because now they're able to use that in their narrative with their engineering teams and their dev engineers and able to say, hey, you know what, the failure rate is so high. Let me tell you how high. It's 30%, which means for every 100 test cases I'm executing, I have 30% failure. So my effort is now 130%. But while building this project plan, we only accounted for 100%. So, we are going to be delayed with this project. We are going to be cost overrun. And so, using your metrics to actually drive a conversation, because you can't deny metrics and data. You can definitely fight the emotion, right? Because it's subjective. Joe, you feel that way, but I feel this way, right? But then, if you throw data at me, I'm like, oh man, these numbers are real. So, let's talk about them. And so, that's definitely change the way we operate, how we collaborate, how we work together. The other thing I'll also say is, you know, whatever we're implementing, I truly believe has to be a conversation from a product perspective, right? Because you've got to understand your why behind what you're doing and then talk about how you're going to do it. And if there is going to be an enterprise outcome and you're aligned with the roadmap, then everybody is going to get behind it. Otherwise, you're in a silo by yourself. And so some of it is just allowing folks to taste the flavor a little bit, allowing them to actually go through the experience, being very patient, really being there as a support system versus just driving. And as leaders, you got to be careful, given the situation, when you're driving and when you're actually taking up backseat and allowing team members to actually drive because there's that time-sensitive aspect around it. Because everybody wants everything now, wants it fast, wants high velocity, accelerate the transformation. You got to, you got to be realistic about some of those things.

So, speaking of why and all that, why do you think being the queen of transformations, most quality transformation initiatives fail?

Oh wow. So, um, I'm surprised you're even asking this question because I don't think everybody thinks about it that way. I, uh, when I'm having conversations, uh, with leaders and I'm tapped on the shoulder, they're like, hey, come join my organization, we want to— we're going through transformation. And you have the experience. We want you to come in and help drive it. After a few minutes of conversation, I actually asked them, are you truly ready for transformation? And tell me how you think you're ready. Do you have the dollars set aside? Do you have the time accounted for? What is your org change management structure? How are you going to actually measure success? What do you want done in the first 6 months? What would success look like to you? And I ask these questions because then there is a realization based on how they're responding. I can tell you whether they've thought this through and realize what it takes to actually get it done. You have to have top-down sponsorship and alignment. Without the sponsorship and alignment, most transformations are going to fail.

Absolutely. Um, it's always a culture thing. It's funny, a lot of people point to technology and it's always the hardest thing to fix is the, uh, the culture rather than the tech, right? I'll just have a new tool to fix everything. It's always like, well, the culture's the same. It's all gotta be the same, right? No matter if you throw AI at it, if the culture's off, like you said, I think it's not gonna work, right?

Yeah, very well said. I was in a conversation earlier with Kavya and Nandini and Priscilla, and this came up. You know, culture is a big piece of anything, like the adoption. Adoption doesn't happen just because you're asking for it to happen. It needs to be driven. It needs to truly be a grassroots initiative. It needs to come from the people, by the people, for the people. You really don't need leaders to drive the transformation. You need them to be there for sponsorship. You need them to really lean in and say, "Hey, I get it. I trust your instinct. I trust what you're doing. It is okay to fail. Yes, we have psychological safety. Let's talk about the good, the bad, the ugly, and the squishy stuff in between." You need to drop the F-bomb. Go ahead, drop it because that's how you're feeling right now. I'm here to truly ensure we succeed together, right? So, that is really, really important. I've said this at the top of our conversation. This, being in technology is not for the faint of heart. You got to have the courage to navigate it, the courage to stand up, speak up, advocate, because you're trying to build quality in the product you're trying to deliver. At the end of the day, your customers are the important. You want to make sure production is stable. It doesn't happen by not thinking about it holistically and strategically.

Absolutely. So, you mentioned a few things. I'm just curious to know if someone was going to start a quality transformation, say, in the next 12 months, where would they— where do you think they should start and what should they avoid? I know we talked about culture, but is there any tangible things? Should they start here? Don't do this, avoid this, right off the bat.

You know, if it is a greenfield opportunity, my take would be start with AI-native testing.

Yep.

And take advantage of the efficiencies that you can see right off the bat, right? Bring in a strategic partner, someone who has the platforms, the tooling, the, the expertise, right? You don't want to navigate it all alone from a space of unknown uncertainty, but bringing in, bringing in an expert and leveraging their industry expertise, hearing their stories and, you know, their case studies, talking to some of the customers that they've worked with, because a lot of partners these days are claiming, you know, hey, I'm AI native and I can do this at a fraction of a cost. And You also have to think about the dynamic around it. For example, if I was in an organization that is primarily conducting business in North America, I wouldn't want all my quality engineers to be sitting all by themselves offshore somewhere. I would like for them to at least be in the same time zone and have real conversations with real people. People across product, engineering, ops, and, you know, feel like they're in the know of what they're building and what they're doing on a day-to-day basis. And using Agentic AI will just allow you to have probably more efficiency, like you're trimming the fat, but you still need the intelligence. You need the human in the loop. For it to be meaningful. I mean, you're just basically taking advantage of the, you know, the 3A framework, your adoption, arbitrage, and aggregation. But you're not moving away from a whole job function by any means. And it's kind of critical to kind of think about those nuances, you know, where is your business? Who are you? Supporting, where are your customers, are you available to your customers? All of it is not going to just be a robot. No.

Love it. Love it. All right. I want to start wrapping things up with some rapid-fire questions. No, it depends. All right.

Okay.

Quick question one. One metric every quality leader should track.

Gosh, I'm going to say production stability.

Okay, cool. Cool. Why?

I mean, that tells me how I'm serving the customer. You know, if I'm working at a store, right, and I have an outage, that's money I'm leaving on the table, the impact that I'm causing to the customer, and the negative press that I'm going to have to deal with after the fact. Too close to the customer.

It's easy to forget about the customer, huh? I always think about the tech, the quality, you know, and it's the customer that matters most, right? That's so true. That's a good metric. Love it. Love it. All right, most overrated practice in test automation.

Yeah, test automation itself is overrated. It's overrated in the sense that there's this expectation that you have automation and works with a button press, and that you go from, you know, developer desktop to production in minutes. We all aspire to be Amazon, Google, Microsoft, it takes a lot to kind of get there, understand your tech stack, etc. And so, there's a little bit of, hey, can you actually be more practical about what you're dealing with before you convey those expectations? Because what that tends to do is constant recursive conversation around, I don't understand the effectiveness of your automation, If you have 80% automated regression tests, but it takes you 2 days to get through it, you're forgetting about the environment constraints, the data constraints, etc. So every enterprise is different. And I've seen in large enterprises where there's legacy as well as new tech, that's most overrated.

Right. Biggest waste of time in large engineering organizations.

Ooh. The red tape. The red tape, for sure. Oh, I've had engineers tell me, peace out, I'm leaving because the red tape is holding me back. I got to go to a smaller startup where, you know, I can actually see my outcomes fast. You know, I want to realize what I'm doing now versus 10 years from now. So, yeah, red tape.

One change that delivers immediate reliability gains.

I'll say collaboration and transparency. So continuous communication. I think just having the awareness, hearing what we're working toward, and all of us working toward the same thing at the same time. Alignment is kind of the way you're going to get to the finish line. Otherwise, Joe, you're working on something and I'm working on something else, and now our roadmaps are misaligned and we're not getting to the finish line together.

Okay, Sunita, before we go, one piece of actual advice you can give to someone to help them with their quality transformation efforts, and what's the best way to find or contact you?

Oh wow, um, one piece of advice, uh, embrace change and, um, have the courage to continuously learn and grow. Very, very important. I think that'll keep you at par with the time, right? Because you're going to find something new every single day. Change is good. Get comfortable with the uncomfortable. And then, um, what was the other question you asked me, Joe?

Best way to find or contact you?

LinkedIn would be the best way to find me. I'm very responsive and very active. I enjoy telling my own stories. I've made it a thing for this year to post my thoughts and experiences as I'm going through enterprises and the day-to-day. So that'll be the best way to stay in touch with me and interact with me.

Awesome. You can find all the links to this awesomeness down below. Thanks again for your automation awesomeness. The links to everything of value we covered in this episode, head on over to testguild.com/a584. And if the show has helped you in any way, why not rate it and review it in iTunes? Reviews really help in the rankings of the show, and I read each and every one of them. So that's it for this episode of the TestGuild Automation Podcast. I'm JoeBot, and my mission is to help you succeed with creating end-to-end full-stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation, testing, and DevOps. If you haven't yet, join our vibrant community at TestGuild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real-world challenges, we're excited to collaborate. Visit testguild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve. Oh, the Test Guild Automation Testing Podcast. With flutes and lyres, the bards began their song. A tune of knowledge, a melody of code. Through the air it spread like wildfire, through the land, guiding testers, showing the secrets to behold.