Aligning DevOps Teams for API Quality with Temil Sanchez

Get ready to discover some of the most actionable DevOps techniques and tooling, including performance and reliability with some of the world's smartest engineers. Hey, I'm Joe Colantonio, host of the DevOps Toolchain podcast, and my goal is to help you create DevOps toolchain awesomeness. Today we'll be talking all about quality as a team sport, how to align your dev and QA and product with your Agile API team. You're going to hear all about it with Tamil Sanchez. If you don't know, Tamil is a product manager at smartbearer. He's been there for years, I just found out, so I'm really interested to dive in with all his experience there. If you want to learn how to do API testing as a team sport within DevOps, you don't want to miss this episode. Check it out. Hey, before we get into it, I want to give a quick shout out to this episode's sponsor, Smart Bears API Hub. If your team is building or testing APIs, whether it's REST, GraphQL, Kafka, or microservices, you know how complex it gets managing designs, docs, tests, governance and collaboration. And that's where SmartBear's API hub shines. It's an integrated platform that combines all the things you need, like design with real time collaboration. API Design with built in, linting and governance has contract testing to help you catch breaking changes before they hurt you. In production, you can explore interactively with test endpoints from the browser. No local setup is required. They have a portal to automatically generate and maintain branded API documentation hubs. And you can test auto generation and real functional regression and integration tests built to plug right into your CI CD pipelines. You get all this power for one unified interface. And the best part, you could try it out for yourself. Now, no commitments, just sign up for a free trial. Jump in and explore how well it fits into your workflows and see where it might save you time and reduce risk. Support the show and check it out using the special link down below. Hey Tamil, welcome to the Guild.

Hey Joe, thank you for having me here.

Great to have you. I guess, you know, before we get into it, like, can you tell us a little bit about our journey? Like how, how you ended up in, you know, doing product management for technology and how do you get into DevOps? All the things. How'd you get here?

Oh yeah. So started my career in professional services and did a lot of things here and there with QA engineering and some, you know, development work. I've really quickly learned a lot about web services and automation and using SOAP UI in my really early days and, you know, as fate has it, I ended up at SmartBear supporting the tools that I use in my day to day work work.

So interesting. Smapier had a SOAP ui, they had an open source before many of these other solutions came out. I'm just curious to know, you know, it always feels like API testing is about to take off. And from my perspective, maybe just recently I started to take off a little bit more. Of some more interest, what are your thoughts on the adoption of API and API testing? Has it been flatlined? Is it a slow, gradual process or is this something that people don't even know enough about and they're missing out on testing it?

Basically, I think over the years it's really taken off. You could see it just by the amounts of different flavors of APIs that you see in the industry. Right. As I mentioned, I worked with web services initially. Right. It was all about service oriented architecture. And then that quickly became people had to find easier ways to use APIs. So you saw what REST took off and then there was a problem with rest. There was no standard definition. And that's when swagger came on. And nowadays we see newer protocols or newer ways of interfacing with backend systems and data. Right. We see GraphQL and event driven services. So there's a huge need for exposing data and functionality. And APIs have really not gradually, I'd say tremendously increase in usage.

I guess another thing that always fascinated me is IOT has always been a dream that hasn't taken off. And it seems like APIs and IoT are slightly pretty much tied together. I would think, and I would think in the future, you know, I know a lot. I was talking to someone at healthcare company. A lot of these mobile devices, a lot of things are coming out. The FDA is allowing things to come out that allows you to, you know, check things from different devices that are not traditional, you know, computers per se. That's going to require more APIs. Do you see APIs being tied to IoT?

I definitely feel like yes. I mean, there are, I would consider some of the IoT protocols that are used APIs. I think the biggest issue with that is data exposure and security and vulnerability concerns in the industry. Once, you know, we figure out a good way to transfer data from IoT devices to others without being fully vulnerable, I think things will take off a little more and they'll become a little bit more attached to APIs the way we see them today.

Absolutely. So inherently our APIs more unsecure. I know I've talked to a lot of people that say a lot of people don't realize how people can hack into your API because you can't see it. People think you can't get to it. But a lot of people can do sniffers and all these other techniques. So do you see security as being still a concern when people are doing API development?

So it has been a concern, it is a concern. That's why there's layers of security over APIs as we know them today, especially with the most types of use APIs. Like there's definitely, you know, different authentication, you know, multi factor authentication that happens, and different algorithms for securing who's connecting to the APIs. So you'll see layers upon layers of, of these services to ensure that APIs are secure. It's been a concern, but I think the industry has really has done a good job of, of making sure they're doing the right things to put them behind walls so that again, they're not accessed as easily.

Nice. So I don't know if I've ever heard of an agile API team. Is this like a microservices team or like what's an agile API team?

So I wouldn't think of that any differently from a regular team except that it's, you know, it's focused around APIs in this day and age. You know, we've heard breaking down the monolith for ages now. That means that product teams and development teams sometimes just focus on these microservices and that's why we see that.

Right, so it's slightly different. I guess. So if someone's working with APIs in development, are there any common, I don't know, collaboration breakdowns you may see between devs, QA and product teams when they're trying to really build APIs in an agile type of environment?

Yeah, I see know a lot of issues with fragmented feedback loops and developers in general and QA engineers working in different platforms using different tools. So this becomes really difficult to, you know, deliver great software sometimes. So that's very common.

So is the reason why, and once again, I just assume with an API, you may be consuming an API and a team may not know you're consuming their API. You may have multiple APIs talking to one another. You might not even know where with maybe like a, like if you're doing functional UI testing, you know, pretty much what you're using. Does that make it harder to collaborate or even communicate if you're between teams?

Oh yes. Oh yes. So it definitely makes it, you know, very, very difficult sometimes, you know, testers aren't even part of the initial stages of these conversations, of these API. So it definitely, you know, feels like eventually there'll be incomplete coverage, you know, late stage bugs and unnecessary feedback loops.

So I know a lot of teams, regardless of what they're doing, you know, there's always this push between technical and non technical roles. But with API testing, do you know, is there a way you could build maybe a bridge or get over the gap between technical, non technical roles when you're working across say the API lifecycle?

Yes. I think, you know, one of the great things is to use platforms that are non technical friendly. There's tools now that take in for example, an OES definition and generate easy to read documentation. And there are cohesive platforms that really bulk this out in a really visual way for any, you know, for just about anybody. And one of the most important things is that these assets and when I say assets, I mean like testing assets, documentation assets, design assets are viewed in a very cohesive manner and you could even see the lifecycle changes of the API across these platforms. So that's really important to really bridge that gap.

Another thing I would think is it's hard when you don't have maybe a platform that everyone can go to. So back in the day I used to use SOAP ui, it was local on my desktop. I do a request, get a response back, test the result and then you know, it wasn't like a full life cycle type of thing throughout the SDLC for testing an API through all it's throughout all the different areas that you need to do it. Over the years the smartpet built anything to do more comprehensive maybe life cycle of API testing along the way and maybe you could break down what, how it fits within SmartBear nowadays, how a team could bridge the gap using maybe a platform to help them with it.

Yeah, so there's definitely many tools out in the industry and you know, SmartBear is definitely one of the biggest components of trying to bring teams together in a cohesive manner. We really recently this year, earlier this year we released what is called the API Hub. And it really consists of a couple of different areas, right? There's design, there's the design phase which helps users generate governance, create governance and standardization across your design portfolio. When you're building APIs then we, you know, we also bring in there the documentation portal piece with a product called Portal that helps you, you know, define what these APIs do in a meaningful manner so that it can be adopted and easily viewed. Right. And if a developer is trying to consume an API, he knows exactly how to consume it, how to authenticate into it, how you know, what type of data structures it supports. It also has an exploration feature which allows you to interact with APIs really quickly. Again, this is, you know, when interacting with APIs or trying to consume them, it's like a trial and error phase to really ensure how to use it. And then we, we have contract testing which should never be an afterthought. Right. Contract testing ensures that there are no break in changes when you deploy new versions of your APIs to your existing consumers. And lastly, we have a testing capability in there as well that allows you to create really quick automated tests and pass them through your CICD pipeline. All these capabilities and features are within one platform and we're calling it the API Hub. So it is really bringing teams together and really trying to cover that cohesiveness of the end to end API lifecycle.

Absolutely. Yeah, I love that. I think having a platform that you can manage an API through development all the way to production be really helpful. Is any piece along that way right now that maybe people don't know that they need that this hub can give them. Like you mentioned, contract testing is that often overlooked or is there another area that's often overlooked? But now if they were in this type of platform, they'd be able to really get, get at an area that they may have known, not known they had a risk in or something.

That is definitely the hope, Joe. Yeah, so contract testing is something that is picking up in the industry and people are starting to see the value. But just thinking, you know, about you're serving APIs right? In the nature of the APIs is to be cross functional, used by different teams for different purposes. And in the world we live, there's continuous change obviously. Right. So it's very, very important that you know whose APIs you're breaking, right. Whose integrations you're breaking when you make small changes to your APIs. And that's exactly what contract testing solves for. And now having it part of the API Hub platform is definitely will give it more visibility and make sure that folks developing APIs and microservices really understand the value of contract testing.

I also would think, you know, we always talk about shift left, getting testers, you know, really involved as early as possible and especially I guess in the API testing would be even important. As well. Does the API hub maybe help testers get looped in early in the development process of an API so they're not caught off guard or near the end?

Yeah. So one thing I'll say about that, it definitely helps a lot with that. Also, quality is everybody's job. Right. So shifting left to your point is something that's really important. And does API Hub help with this? I'd say yes. Right. So it all starts with the specification. If you know what state the specification is on and you're in the same platform, you understand when changes happen, you understand what tests are no longer or when tests become stale and you need to update them. Right. So it definitely encourages testers to stay up to date and it helps understanding, you know, what changes have been made to the API so that they can do their due diligence of updating their regression tests and creating new tests and so on.

Nice. So how does this work? Does someone have to log into a portal or do they get alerted? Just curious to know, if there is a change in API and it affects me, do I have to keep logging in and looking or is there an alert system? How does that happen?

Yeah, so right now we're working on more enhancements to make the alert system, you know, alert when changes happen. But for the time being, what testers can do within the hub is they can create tests directly from the definition. It creates a connection behind the scenes. But they're in the same context. Right. So that's the feature that we that helps today is you're within the same hub, you log into a portal, it has all these tools that we talked about. You can create your test directly from design, and when changes occur today, you can, you will not be alerted. But it is something that we are planning to do in the future.

So do you also find that this also creates tighter feedback loops across the teams?

It does. It definitely does. And I think that's one of the pains that I see many organizations having today because of many, many feedback loops. Right. Developers and QA engineers as an example, not understanding where the issues is. So QA engineer finds a bug somewhere and says, hey, I have a bug. Developers may say, hey, you don't have the test set up correctly. That's definitely a huge pain point. And the API hub does help with that. Just because we're talking the same language. Right. You're inside the same platform and it's easy for developers to say, hey, just log in and look at the test that I executed. Right. So it definitely encourages or helps with Less feedback loops.

Nice. So I know you've been involved with API testing for a long time. In the pre show you told me about SoapUI. I'm just curious to know over those years at SmartBeer, have you or do you know any like real examples where maybe across team misalignment delayed in API delivery. Aware collaboration saved the day with Smart Bear products. You mean Smart Bear products and that maybe. Yeah. Someone company had an issue, but using your product helped them do X or Y or save them from something.

Oh yeah, disaster.

I don't know.

I've seen this over and over. We've definitely helped your organizations deliver faster software. Again creating we were SOAP UI was one of the initiators of like API automation without having to write code. And you know, over the years I wouldn't, you know, have a particular case in mind, but it's just been an overall definitely helpful.

Nice. So how about signs? Are there any signs where teams look for that indicate maybe they have a current API quality process issue that isn't working for them as a team sport?

Any signs in terms of like current issues?

Yeah. So say you went to a company and you, you look at the process and you're like, you know, they're gonna. This is definitely not a team. They're not working as a team here. Yeah, this here's an issue here or something.

Yes. Well, the first, you know, the first thing that I've found in some organizations is just lack of documentation of APIs. There's a lot of. In my earlier days as well, like it was very difficult for QA engineers to get going. There was missed deadlines, a lot of quality issues and APIs. Just because it, you know, it takes a long time to understand how an API works if there is zero documentation and you know, you don't know how to how what data structures it supports, how to authenticate into it. And all that takes a lot of time. So quality is. Was definitely a big issue with organizations that didn't have tools that help them build documentation that help them build a centralized, cohesive network of assets that QA engineers can reuse.

Perfect. I know earlier in an answer you said quality is everyone's responsibility, which I agree with once again. I keep going to this platform because I think it's a great idea. A lot of times QA is seen as a gatekeeper rather than a quality enabler because you have a platform now where everyone on the team has a vested interest or they can. They have insight into what's happening along the way. Does that Help enable maybe this team first kind of a mindset for creating APIs.

It definitely does. And it encourages, you know, the agile process where you're involving stakeholders really early on. It encourages everybody to be aware of what's happening with the API, what state is in. It also encourages folks to start testing and even integrating earlier by, by using mocking capabilities that the platform supports. Right. By you knowing where the API stands in terms of the API lifecycle, like is the definition already available? You could start mocking things and advancing your work. And it again it really provides that level of quality where it's not becoming an afterthought, it's becoming part of the actual entire process.

So that's interesting. Once again, don't know if this is happening. Do you see developers then create maybe a mock to help them with their testing and then when a tester comes in like oh, there's already a mock here that I can consume, does it help facilitate that type of collaboration as well or is that not there or not possible?

It does, it definitely does.

And that's what we try to encourage, reuse of assets. If something's already created, it should be visible and you, you shouldn't have to go out and have to rethink. The same things happen in the API space. Right. Sometimes you need to build something and you don't know if an endpoint already exists, so you go out and build it. It's that discoverability problem that APIs have also occurs with mods and other assets and this definitely will help reduce that.

Nice. I know a big thing also is dashboarding and metrics. Is there any built in like main, like API quality metric, like you're 80% green or something? I don't know. I'm making this up so that people know where they stand with their API quality before they release it.

So the functionality is non existent. But it's something that I'm, I'm personally really looking into. Like coverage is really important. It just gives you, you know, an understanding of, of how much of an API is being, you know, tested. Similar to code coverage. Right. And unit testing we again the functionality is not existing but it's, it is something that, that we're looking into. It's really, really an important piece of having it as part of the platform.

Nice. So it's been a while since I've done any cirrus API testing. Once again there's always been a dream that you can have your product managers or your non technical people help you with API quality API testing. Is there anything in this Hub that helps maybe a product manager contribute to your API testing or quality?

Yeah, so, so our testing, our newest testing capabilities is, you know, is really easy to use and I know this is something that's sold by many products, but it is something that a product manager can go and take out. I mean, it's as simple as, you know, clicking the design portion within testing capability and just looking at all the endpoints and choosing what endpoints you want to test. So it definitely is something that the product managers can do, especially within the platform and you know, because of its ease of use is definitely encourages that as well.

Awesome. So is there anything, you know, AI is the rage. Do you see AI helping at all with helping with team collaboration, maybe that you would start building into these type of platforms, maybe with API design or testing with AI?

Yeah. So right now the design portion of the platform has a really interesting capability, which isn't beta, but the capability is just a, you know, an AI prompt that helps you like by just typing what you want, like what type of API you want or maybe just an update to an existing design and it'll basically do it for you. You know, an example I have is like I've done a simple prompt of saying, hey, I want to build a CRUD design for an API that cruds a user and you know, basically generates a design for you. So it accelerates that design process and really helps devs get to that next stage.

Nice. So I know that's probably me, a lot of listeners that old time testers like myself and DevOps engineers that are very familiar with SoapUI, they have a special place in their heart for it, but maybe they haven't done a lot of API testing or recently with Smart Bear, how can they get their hands on API Hub? Is there a free trial? How does that work?

Yeah, there's definitely a free trial. We also have something called an essentials plan that it's a very low entry which restricts some of the functionality, but at least gives you a step in the door. So there are definitely multiple options for users.

Awesome. Okay, to me, before we go, is there one piece of actual advice you can give to someone to help them with their API testing, DevOps efforts and what's the best way to find or contact you?

Yeah. So again, I think we talked a lot about this during this session, but finding the sweet spot of reducing that feedback loop right between teams, ensuring that there's a cohesive experience from, you know, from inception all the way to deployment of an API and figuring that out is definitely by using the same products, speaking the same language, including QA engineers and other stakeholders as early as possible. I think that's the key to deliver great software and to, you know, make it very useful. Useful and adoptable for users. Yeah. So that's it. How you can contact me? Well, definitely you can reach out to me at Tamil Tamil, which is my Twitter handle. And you could also reach out to me via email.

And I highly recommend for folks listening to try it out for themselves using the special link down below and for links of everything of value we covered in this DevOps toolchain show, head on over to Test Guild.

So that's it for this episode of the DevOps Toolchain Show. I'm Joe. My mission is to help you succeed with creating end to end full stack DevOps toolchain awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation testing and DevOps. If you haven't yet, join our vibrant community@testguild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit testguild.info to explore how we can create transformative experiences. Together. Let's push the boundaries of what we can achieve. Oh the Test Guild Automation Testing Podcast with hoots and lyres the bards began their song A tune of knowledge, a melody of code through the air it's spread like wildfire through the land Guiding tester showing the secrets to behold.