API Performance Testing Trends with Phil Voulgaris & Keshav Vasudevan

Welcome to the Test Guild Performance and Site Reliability Podcast, where we all get together to learn more about performance testing.

With your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Performance and Site Reliability Podcast. Today you're in for a special treat because I have Phil and Keisha from SmartBear to talk all about their latest API testing report. So when we get into the trends that affect automation, site reliability and performance from an API perspective. So, really excited to hear about this. Phil is a senior Product marketing manager at SmartBear Software for Ready API, which also includes Load UI. And Kishev is a proven product and marketing leader in technology. He's led product management of two successful SaaS products from SmartBear, including their load Ninja and their Swagger Hub products. Really great stuff. So he's a frequent speaker at Test Guild's performance conferences as well. So you're in for a treat. You don't want to miss this episode. Check it out.

This episode is brought to you by SmartBear. Listen. Load testing is tough. Investing in the right tools to automate tests, identify bottlenecks, and resolve issues quickly could save your organization time and money. SmartBear offers a suite of performance tools like Load Ninja, which is SAS UI Load Testing Tool, and Load UI Pro, an API load testing tool to help teams get full visibility into UI and API performance so you can release and recover faster than ever. Give it a shot. It's free and easy to try. Head on over to smartbear.com solutions performancetesting to learn more.

Hey, Phil and Keisha. Welcome to the Guild.

Hey, thanks for having us. Proud to sponsor.

Awesome. Yeah, I love having you as sponsors. You're the perfect sponsor because everyone that listens to this podcast can get value out of your products. And it's something that I use as well. So I'm really excited to have you all, so thank you so much for that. Before we get into though, Phil and Keshav, is there anything I missed in your bios that you want the Guild to know more about?

I mean, one, I think an interesting tidbit about myself is that I was an Intel Officer in the Navy for six years, and I always equate being an Intel Officer to being a storyteller, which is actually how I kind of got into marketing. Right. Like, being an Intel Officer, you're taking information from a lot of different sources. You're synthesizing it, you're analyzing it, you're trying to figure out what's the story here. And I Think marketing is very much the same type of process. So I always say that my marketing career started off as an intel officer in the Navy.

Well, how'd you escape the allure of security, which is a hot topic. You probably make mad cash. And how'd you get into management or. Or marketing?

I mean, I think. Yeah, I mean I transitioned out of the Navy via B school and decided to kind of get into the marketing and I worked at AT&T after B school and joined AT&T because very much viewed as a technology company, not the old phone company that many people have think of it as. I was on their API platform team. We had a suite of communication APIs that we marketed to developers and then I made the transition to our Internet of things team doing software as a service products for IoT device management and then SmartBear. I'm on the Ready API team.

Awesome.

So I've been in the space for quite a while now.

Yeah, I think it's great to know. Even though people may hear marketing manager and think this is going to be a pitch and it's not, you actually have technical expertise and an expertise in this area. So really excited to have you. Kesha, have anything I missed in your bio?

No, but one tidbit I can add is I used to work part time as a graphic designer for four years and it's actually helped me understand products better because thinking of it more not necessarily in terms of features or functionality, but more in terms of where people are, where do they want to go and how can we streamline everything in such a way that they can get to their end result as quickly as possible. Has helped me a lot in my journey into building good products.

Awesome. So let's dive into building good products. One thing I think smartpread does really well with is knowing your customer and one way you do it is by having these surveys. I think you have an annual survey. The latest one is the API report. For the folks that don't know what is the state of API report for 2020 all about.

Well, really the goal of the survey is, you know, we went out to our community, Spargo Hub users Ready API uses a larger developer community, anyone who's involved in APIs. I actually think we ended up sending this out to about 700,000 folks. Like a ton of responses. I think close to about 4,000 responses to the survey. So just a rich data set and the goal is really to provide insights on how the API industry is evolving and the factors that are contributing to its growth in the years Ahead and it's a really nice cross section of small companies, large companies, developers, more section of responses.

Nice. So I think what's really interesting also is in the pre show you said you released this during COVID 19. So these results, I'm curious to know, maybe we'll dive into the difference between this year's results compared to last year. So.

How did that affect the results you think being released during this time?

Yeah, I mean it's interesting because I think we sent the results, the survey out in about May and so people, I think at that time obviously knew the pandemic was affecting them. I don't think they quite knew what the new normal would look like yet. And so I think for many organizations they were in this sort of reprioritization of their API strategy. They kind of understood that we're accelerating our digital transformation. And I think there is a stronger emphasis on collaboration tools than we've ever seen. I think those results, when I look at the survey results, that was a big theme that kind of struck throughout collaboration in the tooling, how people and processes are coming together. That was on the forefront of a lot of people's minds when we looked at the survey results.

Keshe have any thoughts on what you saw compared to last year?

Yep. Yeah. I think the other interesting thing which I saw was we've seen more people actually saying that they develop and consume APIs versus and that percentage was a little different, a little higher. And I believe it's to Phil's point, people have started to reprioritize their API strategy, but also in anticipation of a higher workload because now that people are stuck at home, it's much harder to actually go in and buy stuff. And in the retail environment, for example, you want to order it online similar to, you know, if you want to have good foods from restaurant, you order that online and you get it delivered. So it's not just in the B2C world, but we also saw that in the B2B world. So and I think that's interesting because now there's. Because in order to deliver great experiences, your API strategy is crucial. Again, holistically, when you think about it as a large organization, this result speaks to what we're seeing in the market itself.

I could be wrong. I'm thinking having an API strategy could help you be more digital. So if people were struggling with taking their brick and mortar business online, having an API strategy would make it easier to be extensible to maybe other services like Uber or things that could integrate with them. Did you see anything like that or is that. Am I thinking the right way?

Yeah, absolutely. So the extensibility and the platform perspective of the business itself is critical. Now, I don't necessarily know if that's changed over the last three months because of COVID necessarily because that is a long term play. But in general, the organizations that are truly thinking long term and are truly thinking about how best we can not just build a great service, but also to be able to satisfy as many jobs for the end user as possible, that comes with having good partners. And the easiest way where digital partners can come together and build holistic platforms is through the extensibility and usability of good APIs. So that's something which we've seen always in SmartBear. And I think historically that trend is always moving upwards, meaning more people are actually thinking about it as a platform and not necessarily as individual products itself.

Yeah. And we did see a shift. So like one of the questions we asked was, you know, what is the biggest drivers in your decision to develop APIs? And it's partnering with external organizations saw about an 8% uptick. Absorb content data futures from external products saw about a 7% uptick. So we're seeing this trend of we want to partner, we want to get data out there. We also want data to come on our end too. And so you're just seeing a lot of companies just rethink how they're exposing their data and how they're ingesting it too. And now's the time to probably do some more of that.

Absolutely. One thing that stuck out to me, and I wonder if this is if this was normal or not, is more developers answered the survey, 38% compared to QA. DevOps to 12%. And I'm curious to know, do you think maybe that's because developers now have extra pressure to start creating APIs? Is this normal? Is this what it was last year? Do you know?

I think at a high level it makes sense that more developers, we would see more developers in the survey because I think developers are seeing a lot more testing responsibility. And so I think when we think about our own customer base, even from smart bear tools, developers is a larger subset than in years past. That trend actually makes sense when I think about it that way.

Cool. Yeah. For some reason I forget SmartBear has been growing by leaps and bounds. And so you're not just a testing vendor anymore. You really are taking more of a holistic approach for sure. So I guess diving into that holistic approach, I thought maybe Go into a little bit into maybe trends that were pulled out of this report. So any key trends that really stood out to you? First off, before we dive in, a little bit into maybe the nitty gritty.

Yeah. I think one question that was really intrigued me was we asked, in your opinion, which of these technology areas do you expect will drive the most API growth in the next two years? And the top two answers were microservices and DevOps. And when you think about those two things, they're not really technologies, right? I mean, microservices is really like a philosophy. It's an approach to developing an application as a suite of smaller services. And DevOps is really just like a culture of collaboration, ownership, learning with the hope of accelerating your software development. So I think what I take away from that is that it's really about people strategy driving API growth and not about like a single technology like blockchain or VR ar, which scored really low on the survey. So again, it's about this idea of like, you know, API growth is really driven by the top. It's really driven by people by having a strategy of interconnecting this that's, that's really driving API growth. Not a single technology.

And one interesting trend I saw was when we think about platforms, and you said it right, Joe, when you said that more developers are typically building APIs and potentially also consuming them as a byproduct of that. If you have more developers building APIs, there is an important need to standardize on how they're building it as well. Because the last thing you'd want to have is consumers having to bear the burden of poorly designed APIs. You know, when you're integrating with one API, you think about it in one way and then you look at another API and it feels different. And that adds more workload to the consumer, which is what the developers are truly in service of, is to build consumable APIs. And what's interesting is this is something which we've seen over the last four or five years. The open API specification is still the most predominantly used standard for designing APIs and organizations. Good. The vast majority, about 33%, have said that they do use internal style guides. Right. So that means that they are using some sort of framework in order to design these APIs. But then we asked another question, which is.

How would you rate the quality of your API documentation which comes directly. If you have a good, well designed API, you'd want to think about documenting it so that it's easy to consume and understand as well. And only 5%, I believe, said that they're happy with it. You know, that they think it's very good. The majority still think that it could be improved and there's areas to improve and they're not necessarily as satisfied, which speaks to how important it is to not just think about it as, you know, developers building it. All right, now it's time to do something else. And let's just go back to our work of building. It's important to think about it holistically from an outcome perspective, which is ultimately you're building APIs so that they get consumed and to consume those APIs. It's important to think about it from a design perspective as well as a documentation perspective itself. So that was also very interesting.

Steve Jobs, I think I misquoting him, but I think he said something like customers don't know what they want. So because you speak with a lot of customers as well, even though a survey says something, is that the same thing that's lining up what you're actually seeing in the field for people who are struggling with like, is DevOps and microservices really what people have been telling you, oh, we're struggling with this. We need a solution for it before the survey. Or does it line up with what you've been hearing in the field compared to what people actually saying they want in the survey?

I think it actually lines up pretty well. Interestingly, you're absolutely right. I do think when you get these surveys, you got to take with a grain of salt. But it's important that this quantitative data I actually does think match the qualitative data of just going out to the field, just hearing customers talk about, well, we've got a ton of internal APIs and there's no standardization amongst them. And we need to do a better job of testing them across our, across our company in a very standard fashion. And everyone's on the same page like that aligns with this idea of, yeah, microservices are driving API growth. So I think I've seen a matchup and I don't know if Kashif has maybe a different opinion.

Yeah, I mean, I think it's, for the most part, I would say, yes, there is a matchup, but there are things that, you know, did catch me by surprise. So, for example, developers actually building more APIs and actually developers also testing them. That aligns with this shift left trend that Joe, you've been talking about for a while as well. Right. I would love to see more performance testing done by developers hopefully that'll change as well in the coming years. But we are seeing that a slow uptick. It's not as prominent like, yeah, 90% of developers are now also testing their APIs, but it is like quality is becoming an important part of that overall process itself. But for me, the same thing which I mentioned before was that the documentation piece and how people still don't think about API documentation as quite important and API documentation is also important because ultimately when you do do that handoff to your QA organization to do the functional testing and then which then gets handed off to maybe a performance engineer to build performance test to it, if they don't understand what the API does, then there's additional time spent talking to your developers, talking to the product managers or the architects to figure out what this does and building test cases around it. And we've seen that, you know, if you look at a question we had, which is potential risks for the quality of the APIs in the organization.

The highest one was, you know, loss of time and resources spent testing and troubleshooting. That's the biggest problem which these survey respondents have said could contribute to, you know, potential risks in the organization itself. So that for me was quite an interesting trend which didn't align as much as I hoped it would with what I'm seeing.

One trend I saw, which I had liked, goes to your thing about performance. How do you measure success of an API getting developers to do more performance? The top one is performance and the second one is uptime and availability. So the podcast is performance side reliability. You said developers doing more testing of it, but yet in my perspective, they still kind of lag behind and actually measuring it the way they need to be measured when they're doing testing. So are you seeing that disconnect as well as though even though developers are doing it more in testing, but the measurement or rate of criteria, the success for the API is performance and uptime, yet that's not what they're really focusing on with their testing efforts.

Yeah, I mean, I truly think that this is something which we've seen the same exact trend in the previous results as well. People who've responded to this and people whom we've talked to as well indicate that performance of their APIs is one of the biggest identifiers of success. However, just the general understanding of performance, there seems to be a disconnect many times, you know, I have, we have developers in smartware, you know, we have a big, healthy software team, and most of them, when you ask them performance as well to some chunk of them would say, yeah, as long as, you know, when I'm making a call multiple times and I'm able to get the response and like, whatever the SLA is, I'm happy with that. Whereas when you think about truly what performance is, is to understand how responsive the API is when your servers are stressed by concurrent usage and that understanding or that gap in understanding comes from, I would say, a lack of awareness and knowledge and also potentially a lack of tooling that could help them overcome that barrier and make it much more easier. And smartbra, as you know, we've had a performance testing product offering for a long time now. Load UI has been operational for a very long time. We've launched Load Ninja two years ago and just early this year we launched API load testing as well in Load Ninja to help overcome this, to allow people to, you know, especially developers and functional testers, to be able to create performance tests seamlessly off the fly. The ability for Load UI to consume SOAP UI scripts is actually a big reason why we have so many developers within the ready API ecosystem actually doing performance tests, not just seasoned performance engineers itself. So our hope is that by investing in our performance testing portfolio of tooling and also some of the work our marketing team, like Phil's been writing and blogging about, performance testing, best practices, webinars and so on, we can bridge that gap of knowledge as well as solutions and functionality so that users can get to identifying that metric of success, which is performance.

And I think it's interesting that, you know, you mentioned it right. Performance.

Uptime availability is the largest measure of success of an API. I think you correlate that to some. Another question we asked was, you know, how do you interact upon encountering quality or performance issues with a third party API? And we saw a large uptick in people who are willing to switch to an alternative API provider. So we're kind of at this inflection point where I think it's becoming table stakes now. The cost to switch is much lower these days versus the cost to stay with an underperforming API. And I think the server results bear that out. And I think that's something for people who are building out APIs and making them available to third parties, our partners, we need to be wary of that.

Great point. So I guess what I also was surprised by is like you mentioned, microservices, DevOps were the key areas that people think it's going to grow in two years. Machine learning and AI was fought behind. Internet of Things. Internet of Things has Been people have been talking about that for years. But.

Do you see any development in machine learning in the next few years? And if so, do you have any solutions that would be able to handle it? What do you think the concern would be with AI and an API from a testing perspective?

So yeah, so I think AI has become. I would put in some of my personal opinions based on my research and correlate that with some of the findings that we have here. AI and machine learning has been spoken more of by analysts and media over the people who are actually in the weeds trying to fix the potholes that are in the present. Right now, AI and machine learning is an objective that we as an industry will eventually get to. And some industries are way ahead than the others than others. For example, OpenAI and what Elon Musk is doing with Tesla and self driving and everything else. There is to some degree some level of learning that's involved as well. And we are also seeing that to some degree in the testing world itself. But in general, if you think about the core aspects of your day to day as a developer or tester, you're doing less AI and you're doing more. I need to operationalize this. I need to have developers and operations working together. I need to have multiple components talking to each other through like the microservice architecture in order to deliver great functionality to my customers. That's the forefront of any developing organization because I always think of engineers as people who are trying to fix potholes that are present for them at that given instant and not thinking about what will happen four or five years. And that's where the analysts and the media come in and build that buzz. So I think AI and machine learning is definitely something which will happen. I'm actually pleasantly surprised that AI and machine learning is still 4 and 4th place, the Internet of Things.

I'm hopeful it'll increase, especially with Alexa and all these different machines talking to each other. But I'm not surprised that it's third. In fact, I'm pleasantly happy that it is in the third place because it's right after two of the most important problems that API solves, which is the microservice being able to talk to different components together and also operationalizing developers and operations and making them deploy faster and so on.

Now Kashiva, you did bring up before and the report brings up as well, documentation being an issue with APIs. How does that affect people from performance and say, reliability perspective? Is it because they don't know what it's supposed to do? Then it's hard to really capture realistic performance measurements from it. Or what are your thoughts on how documentation actually affects the quality when it's lacking like that?

That's a good question. I would say.

Ultimately it's about developers are trying to solve for the job of building APIs. And then in an organization, you could either have a QA team or the development team itself running their performance tests for these APIs. What's important is ultimately whatever has been built needs to be understood by the performance testing organization, or the QA organization, or even the development organization in order to build those test cases, in order to build out your scenario, configure your scenario, and finally run them and analyze the expected results. So the key part over here is being able to understand the API and trying to minimize as much back and forth in order to get that understanding done outside of just the real benefits that API documentation brings to end. Consumers of the API, in this small microcosm of the development of the API, the consumer is the performance testing individual. So they need to be able to understand how this API works and what the expected result is in order to be able to coherently build tests, maintain tests long term when the API gets updated, and also be able to paint performance testing metrics in a way that reflects what the expected behavior is. So the more jumbled up or built on the fly approach you take, it would work in a small organization where there's like less of a communication barrier. But in our distributed world, where, you know, we have so many people working across so many different geographic boundaries, especially as your organization scales, this approach would need to be changed. So it's really, this is where the, where people have been talking about it for so long. The design first approach helps because now you're thinking about the API not as a functional component, but something that is built in order to be consumed by someone. So you're thinking about design first. And luckily with frameworks like the open API, you can generate the documentation out of this itself, right out of the box. And that helps long term because now performance testing teams and functional testing teams don't have to wait for the developer to build out the API. You can like just design the API, get the documentation out of the box, mark out some of the dependencies, and start building test cases without the backend having to be fully built yet. So it really helps the team deliver on the promise of DevOps to some degree and shift left and delivering value to the customer faster.

Yeah, and I mean, I think again, one of the biggest challenge that API developers, testers face is like standardization, standardizing documentation is a part of that. Documentation is very much the like entry point to a lot of people to consume your API. And so if you know at a high level, if your documentation is a poor quality, what does it mean for the rest of the underlying API behind it? And so yeah, you want that first experience to be a great one. And so the documentation, standardizing it, making sure people understand it easily adjustable is an important part of quality, like a macro level as well.

I like to get your opinion on this. I was disturbed by the answer to this question. What would you consider to be the greatest potential risk of poor quality APIs to your organization? You would think your customers start abandoning you, loss of contract, loss of money, you probably lose your job. No, the number one concern was loss of time and resources spent on testing and troubleshooting. That's what people care about. So I mean I understand it, especially with Agile, you're trying to ship quicker and faster, but it's kind of concerning. So are there any tools or techniques that you think maybe people aren't aware of that maybe SmartBear offers or other vendors that could help them get over this particular risk so they could stop focusing on things that really do affect your customers and your company's bottom line?

Yeah, so it's actually surprising, but it's also indicative of a larger problem within the industry. Right. Where we are so trained to think about our problems and not necessarily holistically about what the organization would face. And if you look at Most of the APIs that are built are internal APIs, so they're consumed by other developers. So people are not thinking about oh this, the consumption of my peer of this API would have indirectly affect the experience of my end customer of the product I'm working on. They think of like oh Phil is going to be consuming this API, he's going to be giving me feedback. Now I have to test it and build it better. So that's what they're thinking. So it is indicative of I would think a larger problem of how we're not thinking about customer outcomes and are more focused on what we are doing on a day to day in terms of how we can train people on this. I think it's a mix of evangelism and understanding and training. Which is the good thing is Cucumber, which is a project sponsored by Smart Bear. It allows people to start thinking about it from a behavior driven perspective, applying the BDD framework of testing itself and enabling and empowering developers and testers. To think about it not necessarily as hey, who's if I do this, someone else is going to start complaining, but more of like what is my end customer trying to do and how can I start building and enforcing those test cases within the API or within the test framework that I'm building as well. In addition, the Design first approach, which is promoted by the Open API spec, formally the support of the Swagger Open Source tooling and Swagger Hub allows architects to be in the forefront of the development process and start thinking about your APIs. Not necessarily a small components, but also like thinking about it holistically as a consumable entity that other organizations and other teams will be starting to use as well. And with Swagger Hub, there's features where you can start giving comments directly in the API layer right in the design spec. So that way you are involving people who are on the business side itself, not just the developers to start giving you feedback. Because ultimately the utopia is an API should be consumed by someone who's not technical as well if you can deliver on that value. And I quote Stripe, everyone in the industry will probably quote Stripe as this utopia or this great example of good documentation. But Stripe, anyone can, you know, with basic understanding of APIs can start building out functionalities on top of that. And finally, I would say the way the Ready API platform is integrated with the Swagger Hub and the Open API tooling ecosystem in general, allows testers to pull those designs from the Swagger Hub platform and start building out test cases, mocking out any dependencies and either creating rich functional tests or concrete performance tests right out of this. So that way you're like now empowering the entire organization across the lifecycle to start thinking about your customers and how you can enable the success of your API to get your customers happy and delight them with good experience.

Yeah, I think that type of data, especially that loss of time resources spent, has sort of had us rethink about us. You know, what do we double down on and double downing on? Collaboration and integration are two key areas. So what are the tools that developers currently use and then like GitHub, which kind of brought up in building a better workflow within the tool itself, within Ready API. So you're not losing time going in and out of different environments, but then also making sure that we have a better integration with our own tools. SwaggerHub and Ready API are talking to each other and you can import data and there's a cohesive environment for these two different teams, but bringing them closer together, I think that's how we've taken this information rethinking, what do we double down on? How do we integrate better with the tools developers use and connect the dots and build a collaborative platform for developers and testers to talk to one another?

Absolutely. I guess on the sunny side, there's two questions here. I don't know if I believe them. Less than 20% of the time they push issues to production due to an API. So performance issues, things like that, less than 20% of the time. So that's pretty good. Or 10% of the time or less. And also most organizations have a formal API testing process in place. So maybe it sounds like people have a handle on testing and a process, but they get bogged down in the actual time sync of doing the testing itself and collaboration. Is that what you get out of that as well?

Well, I also, I think the other thing I get out of it is that larger organizations are putting more priority on it than the smaller ones. Because I think the data you're referring to talks about how, you know, if you've got 10,000 or more folks in your organization, then yeah, you need a formal testing process. The smaller ones aren't there yet and that's one area. But at the same time the data shows that they want to formalize it more. They want on their roadmap, but they're just not there yet. I don't really have much more to riff on that.

Yeah, cool. Also, which I found interesting, is a formal monitoring process in place as we shift left, we also shifting right as well. I think that's a big area that a lot of teams miss out on is actually finding issues in production and then feeding the information back to the organization so they can get better on it. So any thoughts on monitoring production? Is that something you see a growing trend in or something you would focus in on with the tools to help them with?

Yeah, I would say that monitoring, I think is. Doesn't get enough love in the API world to some degree. I mean, people are happy with just enabling like, you know, some sort of basic monitoring in place, like, you know, Cloudfront or something like that. And once you get alerts, you're left with like comprehending that data and then trying to correlate that with what's actually getting affected. Right. So we do see a lot of real user monitoring that's been in place, but from a synthetic perspective, you know, which is I think should be just as important as real user monitoring, but a little more harder to set up. That's probably where there's a growing opportunity for organizations to be ahead of the game and not wait for a real customer to experience this, but actually have alerts set up that there's a constant automation of pings to your APIs to make sure that they're functioning right before a customer actually faces it. We do have tools like AlertSight which allows you to set up synthetic monitors. And again, our approach from an organization perspective is to make sure all of these tools are integrated with our core offering of delivering quality APIs from the forefront. So Ready API integrates with Alert Site so you can now be able to take your tested APIs and then start building monitors off them. And in general, Ready API also has server side monitoring as well as you're running the test itself, though of course that's not production based monitoring.

By taking advantage of AlertSite's synthetic monitoring, but also being able to integrate with real user monitoring tools like New Relic and so on allows the tooling that we have to flexibly get deployed in the organization's workforce without having them to subscribe to a monolithical approach that we are enforcing on them.

So this is a very extensive study. 60 pages. You have seven key takeaways. I'm going to just give it a teaser so people can go and get the report themselves, but I want to hit on maybe one or two of the key takeaways and folks should get ahold of the report themselves to see everything. One thing I see a trend on with API is Kafka and things with the growth in like Graph, SQL and grpc. Is that a trend you saw as well as a key takeaway that people are struggling with? Maybe the APIs are all different so the standards are different and being able to keep up with that as well may be something that is holding them back.

So in terms of like we are seeing an uptick in event driven APIs, that's definitely there. GraphQL and so on. I think Facebook GitHub have done a good job of showing people the benefits of using this approach and because especially Facebook with its identity management API. So many apps that we use on a day to day basis integrate with the sign in and sign up functionality is done through Facebook. So a lot of developers are used to the ease of views which GraphQL promises. So definitely we do see that in terms of.

Where people would aspire to be and interesting trends that are coming up. But I still believe that there's a large set of the industry which is on the rest world and there's still like rest is Something which is proven, it's not avant garde as much. It's something which developers are used to building and testers are used to testing and consumers are used to consuming and it's using similar protocols that we're all used to as well. So I think that's where people are still, if it's work, don't break it that approach. But for more of the forward thinking organizations, especially in, you know, in today's world we're seeing posting, you know, shipping services and so on as well, starting to use a lot more of the event driven APIs. And it makes sense, right, because you'll need to have.

That layer of communication needed in order to get items and alerts faster. So we're seeing that trend and that's probably also because of COVID and everything else. But in general people are still in the rest world and more forward thinking organizations are investing in GraphQL and Kafka and so on.

Awesome Phil, anything want to add to that?

I mean I know in our roadmap we've got event driven architecture as a key point coming out in the next quarter or two, so to catch that point, yeah, rest people still know it, love it, but these other new protocols, I mean we're developing to them because we are seeing a shift in them and more an increase in usage for those APIs.

So yes, yep, I've been speaking to a lot more people doing big data testing and a lot of it is using GraphQL and also Kafka and all those types of technology. So definitely a growth area there. Okay Phil and Keshav, before we go, is there one piece of actionable advice you can give to someone based on these results that would help them with their performance testing efforts? And what's the best way to find contact you or get our hands on this awesome API report.

So I would say that performance is something which we as an industry and all the practitioners involved in building APIs and delivering good products agree on as one of the highest measures of success for the quality of an API. And this directly impacts the end customers that are, that are consuming the products that's built on top of these APIs. So it's really important to start thinking about APIs not necessarily as code level components that talk to each other, but more of value delivering layers of data transfers that allow customers to be delighted by the services that you offer and also allow partners to start extending on your existing platform. So performance is very important. And if you start thinking about performance not as something which you do once in three months, once in six months, but actually bring that in the forefront of your QA process. It actually enables you to be successful in the long run. That comes with a mix of understanding how to go through this and also understanding how to use the right tooling. Luckily, SmartBear has tools that allow you to build these performance tests right out of your functional tests from the API layer. We have that in the ready platform where Load UI, which is our performance testing desktop tool for APIs, seamlessly consumes any SOAP UI test that you have and then spits out a performance test right out of the box in the most quickest way possible. We have a Load Ninja which does the same on the UI side as well. And so by taking these tools and also building on top of your knowledge base, you can achieve this objective of truly ensuring your API's performance and giving delivering on the value that your API is promising for your end customers.

Yeah, I think just do it. It's amazing in this space. Sometimes I feel like we're not even selling the value of a particular product, but just the value of performance testing in general to customers and think they've got a good sense of like oh well you know I can do it more project based. I don't have to do a continuous performance cycle. I know when the spikes are coming up, but never know when like the COVID hits and all of a sudden your e commerce sites get overflowed because you know all the retail shops are closed. Like you just never know. And I think to be able to have it as part of your testing process is key moving forward. We live in an uncertain world, so to bring that certainty to your process is key.

Thank you Phil and Keisha for your performance testing API awesomeness links to everything in value we covered in this episode. Head on over to testguild.com p and while you're there, make sure to click on the Try them both Today link under the Exclusive Sponsor section to learn all about SmartBear's two awesome performance testing solutions, Load UI Pro and Load Ninja. So that's it for this episode of the Test Guild Performance and Site Reliability Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack performance testing awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild.

Performance and Site Reliability Podcast. Head on over to testguild.com for full show notes, amazing blog articles and online testing conferences.

Don't forget to subscribe to the Guild to continue your testing journey.