How to Test Any API Without Documentation

After all the guests I've spoken with over the years, I would guess 9 out of 10 testers only test the easy part of the APIs. The bugs that actually take down production are hidden in the part they skip every single time. So today you're going to see exactly how to catch them. Welcome back to the TestGuild Automation Podcast. I'm Joe, and my guest today is Lutis, who has been breaking software for a living for almost 20 years and testing APIs since 2008. He got tired of writing the same boring API tests over and over again across every company he worked with. So he built RentGen, an open source tool that does all the nasty time-consuming testing for you. The undocumented APIs nobody told you existed, the edge cases nobody has time for, and the security headers everybody forgets. So in this conversation, he walks you through it live, showing you how to actually find APIs hidden under the hood of any app and shares the one piece of advice that will sharpen your API testing immediately. Do you want to stay tuned all the way to the end? You don't want to miss it. Check it out. Hey, Lūdas, welcome to the Guild.

Thank you, Joe. Excited to be here.

Yeah, really excited to have you. We were talking on LinkedIn. You were talking about API testing and how happy path testing has been really is not a good thing within the community. And you actually had to working on a solution to handle this. So before we get into it though, a little bit more maybe about yourself.

Okay. So this is Lūdas speaking. I am originally from Lithuania, but now I'm located in Abu Dhabi in UAE. And long story short, I am like breaking software for a living for almost 20 years and including the APIs. So I don't know if someone is not familiar with the API, but this, they have really simple explanation. It's basically the user interface, but just not for the human, for the different software. And my favorite example is a restaurant. You're sitting at the table and looking at the menu. The kitchen where the food is prepared, right? But you can go to the kitchen directly by yourself. So instead you tell the waiter what you want from the menu and the waiter brings you the food from the kitchen. In this example, the menu is what is available, the API documentation. The waitress is the, like in API, is the messenger who gets your request and brings what you need from the kitchen. And the kitchen is like a server, a system, a black box for us. It does all the work behind the scenes. So yeah, so an API is like a waiter who lets two programs to speak each other and you don't need to know anything about the kitchen, how kitchen works and what they are doing. So, and usually I'm not telling that at all, but it's usually the, what I'm seeing in the market, how the API is tested. So usually the testers is asking for the API, what is available from the menu. And sometimes not always asking really politely what is not available from the menu. But almost never no one asks everything from the menu. No one tries to sneak into the kitchen. No one tries to throw the menu into the waiter's face. No one tries to order all the menu. No one tries to order something that doesn't make any sense, so on. So that's why exactly I created a RENGEN, a tool that do all this kind of nasty, boring testing for you. Why? Because it's time-consuming. And it's not interesting for you. So it's very important. We have two situations. On the one side is the API business logic. Yeah. What is important for all the QA to test and the interesting part, what to test, how to break the business logic. And on another side, it's boring stuff. HTTP status code, pizza mindset, hygiene of the APIs. If there is no any token, it should be 401. And if something, if you're sending bad request, it should be 400 bad. If it is not here, should be 404. You need to know all this kind of stuff and it's really boring and it's not really interesting for you. So maybe let's move to the real example. So do you have any questions?

Yeah.

First, before we get into the real example, which I will, I can't wait to see it, is how do you know when to do API testing? Like, how do you even know that you have APIs or what APIs to test? Or I think a lot of times, because you don't, it's not visual, like a GUI UI that they're over often overlooked. So how do you know if you're a tester, you're listening? Or it's hard to even get started?

This is amazing question, actually. Very good one. And the simple, simple answer is sometimes you're building API like a product, right? You should just, you're building an API, for example, Google building the Maps API for everyone else, for the Waze, for the everyone like Uber using the Google API. It's like a service. So yeah, this one is clear, but yeah, but sometimes you are like a tester, you are receiving the application. And, and no one tells you that the developer is building the API under the hood, right? So yeah, and this has come just from the practice, from the other SAs, from other senior people who will tell you, look inside what is under the hood, right? Inspect, open the DevTools, try to send a request and you will see that something is happening in the DevTools and you will be able to just copy the request and to paste into the any one, any tool you want to, for example, I know Postman, Insomnia, whatever. And start testing it. So yeah, so, but basically no one tells you. Yeah, it's a very good point. And when you're building like internal APIs, usually, like most often, no one is preparing the OpenAPI documentation. No one is like giving any documentation at all. It's just internal APIs. No one's thinking about this.

Okay, great.

So yeah, I know it's hard to get up.

Does AI help? Have you seen people saying, here's my API, can you explain what it's doing?

If they don't have documentation on it, is that a good way to start?

It's very risky. Yeah. I also using AI every day. So sometimes you can get a very great answer, amazing one. And you say, yeah, it's working perfectly. And sometimes you can get like shitty answer that is a little bit. And usually you will get something in the average. So yeah.

All right. Yeah.

I thought maybe we'd do a little show and tell. I know you created this awesome tool. So maybe give us maybe a walkthrough of maybe how to get started or what you want to show for this.

Yeah, exactly. So it's very related to your previous question. How do you know that it's like this application have an API? So I was a lecturer at TalkTechnology for the testing. And yeah, so we have many of the applications that I give, I gave to my students to test. So let's look into one of them. This is an application, uh, API Calculator. Of course, it doesn't make any sense to have a calculator with the frontend and backend, right? For sure. This just doesn't make any sense. So you can build the calculator only on the client side. And, but the goal is just to show why it's so later because everyone knows the functionality. So you don't need to learn anything. This is just for purpose for the, for to understand how to test, right? So yes, the goal of this, of this application is to understand how, how actually the testing of the user interface is limited, are limited for you and how the API, the backend testing even if it's, it is one, just one request, one endpoint, it's time consuming and really hard to do it. So, really well. So, if it's very simple, like two numbers, why, if you provide one number, another number, and just calculate and I will get a result, right? And here is the operators. So, add, subtract, multiply, and divide. Everything working fine here. Divide, subtract. So usually, percent, like 9 from 10, of course, testing the main flow in the functionality. They're providing the numbers, some providing negative numbers, decimal numbers, trying to divide by zero, providing the testing of this, all this, all the paper, and that's it. And it's ended. And only one from the 10, from the 10 is like inspecting and opening the DevTools. So if I open the DevTools, going to the network tab, and we'll press calculate again, I will see the request here. We see that the browser is sending to the server two numbers, operator, and getting the result here, right? So what I can do, I can just copy this because we do not have— because the developers implemented this application without any API documentation, right? This internal, our internal API, it doesn't need to be exposed to the externally. So yeah, so there is no documentation.

All right.

So what's happening, just to make sure we understand, is you have a UI and it's a calculator, but the business logic lives in the API that you don't know about. But using this, the, the, to actually look in the Google Console, you're able to see that it actually is calling an API that you can then bypass the UI and test the API directly since the business logic lives in the API, correct?

Exactly. This is the point. Yeah. All the business logic, all the logic is in the backend. We're sending the request. So yeah. So basically Basically you can implement the client, the logic on the client side and on the backend. There is a big discussion about what can be implemented on the client side, learn them on the server side. But yeah, so basically, like I mentioned, it doesn't make any sense to have local little logic on the backend, but this is just for the learning. Yeah. Yeah. So I'm copying the request, right? And that's it. I can import this request to any tool I want. I can import it to the, I can import it to the Postman, Insomnia, whatever. And here's this, uh, Ren'Gen. My PretaTool. So what it can do, I can just import this request and I will see, yeah, all together somewhere it should be, there is a cookie and should be API key somewhere. It doesn't matter. There is API key and yeah, we can see that it's providing session, two numbers and operator, right? And from this, from here, it's very similar. The testing becomes very, very similar, like like you're testing the user interface. You have 3 fields basically. One is operator, a numerator, right? And 2 numbers. You can provide something invalid, for example, test. You can provide very huge number that will be, of course, out of the boundaries, maybe anything you want. So if we send a request, I see that it's returning the response. Yeah, it's everything is working fine. So if we provide, for example, something invalid, the operator that does not exist, it should return me, yep, it should return me 400 bad request and invalid operator, use add. Add, subtract, multiply, or divide. So this part works like any other API client, very similar. So if I want to test this manually, it'll be like really time-consuming. So I'm providing all the numbers manually and then operators. I need to think about it really how to do it. Joe, I want to ask you a question. How many, how do you think, how many test cases will be enough here.

Let's see. It's a trick question, probably. You do a positive and a negative, 3 fields. I'd say about 10.

Something like that. Yeah. So like an average to be on 10. Actually, it will be depend on the system. In this case, this is just calculator. So I will say it's enough just to check if it's working, right? It doesn't make any sense. Yeah. Yeah. But if you exist, like logic calculations would be used, like in some sense, systems and then, I don't know, some, some system that need to be like operating 99% or like per hour. Yes. So you will need different kind of the testing, right? So it depends on the importance of the system, but yeah. So, but on average, it'll be like all the testers will be able to test this like in 10 or maximum 20 test cases. And what you will do, you will put it in the Postman on the playwright and put this negative and then slow test cases. Put some assertions like you're getting 400 bad request, putting assertion that you're getting error messages. And yeah, but then you import it into Ranjan. I do not want to do this. I do not. I want to have a main flow maybe in the— so it's very important that Ranjan doesn't replace any of your existing automation and it does not replace my existing automation. It's replaced the thing that needs to be tested before the automation starts or the hygiene issues. So, okay. So important. I'm sending the request. And now the request is correct. So, the engine will tell you, request looks good. But again, now it has a valid request to work with. Before, before generating tests, quickly review the mapping fields to make sure everything looks right. Okay, I saw the request, it's not for me, no need to show me this request again. I'm pressing to review and generate. And now you will see here, immediately after the request, you will see that, okay, RenGen identified that you have number, you have two numbers and one operator, some parameter. And this one is, RenGen said this is a string and this two is a numbers. And you can, like in the message, you need to remap them if engine decided that it's wrong. So you can open the dropdown and say, no, this one is like actually email. Of course not email, it's a number. I believe it was a number. And that this one is not a string. We know that only 4 values are correct, so we can change it to a null. And now the engine says enter a valid value separated by comma. Okay, I think I copied it. So this is my valid value. This is my valid. And numbers. So set minimum maximum range for the boundary test. If you will not write it in decimal, it will be like an integer. And if you put zeros, it will be like a decimal. Okay, so what is the boundary? I don't know what is the boundary. I need to go to developer to ask what is the boundary. Okay, let's put huge number. Doesn't matter now for the demo. So okay, so this is the demo. So RanGen knows what is your good request and you need to be to do a little bit of mapping. Okay, let's say the mapping is done. That's it. I am pressing generate and run the test. You will see it will generate 78 tests. Yes, this is a security level and it will check the header. And now it's running also data-driven stuff and you will get also performance just out of the box. So Ranger knows what is a good request in the standard. If I'm sending the number, I should receive the 200, okay? If I'm sending something bad, I should receive the 400, right? Bad request. Oh, I do not forget to mention this one. This one means that check That means it's mandatory. GenGen generates tests based on this setting. So I mark this as mandatory. Is it true? This is really mandatory field. So if it's true, GenGen will try to send the null value and it should be response with 400. Yeah, so no sensitive headers, for example, see that a cache, sometimes developer is leaving the version, like from the, all of us. Recommendations, it should be removed. So this one is passed and some, something is wrong. And like a tester, I do not understand what does it mean. So what does it mean? No spaces. What does it mean? And there is a killer feature. You don't need to understand what does it mean. There is a copy bug report. You just copy this one and you can paste the record to the Jira, Trello, or whatever you are using. And you have prepared the bug report and it'll be everything explained why it's a bug. It'll be the request. Would be the provided header, severity, everything by standard. Let's go to the tool. There is also like a test, like a large payload test. Also many of the testers forgetting to do this. So let's try to send something huge. And we see that my server is accepting this. It's waiting for, like now I'm sending like 10 megabytes. It should be immediately rejected with the 413, fail to load too large. Now we see that it's accepting. And the good part also that you're getting the performance insight. So because we are running a lot of requests, right? We're running different requests with a negative inflow and we're getting the response, we're collecting the response time so we can measure the performance. So media response time. So this one is green. Ping latency, we're doing the ping. So it's good, high. There is a bug also report. Signs, and you can do actually the load test. You can getting the load test out of the box. You can select thread. Of course, it's not full-scale load test, yeah, you need to understand. Just quickly check if you don't get any database locks, or just to get insights like a random gen, like a Nixray, right? To check your API before you're doing your load test, before you're doing your whole automation. So yeah, so also we are checking that there is no any huge responses. For example, it's also by the good practice doesn't make— it's a bad if your request response is bigger than, for example, 150 kilobytes. Yeah, and there is like a data-driven test. You see that, you know, that is this one is like a Num, right? It's a device, it's like generating, adding spaces in front and the back and checking that it should be a bad request. You cannot accept it, or it should be trimmed, or return bad request. Yeah, okay, bad request. I didn't provide the correct value for the boundaries, so You see that it checks, it's provided a little bit bigger one and expected 400, but it received 200, right? So yeah, so need to provide correct value. So yeah, sending all the data, for example, number, this one. Yeah. Oh, it was, you see, it's sending the some really nice, really random string and it was 200, but the response body is not a JSON, right? It's not a valid message. And the 200. Yeah. So a bunch of tests that you can get in out of the box just with one run and the good—

all right.

So let's summarize this really quick. You do one call, you do the mapping, and then it automatically will generate all kinds of tests for you that include both performance and security, OWASP type tests, boundary tests. So not just a happy path test. Is that what you're saying? So you just give it. To call, what the parameters might be, maybe some boundaries, and it automatically generates test cases for you automatically.

Exactly. This is what I'm doing. Yeah. I, how I decided to build this tool, because like I mentioned, almost 20 years testing an API. I started testing APIs from 2008. And I saw the pattern in every company I am doing like 60%, of course, separate, I already mentioned it is a business logic. I'm not testing business logic because usually the API have all the CRUD and like, you Many APIs are connected together, but we're not testing it here. So it lives in your current automation. So, but this part I also was doing and I was writing in the Postman, the Playwright and the, all the frameworks. I saw that I'm writing the same test only for the different APIs, but it's basically the same thing. So I was reusing the same, I was reusing the same, the same automation tool, automation framework, and just changing the endpoints. Payloads and doing the same work. And it says, wait a minute, it's enough. Let's create one tool, universal, to solve this problem. So yeah, that's just how it—

Did you say you use this for sanity first? This is the first test you'd run, and then you do more complicated, like calling one API to another and making sure they work. Is this like the first step one, and then you go and do whatever you do for your other set of tests?

Yeah, exactly. This is the first thing, but you But, but I want them to see from the developers. So actually developers are running this before they're giving to me. Yeah.

All right.

So a little more about the tool. Is it free? Is it open source?

How does it work? Yeah, it's free and open source. It's in GitHub. You can find it and there is like a landing page, uh, rengen.io, and you can go to the GitHub and download it, the latest version. Here it is, the latest release. Yeah.

So thoughts on why someone would use this?

This is, I would think this would be a no-brainer then.

If someone's doing API testing, first thing you should do is run it through this since it's free. And then once you have, okay, all these edge cases out of the way, then you could focus in on maybe more advanced scenarios, correct?

Yeah, exactly. This is what I'm calling. So you don't want to spend time on this kind of thing because it's not interesting for you. So you just want to run it and yeah. And actually, for example, in this case for the calculator, I'm running and maybe I am fine with all these bugs. They are not important, mentioned. The system is not important. So it's not important for my to fix everything. But then I'm doing like, I don't know, when developers like introducing some new feature, changing, doing some changing optimization or whatever, and then doing the regression, I can run it before changes and after changes and compare the result because you can export it to the file. You can run it again. So, Renjin supports also environment. You can create different environments, like in Postman. You can select from the— you can run it from the production, and then run it from the test environment, and compare the results.

All right, so can you run that in CI/CD then? Someone does a small change to an API, and just as a sanity test, it kicks off in the CI/CD, calls Renjin, and runs, and you're like, whoop, nope, don't go any further.

We're thinking about CI/CD. Theoretically, it's possible to run because Not now, but we are working on this and I have an idea how, like, how we want to implement it. I don't want to implement, like, something, I don't know, we have some Newman, some specific tools. I want to be, like, really easy to do it. Like, I'm thinking about to export all the tests automatically to the, for example, Playwright project. It will be very easy. You can prepare the template and then combine all the tests because you know what is the request, what is the response, what was searched, everything is, like, known for the tool. So you can pre-fill and like you will get at the end like a zip file, you just unzip, run npm install, npm start run, and it'll be running and you can put it on the CI/CD. It's possible. So we are thinking about it, how to implement it. So this is, will be one of the solutions.

I know a lot of people sometimes work in regulatory environments, banking, healthcare.

This isn't cloud-based or AI-based, it looks like. So it seems like it'd be safe for Is this because it's all local?

This is totally local. Nothing, nothing is calling back to the server. There is no phone call to the server. There is no server. We do not have any server. That's why it's free because it doesn't cost anything for us. I even host, I even hosting my webpage on the GitHub for free. So it's zero cost. Yeah.

Has, have you shown this to someone and they try to like, oh my gosh, I found this issue. I didn't even know existed. This saved me. X or Y or whatever.

Yeah. So actually after building the MVP, the first version, I run it. Okay, so let's try it. And I went to ChatGPT, opened the dev tools and grabbed the first request. It's like PPL, I think, something like that. It was a bunch of parameters. I do not understand what does it mean. I run with the tool and I found a lot of issues. I wrote an article and ChatGPT fixed 3 of the issues next day. One fix was wrong and they fixed it again. I run it again. I wrote an article again and then LinkedIn., and they fix the content again.

If a developer has, say, already has like Postman collections, should they still care about Rengen? Can Rengen still help them?

So yeah, so Rengen is not a Postman replacement, totally. So we're, I'm not planning to implement any scripting at all. So Postman is one of the best tools on the market for the API testing, for the scripting. They have assertions, so you can do a lot of custom things. So no, the Rengen comes before all your automation, like I mentioned. If you're just running Rengen, And then you start doing your automation.