Web3 Test Automation with Joshua Gorospe

Get ready to discover the most actionable end to end automation advice for some of the smartest testers on the planet. Hey, I'm Joe Calantonio, host of the Test Guild Automation Podcast and my goal is to help you succeed with creating automation awesomeness.

Hey, it's Joe and welcome to another episode of the Test Guild Automation Podcast. And today we'll be talking all about Web3 testing with Joshua. If you don't know, Joshua is a tester with over 18 years of experience. He works as a QA tech lead in the security space, so he's a lot of experience with this. He also has worked in agency and product companies, so he knows a lot of different companies and how they work. And Joshua loves testing and has insatiable curiosity for researching anything related to testing. Currently, he's most interested in experimenting with blockchain Web3 technology, security testing strategies, machine learning, model based testing. All topics that I hope to touch on in this episode. I first met Joshua at RoboCon 2023 that I hosted this year and I really thought his expertise would help you. So you don't want to miss this episode. Listen up.

This episode of the Test Guild Automation Podcast is sponsored by the Test Guild. Test Guild offers amazing partnership plans that cater to your brand awareness, lead generation and thought leadership goals. Get your products and services in front of your ideal target audience. Our satisfied clients rave about the results they've seen from partnering with us, from boosted event attendance to impressive roi. Visit our website and let's talk about how Test Guild could take your brand to the next level. Head on over to test guild.info and let's talk.

Hey Joshua, welcome to the Guild.

It's great to be here, Joe. Thank you for having me.

Great to have you. So, you know, sometimes I botch the intro or botch the bio. So is there anything that you want the Guild to know more about that I may have left out in your bio?

No, you've touched on all of them. And just to let the audience know, basically a fun fact is your articles from back in, I think 2016 or earlier covered robot framework. It actually inspired me to take it more seriously because at first I had found robot framework just casually looking around on various. I'm sure You've heard of GitHub awesome lists. I actually found robot framework in one of them. It was specifically about automation and at the time was just getting off of another company that was more into Gherkin style. And not that there's anything wrong with Gherkin, it wasn't really for me. So I wanted to learn more about an automation stack that could just be more natural flowing, more natural language oriented and robot framework fit my style because I was going to go into more of consulting style of work after that specific time in my career.

That's awesome. I'm always surprised or excited when I host robocon because there's so many things that robot framework does I don't even think about or forget about. Like it can help you with security, testing has plugins. It really is a full ecosystem almost for any type of testing as that we've seen. Also like if you have robot framework in place and you know how it works, you can start integrate all types of testing into your testing suites.

Absolutely. And the interesting thing is when I started working in consulting, I was working for an agency at the time and one of the things that our clients were always after is we want to stand up automation in our project and also we need to do it quickly and we do it in a way that can easily, easily I'd say off ramp the work when the time comes because that's the nature of consulting is eventually you have to hand over the product and it's easier to do that when it's in human readable format. True human readable format, which I think robot framework is perfect for.

Absolutely. And sometimes when people as testers maybe they need to get involved in security, maybe they seemed overwhelmed. It almost seems like you said, because of Zen robot framework, it is human readable, maybe it makes it a little more accessible to the team. Is that what you've seen also?

Exactly. And there's a bunch of situations where I was asked to do, okay, test this thing here and all right now quickly turn around an automation part that we could get out to our team as quickly as possible and tie that into a CI platform too. So I started out with like just a bunch of let's for example some backend tests that were just curl commands that I was using to explore through a system, learn how it works, learn the responses. And then as I learned more of it, I just wanted to just quickly throw that together using robot frameworks libraries and just get that into a CI platform as quickly as possible because they really want to see results metrics and sometimes in some cases they want to be handed like a very simple X unit style report, say show me all the passes and fails and then move on to the next thing. And you're just bang out those projects as quickly as possible for our clients because you know, they care about how many Things you're building for them. What's the next sow look like?

So, yeah, so I guess speaking of clients, you know, you also, you have a kind of a cool niche, you know, automation, testing. And you also know security, which I think is kind of a good combination to have. Is there a reason for that? Have you seen more companies needing a demand for security? Because you just mentioned, you know, you need to be kind of billable as a consultant. So is security something you see as a growing demand for testers that probably a good area for them to get into.

I agree, I totally agree. I mean security is, it's just basically one of those areas that I feel is when I was working in a client. So I'm putting on my client hat now. Client work hat. I noticed that it was an interesting place to find the exploratory bugs, like just hard to find edge case. It's interesting because it depends on the sector you're working in when you're doing client work. But you could just find the most interesting types of problems when you're just kind of looking around casually and then when you catch the scent of something really off, you just go into it kind of deeper and then you go into that domain and say, oh, wait a minute, there's a vulnerability of this type going into this, like, I don't know, building automation protocol. So from the 80s and just pulling from my own experience of when I had to help test a type of building automation protocol that dealt with elevators or sort of like building systems type of things and believe it or not, those are, they're super old and antiquated and there's certain things that you could do on them that are a little bit iffy and they don't feel right. So yeah, just challenging that assumption with those like different types of, well, I guess they're called CVEs or I forget the actual terminology here. But anyways, certain types of vulnerabilities, going into them in the deep end and researching them. Security, really security is a lot of research because the bad guys are always ahead. They're always going to definitely get to something. It just takes time. And when you give them time and when they have the opportunity, it's just going to happen. So we're always coming in after the fact, finding things out and how they did it.

Absolutely. And this just come into my head. I know with functional automation a lot of testers like, oh, I'm going to be replaced by AI or automation. It seems like with security, if you really have a tester's heart. You get to do all the testing because it's so hard use tools to help you uncover things, but you still need to use your tester mindset to actually, like you said, exploratory type testing to uncover a lot of these things.

That's right. And I feel like the automation actually benefits a lot from the exploratory testing. Just learning about what this product in front of you, this thing in front of you, whether it's hardware, whether it's software you're interacting with in the cloud, learning about it, seeing what it does, testing its limitations, finding where that edge is and pushing it a little further. So yeah, it's very important, I think.

Awesome. So you know Web3, I've been hearing about Web3 for at least four years now. I don't know why, for me, I haven't seen it really taken off yet, but maybe it's just because people I speak with. Is Web3 something you see about to explode? Because I know AI had been on the horizon for a while, people like, Ah, AI. And all of a sudden ChatGPT came out and there's an explosion. Do you see Web3 at some point really coming to the forefront at some point with some sort of new innovation like that?

Definitely. I feel like that web3 as like what you just said about AI and chatbots and stuff. I mean, chatbots. It's funny you mentioned it again. When I was working in agency where chatbots were already a thing back then and that was around like maybe 2017, 2016, there were then a fad that was being picked up and clients wanted that more. They wanted that automation that will just interact with their customers and get data out of them and then get them onto the platform and then get them wanting to use the platform more. So I like that it's being picked up again and it's actually seeing utility in terms of, well, chatgpt. You know, I personally have used it myself to say, hey, how do I write this little bit of solidity for this type of protocol or this type of show me how this bit of solidity automation works. It was very good for that. So naturally though, the other side of it is of course, you know, these all kind of work in, I guess, a synergy and hand in hand, I think that, yeah, Web three will pick up more because you have that assistant to tell you how to write a little bit of solidity code it because like, you know, here he is or here it is, it's right there, it's available to you. But yeah, Web three though, it's funny you mentioned that too. I mean, I feel like if we all look at the situation around us right now, I mean, trust in banks is a little low in these times. I've personally, I mean, this is not, let me preface this by saying this is not financial advice. This, of course, this is Joe Show. This is not about financial advice. Yeah. So my predictions are that from 2008 there was a low point of in generally in the economy when these things happen, generally there's a trend. I encourage people to look at the history and watch what happens to these different blockchain technologies as things start to get a little more interesting in the financial sector and the economy. Yeah, honestly, I think that Web3 will definitely pick up in the years to come. For sure.

Great point. So you mentioned Solidity a few times. I'm not sure how many people on this show know what Solidity is. What is Solidity?

Well, Solidity is basically a Turing Complete programming language that is used for creating smart contracts on EVM compatible blockchains. And I believe it's the most heavily used smart contract language. There's different types. There's Haskell, which is used in Cardano, and there's also Solana which is I believe is using more of a Rust syntax. I'm not sure, I haven't tried Solana yet. And by the way, I'm still learning myself. Solidity and in general is the most heavily used one. There's other more Pythonic ones that I think. There's one called Vyper which is a little bit Pythonic. It's more of a Python syntax. When I say Pythonic, it's more of a Python syntax, I believe. I haven't looked into it myself, but there's a bunch out there. But Solidity is the most heavily used right now. So if you're good with JavaScript, Solidity may, you know, you may find it very easy to work with.

Nice. So you spoke how robot framework has a lot of different plugins and I also, I think, I don't know. Did you create this? I saw a robot framework. Solidity testing toolkit. What is that?

So basically it's my first dive into self teaching because I'm a very, I guess, self motivated learner. I've been used to just being thrown into something by myself in both clients, client work and product work again. And this is my attempt to on my downtime pick up something new. But also kind of like my own curiosity because I'm in a number of Defi protocol projects in Terms of as someone who played around with the DEFI protocol and by the way with decentralized finance protocols is what I'm talking about. I've researched them and I've also gotten into them and also gone into their discord communities and actually spoken to a couple of people in their developer communities just to learn like what it is they're doing, what are they used to interact with these dapps basically decentralized apps. So what I did was I first when I find out a new technology, one of the things I try to figure out because I love robot frameworks so much, it's basically my way of figuring out and learning how to bring in these technologies into a type of RPA flow. RPA flow is may say by RPA robotic process automation. So that kind of is my entry point. And then I start playing around with their static analysis tools learning about like what is it they use to check the, you know, the code smells and also how do they check for vulnerabilities or security? Because that's stuff that I care about. And yeah, it kind of like I guess snowballed into something bigger. And then I found out from a friend of mine in a DEFI community that I'm often in Discord. Web3js and ethers js basically what those are libraries to interact with blockchains. So they're client libraries that allow you to basically interact with those blockchains that are out there, for example Ethereum. So with ethers JS is essentially web3js was the first ethers JS was kind of like someone created a slightly different one. I want to say which one is better or which one's worse. But I like ETHERS JS because it's associated with one of my favorite new, let's call it smart contract development environments called Hardhat. And it's a really great library. And after I found that library and I learned the basics of how it's being used to interact with Ethereum and any EVM compatible blockchain. By the way, I found out that there's actually a remote library interface library for robot framework for JavaScript because ethers JS is basically it's an NPM package. You just NPM install it. So I was studying how just any old developer goes in and writes a little bit of JavaScript code and brings in ethers JS into that. And I was like okay, now I see that it's just this bits of JavaScript and I found this and basically the I guess I'll provide these links to you later. So if you could share with your audience. It's called Node Robot Remote server by a GitHub user named Comic. So I'm crediting his or her work because this is the heart of like how I was able to get robot Favorites to the testing toolkit off the ground. Well, it's one of them. This is one third of the bigger picture of what I'm trying to build out. I hope that answered. It's a really long answer. Sorry.

Yeah, no, that's great. So I guess like I said, if a lot of newbies are listening to this. So you would use the solidity testing toolkit to help you test smart contracts.

Yeah, so it's a beautiful thing because this toolkit encompasses a lot of many other tools into it. And what I'm trying to do is kind of many phases of work kind of like combined into one. I want my first phase to be, I want to learn some, just enough solidity to just get it a simple token contract out there, deploy that and then the next is to learn the deployment next phase is to learn how these deployment scripts work, automate that out. It doesn't have to be just about testing. I also want to do rpa, which is a generic approach towards automation with through robot framework. RPA styled deployments, RPA styled security checks. Get them all into like a very nice and neat. Because one of the things I'm personally a fan of robot in robot framework is that log HTML and that beautiful test blogs and test reports. So get those all in one place with that and also figure out some very basic things about how to test these contracts once they're deployed. Because essentially with these tests they're in the web free space. They're called interactive contract tests. So you're essentially interacting directly with the methods of that deployed contract. And by the way, when you deploy a contract, it's out there forever and there's no way of taking it back. It's going to be always out there. And it's also very important that you consider the security of that, which of course I'll get into a little later. If you're into it, it might go on a little longer.

But yes, that's a good point. Like I just thought for some the nature of blockchain is security. It's built on security. So are there common security flaws that people need to think of when they're developing like what kind of security things could be exploited with blockchain that you should be worried about or you should be looking at?

That's A great question. It's very important to, when you're dealing with any blockchain, understand that when you pay for a change to the blockchain, when you pay to get something done on the blockchain, you're essentially paying gas and you have to. When you do things in the blockchain, you're paying gas and it's out there. And that's a transaction. And that's basically think of the blockchain as just a ledger that anybody could see. It's basically a giant backend. So everyone can see that you did that transaction. And essentially there's advantages, of course with that. And of course there's you know, whole suite of like uses and unknown use cases that we could do with that. And the decentralized nature is there's no centralized entity that owns that. Everybody kind of owns that, but also everyone can see where it came from and see where it's going. And of course the disadvantage of that is people can do a replay of that across many other blockchains too because, well, when you deal with a Web3 dapp, for example, you have to have some kind of Web3 enabled wallet like let's use, for example, MetaMask is one of the most popular ones. And Once you use MetaMask on either a test net or with a test net, those tokens, those coins, those layer one protocol cryptocurrencies like Ethereum, they're worthless. They're not worth anything on the testnet. But on the mainnet of course they are what they are and they have a value to the general public. And the thing is, what's done on that test net can be replayed on that main net. So that's called a replay attack. And also replayed across many other EVM compatible blockchains as well. So that's something to think about. When you interact with the blockchain, you are in the public eye that there are things and people that are constantly watching that. If you could think of them as apex predators and your little fish just doing little things and just throwing out transactions, you know, just to test, you know, you might want to think about maybe taking that locally and consider development environments like Hardhat and Truffle. These are all basically development environments for Web three for EVM compatible blockchains.

And I believe the robot toolkit you're creating actually helps you. It has a truffle library, I think. Is that correct?

Yes, that's right. That was the next step I wanted to get out as quickly as possible was after I had played My first step was in taking a dive into Hardhat because Hardhat is, in my opinion, one of my personal favorites. I like it a lot. It's very easy to dive into. The plugins are easy to install and easy to learn. And it's just under the hood after you get the ETHERS JS plugin, getting it in there and installing it into your local. A simple matter of just using your ETHERS JS coding knowledge to get those keywords started. So yeah, if people were to like have that initial experience with Hardhat, which is, by the way, one of the more. They're both popular, but actually both are very. Are backed by very big companies. I believe Truffle is backed by like JP Morgan Chase. Hardhat is backed by, I think there's. As a foundation, I think that backs it. Open Zeppelin. Yeah, Open Zeppelin is one of the most popular. It's called Frameworks for just general solidity development and many other things. And they're very big nonprofit. I believe I have to look them up a little more. I'm still learning. And they're used by many people who have deployed other defi protocols and it's used a lot in the Web3 space.

Nice. So, you know, I know with general security, a lot of times people miss out on simple things like when they check in code running a static analysis or something or a Vulnerability scan for Web3. Does this tool help with that as well? For people that want to maybe get it part of their pipeline built in.

It will at some point. And I'm trying to pull in more of the, let's call them the popular security testing libraries and tools. And the great thing about Robot Framework is it just wraps so nicely around those. And once it's in an RPA process, as long as those tools are installed on the same environment that Robot Frameworks is installed in, it all just will come together really nicely. But yeah, that's the goal. That's the. The next phase of my development of the robot framework Solidity testing toolkit is get more security tools in there.

Great. And I think I also saw something about bot utilization. What's that?

So one of the common things that you see in web3 is different uses of bots essentially that are connecting to the those blockchains via RPC nodes. So with an RPC node that is your. Basically it's a client software that allows you to get to that blockchain, communicate with it, read from it. So I said before that you have to pay to get the, to make a transaction on the blockchain Other interactions with the blockchain are free to read from. The blockchain is free, so you don't have to pay for that. You could just read it anytime you want via an RPC node as long as you have an RPC node with alchemy is one and also Infura is another and there's many other. I haven't really counted basically web3 infrastructure as a service platforms that serve up RPC nodes which are essentially any kind of client software that runs consensus layer execution layer on their machine. They could get into the blockchain and read from it or write to it.

Cool. So you mentioned a replay attack in your previous answer. Sorry, I just jotted it down and just jot my memory. So how do you know that's ATT and CK you should test for? Because I know like Metasploit has built in exploits where, you know, common things that people that are known. Is this something similar that? Is there a tool that helps you with knowing or here are the common type of attacks or exploits that can be done on a blockchain or Web3 type of application.

I think there's. I'm going to just use GitHub as an example. There are basically vulnerability list security lists out there and replay attacks is like there's just one of those. I'm sure you've heard of OWASP OSP top 10. I think that's like kind of like in that top 10. And yeah, it's definitely something that haven't found the definitive list of security vulnerabilities yet. I guess a consensus they have their own actually list of GitHub or open source projects and I know that they have lists in there somewhere. I might be mistaken, but they do have a huge collection of different open source security projects consensus and I forget where to find it specifically, but definitely it's just as simple as going on GitHub. Look up a nice popular security list and you'll catch wind of those types of attacks.

Yeah, awesome. So Joshua, you know, I'm always surprised once again when we talk about robot framework, how maybe it's just me. I would think it would be more utilized but when I interview people, not many people mention it. So what's the future of robot framework? If someone's hearing this is saying well I haven't really used it, I haven't heard of it, should I invest time in it? What are your thoughts on the future of robot framework, the community and things like that?

I think that the future of robot framework is bright. I Also think that like many people out there getting out of school per se, or many people out there like who are just starting out as engineers and they happen to fall upon quality engineering or just QA work or just a software tester, they're going to see the value of being able to put something together very quickly and something that just has a humongous suite of community built tools and community built plugins, community built libraries and community build basically ID plugins and etc. So there's value in that. And when you see that a community is very passionate about keeping that alive, I think they'll understand that's kind of what you want to go for. And I take for example what I saw happen with Smart Baron. They're the gherkin developers, I believe they were. Some of them were like let go. And yeah, that's the thing. And when something is not owned by a centralized entity like the Robot Framework foundation, no one company ever could own that. It's all very tight knit group of many technologists all over the world taking, maintaining that and basically ensuring its future growth.

Great point. Okay Joshua, before we go, is there one piece of actual advice you can give to someone to help them with their automated security testing efforts and what's the best way to find or contact you?

So my best advice to you if you're going to get into security and also building tools for it, which also involves automation, is definitely keep an open mind and also try not to pinch yourself in a corner. Well, I can only write this in Python definitely be open minded to many things and try to be as polyglot as you can. And yeah, and you can contact me on LinkedIn. I'm on LinkedIn and it's very easy to find me.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com a441 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world, we'd love to have you join the fam@testguild.com and if you're in the DevOps automation software testing space, or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the Guild community, I'd love to hear from you. Head on over to test guild.info and let's make it happen.