Open-source Stack for RPA with Antti Karjalainen

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we'll be talking about Auntie all about Open Source Stack for RPA and RPA in general. It's a really hot topic that I think anyone in automation needs to know more about. Anti is the CEO and founder of RoboCorp, which seems to be a rapidly growing company. We'll probably dive into that as well. He's a Robot Framework foundation board member, software engineer and Open source enthusiast. I think you need to listen to this episode because I think rpa, as I've mentioned before, is a growing trend and I think the skills we have as automation engineers is only going to help in this area as well. So open up your skills to make you more employable in the future. So you don't want to miss this episode. Check it out.

The Test Guild Automation Podcast. This SAS is sponsored by the fantastic folks at Source Labs. Their cloud based test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial. Visit testguild.com sourcelabs and click on the exclusive sponsor section to try it for free for 14 days. Check it out.

Hey Anti, welcome to the Guild.

Hey Joe, thanks for having me.

Awesome. Before we get into it, is there anything I missed in your bio that you want the Guild to know more about?

I think you nailed it. I started out as a software engineer myself, wanted to do something more entrepreneurial, got into consulting at some point and then, you know, that took me into a project called Robo Framework where I've been heavily involved there and through twists and turns ended up founding this company called robocop a few years back. So that's kind of the background debrief.

Very cool. So most people, when they think of Robot Framework from my audience, think about test automation. So how did you get involved in RPA then if that was the kind of system you were dealing with earlier?

Yeah, good background question. So I saw RPA starting to pick up speed in 2016 already. So it's early days for RPA. It just happened to bump into it and at an event and then, you know, what I saw there was I instantly recognized that RPA was on the technical level really similar to test automation. So in RPA you try to automate an application to perform business tasks and in test automation, well, you automate apps to, you know, validate them. So the things that these automation systems do at the end of the day for applications is really, really similar. The goal is a bit different. You know, where it lives inside of an organization might be different. And how you control, manage and monitor, orchestrate, that is a bit different. But really the fundamental thing about what you want to achieve is it looks the same for an engineer. So I got really, really curious when I saw RPA for the first time and started exploring. I basically called up everyone I could find who were doing RPA at that time and tried to interview them and wanted to know more. And so that got me started. You know, background in Robo framework was the perfect starting point to explore rpa. And what I couldn't find in RPA was there wasn't any open source in there. So somebody coming from an open source software development background, it just felt wrong to me in a way that open source being the biggest enabler of software productivity over the last 30, 40 decades. So why don't we have that in this growing and vastly expanding space called rpa? So that was my background.

Absolutely. And it's curious you mentioned that. I know being a beginner to rpa, the first tools I hear about are vendor based tools like UiPath and tools like that. So do you think it's just natural that like in test automation when I started 20, 25 years ago, it was all vendors and then came Selenium. So is this like just the natural progression that you see started off with vendors and now it's going to, you're starting the open source type of revolution?

Yeah, absolutely. I mean if you ask the CEO of an open source RPA company, that's the answer that you're going to hear. Obviously my thesis was that it's going to be the same. So we are in the begin of this new field of automation, rpa. We're going to see the first coming, the vendors making it acceptable, making it something that people want to do. Then we're going to see a wave of disruption by open source technologies like RoboCop that really unleash the potential of developers everywhere onto this field. And that was exactly my thinking when I first saw rpa that hey, this is an opportunity to do all over again what we saw in test automation to rpa. But I think this time the rate of iteration is going to be faster. With test automation it took like a few, few decades, you know, and now it's going to be like, you know, five years I guess.

Another parallel I've seen is when I started off test Automation, it's all offender based, you know, it's just drag and drop and record and playback and you don't need a developer, you could just have a business user do it. And it's kind of the same vibe I'm getting off RPA as well. Is that what you see as well? You seem to be geared once again like what they did with open source, geared more towards developers. Is that the differentiation that you've been doing as well?

Yeah, yeah, exactly. So when I first saw rpa, I started exploring the technology and kind of took the claims at face value as, you know, look what the vendors were saying, hey, this is a drag and drop tool. It's so easy and simple that your accountant is able to automate away their tasks. And I thought that, man, these guys have really figured out something that we couldn't have come up with on the open source space in test automation. Like have they finally cracked the challenge of making non technical folks being able to automate? And I started digging in and I started, you know, meeting people who were doing RPMs. Like I never could actually find an instance where sort of non technical person would have successfully automated anything with these, you know, vendor based RPA tools. So it kind of struck me at that time that hey, this isn't, you know, any different than test automation. At the end of the day you have this Persona of, you know, an automation developer, an RPA developer, test automation developer, you know, the automation engineer, if you want to call it that, that comes in and solves these automation tasks and they are really, really similar to test automation. At the end of the day RPA might have longer running tasks, but thing that you build as an automation engineer there is really the same.

So I guess we should just maybe explain a little bit what you explain how you would explain what is rpa because how is it different from test automation? Is it test automation just validating something and RPA is just driving something to get a particular outcome from a business perspective. Like how do you explain what is rpa?

Yeah, so let's start with RPA basics. So you know, we have a lot of different types of automation in business processes. You know, there's workflow, automation, like tools like Zapier, et cetera. They're not rpa. RPA is about, the way I see it, it's about the robotic process automation. So the robot is the key here. Robot meaning an agent, essentially we talk, in short, we call them bots. So bot, the bot is the key thing. So it's about building these automations that can act as a human user would on any IT system out there to perform tasks on that particular system. So think about something like, you know, typically you start with tasks in RPA that were previously done by human employees, something like entering orders into an ERP system, they come through an email, you know, you pick it up, you maybe you read a document, then get the data off of that document and then enter that into an ERP system. So you could build a bot that can like log into that ERP system with their own credentials and start using the UI to punch in that data. You know, you might do something like month end reporting where you used to have people having to go into financial systems or like accounting systems and extract report after report and consolidate them and validate and send them to their auditors, etc. And now you have bugs to do that. So the key differentiator in RPA is that ability to run an automation with your user's credentials on a specific environment, at a specific time, based on a specific trigger. And you might oftentimes have internal business applications that require you to be running inside of a company's network to be able to access that. If you kind of compare it to test automation, then you're working with live systems. So you don't have your test environment, you can't do various setup and tear down stages. You need to actually be able to work on live systems. And if something goes wrong, you don't just fail. So that's kind of the easy part. In test automation you try to make it fail, but in RP you try your best not to fail on anything and you build in a lot of error handling in these steps. So that makes it really different from kind of the development perspective. Then the orchestration systems themselves, they look a bit different. So in RPA you have a concept of the orchestrator, we call it the control room. And that's essentially, you know, really job specific. Jenkins if you wish, or CI server, that's a system that can run those bots, then inside, in production, deploy them where they need to run, monitor them, maintain and collect reporting and analytics. So that's kind of in a nutshell what the differences are.

So you brought up a good point about reliability being critical because you are probably running business critical operations. You could be creating a policy for insurance and in production or whatever. So it has to be reliable. So I know with test automation there's certain factors that can create a script not to be reliable. The way it's written, timing isn't done correctly. So what Are some of the issues you find make RPA processes flaky or unreliable?

Yeah, yeah, they have the same kind of reasons that you see in test automation. So, you know, try to do record and playback and doesn't really work out. Some of the vendors are really like making wild claims about their record and playback capabilities. But it's all the same story again that we've seen before. So what you need to be able to do in RPA to make it reliable is to build it on essentially like event based. Don't do like static weightings, weights and these kind of things. Just, you know, the basic principles that you see there. And you know, with RoboCop how we approach it, we have a concept of the best path automation. So we use Robo framework as the underlying technology because that allows you to access the application in question on, you know, multiple levels. You can obviously interact with the, the visual layer itself, like to image based automation, just like matching screen regions. That's really the most fragile way to do it. Then you can go into locators like UI elements, et cetera. You can go into APIs, you can go into all the way to the data layer itself into database access. So you can mix and match these different modalities of accessing the application and always find the best path for that particular automation. The same way as you would approach, let's say test automation, where you want to modify the application through the UI and then validate it against the database. Same here applies, but in a kind of a different way where you might do one action on the ui, but if you have more robust way to perform that through the database, go ahead and do that. So that's really an antidote to the brittleness and the flakiness.

How do you know the robot you created is doing what it's supposed to do? I guess, I guess with testing you always have a test at the end to say, to validate, to verify. Did this work? Do you always have a check with the robot that does it after it does a process? Or is it just a manual interaction where a human goes in and verifies it?

Yeah, good question. So actually you can oftentimes build these bots in a way. Well, in RPA we talk about two different kinds of bots. So you have attended bots that run on your user's desktop and do stuff there. Then you have unattended bots which run in the back office and completely autonomously perform tasks. So with the unattended stuff, you would typically also kind of want to validate Your actions after the fact, make sure that they are done the correct way. Not everyone will do that. So that's a good point to maybe raise while you are performing update. That's that you know, if you complete a business transaction, make sure that that's actually completed. But in the attended automation case where you have the, you know, the human user watching the bot do its thing there, you typically just leave the final commit stage out and let the human confirm that action that the bot has done.

Cool. So I guess we talked a lot about RPA and what it's doing in production. A lot of things I hear with RPA as well as business users who create rpa. And I know your company takes more of an angle than you think developers should be. I think your company takes the approach that developers should be more involved. I guess my question is who should be doing rpa? Is it both? Is it a combination of both? Is it always a developer? Is it a tester? Who. Have you seen companies succeed with rpa? What approach do they use?

Yeah, you know Joe, we have some numbers that tell us that 11 out of 12 RPA projects fail to scale beyond simple POCs, proof of concepts. But the projects that we see scaling to use dozens or even hundreds of, in some cases thousands of bars in production, they've always organized the RPA function through something called in the industry as CoE center of Excellence, a fancy way of saying an RPA team. So we have these RPA developers there like who do this day in, day out and build these parts and then maintain them. And they might have RPA business analysts roles who collect the requirements from the business users on these tasks, identify opportunities, etc. Again, something of an RPA jargon term would be PDD. So process definition document. This is a document that essentially a spec that tells you what the bot should do, what the process in question is. So your analyst might collect the pdd, then your COE developer might implement that bot and then run and maintain that. Then you have a concept called the citizen developer. I actually feel a bit funny talking to test engineers about RPA because I'm explaining all these terms that are kind of RPA specific although be familiar in different contexts. But the citizen developer is now somebody who is sort of less technical Persona building out automations. This is really a Persona where that you target with this low code, drag and drop type of tools. And then theory is that a citizen developer with the support of the coe could produce automations. That model is still, you know, juries out there to be really able to Say if we see citizen developer led efforts that are actually going to produce value, we take the approach that we really focus on the automation developers. You know, companies like Microsoft entering the game with Microsoft Power Automate desktop, they really target the citizen developer with their tooling so they want to enable them to build sort of desktop automations. But yeah, my personal take on it's like strong coe with like you know, management support and alignment, clear mandate to automate, create concrete results of savings and efficiency gains. That's the way to do it.

So you're in my mind and my next question was going to be about kind of Microsoft Power Automate, how they're positioning their tool. I think this came out like fairly recently as a low code solution and this is a trend we'd be seeing in test automation as well. Low code. So I guess the reason why I bring it up, what skills do you think someone needs to have in order to be an RPA developer or if they want to get into rpa, what do you think is a good skill set to have for that?

Yeah, yeah, you know, RPA developer needs to be really sort of track of many traits in a way that you need to understand how IT systems work. You need to understand a bit about APIs, a bit about databases. Oftentimes you need to be this kind of forensic developer who reverse engineers some applications that might be like ancient and really janky, you know, going into mainframe systems and automating them and so forth. So you need kind of like this broad skill set. You don't necessarily need to go super deep on any area, but you need to know a lot about, you know, a lot of things. You know, with working with something like Robo Framework. Obviously Python is a good background skill, but you don't really need to be a master Python developer. You know, talking about local, really we started out with this, you know, thesis around the automation engineer and automation development, empowering them. But we come to realize that, you know, you know, a shortage of people who I who have that skill set, this kind of perfect skill set. But there are a lot of people who want to get into this space so upskilling into something like Robo Framework and Python. So we are actually right now building a low code tool set on top of Robo frameworks. Even though like two years ago somebody would have said that to me, I wouldn't believe it. But we came to the realization that, you know, even automation developers, they oftentimes want to work with something that's, you know, fast to make something actually work. And happen. So our thesis around low code is that it can be a pathway to automation where you upskill into something like Robo framework and Python. It can be a, you know, speed boost for the developer. And we want it to be something that allows you to move between low code and pro code, as we call it. Like you have no code, low code, pro code and sort of funny definitions there. But what we want it to be, the automation studio that we are building, we wanted to allow people to move between low code and pro code and back so you are not tied it into something that's proprietary, that's low code only. That's I think you get oftentimes handcuffed when you build something with low code. It's appealing in the beginning, but you really don't have that sort of high ceiling to grow into. You get capped at some point with what you're able to build.

Absolutely. And I know you spoke at actually robocon this year and your session was on the open source stack for rpa. I know you talked about rcc. Is RCC the low code solution or is this something else?

Now RCC is a tool chain that really ties a lot of the open source stack together. So the way we want it to be in the open source stack for automation, how we want it to be is essentially something like similar to when you start developing a web app. You don't start with blank HTML page and JavaScript, you start with a framework like Ruby on Rails for instance, so you can focus on the actual automation task at hand. So we want the automation stack that we built be the same kind of thing for RPA developers. So you focus only on building that bot and you have all the scaffolding, all the ways to deploy and package it around you. So RCC is a project that does a lot of that heavy lifting of managing your environment, managing your packages, the optimal way, being able to deploy bots wherever they need to run and bring that whole thing together. So RCC is used by all of our developer tools. So our VS code extension uses that had JupyterLab extension use status, etc. Even the bot runtime environments that we have for deploying these parts through our control room, they all use RCC under the hood. And I think it's a great solution that you know is obviously open source. So I'm talking about open source tooling here. So I think RCC is really the unsung hero of all the things that we build that more people should get into and try it out because it'll so many headaches of just how you deploy a Python based automation routine running in a fresh new environment.

I'm just trying to relate it to test automation. It sounds almost like Maven. Like when I used to use some tools like Serenity with Maven, I'd use Maven. I can use some commands to create a test project and in that test project it would contain all the dependencies I needed to run it. Sample project, all that types of things. Is RCC similar to that?

Yeah, you can think about it like that. The name used to be RoboCop CLI or something, but we just named it to RCC because it was doing so many things. It's based on actually Go. We've written it with Go go as a single binary that you can just drop in and put in place. And you know, when you want to create a new project, run an existing project, it'll just handle the rest. You don't need to even have, you know, Python or any tooling other than RCC in place. You just drop that one binary and then let it do its thing.

Nice. So if someone's listening to this and like, well, I've never heard of rpa. This sounds like something I should get into. It sounds like RCC may be the first step that they could use to to get their system up and running to start experimenting. Is this more for a developer? Once they're kind of more familiar with.

RPA, both will work. You can go to GitHub.com robocop RCC and just download that and try it out. RCC init. You'll create a new project that comes with Robo framework files and config files and everything in place. And then RCC run will allow you to run that. But if you want to get something kind of easier in place, if you're working with VS code, go to the Marketplace and search for RoboCop. You're going to be able to find the RoboCop VS code extension that comes with RCC. So that'll allow you to create that new RPF project run and debug it as well while you're there. Also download the Robo Framework language server support. That's going to help with syntax highlighting and all that. But then yeah, those are really the easiest ways to get started if you are there.

Nice. So we touched on a lot of kind of like what a buzzwords in test automation. We talked about low code and business users using it for automation, not needing developers, all that type of thing. Same thing we see in test automation Another term I've been hearing more about is AI and that how AI, you don't need a developer now because back in the day you did, but it was all snake oil where you actually did need a developer. But now they're saying AI is really going to fulfill the promise of record and playback and that type of thing. So do you see AI being a player in RPA or is AI already embedded in rpa? Is it because it's a newer type of approach? What are your thoughts on AI with rpa?

Yeah, I mean when I got into RPA they were making these claims about, you know, you have this AI watching your, you know, users do their thing and it's going to learn and all that stuff. I've never seen that actually happening. And working out AI can be a part of, it can be part of the solution. Oftentimes what you have is something called intelligent automation and that's really a people will throw around this XPA so something process automation, that is rpa, ipa, something like dpa. So intelligent Process automation means that you kind of infuse your process, your bot with some intelligence, meaning that you have a decision point where you might use a machine learning model to interfere something, might use nlp, might use OCR that categorizes IPA already. So that's one place where RPA kind of overlaps with some sort of AI related stuff. RPA tools themselves typically don't contain AI components or not very sophisticated ones that I've been able to find. We as a company, we have an AI project ongoing. What we try to do there is try to come up with a deep learning based model that can look at an old user interface, something like Oracle E Business Suite or SAP GUI and look at the interface and, and be able to say that, okay, this is a button, that's a text field, that's a radio button, and these are the labels for those. So essentially teaching a model to look at a UI and be able to find the right buttons that we want to press or the right fields that we want to interact with. That's an area where AI can actually be infused into rpa. But oftentimes as it is with AI, it's not something that will be for really like a novice user. So I don't need to know anything about anything. I just let the AI do its thing. That's really not the case in rpa. The way we are using AI in this instance is really using it as one type of locator. You know, instead of XPath, you use AI to locate a button that you need to interact or compose a tree model of the ui. So I mean I'm not saying that AI isn't there like that we don't have any solutions that have like fancy AI. Oftentimes it's not actually the RPA tools themselves, they are very deterministic. You might have AI in places like process mining tools, task mining tools. So these are actually things that come before rpa. So these are tools that are put in place to monitor human users, infer users patterns and then build a model out of that. And you can use that model to determine like what are my opportunities for automation, what are people spending their time on. So there you might have models that do that inference.

Awesome. Okay, anti, before we go, is there one piece of actual advice you can give to someone to help them with their RPA solutions or efforts? And what's the best way to find contact you or learn more about RoboCorp?

Well, you know, I would start from RoboCorp.com docs in particular there's two certification courses, so level one for beginners, level two for more advanced RPA developers. And those are free, free for everyone. So if you're interested in rpa, go check them out and take the course. You're going to get a certificate that you're going to be able to post on LinkedIn on your profile and that's a great place to get started with open source RPA tooling for free and really good start for your path in your journey in rpa.

Awesome. Do you have an actual piece of advice someone can use actually after they're done with the interview, if they're dealing with RPA of how they can maybe put into place right away to maybe help with their efforts.

Yeah. So RPA has so many different application areas that it's hard to nail down anyone. What I tell you is what you don't want to do with rpa, like oftentimes see people wanting to apply it to these like tiny, tiny fragments of tasks. Like you know, I need to fetch this thing from my email inbox and it's annoying but that's typically not where you will get, you know, a lot of benefit out of rpa. Look at where, if you're working in an organization like where are people spending their time on? You know, go and ask around. You might, you know, make friends with your financial controllers. You might find that they're actually spending their time and really, you know, grinding routine tasks that you didn't even realize that exists. So there's usually the biggest opportunities for RPA instead of like looking at your own desktop.

Thanks again for your automation awesomeness. For links to everything of value we covered in this episode, head on over to testguild.com a52 and while you're there, make sure to click on the Try for Free Today link under the Exclusive Sponsor section to learn all about Sauce Lab's awesome products and service. And if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep it. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.