AI for Automation Testing and RPA using Aito With Tommi Holmgren

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we're talking with Tommy all about AI for automation testing and RPA using ITO. Tommy has over 20 years of software and tech leadership positions under his belt and with two startup exits. He has a lot of experience around up and coming technologies and really has a really pulse on what's going on here. Now he's building the next generation machine learning tools for RPA developers using AI and I'm really excited to learn more about the technology he's created. But before we get into it, definitely check out a few words from our awesome sponsor.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs. Their cloud based test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial, visit testguild.com sourcelabs and click on the exclusive sponsor section to try it for free for 14 days. Check it out.

Hey Tommy. Welcome to the Guild.

Hi everyone. Hi Joe.

Awesome, Great to have you on the show. Before we get into Tommy, is there anything I missed in your bio that you want the Guild to know more about?

No, I think it's like interesting coincidence is that I was really heavily involved in testing and software test automation, but that was like maybe 15 years ago and the automation was really in the beginning and now it's like kind of seeing the same stuff with the RPA related topics that were popping up already a while ago.

Absolutely. I didn't actually know you started off as a test automation engineer. So I guess how does it fit compared to rpa? A lot of people get them confused. Is it the same thing or is it different or can they be used interchangeably?

I think for me, my background started as a database engineer, not as much as a testing engineer, but I ended up working with a software test consulting company for quite a while. I see probably the same platforms I use, which is kind of natural. Same tools I use. People get involved in both of the fields, test automation and rpa. So I think the tools kind of drive the trend here.

Very nice. So you know, one of the trends obviously is AI and especially what I've been seeing in functional automation is AI with automation testing. So I was doing a search, I do a report every year on the biggest trends for 2021 and I came across your technology just because I did AI with automation and I thought it was, wow, I've never heard of this. And you had a cool example of how to use this robot framework. So I guess a high level. Could you just tell us a little bit what your company does and how helps with rpa?

Yeah. So AITO is a couple of years old company. We do have some background with the consulting business in the Nordics and in Europe. So we've seen the digital transformation projects within large companies for quite a while. There was an innovation in machine learning, how to make it, how to make things easier. And I think our track is in a way very typical with startups. We thought that we were doing something else in the beginning and we were then drawn into RPA field by our customers and users. But essentially we make it super simple and easy for RPA engineer or software robot engineer to deploy machine learning as part of their workflows basically on any RPA platform that they might be using.

All right, so maybe I'm wrong. It's almost like a software as a service type thing where any company can consume your solution with an API and implement it with their technology. Is that how it works?

Yeah, it is a software as a service solution. It's basically an API. That's all we provide. So essentially that API provides predictions and a typical use case would be replacing. So instead of writing a complex rule base in the automation, you use machine learning to replace those rules. So typically there are decisions made by humans in categorizing things like putting let's say customer service or IT service just tickets into right categories or automating, purchase invoicing and so forth. So there's obviously like a number of possibilities, but very typically either as a product replaces some decision that either is done by human or needs a massive rule base to be automated.

So you would think with RPA solutions they'd have some sort of AI baked into it. What does this give them that you know, maybe a typical RPA solution doesn't have, or maybe you could use this to augment maybe what they have already.

Very typically when a RPA team or a let's say enterprise customer starts looking at the more complex automations so like covering processes instead of a simple tasks and they encounter a use case that requires machine learning, I think the first reaction always is, hey, we need data scientists, we need to make a data science project out of it. But I see the field of machine learning kind of dualistic in a way that there's of course the research custom aspect of machine learning where you need the data scientists. But a lot of the automation related needs are quite repeatable and you can use products to get your use case done with way less costs and way less effort and the smaller teams. And that's where we provide our tool for the RPA engineer. And in the projects there's typically no data scientists needed to do the machine learning parts.

Very cool. So you know, I know you also work with Python. So a lot of my audience uses a tool called Selenium which does browser based automation. And I always get asked how can I add AI to my Selenium test? I know they have a Python language binding. If someone was using just a straight up Python language binding, can they use ITO to do what they need?

Yeah. So at ito, we have Python SDK. So we offer everything, all the functionality over Python, like a fully featured Python SDK. So you can basically send in the data sets, manipulate your data, make the predictions using using our Python SDK. Now I think if there's any users of Robot framework or any of the commercial tools that are built on top of a robot framework, we've shown to work really well in that ecosystem which is basically Python based.

So do you actually have a lot of users that use the robot framework and if so like what do they typically use ITO for?

So that's I would say one of the kind of up and coming ecosystems. Obviously we see a lot of usage in commercial enterprise tools like UiPath, especially being from the Nordics. UiPath has a really strong market market share here. So like a lot of the corporate users would be using one of those tools dominantly uipa. But in the discussions with the customers and RPA consultancies, I think there's like a growing interest towards open source tools like Robot Framework. And there's like some really interesting companies around that ecosystem like RoboCorp from our native Helsinki and a couple of others who kind of pushing that ecosystem further. And I think there's a really interesting alternative to commercial tools.

So I know when I speak with fellow engineers, a lot of times when they hear AI, they think they have to really be AI experts. It's overly complex, but it seems like you also use kind of a unique approach using like a database almost that most developers are familiar with and an API and doing queries. So how much AI does someone need to actually be successful using ito?

If you continue with probabilities, that's like a starting point and that's actually one of the most complex things we've seen and we're trying to constantly make it easier and easier. We take an approach that we want to reduce the complexity of machine learning, not expose everything that is inside the box, like inside the data science box, but rather we bring the elements that are necessary for a automation engineer that they need in their decision making. Like what are the confidences and probabilities? If you can work with those, I think you can achieve a lot already with the tools that are available in the market.

So how flexible is it? I'm just thinking of Selenium. A lot of times people have issues identifying elements on a page. Could they easily develop their own algorithm that figures that out? How would they do that?

Our use cases mainly revolve around tabular data. So we replace the element that needs a human review or a human decision like a typical use case. And I think the most popular in our customer base at the moment would be automatically deciding the general ledger accounts of a incoming purchase invoice. So we don't do the OCR or we don't do any of those parts. There's a lot of other tools for that available in the market. Once OCR has read in like extracted elements of an invoice, we basically can predict either. Can predict things like what's the cost center, who's the human reviewer that this invoice belongs to? What's the GL account where the invoice needs to be filed in and then the automation takes care of the of filing the invoice into SAP or whatever tool is in use in the enterprise. But we are simply that prediction endpoint, the API that can provide that the prediction of a missing data point or value based on the historic data.

So I don't mean to keep bringing it back to functional automation. It just happens to be the space I'm in. I know a hard thing that a lot of people struggle with is test data. So I wonder if this could be used to. If you're trying to dynamically create an automated script that's interacting with the website, if it can be context aware enough to know, okay, you're on a patient page. So therefore for here's some patient data I can predict you're going to need or help you populate using Ito. I don't know, just use a case that pops in my head.

We have not done that such a case. Maybe some creative engineer will figure out.

How to do this.

But I think we've been a bit further away from purely like test automation use cases. It's been more like RPA use cases where the typical need would be a prediction of what is the action for to be done. With the data that is at hand in the automation. So I think it's kind of an opposite or different paradigm to use it in test automation that way.

So it can help you with things that maybe you think you couldn't automate before. It's not necessarily testing like I did get an example of cleaning up a CRM. Steps are a lot of things, things that would take a lot of effort to manually do. Not testing wise, but automation wise.

That's a really good example of a use case. For example, in the CRM we have a user who's using I do like manage the cleaning of their of their CRM. So a typical problem would be that you end up with duplicate entries in the CRM or any master data, basically. So what they do is that they double check using ITO when somebody's trying to enter a new data point into the CRM that is it likely to exist with slightly different variation of text. Like for example, if the name is misspelled in one place where one place it's incorporated and another place is Inc. And New Jersey is written NG in some other place. So IDA can be used to identify those matches and help retain cleaner master data or records like CRM.

All right, so here's a good user case then for functional testing is a lot of times you need to do like tear up and tear down actions where the environment needs to be in a certain state or data needs to be cleaned up after each run. And it seems like this might be a good solution to use for those type of activities where it's not necessarily a test, but it's a manual process that you're currently doing to get your environment or data in a state that then can be consumed by your automation script.

Yeah, that might be a better match for functional automation.

Yeah, nice. So how much overhead does this add if someone's worried about time? Does it consume a lot of extra time to run or how does it work?

The basic paradigm of IOTOS usage would be like it's a SaaS solution. So everything runs in cloud and you get a API endpoint for the predictions. So there's typically like two activities that you need to consider when using us. So there's one activity that feeds into data points like key the ITOS data set that is basically a replica of your master data that is relevant for the predictions. So you need to keep that ongoing up to date ongoingly. And typically our customers implement a separate robot for that purpose that takes data from whatever is the master data, puts it in. Then the real time part of usage would be the prediction or like the inference part. And that's also. That's an API endpoint. If you have say typical automation data sets, they really are huge, so they're like quite reasonable. Talking about maximum gigabytes of historic data. The prediction response times are somewhere between 100 to 300 milliseconds. So when the automation workflow runs, it's quite quick to return those predictions. We're talking about milliseconds, not even seconds and that kind of times what we've seen. It really is an issue because the RPA workflow it often is not real time. So it runs in the background with scheduler scheduler or it's triggered in some way. So it's not like human would be waiting for the activities.

Nice. So I guess where prediction and inferences a lot of times comes up. You need a lot of data to train in order to get something to work. How does that work with ito? Does it take care of that for you or is it a different kind of type of AI where you don't necessarily need a large data set to make decisions on?

It's a great question. It's very commonly asked by the customers and the users do like what do like I have this data, it might not be enough. And I think that us humans have been kind of a lot of the news and a lot of the article and talk in terms of AI tends to be focused on deep learning and like big data related topics which obviously need a lot of data like data for you to have any reasonable predictions and the accuracy. We operate naive Bayesian algorithms and we've shown to work quite well with rather small amounts of data. But of course you still need data. Your data needs to be somewhat high quality. It needs to have something to predict from. Of course it can't operate out of nothing but like typical kind of returning back to the same invoice categorization or invoice invoice prediction case. You can do a lot. Even with a couple of thousand of previous invoices you already get towards good predictions but obviously more you have the better than the results.

So I guess we're at the scenario of invoicing. Is there a way you can run this like in Jenkins to it's a job that look and say oh, invoicing, I see you have this incoming invoice. Let me match it for you automatically. Is that a common user case where someone's running it on a certain timeframe to do certain functions?

Yeah, quite often. For example those invoice cases would be scheduled daily. So all the invoices that came in the previous day are automatically categorized and processed below 8am in the morning. And then the ones where the prediction confidence is not high enough for automatic processing will be then sent to an accounts team for a manual review and manual processing. So it's like very typical that the decision or the prediction made by machine learning, the automation workflow diverts into two paths. One is automatic and one is manual.

So do you have any real world user cases where someone's done that? How much time have you seen saved using this type of approach?

I see the time saving in two different ways. There's obviously the underlying customer use case and the savings potential that comes from for example automating a process or task in a company. And those savings are pretty big. We're talking about massive potential of time savings in e.g. accounts payable or customer service teams. But for us the value that we bring is typically the time save in the process of implementing the whole automation. So our benefit for the RPA consulting company or the RPA team comes from easiness of implementing ioto. And we talk about a design driver of being 10 times faster to production than using custom made machine learning algorithms.

So I think this would be an easy sell. How hard is it to convince people? How hard is it? Once for someone's listening go, this sounds awesome, I want to try it. How hard is it to get started?

I would say that our initial struggle was that we were rather technical. You need to learn to use either's query or you needed to learn to use either query language. And that created kind of the barrier of getting on board quickly and getting like seeing the value super quick. So what we've done within the last months and we are rolling out new features continuously, we've tried to like lower the barrier of starting as low as possible and show the users value of IDA super quickly in the early stages of usage by giving them like a visual charts on the automation potential and savings and data contributes to their predictions. So by bringing those tools and UIs available, we can really show quickly to the users that what they would be able to get out, what's the value and then take a next step to actually implement the workflows.

I was just curious to know how much is the awareness out there in the market, you think? I mean since you partner with these RPA solutions, they must be able to say hey, you can use ITO to take care of these other user cases.

Maybe so even though there's a lot of talk about RPA and the biggest growth in enterprise software in the past couple of years, I feel that a lot of the usage is still quite basic. So a lot of the RPA and automations are still very rule based. They are quite basic tasks that are being automated and I think only within last year the machine learning driven automation that allows to cover larger chunks of the process or more complicated processes, that's kind of happening as we speak. So the market is definitely maturing as we speak and those use cases are becoming available. The customers are getting more interested in the use cases continuously and I think in that way it's really good time to be out with the machine learning related tool and it's definitely easy to generate interest. There's a lot of companies who are looking into it at the moment. And I think the trick here is to make the onboarding and the uses as easy as possible so that as many as possible can deploy their use cases quick and easy.

Yeah, I think there's still a lot of skepticism for some reason in this space. How do you help people get over that skepticism? Like how accurate is your solution? I think you said a lot of people using it for easy things, but it probably could be applied to more complex things like how complex you think you could get using a technology like this.

So how we tackle this problem or topic would be with the transparency. So we transparently show how well item performs in your case. So the usage flow that we are deploying as we speak, it's a new first steps into any new use case using Aida. So you can basically drag and drop a data set into our console and we run some evaluations on the data set against your prediction target. So chosen column from your data basically. And we transparently say that out of the box you can get to this level of automation or write predictions. This many errors we would be making and this is the amount that falls below the confidence threshold. So you see those numbers. And of course like some customers say it's not good enough and then they have always an option to go with the customer solutions with data scientists. And you can definitely find a way to make more accurate models using specialist resources. But our bread and butter would be those cases that you can actually implement without a single like a data science line of code written in your case.

So I guess that's a good point. Say someone tries using Ito and they have a user case they've been trying to work on. Do you offer like a tier where they can get that type of data scientists help or is that something in House they still need to get we don't offer that.

So there's plenty of great data science consultancies in the market and we are happy to help our customers to find a good one. We've decided to work solely on the product and kind of put all our efforts into making our product as easy to use as possible and kind of find use cases where it works really well and leave the consulting and custom made stuff for other people.

Fair enough. What protocol does it use? Is it just HTTP?

Yeah it's just HTTP and the current version of the API takes so basically you put a query like very database SQL type of a query in the body of the HTTP call where you say that from this data set where you know these knowns you want to predict certain feature, certain column. But we are also in that field. We are introducing a like kind of a simplified version of the API which hides the need of using the body in the core completely and everything will be in parameters.

I think I also forgot to ask is this an on prem off prem solution is that both if someone currently.

Cloud only everything in aws, simple decision of speed and easiness of our own development deployments. But of course this is something that we are looking into in the future to offer on prem possibilities.

You're pretty here. You're early, you're not early on, but it seems like you're one of the first companies out here that I'm aware of with a solution like this. Where do you see the future going and what's on your roadmap? Can you give us a little hint of maybe where you think you're going and what you see maybe could be possible if people get on board.

Now.

I think our roadmap, if you look at the short term we really want to bundle and integrate our offerings super well like a mainstream RPA platform. So that usage of us will be like a couple of clicks from UiPath or from RoboCorp or robot framework. So then it would be a breeze to start using machine learning predictions in those RPA platforms. That's absolute like priority for us. Now looking a bit longer ahead, we currently support only classification so you can only do classification type of machine learning with IDA today. And we have a lot of ideas how we can start broadening the spread of possible use cases based like based on offering either machine learning algorithms or like a prediction targets like regression or something else.

So I guess the reason that you start with classification, I assume that's the biggest user case that you saw to start off with maybe it's kind of.

A dual answer to that. One would be that the technology that we've created was just most suitable for classification. It's like a Finnish approach, engineering approach first, like a technology intensity, if anybody knows how to use it. But the second is that we've also seen that in RPA world a lot of use cases kind of boil down to classification. So you need to predict a category of a something like urgency or team where the customer service ticket belongs into and so forth. So I think a lot of use cases, even though it doesn't sound like classification at the beginning, they can be still implemented with the classification type of predictions.

Nice, Tommy. I have my last question before I ask that question though. Is there anything I missed that you think I should have asked that you think people need to know about?

No, I think you didn't miss it. But I think what's really important to enforce is that like I see that RPA teams, they really have the tools available and it's not just us, there's other tools in the market, like you can even look at the offering from Google and AWS and Microsoft. But there are tools to implement machine learning super easily nowadays. And I think we still hear in many cases that when like the question of yeah, we need machine learning for this use case or we need to implement some AI to get this done, the instant first reaction is that hey, we need to get our data scientists in and we need to make a massive project out of it. Now I don't think you need to make a massive project out of all of the things. So I think that mindset of hey, can we use the tools out in the market which do the simple things for us with one API? I think that's something we kind of want to preach.

Absolutely. And I think I forgot to mention it is a paid solution. But you do have a free plan where people can use it as a sandbox. So if they're thinking, hey, I should try this rad before you get your data scientists involved, try the sandbox out yourself and see what you can get before you scale up or get them involved.

Yes, we have free sandbox tier so you can go either AI, create your account or console, you can get the free sandbox. The new features that we are bringing that basically make you a full on evaluation on your data set and prediction targets and the potential accuracy, those actually will be completely free. So our business, well, we only charge for live predictions. When you operate something in production, that's when you pay for either. We keep the whole first stages of the usage to get the evaluation of your data set and predictions that will be free.

Awesome. Okay Tommy, before we go, is there one piece of actual advice you can give to someone to help them with their AI automation efforts? And what's the best way to find or contact you or learn more about ida?

Tangible advice? I think take an API and try it with your dataset. That's the thing to do. You can reach me, you can find me LinkedIn or Twitter, or you can find me through Tommyida, AI email, whatever you prefer, I'm there. I'm ready to help.

Thanks Tommy for your automation awesomeness. The links to Evident value we covered in this episode. Head on over to testguild.com a341 and while you're there, make sure to click on the Try for Free Today link under the Exclusive Sponsor section to learn all about Sauce Lab's awesome products and services. And if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.