Visual Programming for Automation with Duncan Brigginshaw

Five, four, three, two, one.

Drop that.

Welcome to the Test Guild Automation Podcast. We're all getting together to learn more about software testing and automation with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today, Duncan Bringenshaw, the co founder and technical director of ODIN Technology, shares the benefits of using the scriptworks automation testing framework. So listen up and discover a unique way to visually program your automation for your web, mobile and API tests. Check it out. The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs. Their cloud based test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial, visit testguild.com sourcelabs and click on the exclusive sponsor section to try it for free for 14 days. Check it out.

Hey Duncan, welcome to the show.

Hey Joe. Hi. Thanks for having me on your test talk.

Awesome. Great to have you. Today I'd like to talk about codeless test automation. It's something I've been hearing more and more about. It's actually an older technique that I heard about, you know, in the early 2000s, but it seems to be really gaining more popularity again. So I'm just curious to find out your views on this and maybe anything else you have thoughts about test automation. But before we get into it, just tell us a little bit more about yourself.

Yeah, so I'm based out of the UK in a small medieval town just outside London. I've got a young family, got three beautiful young girls, and in terms of my career, I've been coding probably since about the years age of about nine years old. So I think I wrote my first computer game on a Sinclair Spectrum back, you know, kind of, I'd like to say 15 years ago, but it's probably more like 35 years ago.

So. Yeah. And then off the back of that I got my first job, I suppose in it when I was about 15 as a summer job creating a database for a large charge card company. Went on from there to join IBM's internship program, Bachelor's degree in computer science and then rejoined IBM as a, as a developer sort of in the late 90s, I guess. And during that phase they took quite a rounded approach to the way that.

Their graduates were taken on board really. So in that I did roles in development, in testing, in a number of different facets of the software development life cycle as it was back in the 90s. Left IBM to join a software test consultancy who back then had, I suppose the foresight to hire developers into testing consultancies in order to further their automation practices. And I ended up leaving there a number of years later and starting my own ventures, a little software development company delivering consultancy to Microsoft and EA Games amongst others and then hooked up with a co founder of Odin. And that was about sort of 19 years ago or so now. So that's. Yeah, that's a bit about me, my background, very cool.

You know IBM was the place in the 80s to work for. Did you learn anything from that that you've always taken with you the rest of your career?

I think a foundation in programming principles. I think IBM were very diligent in the way that they treated graduates to give them fundamentals as part of their sort of first early years since joining, as I was saying about putting me in different roles as well as just development, I was doing manual testing roles.

Automation roles in that day and day as well. So yeah, it was a really exciting place to work actually and I was lucky enough to work out of their R and D labs doing research and development on formal methods and things like that. So it was quite a, quite an exciting place to work at the time.

That's very cool. So since your internship till now you've been involved in software development obviously. So are there any main challenges you've seen with automation testing that we've always faced since you started?

Yeah, I think fundamentally, as well as all the sort of planning and approach and all the wrappings around the actual physical act of creating test automation scripts, one of the biggest challenges I've seen is the kind of gap between skill sets and that's been prevalent since I started out in automation back in the late 90s where the people who had domain expertise, testing analysis skills, who were deriving test cases were unable to, to get the best benefits out of the automation tools that were obviously in those days and still mostly are very code based. So there was this sort of gap between skill sets to deliver automation. You needed really to know how to code. But most of the QA departments and organizations that we worked with had focus on domain expertise and testing skills rather than coding skills. So I think that's probably been one of the biggest challenges I've seen in successful automation in organizations and I don't think that's changed over time.

So Duncan, that brings us to our main topic on codeless test automation. I guess before we get into it, how do you define codeless test automation? I think there's a lot of misconceptions around it. So what do you.

Yeah, absolutely. I mean codeless.

I know from sort of my output, certainly on LinkedIn and social media, that codeless tends to be quite a divisive topic or divisive term. Very much like the term frameworks were a number of years ago. People have different views of what codeless is and those views tend to sort of color their opinions of others when they're talking about codeless. For me, I think codeless is really a movement around creating abstractions to aid in those with less technical skills, lower coding capability to get involved in building test automation scripts. So providing some level of abstraction that they can use without needing to go down the route of picking up just coding languages, but also all the ancillary tooling and skills that are required to get the most out of test automation. So for me there's usually always some code somewhere and I think that's sort of been teased out over my engagement with the community recently that it's not truly codeless, but it's a way of abstracting and providing a mechanism for non technical staff to get involved in that automation process.

Very cool. So I know a lot of people have tried to go like 100% codeless test automation per se.

Any cons you see with that approach, the 100% codeless. I think there's always a requirement to have extensibility and customization in any codeless approach. I know that there are certain vendors that have gone down that route and have had to add in the capability to add additional coded steps into their UIs so that there's always that need to customize. And I think probably that comes out of the fact that UI technology, which is where a lot of this is focused, is on UI automation. UI technologies move so rapidly and vendors have always struggled to keep up with that. So being able to provide a level of customization and ability to actually get under the hood and work with code is pretty important in any of the codeness type approaches, I suppose.

So I'm always curious about though when someone talks about automation, I think that is software development. So when people think codeless test automation, I'm thinking if this worked, why isn't there more codeless development software? Is there developers out there creating software that's codeless?

Absolutely Jade. I mean, if we look at the low code movement. So in software development and app development.

The term low code is often used instead of codeless or in preference to codeless. Probably because of the reasons that we've just been discussing that codeless, you always need a potential to add more.

Coded aspects to a solution. So the low code movement with companies such as Outsystems Bubble is, I mean that the huge industry there, billion dollar companies bubble is taking a large round of investment recently, all aimed at delivering software without the large volumes of code that are required to deliver typical software development exercises. So it is a movement definitely in app development as well.

Very cool. So earlier you mentioned abstractions as maybe one of the cons of codeless automation. People don't know how to do it correctly. So how do you go about creating proper abstractions when you're using a codeless typ solution?

I think in terms of an abstraction it's got to be accessible to.

A non technical tester first and foremost. They've got to be able to articulate the steps required to perform an automated check or test against an application in a form that is manageable by them and maintainable as well by them. So there's been a number of approaches to codeless automation over the years. Certainly it's not something that's new. If we look back at tools like I guess Test Partner, the old Compuare tool, I don't know if you remember that one.

It had a form of creating test scripts in a visual fashion in a kind of tree view type approach to adding steps to a test script. UFT has the same kind of interface and.

In terms of.

Creating an abstraction, they aren't new in these tools. What I suppose has shifted is that people are now looking at the non commercial alternatives, the open source community, particularly with reference to Selenium and Appium, and they're starting to build new abstractions around those software. One of the ones I suppose that's most common is a table based abstraction where people define their test steps in tabular format. And that's a very popular approach, probably stemming from the working keyword frameworks many years ago. But in terms of creating an abstraction, if we go back to that question, I think you've got to create something that's accessible to a.

Non technical user who doesn't necessarily have experience in code, but also has the flexibility to articulate the nuances of a testing process. It needs to have.

I wouldn't say coding constructs, but coding like constructs to deal with complexity in UIs. So some examples I tend to use are decision trees, given a certain behavior in an application.

Perform a particular action.

It's one of those areas in test automation where I've kind of changed my view over time in that I always tended to be of the view that we shouldn't be using decision trees in automated test scripts that a definitive path should lead to a definitive outcome. But I think the way that modern UIs behave, certainly unexpected behavior does happen. And.

Examples may be around security checking, for example, entering passwords, that kind of thing. At random points during the script, the tests need to be able to react. But also simple things like just being able to combine data or extract sections of data out from a on screen presented string. So to tease out the account number from a message box, for example, all of those need certain levels of code, like techniques that are more difficult to articulate in a table based form or tree view type form. So it's that balance abstraction that's accessible but also flexible and.

Gives people that power to articulate a level of algorithmic construct.

Nice. And I guess that's one of the knocks on codeless automation. Back in the day. When people hear codeless automation, I think sometimes first they think of capture and replay and also keywords which we mentioned, they've been around for a long time. How's it different? It's been 20, 30 years since that technology has been out. How's it different today that makes codeless automation a more viable solution than it was in the past?

I think there's been a movement towards.

More graphical representations of.

Test automation scripts. So if we were looking at say the UI technology back when we're looking at UFT and Test Partner and tools like that, UI technologies were quite rigid. So we had tables and tree views and.

Capabilities like that that people were used to working with. I think UI technologies moved on to the point where it's so flexible that we can create some really great visualizing and.

Visual type approaches to the same problem.

They've existed in rudimentary form, I suppose with QTP's action view or UFT's action view. But UI technology has brought about some visual representations that are more flexible and more powerful in my view than the table and treeview based abstractions. So I guess I'm referring mainly to the kind of executable diagrams that we're seeing coming from certain projects out of IBM, mit, those kind of institutions where people are looking towards abstraction to simplify either app development or.

Process automation as well. So I mean, we can touch on RPA as well as being an area where low code or codeless approaches is rolled in from the start.

So, Duncan, the way I came across is I found a paper about codeless test automation and it talked about this concept of visual programming. So you've mentioned visual a few times. Is this a phrase you've coined visual programming. Or is that what you're getting at with the approach?

Yeah, absolutely. It's not a phrase I've coined at all. I think visual programming has been around quite a long time now, and visual programming styles tend to be used as an abstraction to create app development or process automation. Visual programming is essentially using some kind of design canvas to join and connect elements to form the process or flow of an application. There are types of visual programming environments based on executable flowcharts. Creating a flowchart to describe a process. In this case it could be a test automation script and then executing that flowchart. There are also.

Techniques, particularly out of the IBM Stable. The Node Red project uses.

Interconnected blocks to create similar structure to a flowchart with a little bit more flexibility in terms of inputs and outputs. And that's been used for process automation in robotics, in Internet of Things development, and a number of areas now are adopting that. The reason that the visual programming approaches work.

Well is that they have that flexibility to create algorithmic constructs in their design, built into their design so that you can loop back over a process, you can, you can branch your decisions, etc. And one that's sort of, I guess, that most people will be familiar with is the. The concepts of block coding. And block coding is something that came out of the.

MIT and Google stables as a way of getting.

Essentially to get people involved in creating code, to educate students, to educate aspiring coders, and for them to get used to the capabilities of a programming language, but in a more accessible form. And it's quite cool. It's based on the indent structure of code, so you can wrap blocks, you can assemble them and connect them in quite creative ways to create an algorithm just by literally dragging and dropping these blocks onto a canvas. And people may be aware of projects like Scratch and Coda. Org that use these to educate young aspiring programmers in the kinds of algorithmic constructs that they may use later in their careers or just may need to be aware of when they're entering other domains.

How's this approach different than bdd? Would you consider BDD code list? Or is visual programming completely different concept?

I think it's a different concept aiming at the same problem domain. So BDD is.

Obviously the ability to use a ubiquitous language for a single source of truth across the whole development process. So typically it's associated with Gherkin. It doesn't have to be. You can define your own ubiquitous language that's more comfortable for your teams, and it tends to be associated with tools like SpecFlow.

As an example or cucumber obviously for automating.

Testing of those specifications. So where BDD I suppose differs is that it goes through, it permeates throughout the whole development process. So right from early stakeholder involvement, everyone is speaking the same language, developers, testers, business stakeholders, and they're articulating their use cases in a formal way, semi formal way that enables them all to speak the same language if you like, rather than having multiple documented repositories of information, specs, tests, documentation for the application, use cases, all the kinds of things that.

Tend to be associated with the software development process. And I guess where BDD is slightly different is that there's in typical usage scenarios, there's still coding involved. So similar to keyword frameworks that keywords are an abstraction for numerous steps within a process. So you might have a keyword like add a user or given I've added a user in BDD terms. And to automate those processes there's still a requirement to create code to add the meat to those, if that makes sense.

Whereas Codeless I guess is coming at it from a different angle. It's saying, well look, there may be a requirement to automate, automate some use cases and we want to be able to articulate all of our step in that instruction abstraction layer rather than just kind of a high level step view.

Very cool. So this brings us to the point, if someone wants to get started with visual programming other tools out there, I believe, do you have a solution? I believe it's called scriptworks. Can you tell us a little bit about that?

Yeah, well, scriptworks came out of a project we initiated at ODIN about two and a bit years ago where we were looking for a way to.

Enabled non technical testers to articulate test scenarios for Selenium. So Selenium is a core execution engine, obviously has been been around for a while now, it's now a W3C standard and Appium also is probably going to follow suit. We shall see. But one of the problems with Selenium is that it doesn't come with much in terms of ancillary tooling. There are lots and lots of frameworks. I think you mentioned in one of your previous talks, you've got at least 48 different frameworks or 40 something different frameworks mentioned, probably most of them around the Selenium ecosystem.

So when we initiated the project we were looking for a way for non technical testers to articulate tests for Selenium. And I think scriptworks is unique in being Based on the block coding approach, which gives obviously the flexibility of algorithmic constructs, but also the accessibility. When we were doing the initial usability testing, my daughter, who is 11 at the time, came in and with a little bit of guidance managed to create her first test within a matter of minutes. It's simple Google search looking for information on her pets, pet gerbils at the time, but it shows that it's accessible with.

A lower barrier to entry than other coded approaches. So yeah, we launched last year and we think it's a unique solution based on that visual programming approach.

Cool. So besides being easy to create, another thing people struggle with is maintenance of tests. How maintainable is this approach? It sounds like it'd be difficult.

So one of the things that has always been a sort of a basic tenet for automation is to make things as maintainable as possible. So maintenance is often one of the biggest reasons for automation failure. So we've built this in a modular fashion so people can define small reusable pieces of automated test process and sequence those together to form end to end type processes or larger test workflows. So modularity has been very key to this. One of the other areas is obviously the use of Selenium tends to be based around defining the way that Selenium interacts with your application under test. So the ubiquitous selectors, if you like Selenium selectors, and oftentimes if we're using something like a capture replay approach, the selectors that are created during that process are brittle and prone to failure. And again, brittleness of test is, is another reason that test automation projects tend not to succeed. So we've been doing some pretty cool stuff with an artificial intelligence approach to maintaining selectors. So using natural language to define your target elements for your Selenium actions. I know AI is a big, big topic, and it's probably a topic in itself for test talks, but we've, we've taken quite a narrow focus on where we've applied it within our product in that the fundamental problem is brittleness of selectors. So if we can leverage modern UI technologies that are much more rich and self describing than they ever have been. So accessibility initiatives, et cetera, have made apps and web applications more rich in the way that they describe themselves to users requiring assistive technologies. So what we've done is used a bit of that and a bit of analysis of the way that we look for elements on a page, we've thrown a predictive model at it that we've trained up on quite a wide variety of application types and web architectures. And what that does is it enables you to specify as you would see something on the screen, for example username field, and it will go off and find the closest match to that. So we think that in terms of test creation and maintenance, makes it more resilient to change and also easier to create. So that's one area we've been working on, but modularity being the other.

So you beat me to my next question was around all tools coming out now have AI in front of it, slapped in front of it. So is this true AI or is it just a fancy algorithm?

It is, it's. It's a feed forward neural network at its heart and it uses quite a number of data points to make its predictions. So we've been training this using.

Sort of standard interfaces, standard web interfaces, application mobile app interfaces as well, up to a level. But then it's in beta at this moment. So users can try out the generic model that we've created, but we've also got in there the ability for them to retrain the AI as well, so that you can go back and say, actually that was incorrect. You made a prediction there that that was the field I was looking for, but actually it's this one. So you can build up and retrain your own internal models as part of that. Yeah, I agree, everything's got AI. We're keeping it relatively low key, but we wanted to explore how this could help one of the big fundamental problems in automation, of brittle test scripts.

So I guess another downside, some people, when they talk about these solutions, does it use Selenium under the covers or if someone uses a solution like yours, are they locked into your ecosystem where they can't port it over at some point in time?

Yeah, so it does, it uses Selenium and Appium. So we've attacked both web and native mobile application architecture. We've stuck with the W3C compliant JavaScript versions of Selenium. But the way that our product works is it's slightly different to typical codeless automation tools where they interpret the abstraction. What ours does is it generates code based on the abstraction that can then be leveraged in a standard CI pipeline, or it can be exported out to an IDE and used as code in its own right. So there's a subtle difference here. Our product doesn't interpret the block based abstraction. What it does is it generates native JavaScript code for WebDriver that you can use. In other. We've made portability key. Our ethos is our run concurrently run anywhere so you can leverage other vendors infrastructure as a service solutions. You can use visual.

Testing tools like Applitools alongside it so it makes the most of the Selenium ecosystem that's advanced but doesn't necessarily lock you in.

So Tarkin, how long have you been around this company? Two years you said?

No, I mean the initiative for scriptworks we started a couple of years ago, about two and a half years ago, culminating in our first sort of released last year as a company. ODIN Technology have been selling in an abstraction framework called Axe to large financial institutions, federal banks, etc. For going on 15 years I suppose.

And that was based on a table based abstraction working alongside what I would say is the traditional tool vendor lineup, the IBM Rationals or Micro Focuses where we would generate code for uft. We generate code for.

Coded UI or Rational Functional Tester and we're still actively marketing and selling that into into big institutions today.

Okay Duncan, before we go, is there one piece of actual advice you can give someone to help improve their automation testing efforts and let us know the best way to find contact you or learn more about Script works.

Yeah, I think fundamentally the single biggest piece of advice is don't dive in with both feet. Don't just download the latest tool framework buzzword that you've read about on a LinkedIn feed or step back, think a little bit about the requirements for your organization. Testing and test automation is not just about tools as we all know. It's about.

It'S about automating processes and if those processes aren't right in the first place, throwing the latest greatest tool at it is not going to help.

So I think take a step back, read a book, do some research, look at things that are going to help make automation successful before you dive into an IDE and start writing JavaScript or Java code. So I think that's probably the biggest fundamental piece of advice I'd I'd give at this stage.

In terms of getting in touch with us and learning a bit about the ScriptWorx program or application using visual programming to create JavaScript Selenium code. You can contact us at ScriptWorks IO and we offer a free trial tier so people can get used to the product, can ask us questions, trial it, see if it works for them in their environment.

And yeah, please get in touch and sign up for a free account.

Thank you Duncan for your automation awesomeness. For links to everything of value we covered in this episode, head on over to test guild.com alike automation a270 and while you're there, make sure to click on the Sign up for a free trial link under the exclusive Sponsor section to learn all about Source Labs, awesome products and services. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to test guild.com for full show notes, amazing blog articles, and online testing conferences. Make sure to subscribe to join the in the Guild and continue your testing journey. This has been a Joe Calantonio production.