Tackling the First Mile in Test Automation with Jonathon Wright

Get ready to discover the most actionable end to end automation advice for some of the smartest testers on the planet. Hey, I'm Joe Calantonio, host of the Test Guild Automation Podcast and my goal is to help you succeed with creating automation awesomeness.

Hey, it's Joe and welcome to another episode of the Test Guild Automation Podcast. And we have the Automation Cyborg back with us, Jonathan Wright, to talk all about automation and especially tackling the first mile in test automation. He's traveled all over the world, so I'm sure we're going to touch on a bunch of other topics as well. If you don't know, Jonathan is a familiar face to anyone in the quality assurance and test automation space and also the Guild. He's a friend of the Guild for many, many, many, many years. He is recognized as a thought leader in the test community, most recently through his tenure as president of Vivint, which is the world's largest, or I think was the world's largest independent user community. Over 70,000 members, over 190 countries, very involved there. Alongside all this he also has time to host the QA Lead podcast where he regularly interviews leading influencers across industry. Hasn't interviewed me yet though, so have to get on his plate there. Also, in his spare time, Jonathan sits on the committee for the ISO 29119 part two for testing AI based systems. I probably botched that the European Commission's AI alliance and co authored many books, but the one that I think I have here is AI and Testing with Rex Black, maybe the most recent one besides my book, which he did contribute to as well. So if you haven't got that, definitely check it out. He has really great advice on the Automation Testing Scorecard dashboard everyone needs to know about, so pick that up as well. And he is also proud. He's been actually on the Eggplant team for a while now for the keysight Technologies as the Chief Technology evangelist and he's really rocking the role. I love him here and he's working with some high end great tech for automation. If you haven't tried keysight Technologies, especially Eggplant, you definitely want to check that out after this podcast as well. All right, let's bring Jonathan in. Hey, as always, Jonathan covers a lot in this episode. To see many of the things we actually talked about in action, check out his webinar the Road to Zero Manual Testing Leveraging Automation that's happening on Tuesday, August 29th. To actually see how to overcome the challenges of complex and time consuming manual activities. They Also have an excellent panel of experts that are going to be there as well like Hamza Ahmad, Marcus Merrill from Source Labs, Ethan chung and Anna McCowan. Going over a bunch of different things like testing using over 700 combinations of iOS or Android devices, enabling effortless collaboration, version control and continuous integration. And also automated testing for virtual desktops using keysight's eight Plants Intelligent computer vision to test Citrix workspace securely, which helps you reduce test case complexity and risk. To register, head on over to testguild.com key that's K E Y and hope to see you there. Hey Jonathan, welcome to the Guild. Foreign.

To be here as always Joe, you know I literally just got back from India yesterday actually promoting your book, you know, promoting the Guild. You know you've got so many followers around APAC as well so it's great for to be representing. But also you know, I just feel like I've come back from the future again talking about all sorts of really next generation capabilities, Web4, you know, Gen AI, you know, some really interesting topics to kind of COVID today and also some of this new I'm going to not call it mobile device testing. I'm going to call it just device testing because I think this is where it plays nicely into keysight being the largest pure play testing vendor on the planet starting in 1938 with Dave and Bill in Silicon Valley in the garage testing devices. Right. The first ever device testing company in the world. And I think this is where it's exciting. I feel like I've been sent from the future but I've also been sent back in the past and I haven't been told to terminate Dave or Bill Packard yet. So I'm, you know, I'm definitely on my way to that automation awesomeness.

Very cool. So Jonathan, I think that sometimes people may get intimidated when they hear us speak or hear people like you speak about all the newer things going on in automation and it almost gives the perception as if everyone's doing automation and everyone's being successful and doing all the latest and greatest. But one thing I wanted to touch on is getting started with automation testing, especially the first mile of test automation. In your experience, how many people are still struggling with manual testing and knowing which part of these activities can still be automated that actually just starting the journey or is it now since we've been around forever, people all do automation. What are your thoughts on manual testing? I guess.

Well I genuinely don't think the levers be moved that far. Right. You know I spent a lot of time Last week, you know, Mumbai, Hyderabad, Bangalore, talking to people on the ground, you know, and manually still, you know, put the driving force. And I think the difference is that we've seen waves of automation where maybe we even feel like we're repeating ourselves a little bit and maybe we're only at that 20, 30% critical mass. You know, we'd like to believe it's people have got a higher level of maturity and as you mentioned to the school card is, you know, where do they start, how do they get started? And you know, I think that's, you know, without having to be the first person to redrop in gen AI, I think that's where we're going to see generative AI really take off is it's, you know, you aren't going to be replaced by automation, but you know, these technologies are going to give you an advantage that really superpowers your manual testing capabilities. I think that's what we've had to pivot to a little bit because I think this focus of replacing humans with machines, as I mentioned, being sent back from time, that's not going to be the future. Right. It's augmented testing. It's the ability to augment your capabilities as part of session based testing that allows you to be smarter, more intelligent and utilize some of these tools. And that's the difference. You won't be replaced by a machine, but you will be replaced by other people using smarter tools.

Absolutely. And I heard this analogy. Someone said when Excel came out it didn't replace accountants or financial people, it just helped them do the job better. So I see it the same way as well. So why do you think then people are still struggling to make the shift? Is it that they need to get generative AI in their hands that's going to help or do you see it more like. I think a lot of people take like a do it themselves type of approach and not look at, you know, the 20, 30 years like you mentioned. Keysight started 1938, you said. So there's all this history of a tool vendor that's been around forever, that knows best practices worked where enterprises. Yet sometimes people try to say, oh, I'm just going to start from scratch without looking at all this history from all these companies. Do you see that? I guess this is a long question, but do you see that as maybe something people miss out on, is actually looking at past to learn how to do what they're doing now?

Absolutely. And you know, go back to Dave and Bill, right? They were build and test, build and test Build and test. And the difference was it was Gen 1, Gen 2, Gen 3, 3, you know, Mark 5, Mark 6, Mark VII, like Iron man, you know, with each new suit tool came out, suit new capabilities, right? And those new capabilities started becoming part of the products and it gave the people that they provided those tools. So I remember back in 1938, the first device, the 200D, they gave to Disney to create the first stereo channel in Fantasia, right. The next version, the 300A gave them new features that allowed them to extend capabilities out. Now we have these immersive cinema experiences that feel like we're there today as we evolve and so do the tools that evolve with us. The maturity does. And I think I'm not going to kind of go against what the industry might have kind of pointed us to say, look, let's all shift left, right? Let's try and do things, build our own tools, right? Let's be Dave and Bill in that garage and start from scratch. And it's tempting, right? You know, if you look at some of the great frameworks that are out there, like Playwright, like Cypress, you know, you can get started with and you can do really powerful things. The difference is, is your company an automation company or do you want to leave the creation of the tools to people who are working like my Cambridge R and D team who are working on quantum use cases for testing. Right. And I've got two quantum physicists and an astrophysicist working on these kind of next generation capabilities. Do you need to hire those people or is your core business actually what's more important? And I think this is where we're getting to now. It's the same as what we started on with the days of TSL test script language is actually, it doesn't really matter, you know, understanding code and be able to understand the ability to build these things, to have these skills. Absolutely. But actually we're going to see those. I think we're going to see those disappear. You know, I've just literally this morning filled in a. I'm speaking at Gartner next month about Web4 and generative AI offline generative AI large language models. What we've trained ourselves, which is testers GPT, which is our own proprietary large language model which will automatically generate new code, test at the UI API level at 95%. You know, the large language models trained at 4K, but it can do about 85 to 95% accuracy. And you just kind of look at it and you think to yourself, yes, I could write my own zap tests or my own, you know, playwright scripts. Or I could get this to build 95% of it and then I do the last 5%. Or, you know, what's more important with the augmented testing is that it's my intelligence that I, you know, human in the loop, you know, that's the really interesting stuff about this is our domain expertise is incredibly valuable and we don't give it enough credit. Right. We kind of want to believe that, you know, we need to be tool experts, when we don't, we need to be test experts and domain experts. And I think we'll see that shift. And also we don't need to be prompt engineers. Right. You know, I think this is kind of where we're starting to look at the industry and people kind of want to commit to one thing and it was the same as what we did. We loved building frameworks, we loved difficult solutions, but maybe we didn't need to, and maybe we actually wanted to use our skills, which was in healthcare or in financial services, and really enable automation to go that extra bit. Because everyone's getting to that 65, 75%. And I'm going to, you know, utilize the word auto ops, you know, automation operations. Is you starting to see automation everywhere, not just testing. And I think that's pipeline, that's security, that's everything as code. Okay, how do we do that? Right. Chaos engineers, site, real liability testing, all of these new kind of capabilities you might not be experts in, but you can leverage them to build handoffs that are automation everywhere. And I think this is where it's quite exciting going forward. I think there's going to be so many more opportunities for us as automation people to actually start leveraging this technology.

Yeah, I mean, I totally agree with you. It's kind of confusing because, you know, testing experts like Michael Bolton and James Bott talk about how they seem a little down with AI, but it seems like what you're saying is in line with what they've been saying all along is we need to be testing experts, not necessarily tooling experts. So I see the same thing as well. I see eventually we'll get to visual kind of AI based automation that we're not going to have to worry about coding per se, we'll have to understand it, but not necessarily understand how to. We'll just have the machines do it for us once, a hundred percent. So I guess then will that help then? Like when people are starting their automation journey right now, what do you see as the difficulty for them to even Getting started. And do you see these type of like generative AI, these type of things maybe help them get over that hump then to get started?

Yes and no. I think, you know, part of it is that implied knowledge that we have in the sense of our experiences over time, you know, help us solve problems. Right. So if we're thinking of security and maybe we're looking at owasp, they're all disciplines, right? If you've just started in security, you know, the difference between a SaaS and DAS tool might be, you know, quite confusing to you, but once you understand the principles within your trade, like you've said, Michael and James, you know, they've been talking a lot about large language models. You know, there was a great example with my friend who I did a book with the experiences and test automation, Alan Page, who said the same thing. He started looking at large language models and said, saying, well actually it's not actually that great for testing. And that's because people haven't trained a large language model that is specific for testing. And that's what we've been doing is we've realized quite quickly with things like the EU AI act that these, you know, public services will disappear because of legalization. You know, you've seen some of the copyright violations they've had. You need to train your own specific GPT. That is a legal GPT is a healthcare GPT. It's a testers GPT, it's a security GPT. You're going to start seeing the evolution of these commercial models of large language models that imply those knowledges. Which has a Michael Bolton mode, right. You know, my good friend, which he had on just the other day, Jason, you know, he's done that with testers AI, right, Is how do we embellish the Persona of a accessibility expert? Because I don't know the W3C, you know, standards by default, why don't I train those W3 standards into a large language model and then create a Persona around accessibility and then use visual GPT and some of the visual capabilities to pass it. An image, not text, an image and say what should I test? What kind of scenario should I be thinking out? High contrast mode, you know, how do I address those kind of complex things that isn't in my wheelhouse. You know, you might more for generalist. And we've been kind of pushed away from the specialism area, but now we can have a specialism which could be business focused or heavily business focused or architectural. And then we can build on that. We hand off to other teams, right? Say, okay, well I didn't know enough about this, so I pass this on to you. But here's the foundational information, here's how I've expressed the problem, whether it be performance engineering disciplines where I'm defining better non functional requirements. Maybe it's me looking at open telemetry and observing test observability to say I'm seeing something happen, but I can't get under the hood any lower because I don't understand it. How do we work together and hand off some of those, what was chat ops and this kind of, this view towards conversational AI? How do we start passing through divisions to work more efficiently, become more lean? And I think that's the thing is the barrier of entry will be lowered. People who already got skills will have either more superpowers, the people who are on that journey and maybe need support to kind of get to a point where they understand the fundamentals. Again, just like Googling it, probably not the best idea. You still need some kind of structure around there. You need to listen to episodes of, you know, the Test Guild and go, okay, I'm really excited about this new trend. It could be API testing. I need to understand a little bit more before I just pass a swagger or an open API 3.1 spec at something and say generate me all the API tests because I need to understand, is that right? And I think these are the fundamentals where we've been talking about and the ISTQB and other foundations have been talking about for such a long time. My good friend Paul Gerard has just created something called the Test Engineering Society which he's started building this capability where it defines from every single different type of group, whether you're a content driven one camp or you're in an ISO or an istqb. What is pairwise testing? What is exploratory testing? Tell me these definitions get me started because I need to progress my career. And what he's done, which is remarkable, is he's actually made it so we can get a chartered status as a tester, recognized as a tester based on peer reviewed from other testers, not just I've set an exam, which we know ChatGPT can do now. It's actually recognizing that I'm an engineer and I can get a C engineer or whatever that might be to say I know the fundamentals and I want to do professional development to get me to the next area. And that might be collaboration, it might be particular discipline that you want to specialize in. And I think that's what we've been missing is we've been missing the support and recognition of formal recognition within the testing, engineering, quality, engineering landscape.

Absolutely. So, you know, I just want to go back to what you said where we're going to be focusing more on testing and just having the tools leverage do the heavy lifting for us. It's almost like a seesaw. When we started as well, we had tooling, then went to open source and open source, like, oh no, you're developers, you need to be a developer moving away from that tester mindset. And now it seems like it's swinging the other way again. So, you know, an old debate, I mean, the debate that used to go on or still it's actually going on more now is build versus buy. Because people have, the newer people that have come up have come up where everything seems to be open source and they didn't realize before older guys like me, there used to be tools that took care of everything for you and you didn't have to worry about it had baked in, you know, best practices into the tooling. So do you still see that build versus buy or do you see that now as something that's hard to tell someone? Hey, look, yes, I know Selenium's awesome. I know you could do a lot with all these other toolings. We're not saying it's bad or not to use it, but maybe you could use another solution that sits on top of that, has all this knowledge baked into it.

Yeah, probably. Same as you. I saw an interesting post about the Selenium foundation yesterday and you know, myself, Jason Auburn, Tariq have all talked about Project Carbon, you know, the cure for selenium poisoning. Because I think we might have gone too far down like we did with mercury and kind of got to the point where we realized Selenium is just for web browsers. Right? Yes, that's great. But you know, how many web browsers do you really use every day now? Yes, you probably did 10 years ago. But now we're using apps, we're using generative, we're using web, three technologies, decentralized Internet, you know, messaging, layers, Kafka. All of those aren't browsers. Right. And so I think part of it is we've done a really great thing which is we've created a W3 standard for web browser automation that's really going to help things go forwards. But actually I think we've realized that enterprise scale of automation is much, much more. And scaling automation is hard. And I think part of what we're seeing and we've just finished, I've got a meeting straight after this one with Bloor. I had my IDC call this morning and the results that they started looking at are, you know, there's a shift towards this build versus buy because it's really quite difficult to justify especially in this tech permit crisis at the moment. Resources spending time doing, maintaining scripts. And then I would argue, and this is probably the bit which I will kind of reiterate a little bit is that just having test scripts is slightly meaningless. You know, if I've got a thousand scripts, what does that mean? You should have 2000 scripts or should. And I know we covered this in this open, our open testing session is we need to understand from a different dimension, we need to know from business orientated metrics what does this mean for my business. I've got more confidence that I'm able to release earlier. I've got more confidence that my products are the quality that we have defined organizationally wide. Scaling quality upwards, you know, shifting up, you know, understanding the dependencies or even potential for, you know, failure chaos engineering, you know, all these new types of challenges and we're addressing our customer base, the customs at the center of everything we're doing, we're getting the constant feedback. You know, we're able to bring features to market that are relevant, that are, you know, we're observing from, you know, different shifting right approaches that they're using in the right kind of way. And I think it's so complicated that we've been tempted to just say okay, let's look at a linear journey and cover that linear journey off like we did in the 90s. And I think now that is irrelevant, you know, it's now having a look and changing the game. And so you know, what I'm going to be talking about at Gartner, but you know, I've been talking quite a lot about is this automated test assurance is how do we create a, let's call it a scorecard. You know, you mentioned it earlier on, but giving companies a grade, you're currently at C which means you've got people process technology, you've got maybe security, maybe you've got your pipeline stuff. You're at certain levels. So we're able to say okay, you're at B but you want to be at A. These are all the things that you need to be able to do. This is how we need to mature as an organization wide company is how we have to scale, you know, how we bring all of those components mobile, whatever else it was performance how do we bring all of those up to the level they need to be now aspirationally we've seen from every device on the planet it'll have a CE stamp on it, you know, a certificate to say that that product will work, it will work in certain circumstances. We need to do the same with software. And we've been you know, in the eye of the beholder for a second. We've been kind of confident, well you know, salesforce or you know, workday. How do we know that they've not lost all their engineering capabilities, they've not bothered running any of their automation scripts or performance scripts and they're going to release something into production that's going to break all of these businesses critical systems. We don't, we have no idea what 4 1.5.7 is going to bring and the impact it's going to have. Whereas the DoD have actually just created software modernization bill what says we'll only do business with people who are here rank A because they know and they've learned maybe the hard way that actually software vendors are building on more and more open source and that in actual fact the complexity scaling out of control and if something goes wrong those of systems are in their environment and they have to maintain them and if they don't understand the maturity and level of which you're delivering, the quality of the the engineering that's been brought behind it, then there's a chance that those critical systems are going to fail. And I think we're going to see a lot of this and I think this might be the next wave of testers right Is or quality engineers is for us to be the ambassadors of that to say yes, I'm not a security expert but I know we're ranking based on a tool or a platform or whatever it may be that's saying you've got a D. And the risk to our business could be we're going to have a data breach and be fined £20 million or even worse we could have brand damage that could mean that we lose thousands or millions of our customers. And so we have to change the game, we have to change the conversation. We have to change if I I'm calling it the A model but which is needs a better name but assurity at the top assurance software assurance is at the top at the bottom is just in essence just testing devices whatever that's all been yeah easily consumed now it's very commoditized in the middle there's Tools, there's lots of tools, right. We're not going to say everything's a hammer, everything's a nail. You will need different tools, just like an engineer would, but they're also irrelevant. What's important is what these tools are doing and we covered it with the open testing and the, you know, OTAP stuff is they need to be able to provide insight and analytics and predictive capabilities that will say I'm giving you insight to say you're missing this component or this kind of type of testing and then look at remediating that as quickly as possible time to quality, you know, those kind of metrics which will really be differentiator in the future. And it sounds really complicated, but actually it's really simple. It's quality everywhere.

Absolutely love that. So I just want you talk about complexity. I want to dive into three main points really, really quick about complexity. We mentioned Selenium is just for browsers. There's all these other devices when you're testing at the enterprise level. So I want to cover three areas that the sometimes people fail at the enterprise. One is mobile testing. So maybe talk a little bit about any challenges with mobile testing and how you see how maybe using solution like, I don't know, like a plant could help with that.

Yeah, or Source Labs, you know, Source Labs be the big, you know, supporter of your medium, your platform for a long time. Right. And I spend a lot of time with Mark is actually our September testing keysight World event is going to be talking about exactly this mobile testing or device testing and how to scale to enterprise wide automation. And you know, I interview with him and we're actually in Malibu at the time and we're, you know, proper filming and we're talking about what are the challenges? Right. What's the challenges now? And we, I think that they're different to when we were looking at cross browser testing and we were looking at just mobile testing and said oh, and even the arguments with things like physical versus emulators. And I think what we realized is that there's three layers. The first layer is application layer which we've talked about for many years, application under test. And you can think of that as a, an IPA or APK file that is delivered to a platform of your choice, let's say Apple or Android or even Reality Pro. It doesn't really matter, it's you know, it's an app. But underneath that is the system. Right. The system is that Reality Pro which has got lots of sensors and you know, lidar and positioning and all sorts of other things for xr, probably written on Unity or could be just a standard Apple device, which again, has to work with other components. It has to work with the connected car. It has to work with the Bluetooth profile to stream the audio onto your car to X, kind of your, your connected car IVI system. You know, part of it is that system of systems is dealing with, in essence, all of the software that's on that platform. And we refer to that as a system under test. And that layer means it could have multiple applications, multiple systems of systems, and then underneath is device under test. And that's the, the dreaded HAL, the hardware abstraction layer, where we've just like JVMs, we've just said, let's ignore that, right? Let's ignore the hardware, let's only care about the software. But there is no software without hardware. And so the device under test could be literally different versions of firmwares off the modem, the Bluetooth adapter, it could be the gpu, the cpu, whatever else is on that device. And we've kind of ignored that a little bit. And even when we've had it, you know, we've not said, okay, well, what's the profile? You know, if we're thinking about GPX and geolocations or the stuff that I did with MIT during COVID was Bluetooth proximity for passing over those handshakes. Well, in Android, I think of it as the biggest turn for our industry ever, which is going from IBM to IBM and compatibles, is everyone's making Android phones. You know, there is no restriction on, oh, everyone needs to use this particular type of modem or Bluetooth is that the device has changed so much that actually, how do I test that? And, you know, I was presenting the other day about Wi Fi 7 and 6G, which we've started testing. And so Wi Fi 7, you'd say, oh, well, just Wi Fi 7 is just another standard, but it's not. It's a completely different thing. 48 gigabits per second, we're going from 230 channels to hundreds of 160 MHz channels that allow a bandwidth up to 48 gigabits per second, which is Thunderbolt 3 speed. So suddenly we're saying the latency is reduced. That means it's going to be quicker from a performance perspective. We've got bigger throughput. That means there's bigger loads of data coming through. If we take 8K, you know, an 8K video is anywhere between 400Mbps versus megabytes per second. You know, suddenly it's like, okay, we're getting this huge throughput and it has to be less than 20 milliseconds between, you know, the input to the, in essence input lag, but motion to photon input. Otherwise we see delay. We all look at it and go, that's buffering. Or it's slow or it's tearing. We don't know if the battery level is causing the tearing on the device because the GPUs being throttled, not the CPU. But the GPUs can't do the AV1 format or the H265 format because it can't decode it fast enough. There's so many levels of questions that we don't know because we never got down to that level. And what we expect is that it's going to be the great experience on every type of device in every circumstance. Whether or not I'm on a really poor 3G connection out in the middle of nowhere like I was the other day in India in a tuk tuk, you know, where I'm going between beacons where I've got, you know, millions of people all connecting in a very, you know, centralized city. To me being here today with my fiber connection and my Wi Fi 7 and my 6G line of sight, 3GPP v17, 18 of, you know, 5G max protocol which supports XR profiles. All that complexity we've just ignored and I think, you know, that's going to start causing us problems in the future. We don't need to know it, we just need to know how to test it. Standards are there to help us, like 3 gpp for mobile testing, but you've got to be able to define that because the, the apps of the future aren't going to be just, you know, teams and teams fatigue. They're going to be Windows Mesh, you know, or Microsoft Mesh, which I've already started using, which, you know, has spatial positioning for, you know, environments where you can work together and whiteboard. And therefore now, you know, it's got to go to the team server, but, you know, it also needs to be able to work with all those different types of devices. XR AR, Google Glasses, Apple Glasses, 2025. You know, the world's going to change and we need to change with it and we need to think about how we test completely different. And no selenium was harmed during that conversation, right? Yes, maybe some Appium was used, but under the hood, the telemetry, crash analytics, you know, backtrace from Source Labs looking at, you know, every single Nintendo Switch crash that you get or every single, you know, Unity framework crash. If you're going to think of the new XR devices, correlating that, looking at it and saying, well, it's because it's got three gig of memory on this phone and it's got a memory leak and it's requesting memory and now it fails. You know, how do we possibly know that by what we're doing today? We just don't have the visibility, we don't have the observability and we don't have the enterprise scale to test it.

Yes, right. So you mentioned a bunch of times and you know, automation is kind of easy, it's the scaling that's hard. So really lightning round two quick things. So I see a lot of people struggling to scale because the CICD pipelines, the complexity required to, you know, a lot of manual task management going on. Any quick thoughts on CICD pipelines and.

People can do it better.

Again, CICD seems to be something. We've gone through these waves, right? And this is why I said autops is that we've gone through these waves of kind of trying to attempt to do cicd, trying to have access to the right environments on demand. You know, we struggled considerably. We believe that maybe Jenkins is the solution to our problem. But in actual fact, multi cloud hybrid deployments make things, or even mega cloud make things infinitely more complex. You know, infrastructure, platform, all of those levels. I think we're very tempted to do things in silos. You know, let's do all of my testing for API and put it into a pipeline for the team that are doing API, not doing what I know you talk a lot about, which is this true end to end enterprise testing where we're going between a mainframe, a physical car or whatever it may be all the way through to SAP workday, but you know, the workload, automation, the low code platforms, everything else that's hard to establish because accessing those different environments is difficult. You know, the maturity of lifecycle virtualization is still low. So being able to mock and stub out those endpoints is challenging. Having access to the right data in those environments is challenging. So CICD continues to still be a siloed activity with limited visibility and value that can't scale. And that's why Autops is absolutely essential. It's scaling, agile scaling, DevOps, scaling automation at enterprise wide scale with insight and analytics and predictive capabilities. And I think we'll see again some exciting new products that fit in that area. But it's not the. Just because you can make it run, you need to understand what the value is, otherwise why run it? Right? And it goes Back to day one of, you know, Borland in 1978 saying if you create an automation test, there's very limited amount of chance that that will ever, you know, find a defect. Because, you know, if it runs once, in theory, unless it regresses, it's not. So I think we need to move that mentality out and say continuous testing, continuous adaptive testing needs to be learning, it needs to be testing new things, it needs to be intelligent testing and it needs to be continuous. And what I would say always on know, you turn your back your machine and we call it a brain. We're actually building physical GPUs, Nvidia GPUs that are actually going to go and do workload to say, oh, I've got five brains. I'm going to go and, you know, test all of these new things because it's learning from the system at the edge, edge ML and then actually go and say I've detected a nuance in this environment. I'll go and create some tests around that. It needs to start having a brain of its own, not just be waiting for us to give inputs because these environments are just not utilized. Right. And everything from a sustainability perspective now is measured. Right. Running the same 50,000 scripts and expecting something new to happen is not the behavior of the future.

Absolutely. And one of the last challenges we've already spoke on and Slim can just do browser based automation. And back in the day I had a test like a Citrix system and I used Loadrunner and had a Citrix protocol like in 2000, 2001. And you know, so I'm seeing tools like Akplant being able to handle and a lot of tools now coming up with visual based automation that allows you then to cross all these devices across the pipeline from beginning to end, skipping from browser to mainframe and all that. So can you talk a little bit about. And this lends itself more to like a codeless type of approach, which tends to be on the rise as well. Any thoughts on someone not necessarily having the coding experience that you talked about earlier? You see like a shift going towards more being a testing expert but being able to leverage maybe a visual based no code approach to help them with these other types of automation, not just browser based automation.

Yeah, I think you're completely right. You know, part of visual AI driven automation is to say it doesn't matter anymore, right? It doesn't matter if it's the messaging, whatever it is, it's purely how do we test at scale? And like I said, you know those trends we've been talking about for a long time, high volume automated testing, high volume mobile testing. Well, how do you do high volume automated testing where you build the ability to do secure desktops on demand? You know, post Pandemic, everyone needs to go through some vpn, they need to go through some Citrix, they need to go through all these additional security wrappers to be able to get access to the system, application or device in the test. How do you make that scale so you can paralyze it and say, okay, I want it to run. I want, you know, the 4 hours and 22 minutes to run in 30 minutes because that's when I need the instant feedback. All of these questions have been addressed and you know, now it's not just Citrix, it's everything. And we've prided ourselves for a long time about being non intrusive. And you know, again, back to being a hardware company, we're building these hardware capabilities at the moment which will process computer vision at the edge. So you can literally just calibrate a webcam to look at a screen. You know, you can literally connect the video adapter off the back of something and actually process the computer vision there instead of processing it on my, my Eggplant client. So that way again, it's this new. Those, those machines that you used to use for mining Bitcoin are going to be used for doing complex automated tasks at the edge within secure environments that provide a secure, encrypted way of being able to access those machines and deal with the challenges of accessing things which are hard to access. You know, in medical devices, you know, military devices, you know, being able to test all of the things. And we're using the Word Lab as a service, you know, and within that lab we're going to have, you know, 6G capabilities, autonomous radar emulators, battery analyzers, all available in demand. It works out the slicing of who's going to have access to that device and that battery analyzer things are going to get very smart and it's going to be very interesting. Again, we don't need to know all that kind of stuff. We want it to be suggesting we should be doing this type of resilience testing or this type of device testing. I'm going to go off and do that now for you and come back with some results and some insight. And I think it sounds like it's the future, but it's not that far away and it is going to change the way the automation landscape is going to be forever.

Absolutely. So you covered a lot of things here. People are probably like, oh man, I wish I could see this in action. I overheard or I have some insider information. I think you're doing an event called Overcoming the Challenges of Test Automation Complexity that you're going to be going over. I think you have Marcus Merrill joining you. Humza. Ethan Chung, who's been on the show, Anna McCowan, can you talk a little bit about this event? I think it's taking place on August 29th at 9am Pacific Time. So maybe a little teaser for folks of why they should definitely check that out because we talked a lot about a lot of this but I think actually seeing it is going to help folks. So maybe you can pitch it a little bit here.

Yeah, no, absolutely. And I'm excited about this event for two reasons. You know, I did a call with Forrester today and I got a session with Diego next month on Generative AI. But actually these guys are going to bring it all together, right? They're going to show you the platforms that we're building, the capabilities we're building. They're going to talk to you about how to address that maturity question which you mentioned earlier on how do I assess and understand what I'm missing. And you know, it's not just FOMO of you know, for automation engineers. This is going to be practical, useful, actionable to your book point of view, actionable automation awesomeness that you can do. So don't miss out because you know, Humza, you know and Ethan have got some great examples. They're going to practical examples of how to help those day to day digital grind, you know, automation tasks. And then Anna's going to lay out that kind of footprint of well, how do I do this right? How do I get this and consume it very quickly and very easily? And I think that's lowering the barrier of entry and you know, getting time to value really quickly with tools and that's what you've got to do. Whatever tool you use, it has to give you value and it has to give you value quickly.

So sounds like an awesome event. And for folks to get there all you need to do is go to testguild.com key and register for this event. I highly recommend you do. Taking place August 29th. Love it. All right, Jonathan, before we go, as always, one quick piece of actual advice you can give to someone to help them with their automation testing efforts and the best way to find or contact you.

Sure. So you know, like I said, go off, have a go with large language models. You know, there's great, great tools out there. I'm going to say GPT for all, which is an offline model which can bring down Dolly and everything else and start running this, you know, building this stuff yourself. The second one would be go and look at some of the future stuff, things like automation fabric, the new definitions of the new world of bringing automation across the landscape. Go and do your own research, learn a little bit more about that. If you get stuck, you know, you can reach out to me. Of course, it's the easiest one in the world. LinkedIn.com in automation is my username, so add me in there if you need guidance on where to go and how to move. Of course there's the Artificial intelligence Software testing book which we can give out for free. We can provide that link. But also your book, which I would be, would highly recommend because every single page has notes, it has an action you can go off today and do. And that would be my recommendation. Go through that book, try these things out, learn and reach out to people. If you need help, we're here as a community to support you. And that's, you know, half the battle is finding that community and people that you can trust and even join the Slack channel, which I know because, you know, Stephanie keeps on posting these fantastic questions of have you tried, rest assured with this, this and this. Can anyone help? You know, that's what we need to do, ask for help. And you know, that's why the community's here.

Thanks again for your automation awesomeness. The links to evident value we covered in this episode, head on over to testguild.com a459 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world, we'd love to have you join the fam@testguild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the Guild community. I'd love to hear from you. Head on over to test guild.info and let's make it happen.