How Vibium Could Become the Selenium for AI Testing with Jason Huggins

In a land of testers far and widely journey, seeking answers, seeking skills, seeking a better way. O the Test Illumination Testing Podcast. Guiding testers with our nation awesomeness. From ancient realms to modern days, they lead the way. Oh, the Testicle Automation Testing Podcast. Hey, it's Joe. And today's episode is one you don't want to miss. My guest is Jason Huggins, the person who first started the original versions of Selenium, Appium and Sauce Labs. And he's back with a brand new project that's already creating a bunch of waves online, vibm, which is an AI based test automation tool. So if you've seen and heard rumors and want to know what VIM really is, how it compares to Selenium, and why the testing world is already buzzing about it, stick around. You'll hear directly from Jason on what FIBM can do, why it's going to be open source, and how it could possibly reshape software testing in the age of AI. You don't want to miss it. Check it out. Hey, quick question. Are you overwhelmed by test tool research? You're not alone. Choosing an automation testing tool can be a rabbit hole with hundreds of opinions, each promising to be the best, but none tailored to your team's specific goals, stack or budget. And that's where the free test skill to matcher comes in. Answer a few quick questions, think test, type, budget, tech stack, and instantly get a personalized shot list from over 300 trusted tools. This will save you weeks of demos and sleuthing. Get expert curated recommendations backed by insights from our podcast, interviews, blog posts and webinars. Make confident decisions faster. Think of this as like a G2 product hunt test Tool Encyclopedia of Test Tooling Awesomeness so you're ready to cut through the clutter. All you need to do is go to test guild.com, click on the Tools menu and select the tool matcher to find your perfect test automation tool in seconds. Powered by the Test Guild community, where automation pros get smarter, faster.

Check it out.

Hey Jason, welcome back to the Guild.

Hey Joe.

Yeah, I think it's been 2018 since the last time you actually officially been on the show. There's a lot of stuff going on around the Internet when I look for Jason Huggins nowadays, talking about this thing called vibing and like, I don't know what's real, what's not, what's going on. I don't know if you're trolling people. So I guess a little thing like what is vibing? What is this all about?

Vibum is Selenium for AI. Like, that's it, full stop. Whatever we did with Selenium for the web, whatever we did with Appium for mobile, we are doing with AI. And if anything, you could summarize. My career is that, like, whenever a platform comes up, I do whatever I do and try to make a little testing tool that rides that wave. Obviously, the AI is the next wave. And so VIBM is my kind of shot at articulating this, turning this weird fuzzy metaphor into reality. Right.

Is it the successor of Selenium in that you're building off, you know, the protocols that Slim has built in, all the hard work that's gone into it? Are you just starting brand new with this?

Brand new? Yes and no, man. Like, I. So the reason why I want to do this, and I apologize to everyone, this is kind of like raw version 1. I have not perfected my elevator pitch, or if you will, we are right now in a building that is like 500 stories tall, and it's a very long elevator ride. By the end of the month, it'll be a. It'll be a shorter building, I swear. But another way to look at this, I only have like these bumper sticker phrases, unfortunately. But another one to say is, whatever I'm doing, this is my webdriver by die awareness campaign.

So, hey, you know, if you hadn't heard about the WebDriver bidi spec, you should go check it out. But fun fact, the Puppeteer team put out a blog post, you know, last year that they went all in on the WebDriver bidi specs. The puppeteer is like full fledged WebDriver bidi. Right. Also, Firefox, they deprecated their CDP, their clone developer tools thing, and they also now support webdriver bydi.

All right, Jason, up. Hold up. For those that don't know what is.

Webdriver Bidi, the summary is it learned all of the lessons, or most of them, of what made puppeteer and playwright amazing. And that was the Chrome developer protocol, the CDP.

And it just made it fast. Right? So WebDriver, the first version, classic they call it, is REST API. You send a request, you get a response back, but it's flaky and it's a. It's an. It's an expensive chatty protocol. There's a lot of stuff. It's basically HDP versus WebSockets, really. It's just websockets, like, so there's kind of a nerdy thing. If you haven't even read WebSockets like, oh, there's more stuff to go in. But the reason that Puppeteer was amazing, Playwright was amazing, was because they're using a websocket based protocol, full stop. And Webdriver wasn't granted Webdriver, sorry, WebSockets didn't exist when we started Selenium and Webdriver and whatever. So like, hey, you know, cut us a little slack, I guess, whatever. But we're rapid, we're not going away. So.

The webdriver by die effectively is standardizing what was a proprietary protocol that only existed in Chrome. Right. So making that thing a thing that works standard. So webdriver bidi is like if you wanted Selenium to effectively, sorry to be blunt, stop sucking one of the things or to don't be so flaky. One of the things you would do if you're going to start over from scratch is use a websocket based way to talk to the browser. That's what WebDriver bidi is. Right? It's been built into the Selenium existing libraries. It's in Selenium libraries, it's in Puppeteer now. There's even an issue on the playwright GitHub repo that they're, they're investigating. You could argue that I'm at a base level, at a fundamental level is that I'm doing a from scratch implementation of the WebDriver by Dice Spec. Just that's just the JavaScript in Python. There's going to be a library that just fulfills that spec. But if you had the luxury of now knowing you have magic AI robot technology that you could have put in there, instead of bolting it on, you could just assume you now have. Occasionally you can jump out, pull the parachute and ask some magic to do something for you. If you had access to that, you would. Then maybe there's some points along the implementation path where you would say, hold on, timeout, let's ask the magic. Right. If that makes sense. Right. And we're all in real time. I have some opinions. I'm not saying I'm right, but I have some opinions and I have some hunches and some intuitions. And so everyone is already kind of preordained that this is like the future. And I've kind of even in my YC application saying like, I'm creating the successor to Selenium. So that's like on me that that's what I kind of promise to people. I hope whatever I hope it is, but I haven't done it Yet.

So, Jason, what about MCP servers? Isn't that the next big thing?

MCP model context protocol. That's not even a year old. That is. Some people are saying that is the thing that will be the merger between the AI and all of the things.

There's a playwright mcp. There's a Selenium MCP that Angie Jones made, which is amazing. I talked with a developer at Stripe and I, and we were talking about the MCP stuff and he said, like, oh, yeah, but it's expensive. It's like, oh, yeah, I know, man. Like the tokens and like, it's expensive. Like, no, no, no time. Like, say more. Right? And it's like, no, no, no. If you have the LLMs, even if you have a crazy, ridiculous $30,000 rig on your desktop, which actually is literally financially expensive, it. It still is slower. And when you're at a scale at like, Stripe.

Asking the AI to do literally everything, like open the page, do think, click a button. Look for a button. Like, that's the thinking thing. Click the button. Oh, I have to think about how to click. If every interaction back and forth is back and forth over the MCP channel, it is expensive in time. And so this is my opinion, and I could be wrong because I'm not an MCP server. Like, who am I? I'm just like. The argument is that I don't think MCP will rule the world unless either they get the time down to zero or LLMs. Some people are making the bet that actually LLMs will just keep getting faster and faster and faster and faster. Like, yes, they are slow. You have two cosmic problems. They hallucinate. So there's a making a stuff up problem and then there's like a slow problem. And everyone is kind of assuming that there's some kind of vague Moore's Law about AI, that they'll get, like, smarter, less hallucinating, and just faster, full stop. Right? But there's also a thing where you could kind of like. Well, actually, especially in software testing, determinism is still kind of a thing that we would prefer to have. Whatever. And also FAST is still kind of a thing. And if there's like, some obvious ways we can actually just. We don't need to ask a robot to do it because we could just see the button on the screen and I know the coordinates for it. I'm just going to click it over a protocol. I don't have to, like, have a round trip through a robot to confirm what I can already see. I hope I'm making sense here, but basically, like, what I'm doing with VIBM is that would be talking to the web, like driving a browser of the WebDriver protocol. And occasionally when you get stuck right now with Web, with WebDriver, it just said bombs out and says, element not found. Good luck, have a nice day. Right? That's what everyone actually makes. Them bail out on webdriver saying, screw you guys, I'm out of here. Then you every go down and download playwright and then it'll say, hey, I don't see the element. But like, hang on, right? And then you play like the elevator music and like, and then it shows up like, oh, hey, the button's ready. And then you continue. Right, but if you were extending my metaphor here, there might be some situations where no matter how long you wait, that element is not going to show up, right? Like, you're just kind of stuck. And.

That'S where potentially you. You fall back to an AI and say, hey, I'm stuck. But I don't. I might not necessarily want to fail the test, right? Because really, actually this is where I kind of get into the stuff that no one knows about. But this is the thing that might actually be the first version that gets released. The reason why I'm kind of talking about stuck, I use this metaphor of like, if you ask Google Maps to get you to the store occasionally as you're driving, it'll say, oh, hey, there's a traffic stop, or I found a faster route or whatever, right? It will in real time tell you there might be a better way to go. And I don't think, well, in a modern context anyone's really kind of is doing, sorry, take this back. People are working on this. I'm now working on this. There's this idea, it's actually an old idea that I'm now bringing back. Another way to summarize what the heck is vibing. There's. It's really two things, right, Is that it's a low level. There's going to be a library that you can download. It implements the WebDriver spec. It's going to be WebDriver by die. Spec is going to be in JavaScript and Python. And then based on all my survey responses, probably Java, because that's actually huge.

At a low level and at the high level, we're taking an old idea that's been around forever in academia, but it's never really had its moment in the sun, but I think it will. I'm going to, I'm going to try to give it its moment in the sun. Model based testing, like it's a kind of a wonky academic thing. I'm rapidly coming back up to speed on this stuff. But it's basically this idea that if you map your app, the automated tests actually just fall out of. It's all really kind of wonky stuff. So people accuse me of like.

I'm just trolling the Internet. There's nothing. It's like, oh my God, there is so much. There is so much stuff that we have to get up to speed and learn on me specifically, but all of us collect collectively. Model based testing works. But its fatal flaw is that it's tedious and very manual labor intensive to create the model of your app. Because the second you do that, you say here's these pages and they're connected over things. It's all graph theory nodes and edges. You have two nodes, so you have two pages. You have a login page as one node and you have a checkout page on the other one. And they were connected through intermediary other nodes. It's all nodes and edges. And those edges in like the academic term that they would use is like a click or a submit button. Like there was an action you took to go from one node to the next. Right? So my metaphor going back to Google Maps, I want what I want Vium to be is kind of like, again, I've used this other metaphor. I'm just throwing them out today. Waymo for web apps, right? So you just get in the car.

Robot takes the wheel and drives you to the store, right? That's what we want for testing. I want the metaphorical equivalent way more. Way more for web apps means I want to like go to my laptop as Mr. President business CEO and say, can people buy stuff on my. On my web store? That's the question I want to ask. And just, just like you would ask Google Drive drive me to the store. What I want for software testing is can people still buy stuff, right? Granted that is probably triggering for a lot of people. Because what I would say though is I'm. Because like wait a second. No, but like we actually have to verify that the button is blue. Like all kinds of weird things actually have to happen, right? I would argue I am adding a new layer to the testing pyramid. And so I'm like, I know I'm triggering to a bunch of people, but I'm also saying there's a new kind of testing that can exist now. It's basically like, like end to end testing, but like Lazy mode. Advanced lazy mode that you can just kind of say arguing, putting my marketing hat on. A marketing phrase like vibe coding, that was also only coined last February. It seems like it's been around forever, but it's only this year. It's just as good of a marketing term as cloud computing. Nothing has. Nothing has been its equal since the phrase cloud computing. And so as far as a way to communicate whatever is going on with the AI world and the merger of AI and coding, being able to communicate to your friends and family at like Thanksgiving table, right? It's a phrase that kind of breaks through. If you're driving down like the 101 freeway and you see a billboard, it's probably going to have the word Vibe coding on it or vibe something, right?

So, all right, I have to ask, why the name? Via, I think it's cool. But there's a lot of people out there that really think the name is an actual joke.

So when people are accusing me of kind of like trolling the Internet a month ago and doing all this stuff, I kind of realized, like, calling this VIBM is a little silly. But also I realized that there's some juice there. Like, there's just. From a marketing point of view, there's something going on. This because like, the word Selenium itself was also kind of a dad joke, right? Here's the other thing. This is effectively my robots are called valet. The last two years I've been using the valet metaphor and the product name for my stuff. Uh. Cause it's a classy way to drive. To drive a car. A valet is nice in French and whatever. And it was like, oh, but. And I'm web and I'm implementing the WebDriver protocol and it's the word driver valet. It's perfect chef's kiss of like a pun that makes sense. And guess what? No one cares. I put the valet thing out there and just like crickets, right? And then so I could have called this thing like Valet AI or Valet Em or whatever. And I just knew that wasn't going to work. I think there was another thing just on the marketing, I could have. This also might be triggering to the Selenium committers and I need to go actually have a coffee with them immediately. But we could have called this Selenium 5.0. There is an argument to be made of, like, what is selenium 5.0? Because whatever, right? And there's precedent for this. Google put a huge marketing campaign for HTML5. They didn't rebrand or fix or call HTML something else they called we're coming out with HTML5. It's got a whole bunch of cool stuff, it's got WebRCC, all kinds of amazing things and it's all HTML5. They did not get rid of the HTML brand me without actually having a council of elders meeting about this. I was choosing like, okay, am I going to go all in on making selenium 5 the phrase or does Selenium effectively have a branding problem at this point? And I kind of needed to coin a new phrase. And right now it's unfortunate reality. But the Selenium brand, every company, every product, every open source project, whatever has come out and they've had to take shots at Selenium to say Selenium sucks in X kind of way, use our stuff because it's better. And the thing is, it's not just a marketing thing. Like Selenium actually did have those weaknesses and deficiencies. I'm not saying it's like they're all just a bunch of liar liars who lie, whatever. Like this is how you market. Like you point out something that is true and you can leverage that into marketing your thing. Right? But everyone's been doing that for, with Selenium for the last 20 years. And at some point those hits that you take, it's like a video game. At some point those, those health points, your, your counter goes down. Whether it was like a legit thing or not. You got dinged, right? And so the Selenium brand is this also because it's been around for 20 years. Some people say, oh, it's fine, whatever. But that's like what your dad uses to test, right? But so I just have this marketing thing where it's like it could be valet. No one cared. I tried it actually. No one cared. I could go have a meeting with the council of elders and say like we're going to go all in on Selenium 5. Maybe that could still happen, whatever. But I kind of thought like to capture basically testing needs like some new juice. Like it like all the juice that could possibly be squeezed out of from Selenium is like it's got all this weird kind of marketing baggage and all kinds of stuff where everyone's taking shots at Selenium.

And I just kind of figured like I'm doing a new thing. But here's the secret. It's actually webdriver, like ta da haha. Pull the mask off. Like the Scooby Doo reference thing. It's like, oh, it's actually Selenium and webdriver under the hood, right? But this time it's actually Selenium and our your pick your favorite AI LLM friend. So that is a low level. It's WebDriver by Die and the AI robots doing the things and when it gets stuck, it's figuring out how to reroute. But to be able to do the reroute, you also effectively need a map of your app. You can't have a self driving car unless it knows where the roads are, right? So there was a whole separate multi year project. Hopefully in this case it won't take multiple years. But Google Maps was almost a prerequisite technology and Google Street View was prerequisite technology to make self driving cars work. You have to have a navigational, you have to have literally you can't just set sail and go somewhere or to not be a metaphor, you can't just bring up ChatGPT and say test my app. Buy a thing like go from. It can open up the front page and then take an educated guess on what the next step is. And the thing is now you're rolling the dice that it's going to be able to take the correct educated guess every time a new page is loaded. If all you do today current technology is go to ChatGPT and say test my site or actually spit out a test.

And even if it has access to all the source code, there is a certain kind of a magic too. Like you could analyze the source code but you won't even really know until you run it how to do that. Right. And so you need a map of your app. And so then the question then is like okay, at one level the thing that I announced, the library, that's the thing that everyone's responding to. It's going to be better at dealing with flakiness. That's going to be built into it.

All right, before we go further, so how does this relate to automation? How do you create this map, say for automation testing?

And so there's a couple of companies that are working on this and it's basically what I'm trying to work on is we are bringing model based testing back, but instead of it being a manual process that you have to go create the model and label all the things, it's going to be automatically generated for you. Just like Google Street View as the way they mapped the world is by just driving a car on every street. They just, they just, they didn't take a satellite view and draw over the lines. They literally just, they just literally speed run it they just drove. Right. And so in this case the not metaphorical equivalent of this is you instrument your app with in a way that when people walk through it, events are getting fired and then those are getting stored in database somewhere. You could call this again not an original idea but like maybe when you combine all these things together, it's a thing. Years ago I used to think about this of like hey, you have Google Analytics, but the it's product, it's marketed to marketers that just want to see which pages that people are going to so we know what keywords they're using. But you could take what Google Analytics is and repurpose it to making that a useful tool to developers and testers. But even more specifically, instead of just showing you pretty graph of like oh, here's your bounce rate on your first page and here's people who got all the way down your funnel, all the way to the end, it just spits out a selenium or whatever flavor of I guess vibium. Now test script, it just spits that out and then you can click run and it plays because you've effectively mapped as people have gone through manually. So there's kind of a weird interplay also between manual testing and automation. It's never black and white, it's kind of gray because of that manual traffic through your site. And it could be in dev or it could be in prod, whatever, all that stuff, all those events are getting fired. And AI magic thing is collating that into a test suite. It's effectively creating the map of your app, but it's also mapping workflows. So not just where the streets are and where the buildings are, but where the paths are. Like where are the most common. What's the best route between A and B, right? That's a thing that you can kind of do without AI. This is like graph theory kind of stuff, right? And then you have, okay, now you have the map, now you actually can again this is actually more statistics and kind of just like code algorithm stuff, not AI magic yet. Which means it actually works. And there are reference implementations of model based testing that you can kind of just use. And it's like before the AI hype wave, it's out there, right?

And you have that and that's the thing you can then give to an AI tool. Basically the self driving car. The self driving browser now can reference that map that was created in a separate process and go from login page to checkout.

So let me just try to summarize it really Quick.

All right, so I've seen other companies that have like a sniffer in production. So it knows real actions users are doing and so it's shifting right to left because now it knows a model. Not only is it a model, but it's a real model of what users are really doing, what they're really touching. And because of that they're able to generate automated tests based on real traffic.

Right? Yeah. So that's that. But also I'm Jason Huggins. So it's going to be open sourced out the wazoo. No secret magic. Every. All of the magic will not be gated behind. Like hey, this is amazing thing and it works and oh my gosh, it's the future. Sign here. Like it's only behind our SaaS service. So just like with the Selenium project where there's reference implementation like, like for every SAUCE Labs there's a Selenium grid, right? Or for every, like the Selenium grid implementation has to exist, right? So and a lot of people do run their own Selenium grid. And also same thing with Appium, they have their own local lab. They do not have to use SAUCE Labs, Lambda Test Browser, Stack Testing Bot. Go down the list, all the guys, right? I haven't even, I can't name them all, right. There's so many of them, right. There's so many SaaS players. But there's also open source stuff you can do. There's like the on prem open source chocolatey granola, you know, Portland approved thing that you can run locally and whatever I'm doing with this is also going to be, there's going to be an on prem reference implementation but that also has implications. That means that we're going to have to make. We're constantly going to be evaluating the. I know it's on open source, but like it's the, it's the open weights. I mean I would love to have if there was an open source LLM, we'll just use that. But the best we can do right now is the open weights versions. There's Quen, there's Llama, there's a bunch of them. We're going to have to first class citizens of whatever we're creating here. There has to be an on prem local version of the entire stack. All of the magic is available that you do not have to sign here like one lump, you know, yearly contract, whatever. What I'm doing with Vium is like there's going to be an open source implementation of this stuff.

So okay, so now What?

I'm just now realizing I've hit a nerve and I now need to just like, pay attention and catch up and actually one actually release some of the software. But there's also a certain level of like, I now need to literally, Avengers style, assemble the team.

I cannot do this on my own. I know there's some kind of like some billionaire VCs are making the joke. Like now we're in the era where we can have a billion dollar business with just like 10, like one employee or something like that. And there's this weird thing where it's like, you know, me and my AI friends could probably do it, but like, I'm too busy shit posting on LinkedIn right now to pay attention to what's going on with my vibe coding friends over on the left. So I still need people to watch the robots from the farming kind of a point of view. I have to get up on, you know, CNN and give out, you know, give the farming report. Meanwhile, we actually have to have farmers in the field doing stuff as well. There will still be plenty of jobs for humans, I swear to God. Because right now I'm like doing the AI vibe coding robot thing that theoretically is taking all the jobs. And I am now freaking the F out because there is now so much work to be done and I'm only just one dude who can't click enough buttons fast enough on the LinkedIn interface, which of course, professionally, people have been coming out of the woodwork saying you should probably Also close the LinkedIn tab and go, go back to work. Right. But. But whatever. That's my story, man.

Awesome. So, Jason, if people want to learn more, I know you. You were kind enough to join me at my first test guild in the Real, Real Life event happening in Chicago in September 18th. Yes.

Beautiful. Thank you for choosing Chicago for that.

Yeah, Excited for that. Where can people learn more? I guess another piece of confusion that comes in is people are going to Vium AI and they're like, this is this. What is this?

We haven't. We haven't even gotten to that. That is effectively, if you re this, the thought process for that was if I started Sauce Labs over again from scratch, I would do a global, decentralized network of devices all around the world and open the source the snot out of it. Right? So it's effectively a distributed network of devices that could run tests. That's why I use the metaphor in various places. Like it's like Uber or Airbnb for testing, because you're using idle computing Capacity around the world. That was my like previous version of this idea. And that's what that. That snapshot in time of. That's what that website VIBM AI is. And then another snapshot in time, fast forwarding a little bit was my YC Vounder video. And then now in the last month I've kind of realized I've got this kind of map part that's also kind of a prerequisite to I'm working. So that is all to say VIBM AI is the going to be the website that this stuff is going to be landing on right now. It's not what it should be. I need to lock that down, put up a landing page or a capture email thing because it's too distracting. But it's also been interesting to see leaving it up, seeing all of whatever people are imagining it is and complaining about it and whatever. And I guess maybe I should be embarrassed, but I'm just kind of having fun with it and whatever. I think the best thing you can do is follow me on blue sky or LinkedIn actually right now. And the other one is, I would say, like people should follow me on Noster because that's also part of phase two. So if anything, the best thing for your audience would be like.

Until there's a better actual home for this thing is see whatever I'm randomly ranting about on LinkedIn, Blue sky or, or, or Noster, like hugs, dot B, sky, whatever. Or you can find, you can, you can find me on LinkedIn. Right? Eventually it'll be VIBM AI will be the page. Once we can kind of get around to it. There also is the vibm. There's going to be a Python package that's already there. That's VIBM. It's just a placeholder. Sorry. There's already actually 200 downloads of that thing and it's literally just like hello World. That's the code, but I kind of reserved it. And also there's a vibm package on npm for JavaScript that. That's there. And I just checked, there were 30 downloads as of today. And also literally, hello World. So there's going to be places if you're a nerdy geeky person, it's going to be where, you know, find software is available.

And then at the, you know, CEO to CEO level, it's going to be VIME AI. Right? Eventually. But right now the website's a little confusing. It'll hopefully fix this by the time this goes live or something. Whatever. I don't know.

Love links to all this awesomeness down below. Thanks again Jason for all your automation awesomeness. Links to everything I value we covered in this episode. Head on over to test guild.com forward/a559 and after almost 600 episodes, if you haven't already, can you please rate and review this on your favorite podcast listening app? It really does help us in the rankings and also lets me know what you're interested in. All right, so that's it for this episode of the Test Guild Automation Podcast. I'm Joe, you and missions help you succeed in creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across all our platforms and over 40,000 email subscribers who are at the forefront of automation testing and DevOps. If you haven't yet, join our vibrant community@testguild.com where you become part of our elite circle driving innovation in software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit test guild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

With lutes and lyres, the bards began their song A tune of knowledge, a melody of code through the air it spread like wildfire through the land Guiding tester showing us secrets to behold.