The Web Browser Is All You Need - Paul Klein IV, Browserbase

Hey, everybody, I'm Paul. I'm the founder of Browser Base, and I am obsessed with browsers. Specifically, one type of browser is headless browsers. And I'm here to talk about how the browser is all you need. It's not attention, it's not mcp, it's the browser, or specifically the browser. MCP Server is all you need. And I'm gonna try and keep it light on slides. We only have 100 to get through in 20 minutes, but you and I, we can do it together. Just stay with me for 20 more minutes. Okay, so first of all, a little about Browser Base. This is what it looks like to build your own browser infrastructure. It's messy and it breaks all the time. With Browser Base, we can let you run thousands of headless browsers in the cloud, very easy to control. And you might be wondering like, well, why do I need a browser? Well, every AI agent needs a web browser. That's the point of this whole talk. So of course I can talk to you about how to scale browser infrastructure and all that interesting stuff, but that's too pluggy. We'll save that for later. But I do know quite a bit about how customers are using browsers as part of their AI agent applications. Specifically with mcp, we have the most popular browser automation MCP server out there. And the reason why people choose browser based to run headless browser mcps, that we make it really nice and really can help you scale your infrastructure. And when you think about why do I need a browser? Well, I like to think about two things. You have AI agents and the legacy Internet. The DMV is not going to have an MCP server anytime soon. My barbershop is not going to open a GraphQL API for me to schedule a haircut. As much as I keep begging John to do it, he's got better things to do. So if we want AI agents to interact with the rest of the legacy Internet, they need a bridge AI. And I really do believe that the browser is that bridge between AI and the rest of the Internet. And this is the unsexy Internet, I might add. It's the Internet that's not gonna get a lot of attention. I've seen countless flight bookers and countless restaurant pickers, and I have not seen anyone do the thing I need, which is a Delaware franchise tax filing agent. Any founders in the room have done that before. Not super fun. And I think people use a lot of acronyms these days. You know, you have MCP, you have A2A open API but if those aren't available, you can just do what could be considered the dumb thing. You could just use a website. And websites are out there. There are plenty of them. There are billions of websites. And when your user is going to prompt your agent to do something, you might not always have a first party integration available. That's where a browser is kind of the integration of last resort, the path that you can take your agent down. If you don't have something, a primary integration already built in, you might be wondering, well okay Paul, cool, I get it. You've beaten this thing dead AI agency in the browser. Well, how do they control it? Well, you can think about web agents and browser tools. First of all, you know, what is a web agent? I want to keep this technical as this is a bunch of AI architects, right? You know, web agents, we've heard about them for a long time. Take a model and then have it generate some code to control a browser by generally parsing the DOM on the page, the HTML and the css. Web Voyager was early here adept. Did a lot of really cool stuff here with their Fuji models. OpenAI had operator proxy by Convergence, now Salesforce, each company, everyone was kind of doing a lot of this stuff. Last year we really got to see a lot of web agents in production. But they were still early days, you know. And Web Voyager was first, you know, it was taking screenshots of a page. It was using chain of thought prompting and then from that it was saying click the button at this coordinate. Sometimes they're doing these things. We'll talk about a second called labeling on top of the page. But I think it's pretty cool because we haven't changed that much from this. There really are two different types of web agents. There's vision driven agents. These are ones that predominantly use screenshots as context for the model. They might do some marking up of the screenshot to indicate what box to click on. Or there's text based web agents who predominantly use HTML as the context of the model. Both have different approaches, pros and cons. Text Web agents use XPaths and playwright code. Some may argue it's more repeatable. Vision models can be more accurate on more complex pages. There's trade offs here and it really does depend on what website you're trying to automate. Here's an example. On a vision agent using set of marks prompting, you can see these little boxes here where you're marking up what you should click and the agent will or the model will return and say click the box label 25 that's gonna help you out. And on the DOM based agent side, there's also ideas of like taking HTML and how can we transform HTML to make it more reliable for web agents. So the accessibility tree is something that's built in every page in a lot of applications where you can take a different structure of the same information and condense it down where you get the same layout but without all the extra div tags and classes. So we have vision based agents and DOM based agents and there's now also computer based models which are kind of like the next step here where, well, let's train a whole model on this stuff. Previously we're just using like the stock image models, but now we can train that model on these things called web trajectories. And I won't go too much into this. There's a lot of great papers out here. I'd recommend this paper link down here about like web trajectories and how you can generate them to do rl to teach models how to not just pick the right button on the page but, but how to reason across multiple pages about the right path to take. But all in all, there's just a lot of innovation here happening on teaching AI how to browse the web. And this stuff is getting good. It is working and you can use it today to help add some sort of extra functionality to your applications if you're making the right choices. And I will kind of add. You kind of want to think about if you want a web agent or a browser tool. And you may have never heard of like what the difference is here, but there is a difference in my head. Like a web agent is kind of like one prompt to many actions. I think OpenAI's operator is a good example of this. You say, hey operator, go, you know, file my Delaware franchise tax. You give it some prompt, it's going to go take many actions. And if you give it that same prompt twice, it might take two different paths to get the task done. Web agents are good. They're like little cockroaches. They're just going to keep trying to find a way to complete your task. But they're a little bit more non deterministic because the reasoning is in their control. Whereas like browser tools are like one action or one prompt. You say click the sign in button. That thing's going to click the sign in button. You ask it to purchase the Amazon item that you want. That's a series of multiple steps that really might be more suited for a web agent. So we have a framework called sagehand that we think is the best browser tool. And it really does depend on what you want. If you know what your workflow is going to be with some high level steps, you can actually use a browser tool to take those steps and translate them into reliable web actions. If you don't know what you're going to do, if you don't know what the prompt will be, a web agent more generically might be the right fit. But I do believe your AI agent needs a browser tool. Another thing I'll add is like you may want to think about with mcp, what types of servers are you integrating? So a vertical MCP server with something like linear, where it's going to give you options to control specific things on a specific task, like create a linear ticket, create assign someone to the linear ticket. Whereas like a horizontal MCP server, you're going to have some sort of permit that can do many things. You know, for browsing, we view it as a horizontal MCP server. You're exposing perms like click a button on a page. Now that page may change, there may be many different pages. But when you have a horizontal MTV server with one server, you have the opportunity to automate the whole web. And these like, you know, you will see vertical NCV servers, which are more direct tool calls, as like an important part of your agent. I'm not saying we replace those with browsers. If you are interacting with Salesforce, you probably should just use the salesforce mcp. You don't need a browser there. But if you're interacting with custom bespoke MCP built by large enterprise that doesn't. Sorry, custom bespoke CRM built by a large enterprise that doesn't have an MCP server, you don't have to go reverse engineering APIs. You can use a browser tool and a browser MCP server to go out and automate that. Does that make sense for everybody so far? Okay, thank you guys. Hearing some heads nodding. Okay, a few other notes on like MCP and all these like concerns I have here. I do think that MCPs are going to have to pass compliance and dynamic tool discovery. This idea that you can plug in your agent to an infinite toolbox of MCP servers is going to be hard for CISOs to get on board with. Right? You're going to want to be able to select which MCP servers make the most sense. And with a browser or a horizontal MCP server, this could be browser, it could be email, could be anything. You really only have to Onboard one MCB server as opposed to an MCB server for each individual integration. Secondly, like, yeah, that's the most important one. Secondly, like, evals, most benchmarks are fake news. Just want to let you know, especially when the company putting out the benchmark is the one that's also ranking themselves. So I would be very critical of public benchmarks of any web agent you see out there or any model you see out there. You really need your own evals. My friend Ankur who runs Braintrust, he supports our evals. I really like them because then we're able to say, hey, actually, for our web agent, we're building which models are the best model for this web task or for this certain website. You can get really intelligent and honest about what you actually need to do to automate the web and which model is the right choice for you. Finally, I think you need observability. If your AI agent is controlling a browser, you need to be able to see what happens in the browser. Browser bundles this out of the box so you can build your own. Taking screenshots, recording history, recording actions, making sure that you know exactly where your agent went to and why is important because let's say your agent is gonna go buy an Xbox and it buys you AirPods. You wanna understand what prompts went into that, what page paths it took, and really break that down. At Browserbase, we include this in every browser. Sessions are recorded, logs are available. It really makes it quite easy. Okay, so that was like a lot on browsers, and I got a ton of time left. But I'll end with this one point. The browser is the default MCP server for the rest of the Internet. If you need to integrate with something, whether that's with MCP and API and there's not something available, you should really consider including a browser, because a browser is all you need. And since I'm doing so all on time, I'm going to do some live coding because I feel like there hasn't been enough live coding in this room today. So I'm breaking it out and let's pull up cursor and bring it over here. And it's going to be so hard to see, but let me try. Give me this, please, love. All right, so we have cursor right here and all of my screenshots. I'm a screenshot hoarder. I apologize. So I pulled up the cursor MCP server, sorry, the cursor controller. And I've written a prompt. It says, create a new raspberry session, navigate to sfpca.org, close out any modals and find a dog for adoption. And in San Francisco return the URL. We'll click enter. And I may have to jump to a browser here really quickly. So you can see it's calling the browseway session tool. It's generated a browser by session right here. And you can look how it's actually making these individual tool calls on the page. If I pull up the session at the same time and scoot this tab over there, we can see in parallel the browser is loading. And as our MCP server is making these tool calls right now, it's trying to close out the modal. It's navigating the website and thinking about how like there's a reasoning model here that's deciding what should I do, what should I call? These tools are exposed. And now if we go here, we have been given a dog with this URL and one of you will be going home with a lucky dog today. All right, can I get a drum roll, please? All right, that was so half hearted, but thank you. Give me a dog. Yay. There we go. Oh, they really want that 200k match. This campaign's been running for a while. I feel kind of pathetic today. But that's a good example of what will happen when you're building these web automations. Sometimes modals will pop up. Sometimes things you aren't expecting may happen on the web page. You need to have an AI agent driving the page with primitives so it can react to any sort of weird changes that happen. And hopefully you can integrate with things that aren't going to be AI native to me. The most important problems to solve in AI right now are the unimportant boring problems. And they're going to require intelligent engineering to both bring the best models to the unsexy problems. And when we talk to customers at Browserbase, they're not just Perplexity or Clay or Kimura. They're also a 55 year old dairy trucking company who has never hired an Engineer in their 55 years until this year. And the first thing they did was use browser base to automate a really painful ops workflow. Workflow. So if I can somehow pull my slides back, I don't know if I'll be able to. I might just have to move this bad boy over here and we'll just go for it live. Yeah, screw it. Let's do this. Okay, so if you do want to try a browser NCP server, it's available today. You can actually Go ahead and scan this QR code and sign up or use this to try adding some sort of automation. It's really easy. If I can use it, you can use it. I promise. I'm going to pause. I'm seeing some photos being taken. Amazing. Great. And finally, you know, if you are looking to join a company that's growing quickly, browser's been around for a year and a half or 30 people, we're backed by some really great investors and we would love for you to come build the future of automation with us. All right, that's all my time. I might have a minute for some questions or two, if that's okay. Yeah, if there's any questions, happy to take them. Otherwise, thanks so much, everybody. Any questions? Hold on. Maybe do hands. That's okay, we'll go front and then back. Yeah, go ahead. Do you use specific model. Do you use specific model for navigation like it's your own model or like when the browser is scrolled and so on and so on, or you utilize different. We are model agnostic. So browser brace is just the infrastructure for running headless browsers as well as the frameworks and tools. We have an MCP server, We have a framework called stagehand, which is like Playwright but better. You bring your own model and you may want to choose different models based on your conditions. If you are doing HIPAA and you need like zero data retention OpenAI, you might want to use one of their models. So you bring the model, we bring the infra. Hold on, I'm sorry, I'm just. Go ahead, say it again. You could just. We just skip it. Yeah, the example we just did, I used. Yeah. The cursor agent mode. I think it's 4.0 max or. Sorry, claw. It's probably 4O sonnet right now. That's why I baked in there. Go ahead. Love the questions. Keep them coming.

How do you guys manage anything that requires human in the loop? Because we have to deal with lot of legacy infrastructure, but it's financial data, so an advisor wants to review it before it goes into a financial planning tool, something like that. How do you bring in some sort of human in the loop interaction or to a browser A and B, if. Because this is financial data, we also want to give users a clear view of what the agent has done. Is there a way we can send that information even if they couldn't interfere in the process?

Yeah, I was just going to hop over the slide. So not only do we have recordings available and they can be turned on or off depending on data sensitivity, you can embed these recordings into a user facing application and show your user what happened. We also have this feature called the live view where you can embed an iframe and show exactly what's happening in the browser. And and better yet, if someone wants to, a human wants to come in and click and type in on the live view, they can do that as well. So it's not just for browser automation, it can be a browser copilot and it's an iframe. So it's a browser inside a browser which is kind of fun to see. Yeah, thank you. I think there's a question over here. Yeah, no, we'll go to you guys next. So my question is kind of two parts. One, have you dealt with captchas at all? And is there and do you see like future websites kind of using similar strategies to defend against like automation tools that it's only using can use their website? Yeah, captchas. So for the longest time there's never been like good bots on the Internet and CAPTCHAs were built to stop all bots, but now there are good bots and bad bots. And at browserbase we do offer captcha solving and proxies. It's something that's needed to browse the web. We have customers that use captcha solving against their own captchas because they can't get their SecOps team to like get through a good way to bypass that. However, I think longer term, you know, my friend Michael at workos was just talking about captchas. Agent authentication is the path to avoid captchas. Most captchas we see at browserbase are when someone's logging in. Once you log in, you know who the agent is, you know who they're acting on behalf of. And I'm really hopeful that solving captures at browsers is a short term solution as we figure out how to do agent authentication on the Internet longer term but down to talk about that afterwards, come find me. I got one minute, 42 seconds. I'm going to. I'm holding us to it. Yeah. So during web navigation like I think my question was related to captchas but during web navigation navigation, does the browser actually detect it's a robot that's doing the navigation and does it increase captcha coming up. Yeah, I think that the way that captcha detection works is often based on your behavior. And what we advise our customers is like listen, you know, even though we provide the best stealth browsing features necessary, in the end if you're a bad citizen of the Internet, you are going to get blocked. It's an inevitability. You can see this on LinkedIn. If you have an agent that's using LinkedIn, LinkedIn measures how many actions you take per minute. And if you're violating that, you're going to get stopped. So we advise our customers is like, you need to be a good citizen of the Internet. First, you need to try and obey robots, text. You need to be careful what you're doing, and if not, you're going to have a really hard time. And no matter what we do at browsers, we can't stop that. We can help with the simple things, but if you're doing something that's against the law, unethical, we don't really want that on our platform.