Ep. 26 - Rolling In The Deep (Research) with Daniel Jacobson | Devs:After Dark

Now, before this, what was the process that someone would need to take to— in order to accomplish something like this without the connector?

Good question. I don't know. I don't even know how it would have been possible.

Hello and welcome to another episode of Developers After Dark. I am your host, Nick, with my co-host Dennis. We have a repeat guest today, which is going to be fantastic because we have some updates on some really, really cool stuff that's happening around the developmental world. But before we get all to that, let me pay a couple of bills and just say, for those of you who are watching us for the first time, or those of you who have heard of us or watched other episodes, of course, you know, you go to devsafterdarkpod.com. That's going to be the place to watch not only this episode, but other episodes that we've done previously. 25 to be exact. But that's not the only place you can catch us. You can also go to the, um, the YouTube channel for the HubSpot Developers. That's @HubSpotDevelopers. Now don't just go there for us. There are, some would say a plethora, some would say a multitude, some would say a bunch of other HubSpot employees, developers from the community, all with tons of content, a wide range, just a wide range of topics. You'll see people that have been on the, you know, on this show, Brooke, you'll see Hannah, you'll see, of course, our guest who will— I'll let him introduce himself if you hadn't seen him the first time. But that's going to be the place to go for just about everything. You will learn new things, you will get updates, things, things that we've launched that we go over. It's just, there's so much there. And please, when you're there, like, subscribe, and just check out everything because it's one of those places you can go. And you can spend a lot of time there just learning a multitude of—

So, Nick, you sound like a YouTube influencer. Oh my God.

Smash that like button and subscribe, you know, send money to my Patreon.

But no.

And then of course too, if you really want to be in the know with everything that's happening in the development world, you're going to go to developers.hubspot.com/changelog. And twice for Dennis, that is developers.hubspot.com/changelog. Not only will you see what has been released as far as things happening in the development world, but upcoming things as well. So with that said, we have a really great episode, as I mentioned previously, because we have a return guest. I like to call him the liaison of everything AI that kind of happens in the world.

Mm-hmm.

I live actually vicariously through him when it comes to learning about AI. Um, but before we let him introduce himself, uh, I did get to earlier this week talk with Daniel Jacobson here at HubSpot, and, uh, we got to do a quick little what I like to call one-on-ones. We're messing around with a couple different recording formats here, but, um, it was really, really interesting because what we got to do was we got to talk about the CRM Deep Research connector, uh, with HubSpot and OpenAI. And it was really, really fantastic because Daniel, uh, got to go in and he got to explain what it was, explain why it was beneficial for users. We got to have a really good talk. So we're going to cut to that You're going to take a look at that and we'll catch you on the, uh, back end of it. Hello everyone out there, all of you developers after dark. We have a nice little special sit-down with a very special guest and we're going to talk about the HubSpot Deep Research Connector. Now our guest, and I'm gonna let him introduce himself, is Daniel. He is a HubSpotter. So Daniel, why don't you tell everyone who you are and what you do?

Sure.

Hi everyone, my name is Daniel. I am the Director of Product at HubSpot and I lead our Developer Product Group, the PM team more specifically, helping to empower all of the developers building on the HubSpot platform.

Excellent. Now this little sit-down is pretty special because we're going to talk about the Deep Research Connector from HubSpot. And knowing that this, you know, Developers After Dark is a little bit of a— it's a great name, but it's not Doesn't exactly catch everyone that does listen to our podcast and watches us on YouTube. There are not just developers, but people who are developer adjacent. We talk about like people in SEO and marketing and just HubSpot users in general. And all of them, of course, belong to that family of over 250,000 businesses that rely on HubSpot. So with this, why don't you explain to everybody what this connector does so that way they'll know as part of that.

Yeah, so we just recently released a connector into ChatGPT, more specifically, uh, for deep research use cases, and it basically gives the 250,000 businesses built on HubSpot, uh, deep research and analytical capabilities powered by a subset of their CRM data. So with this first release, we focused on companies, contacts, deals, and tickets. And we've proven out and ran a bunch of prompts and generated a ton of reports, and you can get some really interesting insights as a result of those queries.

Now, that must have been a pretty awesome collaboration to have to work on or to be able to work on because, like, you figure, I believe the figure was over 75% of HubSpot users already use ChatGPT. So I made the analogy, I think, in the last official episode we had that it was kind of like peanut butter and jelly, right? These 2 things will go together very nicely. But it's— and of course, that's a very simple way of explaining it. But you look at it from like a— and in some of the things that you'll hear is like, it's doctorate-level research, right? And you have your own customer data and you have those things and you're able, like you said, that there's just these deep insights. This is, I mean, it's Needless to say, this is pretty massive. I think one of the questions that would come up in my mind, you know, if I first saw this was, I mean, this, something like this must already exist, right?

Uh, no. So in this case, we're the first CRM to integrate with, you know, ChatGPT for any sort of deep research capabilities. Um, and it was really just, uh, you know, inventing it as we went in partnership with engineers from OpenAI. Uh, we kind of collaborated on the schema and the spec for how this could integrate with deep research to provide useful and meaningful results. gave feedback on the experience and made sure that it all worked end to end with a bunch of different tests and things like that. So yeah, this is, this is a first, especially in the CRM world. We're really excited about it. And, you know, the thing that I have been excited most about, or maybe delighted most about, is hearing stories, learning what prompts people are using, what surprises that they are experiencing as a result of having this new capability to do such deep research and analysis that previously was elusive or only reserved for huge corporations with massive resources to have teams to do some of these things. We're creating a lot more accessibility around all of this type of research.

Was there anything amongst that that surprised you the most? That one of those examples where you were like, oh, I can't believe someone's using this for this purpose. This is fantastic.

So we run some of our app partner programs on HubSpot, and so I've had several of my peers on the product team at, you know, different teams at HubSpot, basically trying to understand through tickets, uh, that come from our partner app partners, um, just like what's good and what's— what could be improved about their part of the product. And so we've had, you know, folks looking at how can we improve our UI extension capabilities for our developer platform, how can we, you know, make our marketplace experience better, all of those types of things. Um, and so that's kind of cool. It's like, hey, like as a PM You don't think of like, oh, PM use cases quite as much, but yeah, there's quite a lot of value in terms of understanding customer experience, customer pain points, and doing almost product discovery and customer discovery work through some of these things.

That's amazing. Is there— so let's say, for example, let's say I am a manager of a customer service team, right? Because we're talking about tickets and customer service. Would I be able to use this if I wanted to say, You know, maybe I sell XYZ widgets, right? And that's my company's business. And I run the customer service team. I'm the manager. And I kind of want to see what the— let's say I'm coming into my busy season, right? Let's say that's in the Midwest here. It's summertime. So it's getting warm and people are a little bit more active and maybe they're going to buy more XYZ widgets. Maybe that's the time when they do it. Is there a— would this be a way are using this, would this be a really great way for me to be able to, like, forecast my tickets or to get any type of deep research into the staffing needs that I might need?

Yeah, so you could run a query to evaluate seasonal trends based off of historical records in terms of ticket volume and types of tickets you might expect to experience, to do some forecasting on, you know, should you expect to see significantly more tickets filed? Does that also result in ticket resolution slowdowns? Like when you have an increased volume of tickets, does that mean responses are worse or slower? Resolution rate might go down, et cetera. You can basically generate a report that evaluates that from a historical trend point of view to help with forecasting and projection of do you need to scale up your support team? Do you need to hire seasonal staff, et cetera? Yeah, totally.

Excellent. Now, surely with something this powerful, I mean, it can only be You can— you have to have like a certain HubSpot tier, right? You can't— not everybody can't just have access to this.

Sure. Uh, in this case, it is not constrained to HubSpot tier. Uh, you do need a ChatGPT SKU that has deep research capabilities. Uh, depending on the version of ChatGPT you're using, you basically get an allotment of deep research queries you're able to run each month. Uh, but yeah, it'll work with any HubSpot tier.

Excellent, excellent.

If you don't even have HubSpot, we actually have the ability for you to create a portal in the process of trying to connect HubSpot into ChatGPT. So if you're trying to invoke the connector, you can go ahead and create a portal through that flow and just get started with it. Although you'll want data in your CRM to be able to analyze, so you'll need to put in some data first.

Of course. I mean, for— it seems like this does essentially solve one of probably the two, not the biggest, but the two things that like stand out the most as far as like problems when it comes to like, for example, You can have all the data in the world, but without being able to have some way to like analyze that data like intelligently, you end up with almost like data paralysis, right? And then on the flip side of things, you need really good data to be able to kind of pull out these sort of insights. So it really just goes to show like the amazing data you can have within HubSpot and the way you can actually access it and the way it all puts together. And then you have on top the work you've done with this collaboration with deep research to make those deep insights. I feel like there are just an unlimited amount of problems for users that this can solve.

Yes.

I mean, as long as it's constrained to, you know, contacts, companies, deals, and tickets use cases, yes, you can do quite a bit. Data fidelity and the quality of the data in your CRM plays a big role. One of the things that I like most about the experience is that it is connected back into HubSpot. So, you know, when generating reports, it embeds deep link citations to the records that it is querying from HubSpot. So for example, you know, say you're trying to evaluate or generate a sales report and get recommendations to improve sales win percentage. So like you want to close more deals and you want to win more deals. You can basically have it linked to lost deals directly into HubSpot with explanations as to like maybe why it was lost and things like that.

Um, yep.

And so that way you can like see like, hey, is that actually pulling in the right data? Well, it is almost always pulling in the right data, but you can see like back into the HubSpot CRM, you know, um, give me the entire history of that deal and give me the full record there and things like that.

Now before this, what was the process that someone would need to take to in order to accomplish something like this without the connector?

Good question. I don't know. I don't even know how it would have been possible. I suppose you could have built a custom app querying our CRM objects APIs and integrating it with some LLM deep research API behind the scenes, things like that. It would have been really tricky.

Exactly. I was setting you up there a little bit on that one.

Might be possible, but good luck.

Exactly right. No, I just got to say, like, that is an amazing amount of work that I imagine that this was This wasn't just like something that you just spun up overnight. This definitely took some time.

It was probably faster than you might have thought. This was like, you know, a really quick opportunity that we seized and we were able to do so by building on, you know, an infrastructure platform that we have at HubSpot that makes it easy to spin up new services and attribute, you know, RPC calls to basically become MCP tools, things like that. So, Yeah, the entire product development lifecycle phase here was around 4 weeks from ideation to having functional end-to-end product, and then a few more weeks on marketing, comms, reviews, all of that and getting it launched. So it was pretty quick.

That's amazing. That is absolutely amazing. Well, I thank you for doing that. I appreciate it because I guarantee you are solving quite a few problems for people and not just solving problems, but you're giving them the insight because that stuff It exists, right? It exists in the data. It's just a matter of trying to find the most adequate way to, one, find it, and then two, put it into a format in which you can create what I would just call actionable items. So this really does take— and I mean, and it's not— and the examples that— and the example that I gave, it was, you know, it was just essentially we're talking like customer service and customer service tickets, but there's plenty of other stuff. And we actually, I'll throw up on the, if you want to take a look, you can always go to, let me expand this so we can take a look at this here. There's not only, we have company news for it, but then there's also an actual landing page for the overall stuff of that. And we'll put that up on the screen in a couple minutes here. But basically, like the, not just the customer service tickets, but there are Plenty of other kind of examples. Like you have sales teams, you have marketing teams, you have customer success, like you had mentioned before, that can use this tool and be able to just extrapolate not just meaningful data, but like making sense of it with the deep research. Was there any— I guess the question that I have for you is, was there any problems that you ran into? while trying to do this that you kind of had to kind of stop and rethink?

I don't know if we encountered any problems throughout the development lifecycle, but there were some things that we really thought about at the forefront that we kind of knew were must-haves. The primary one being that we wanted to enforce user-level permissions on all requests so that only individuals who have access to read or access that data inside of their HubSpot CRM that's the only data that they can generate reports on within ChatGPT. Historically, many of the apps that are created are portal-level apps. Typically, they're like, you know, set up and installed by an admin and they're doing large-scale, you know, maybe data synchronization use cases, things like that. But with the rise of these generative AI tools, a lot more HubSpot usage is at the individual and the user level. And so that was something that we really focused quite a bit on upfront. Most of the work was actually on just making sure that our APIs were up to the task, respected individual user-level permissions, could be exposed as, you know, available APIs to tools invoked by ChatGPT, things like that.

No, that's great because that's going to be— that's a pretty huge part of it, I would say too, because it's just— I wouldn't call it open access, but you definitely want to make sure that the data that you're pulling is actually data that you're supposed to be seeing as opposed to not. So But yeah, and we do have— and now I'm going to put this up here for everybody too— the landing page. If you go to HubSpot, for those listening on audio, hubspot.com/ai-tools/openai-connector. So I just want to say thank you because this is— and I was kind of teeing you up a little bit for the— this has to have been done before. This is the first of its kind. And then I gotta say, it's gotta be really exciting to be working on stuff like this where you're kind of— it's groundbreaking, I would say, right? I mean, I think that's a fair assessment. But yeah, thank you for doing that and thank you for working on that and then coming on to be able to explain it. 'Cause again, this solves a lot of problems and this is just like one of those things where it's like, it just makes perfect sense.

Yeah, I mean, my path to this industry has been originally gadget obsession. And so I'm just like, developer platforms, developer infrastructure is basically always at the forefront and the bleeding edge of what's possible with technology and with devices. And now with this whole AI revolution, it's now the forefront of what's possible with just software in addition to devices. And so getting to work on things like this is really cool and really fun from a personal satisfaction point of view.

Excellent. Well, thank you so much for spending time and doing this little sit-down with me, Daniel. I appreciate it. And, um, yeah, for those listening, uh, we're probably going to jump back and you'll probably be listening to, uh, John talk about MCP Server. So, um, again, I wanted to thank you for sitting down and talking and hopefully, um, we would love to have you on the show in a future episode as well. Because, uh, yeah, I would love to. It's a lot to talk about for sure.

So.

Sounds great.

Excellent. Thanks for coming on. I appreciate it.

Yeah, you bet.

All right. We are back. That was me talking with Daniel Jacobson and for everyone that was, he is the director of product here at HubSpot, and he got to work directly with OpenAI. So it was like one of those things where we had to grab him, we had to talk to him, because, um, I mean, just for me, I thought that was like probably one of the coolest things and one of the coolest collaborations, right? And it was pretty instrumental to be able to have him on and just talk about it and just tell us what it was like to be able to do that. So—

He's a smart freaking cookie.

Oh yeah, and big thanks to him for taking the time to do that too, because keep in mind This is all going on right now, and he decided to take you know a little bit of time to talk with me. And I was just like, "Who am I?" Like I just reached out him out of the blue, and it's like, "Who's this guy? This mohawk guy calling me on to talk about teeth research, right?" It's like, come on. But no, we really really appreciate it because like that was that was that was really really huge. But not the only thing that's really really really exciting, really big, really huge. Our guest, of course, repeat guest. I'm not gonna say we had to bribe him to get back on here because we love him to death. But it is fantastic that he's joining us again. So let me let him introduce himself. John, take it away.

Yeah, longtime listener, second time caller.

Just saying.

I'm John McLaren. I'm a staff developer advocate here at HubSpot. And yeah, I worked with Daniel on launching this connector and getting our landing page and some of our resources out about it. And it's been a very exciting and fast-paced set of projects that are all interrelated. Yeah.

Is it really not— okay, I'll use the word nerve-wracking, but maybe that's not the most apt description. But having to work on something like this in kind of an atmosphere where things are constantly changing in this category, Like, how was the pressure when it came to that?

Ooh, that is a really good question. I would, I would say the pressure to get something out rather than get it perfect was, was much higher. So, uh, what we know is basically LLMs are capable of kind of handling things. If we need to change, say, how the MCP server under the hood works, So related is we have our local MCP server that y'all can download and use, and we recommend using it through npx. So you're always using the latest version, and we have guidance on how to actually do that on our landing page, developers.hubspot.com/MCP. Repeat that one more time.

Mm-hmm.

Radio/video, developers.hubspot.com/MCP. And so under the hood, the OpenAI connector is using Model Context Protocol. And so the cool part is, is LLMs kind of make this a little easier to like get something out there and then iterate on it and not worry as much about, say, like the integration layer with the other companies. Work because in this case it's, it's OpenAI's LLMs going, okay, I understand how this MCP server works. I'm going to connect to it, get, use specific tools with HubSpot, and then I will, I will plan out all the steps I need to take to do whatever research that, that user wanted and get them back a useful result. So if we change what those tools are under the hood, it's not really a big deal as long as the functionality is still there.

Mm-hmm.

So the tools could be different later, they could look different, but the actual functionality, as long as we maintain that, the LLMs can just kind of reason about it, which makes getting, getting it done, getting it out there, Uh, more important than having everything perfect because we can still perfect things after it's launched. So that's the theory.

How long, how long have you been working on like this, like kind of abstraction layer? Like, I know, I know Daniel said something like took 4 to 6 weeks, right, to get this, uh, integration done with them. How long have we been actually working on this? Because that just does not seem— I mean, to do this right, it seems like a very, very short window.

Yeah, so this was the fastest product, like, launch that I think I've ever been a part of at HubSpot. So week 1 of that 4 weeks or so was literally us planning out, like, should we do this? Is this the right approach? How do we approach, like, actually launching this? How do we move fast on this so that we can get it out there, get it in people's hands right away? Uh, and then week 2 was the actual week that we started actual development. The bulk of the work on development happened in week 2.

Okay.

We were already at an executable MCP server at week 2. Uh, and week 3 was polish and testing and, uh, some internal, uh, like, uh, kind of like proof of concept-y stuff. And week 4 was all about how do we like get this in people's hands the most effective, efficient, educational way. So even things that normally would've taken a lot more time, like our landing page, we streamlined every piece of that whole process to get things into people's hands as fast as possible. The video that is on there, Thank goodness for Dennis here because I recorded that basically same day as Dennis edited it. So like the, the, that's how insane things had to go.

So rumor is Dennis is pretty good with video. I just, I, I heard that. I don't, I mean, I just, yeah, yeah, yeah.

It's one of those things. Never expected that.

I mean, yeah. You guys don't hear it. I'm constantly swearing and he's just editing it all out. It's true. It's just true.

It's just him going, I gotta cut this again. I imagine it was probably, um, you said there was that, that, that week or so that you were, you know, there was like not, not a debate, but it was like, you know, is this the right move to make? I imagine that the fact that probably, what is it, around over 75% of HubSpot users are pretty much already using, uh, ChatGPT. That had to have— I mean, that had to have been a pretty big, like, when you're, when you're putting the things on the scales of, yeah, that's got to be one of those ones that's like, Yeah, I feel like this is going to be one.

Yeah, so one of the influencing factors was like, okay, so yes, a lot of people are using ChatGPT, a lot of people are using Gemini, they're using all of these different large language model services. But one of the things that motivated us is we've seen with other companies, like community-created MCP servers spinning up, Some did spin up for HubSpot, but we realized we were like, okay, well, there's a bit of a security issue. Like, we, we want to make sure that when people are connecting to HubSpot, that they are connecting directly to HubSpot so that their authentication information goes to HubSpot.

Mm-hmm.

So that was a motivating factor. Obviously also the utility factor of like, hey, what can we do by just allowing an LLM to like take actions inside of HubSpot account or read the data. So the OpenAI connector is a read-only solution right now. The MCP server, however, is read and write into a few more types of objects than the OpenAI connector. And so what's really cool is like we popped that thing into Cursor, got it all set up, and we're like, what would a developer want to do if they could just ask about the data in, in a HubSpot account? And we were like, well, one of the things that I've done is I want to build like a serverless function and I need to know what the structure of that data is. So what better way to figure out the structure of that data than to just go, hey, give me the structure of the data for a contact record or for whatever custom object I'm trying to work with. I can just literally copy and paste that dummy data or placeholder data to work with as I'm developing. It just unlocked a lot of different cool doors and a lot of them we haven't even seen yet. I know there are some creative things that our developers are going to do with this, and there's a lot of creative things that people are already doing with the OpenAI connector, things that I wasn't expecting, like people trying to create different reports for how they're handling prospects or how they're like looking at their active deals, like These are things that, yeah, in retrospect it's totally obvious, but some, some of these, some of the stuff with the MCP server stuff, it's not like we knew what people were necessarily going to do. It's a little bit like how our APIs are. Yeah. So yeah, right. Like with HubSpot's APIs, we don't necessarily have all of the use cases figured out yet. We release anything, we, we might have some vague ideas of how you might use something. But the coolest part to me, and this is, this is why I love this, this type of work, is, and I just love working with developers, is because y'all figure out weird ways to do things that we didn't expect or design into HubSpot. So like, I remember doing this myself before I joined HubSpot, like pushing HubSpot beyond its limits, and that's the coolest thing to me. So, right.

You think that for, um, something like this connector and OpenAI, there'd be a— I know it doesn't exist yet, at least I don't think it does, but like an anonymized way of seeing how people are using the HubSpot connector, like in conjunction with other connectors, for example, like visualizing, uh, you know, uh, data points like with some other— I have no idea what, you know, Figma or something like that. There's some sort of connector that can, you know, put it in there for you.

Yeah.

It'd be really cool if, like, as I'm researching what I should build as an app, you know, to see an anonymous, you know, this is the trends, how it's being used. Do you think that's something that could be done or would be done?

Um, I think that's an interesting question. I— so I don't know right now if or what exactly we're tracking. What I can say is we do know what APIs people are using, so we can tell what Tools get used. Uh, but that doesn't mean that we are seeing like the LLMs side of it.

No, I don't think we can do that, right? That'd be something that OpenAI would have to provide. I'm just— I, I know you don't have— I'm, I'm quite aware that you do not work for them and you don't have an answer to this, but do you— I just feel like at some point in time, marketplaces and all the good stuff, there's got to be some sort of model for this, right?

Yeah, so what is interesting is in the model context protocol, there is a property that the, that the LLM can provide to an MCP server that basically allows that MCP server to get sort of like feedback, like, like give it some kind of context to what the user was trying to do. So It could end up just like passing the literal prompt. I don't know that OpenAI is doing that. I did not check. I do not— Yeah, I know, I know.

Throw the statue blind. I know that.

All good. But, but so there is some possibility for us to get that data. But yes, you're right. It is definitely on OpenAI right now if they want to publish that data of how people are using it on the LLM side. Um, yeah, yeah, related to that, uh, they cannot train on, on company data, uh, that is using the MCP server.

Do you know— and Nick, I apologize again for taking over and asking all these dang questions, but that's who I am. Um, do you, do you know, um, when or if we will make it writable?

Uh, I don't know on the OpenAI side. I know that it is something that we are thinking about. I don't have a timeline. Right now, the priority was making sure it worked great with their deep research functionality. The MCP server, however, that already supports Write.

Okay.

As we have the local version and coming in the future, we do have a remote version That remote version is how our OpenAI connector is working. Um, and let's back up and say what that—

what does that mean?

Yeah, yeah, great question. So the way that the local MCP server works is basically every time you want to, uh, you like have your AI tool make requests to HubSpot through what are called MCP tools, it means that ahead of that interaction, you need to be running on your machine a local web server basically that goes, hey, I understand what the— this LLM is asking me to do. I have this specific MCP format that it is providing me some information in. Then it goes, all right, I'm going to use the relevant tools script code to basically hit HubSpot APIs. And then I'm going to take that data from HubSpot APIs, bring it back, give it to the LLM so that the LLM can do something. So that's all running locally on your machine. So when you run npx, that server is hosted on your machine temporarily. And then if you If you just used npm install, the same, same deal. The remote version is a whole different game. Instead of it being on your machine, it is hosted in HubSpot's infrastructure.

Got it.

So the positive is there's a lot lower barrier to entry, so you don't have to be super technical. You don't need to be a developer. You don't need to have Node installed. which is one of the dependencies of the local MCP server. So huge benefit. Another benefit that I'm excited about is this concept of user-level OAuth. So one of the things that we wanted for the remote server, which is more for the general user, so the developer-focused local version right now, the way that you interact with it is you need to create a private app. That feels like it's a little tricky for a non-developer to figure out. You gotta deal with secrets and—

Mm-hmm.

And you have to edit a JSON file. That's, that's not something we want regular users to have to deal with. But with the remote server, we can actually change that. So instead, what we can do is we can make it feel like installing an app, that you'd find on the HubSpot Marketplace in a way. There's the permissions screen where you go, it'll go, do you accept giving this AI tool these following permissions? And then as soon as you hit yes, install, then cool, it works out of the box. And when I said user level, so it will be scoped to the user's permissions. It will not have access So say even if there is an MCP tool for viewing contact records, if the user doesn't have contact records access, then the AI agent does not magically have that. In the local MCP server version, because it's based on private apps, it's based on what the scopes for that private app are.

Okay.

This is just a way better way, a little bit more security. And so we're real excited about it.

What if you don't want one of your— I'm an admin on an account and I don't want a user to have access to this. Is there a way to negate that?

So as far as the OpenAI connector, the OpenAI connector requires an admin, a super admin to basically be the first person to install. After that, then anyone on the account can use it. But before a super admin has done it, you just can't use it.

I see.

So, and as far as the remote MCP server, we will probably follow a similar paradigm to that. But I will admit that's still being figured out right now.

How's it work? Let's say, man, Nick, stop me, dude. I just go something.

I like your questions, honestly.

How's it work if like, um, I am a user and my, my permissions has changed? How quickly are those refreshed, uh, through the tokens to ensure that I am not like, I, I've transitioned from sales to service or something like that. When, how quickly will that be applicable to, uh, the new That's a great question.

That would be based on the refresh token for the OAuth connection.

It's a standard OAuth flow then? Okay, I got you. That's pretty quick.

That would be excellent. I was wondering about that too. Actually, those are good questions because one of the one things you don't want to do is to be able to have, based on your user permissions, to be able to ask a question and then have all of this data dumped on you that you're like, I'm not sure I should be looking at this. This is okay. Interesting. You know what I mean? Like, you've always— people, I'm sure, in the development world have probably come across a situation like that once or twice where they're like, ooh, I shouldn't be looking. This is interesting, but I shouldn't be seeing this.

So, you know, that's— I actually want to chime off that. So an example of like data that you'd be like, ooh, I don't think I should be looking at that is So if you have sensitive data, so if you have properties in HubSpot that are marked as sensitive data, then those properties are not visible through the MCP server or through the OpenAI connector.

How are they represented? Are they just like X's?

They just don't get it. The properties just are not visible.

It's like this redacted sheet where everything's just like this criminal data and black market, right?

Yeah. Yeah, my childhood.

It's— ah, sorry, sorry. I didn't know this was a counseling session.

What about, John, what about, um, uh, regionality? Like, um, I know we have hublets in, you know, the EU, elsewhere. Um, how does, uh, OpenAI honor that, or does it, or is that important?

That was a great question. So It's a little bit complicated because there's a lot of regions involved, but the landing page hubspot.com/ai-tools/openai-connector has the latest information. So in case what I say ends up outdated, look there. But the European Economic Area, a word I just learned, abbreviation I just learned, Switzerland and United Kingdom customers will need a ChatGPT Team, Enterprise, or Education plan, and all other regions can use any paid ChatGPT plan— Enterprise, Team, Pro, Plus, or Education— to access the connector. And as far as— let's see— automatically be available to Okay, so the DeepBrain Search connector is also available to all HubSpot customers across all tiers who have a paid ChatGPT.

As long as you have a paid one.

Yeah, right, right.

Well, that's fantastic. You know, one of the things that, um, really struck me as fantastic, because I, I, again, I'm one of those people too where it's like being able to see what everybody's creating with this, whether that's reports or how someone's using it, that this isn't one of those things that's kind of In one particular area, right? There's, I guess you could say there's many, many like go-to-market teams that can use this, use the connector, use the deep research. So it's not just one of those things where it's like, hey, yeah, if you're in support, this is going to be great for you. It's everything, right? It's, uh, save me if I, correct me if I'm wrong, but like you've got marketers, sales, uh, support, um, anything that can basically contact deals.

Right.

Company, like utilizing those, like there is just an infinite amount of possibilities here as far as solutions for not just the developer, because again, like the everyday user as well, because I mean, a lot of this is, I don't want to say it's specifically geared towards users, but like, don't get me wrong, developers also have accounts and developers are also service providers. And these are things that they're telling their customers as well. But it really does kind of provide this sort of blanket, maybe this is the wrong way to put it, but I look at it as like this blanket solution that covers so many different teams within one organization that, yeah, being able to see how each individual one comes up with a way to use this and a way to solve for whatever X would be in this case is just something that I'm, it's really exciting to see.

So no, Nick, when you, when you were talking to Daniel and you said, how did you do this before? And he said, you didn't. You know, like, I think that might've been like the most powerful things. Like, you know, I guess you could write an app possibly that would be able to go through all this data, but good luck and good luck making it not focused onto the most specific niche, you know, possible, like to be able to generalize this and actually still have the depths. I don't know, man. We live in a really cool, amazing, weird world that's moving so fast.

And not just at the speed it was done too, but then keep in mind, like, it was the first of its kind because that was like— I was being a little facetious with my question, but it was like, yeah, you know, I'm sure, I'm sure there's a million others out there just like this. It's like, no, there's— it's like, this is it. This is the first one in that record time with that much consideration put into it at once. It's like, I don't know, you just think back 5 years ago, maybe 6, and this was— even that process was just unheard of, right? It didn't have to be specifically AI, but just that process in general with that speed is like unheard of.

John, I know that this OpenAI thing was quite the catalyst to this, you know, first, like, we'll call this the MVP, I guess, right? Like the first version. Um, I can't imagine iterating as quickly again for V2. I mean, is that—

is—

do you think we're gonna keep this clip going? I mean, it seems impossible.

I, I, I don't want to spoil anything, but I would say that—

Spoil, please spoil it all, man.

We, we have not slowed down at all. If anything, uh, we've been energized by the response that people have had. to both the MCP server and the OpenAI connector. And we are emboldened to keep moving quickly and change how we work. Even just in general, the amount of AI stuff that our teams have to use in order to keep up with everything is insane. We're all having to work way more cleverly I joke that, well, my coworkers within the developer product group have joked recently, and they're like, I'm pretty sure that John has stunt doubles or is part AI at this point because I've had to get extremely clever because this is only one. This is only like one.

Right.

Some of the big projects that we have coming. So I'm really excited for people to see the rest of what we have coming. And some of it obviously is related to— it will be coming around inbound time, but we have longer-term plans that are really exciting. And the funniest part to me is I say longer-term, but every time we decide something, it's like, okay, here's the short-term version of what we're going to do. Here's the long-term version. Long-term version ends up being like 6 months shorter than we expected because everything just started moving way faster.

I imagine you discussed short-term in a matter of seconds and long-term in a matter of minutes, right?

As opposed to—

and correct me if I'm wrong, what you're saying is we could potentially right now just be talking to the AI bot version of John. You could be doing probably 3 other shows Simultaneously. Is that what I'm getting right? Yeah.

Well, uh, you know, you weren't supposed to find that out. That is— Wasn't it?

It wasn't in Total Recall where it was like under the eyelid, right? There's like a mark to show, or isn't that what it was? I can't remember what movie that was, but yeah, we're gonna have to start asking every time we see you in person now. Can you just lower your left eye just a little bit?

Real John has been on sabbatical for a long time.

He's on a beach somewhere just, you know, sipping a Mai Tai.

He's not even here.

Um, no, it's like, just understand why, why else has my background not changed?

And right. That's fair.

Hey, Nick, Nick, when, when you were having the interview, you, you came, you said data paralysis. Is that something that you came up with or is that like, uh, something you heard elsewhere? Because I think it's such like a, it was one of the, when I was rewatching it to prepare for this episode, because I do that, um, I wrote that down because I thought that was a really, really, really important way of putting it.

You know, I will claim that, sure. But I guarantee you, probably not, because I've always, I've always heard it in the context of choice paralysis. So it's like, and I've always, I've always experienced it. I don't think I've ever called it that, but I always experienced it like from an SEO standpoint when it was collecting metrics, like in the real estate world when you just— because I remember, you know, like if you think back like 10 years ago, maybe even at that time, you know, people were collecting data because you had to. Now you did have to, but nobody was saying, well, what do we do with all this data now that we have it, right? Whether that's, you know, even if it was something simple like Google Analytics metrics that you were collecting, you still had to do something. That's why I think like with deep research and everything like this and this connector, it becomes so much more important because you go back to those days and you just remember collecting mounds and mounds of data and you were trying to sift through it as a human being or even a team. And you're sitting there trying to say like, okay, well, how do I make sense out of all of this? this all looks very pretty when you put it on a line graph. You look really, really intelligent, right? And everyone around you goes, ooh, ah. But at the end of the day, it's like, where's the actionable items, right? You'll have a CEO or a CTO looking at you like, great, now that I spent all this money on all these tools and gathering all these metrics, what do I do with this and what's the next move? And if you're just kind of standing there just still showing bar graphs, like smiling, you look really, really stupid. So it was just, but yeah, it kind of came from that because I just remember looking at all that data in the past and being like, it'd be great if I could make heads or tails out of all of this and then multiple tools too, as well. I mean, that's, again, that's no trade secret. The big thing about HubSpot that I love the most is you don't have to go to 50 different tools to do 50 different things. It's all pretty much right there. But when you start adding these other layers of communication on top of each other, like this connector and other things, Yeah, that was always the big fear was that kind of paralysis, um, that you would just run into. So, but things like this solve that like beautifully, you know? Yeah. Yeah. That's where I came up with that. So give me credit for it if you want.

I mean, I just did.

I was, that was for you, buddy. I was, that was like my, my, my Father's Day treat to you to give you a little, like, you know, you came up with something you'll be known for years to come.

Yeah.

I could charge, I can charge money if people want to use it. That's fine. Put it on a t-shirt or something.

But yeah.

Um, it's good.

You know, I got to make a living somehow, you know, this, This ain't paying the bills. I'm just saying. But no, you know, John, I was gonna ask you, is there like going along the lines of like stuff like data paralysis and like problems you run into when you were doing this and you were kind of helping create all of this and, you know, working with, you know, multiple teams, not just within HubSpot, but outside of it, were there any problems you ran into that were kind of, I don't wanna say instrumental because problems aren't really instrumental, but there are problems that you run into that are more They kind of make you stop and like, you don't even have to rethink, but they kind of really change the way, you know, you were kind of moving in order to make sure it's done right. Did you run into any of those kind of— because there's like really big ones that I can, I don't know, I can think of with other things that not just HubSpot but other people have done in the past with big projects or big collaborations where it's like, hey, we had to stop on a dime here because we wanted to make sure, you know, this came up, but we wanted to make sure that we prepared for everything. Did you have any kind of moments? And again, it was a very short period of time. And so, I mean, but was there anything like that that kind of happened where you did stop and say, okay, that's something definitely to consider. Let's make sure we don't— that doesn't get lost in the weeds.

Yeah. So there's been a few different things. So as I mentioned, I'm working on a ton of different projects and many of them you'll be learning about soon, but a lot of them are interrelated to each other. And one thing that we did realize with the MCP server in testing, and this is not specific to any of the LLMs that we've advertised as being great ones to try it with, this is just general. Something that we, we had to change our mindset a little bit with was the way that LLMs work best is when they have the granular control to do whatever you want. And it like, so for example, we, we might be thinking along the lines of like a specific use case.

Mm-hmm.

Uh, normally that is how, that is how software usually works is you go, okay, This is the job to be done that needs to be solved for. And then you work back from that going, okay, I need to make sure that there's like an API, there's the UI that fits this. With the LLM world, it doesn't work that way. You don't think in that way. You think the other way around. You go, what are the tiny little pieces that are needed to solve for a job to be done, but is not the job to be done? Like, it is not The solution to the job we've done. So an example is like, all right, if we want the LOM to be able to grab a ton of data from HubSpot, then we obviously need it to be able to grab like object records. We need it to be able to grab associations, but we can't be prescriptive in how we create the MCP tools. We need to give it as much flexibility as we can. And that's a weird way of thinking. It fits more how we think of like giving developers APIs, but it doesn't fit the— like even with the APIs, we often think about like, okay, here's some use cases that we build around. Instead, it's basically the user can enter any weird request they want. Like I want only contacts that are named McLaren that were born in this date range. And are associated to this weird deal. Like, they can ask for that, and we need to give the LLM the tools that it needs in order to be able to answer that question efficiently.

Do we, do we apply the same, same restrictions, like, you know, like API limits, for example? Is that, is that something that we are actively changing specifically for LLMs?

So we're definitely thinking around how we handle our rate limits in this new world. But right now the MCP server is completely using the, the normal rate limits that apply to if you were to create a private app, because it is literally using a private app. The OpenAI connector is a little special. That's a direct partnership between HubSpot and OpenAI.

Mm-hmm.

But yes, under the hood there are, there are still rate limits, and so that also impacts how we think about things. So what you might notice if you were to like dive into the code, um, the tools for getting objects and creating and updating objects, we're only using the batch endpoints even if you only have a state record. Yeah, that's true. And that just helps ensure that it doesn't need to make 50,000 requests.

Yeah.

But like, if you, I mean, as you just said, I want the McLarens who are associated to X, Y, and Z. I mean, now we're starting to talk about some calls, right? And if there are a million, 2 million records that you have to, you know, parse through, would I, would I actually see, uh, an error in ChatGPT if, you know, I hit that?

Yeah. So, uh, if you were to hit an error, you would end up seeing a, a, basically you would see the, in all of the different tools, they kind of display it a little bit differently, but, but you'll see that it tried to use the tool.

Mm-hmm.

And then usually the LLM will go, it will just directly respond to you and be like, I didn't get something. I'm going to try something else to get that data. Or it'll be like, I hit the rate limit, so I'm going to wait. Yeah, yeah, yeah.

Gotcha.

And then I will retry. What if it's, what if it's not solvable in that sense?

What if they have to actually reconstruct? I guess you just said the first time. So like if you hit the rate limit the first time, it might try a different method of getting it back.

Yeah, it might try it multiple times, to be honest.

Can it catch like a, you know, a set and hold it in queue while it's trying to get the rest of it? Or does it have to be all in one shot?

Uh, so this will depend on the different AI tool, but, uh, largely what'll happen is if there are, for example, if you needed to do 2 batch requests, I'm throwing out a number. I forgot the exact number that it's actually here, but say we had a limit of 100 contact records. Yeah.

Yeah.

I think it's a lot higher than that. I'm just throwing that one out there.

I think it's not much higher actually, I don't think, but yeah.

Cool. All right, so say it was that, um, then you will need 200 records. Say that was what you were requesting was 200 records. So, uh, it no matter what is gonna have to make 2.

That is a good point. Yeah, yeah.

And so what they do is they will store that in the context for the LLM.

Gotcha.

That gets passed to that next LLM, uh, iteration.

Oh wow, wait, wait, wait, wait, wait, wait, iteration within that So if I, if I kept a conver— man, I'm sorry, Nick, again. If I kept that conversation, I came back to it 2 weeks later, would it still use what's in that context?

Yeah, actually it should, unless you directly gave it, uh, some like, hey, only use fresh information. Yeah. And it caught that the date changed, then it would probably make a new request. But otherwise, yeah, it'll use what It had stored in its context.

I think we should add to it, by the way, just post in the developers forum, you know, if there are issues. Isn't that the standard? Just click this button, post in the forums, we'll get to you someday.

Well, so actually that's a great, great idea, uh, and we sort of already implemented it in a different way.

Really?

Yeah. Uh, so if you're using the local MCP server, and I think our plan is to do this as well for the remote MCP server, I don't believe that it's implemented in the OpenAI one, but I could be wrong. So if you say that, if you tell the LLM, hey, I want to get some feedback on HubSpot's MCP server, then what it will do is it will actually give you a feedback form to provide direct feedback to HubSpot. And it's not— the feedback is not passed through the LLM. It is— it literally gives you a link that way. You can trust that the LLM is not like summarizing what you said in an incorrect way or anything. It is verbatim what you, your opinion is, uh, so that we can improve upon it.

Cool.

Cool.

Nick, shutting up. I know I said if we don't watch out, I'll, we'll make this go an hour and a half long. So I'm gonna shut up and let you talk now and I will be quiet.

No, no, we did. We did. But, but no, that was a, that was a really good question though, cuz I, I was actually curious about that myself cuz I would imagine it would treat it like a session and keep And unless you opened up a new one or unless you told it to specifically disregard X, Y, and Z or start fresh every time, like, yeah, no, that's a very— I, I would wonder how long it would kind of keep it in, like, not in the buffer, but like, you know, keep it in the queue, if you will, just to kind of make sure. But, uh, no, no, that, that makes perfect sense. So Dennis always feels like he's talking over me, but he's not. I'm like, literally, like, some of this stuff, like, I'm still learning on the fly as well. Like, as you know, I've, you know, it Full transparency, like I'm ecosystem quality, so I'm not in the developer advocate, so I'm not as close to some of this stuff like on a daily basis. But, um, you know, sometimes selfishly, selfishly, sometimes I get to literally learn on the job while we're doing this. And it's like, so some of these things, it's like they're really good questions and, you know, it helps at least me understand stuff better as well because yeah, like I said, with things constantly changing like this, having somebody and, you know, again, Dennis will say I'm kissing up, but Having someone like John on the team in the department, like that's, again, I wasn't even kidding, living vicariously through you via AI is like, that's true. It's absolutely true because there's some stuff where I'm like, I'll hear about it or I'll listen to it and I'll be like, yeah, I'm sure John's already caught it. I'll ask him later what this is because, you know, I'll read an article or something or something and I'll just be like, yeah, I'm pretty sure I could read this whole article, but I'm pretty sure probably John knew about it like 18 minutes ago when it first, you know, so. But no, it does. The questions make perfect sense, Dennis, so don't ever think you're talking over me. Besides, it gives me a chance to not talk, and that's probably good for everybody.

So shut it down.

Don't worry about that there. But I know we're coming up on time a little bit, and I did want to ask, um, uh, to you, John, specifically, is there, um, with everything we've talked about thus far, and then like knowing that things can change, you know, in an instant, you know, 5 minutes after we hit end, um, Are there things right now that you could speak to as far as like, because I know we had asked, I think we had asked Vikram from Prompt Armor with regards to OpenAI as well. Like, what are people, what are like, what is the one thing or one or two things people right now are missing as far as, because again, there's a lot of information out there, right? There's a lot of iterations on things and with things constantly changing. Are there things out there right now that just Um, people may have an idea of, but they're— especially with regards to like using an MCP server or, you know, like how it interacts with everything. Are there things that people should know right now that they might not, you know, that are just kind of really important?

Yeah. Oh, that is a great question. So I touched on this a little bit earlier, but, um, what I would say is, so the entire idea of Model Context Protocol is kind of like a nebulous, too sciency term for most people and doesn't mean much. And the actual MCP tools bit and versus like the MCP client versus MCP server versus host, like all these words. The thing that I hope that most people understand is that the MCP is the connection between your AI tool and the SaaS product that you want to work with, or it could even be tools that run on your computer could use MCP. But what I do want people to definitely know is like, hey, yes, the open source community is freaking amazing and has like provided these open source MCP servers for a lot of different cool tools. I just I wanna encourage a little bit of caution. Like, remember, if you are connecting your HubSpot account, you are connecting your HubSpot data to whatever tool you're connecting and that middleman that is that server. So you need to make sure you trust that server and you trust that AI tool. Otherwise, yeah, that's like, it's, you're just adding more more companies, organizations, and individuals into that web of trust that you have for your security to your company.

So you're saying, John, it's totally fine to connect to Dennis's Not a Spam Bot NCP server? That's, that's a, that's—

I Swear It's Me Dennis is the name of it.

Yeah, exactly. Totally trustworthy Dennis over here. Totally not going to change your password to password exclamation point.

It's amazing. Well, and final question, which I— for both of you, which I think is the most important: when you guys talk way too fast, do you also say ChatGTP? I, I do. I have done it.

Acronyms. Like, I thought us web developers had too many acronyms before. Again, I also learned, like, so my wife is a teacher. They have a million acronyms and they even created an acronym to make fun of the fact that they have too many acronyms. They call them TLAs, 3-letter acronyms.

Man.

And so, yeah, I think in this new AI world where we are inventing A to A MCP, GPTs, like there's just too many, too many acronyms and we're all gonna mispronounce them like GIF or GIF and we're just gonna have to deal with it.

I try to sound so smart, but I butcher it every time. Every time I do. I know I do. I caught myself doing it.

Yeah. We're just gonna have to get AI to retroactively fix our pronunciation in videos.

I think that's We call that Dennis right now, but like, yeah, it's, um, well, I wanted to say thank you, uh, John, for coming back and talking to us again. We didn't scare you away, which is fantastic. I mean, I, you know, always, um, but yeah, I mean, I think it's really, really important for people to be able to, um, because again, not, not everybody is as in-depth or in the weeds like our teams are, and at least people at HubSpot are. So they're They're not that they're getting this information secondhand, but it's not coming as free-flowing as it does if you were sitting within the walls of HubSpot. So that, that makes things a little bit more difficult. And I do really appreciate, again, like Daniel and yourself and others who worked on this, because I mean, there were multiple teams, you know, across the whole organization that made this possible. So without them, it's, you know, we're kind of parroting a lot of what you're saying and what a lot of Daniel and everyone else is saying too. But without that work, it's like, yeah, it really is fantastic because it's a nice way for people to be able to stay on top of things. Because you can get in, I'll be honest, you can get newsletters, you can get other stuff and you get to, but again, it's like, it's like reading an instruction manual on a thing you've never used before. Yeah. It's going to take some time.

So. Reading, reading the documentation on MCP alone, I was like, oh yeah.

Right.

So I get it. It is. You have to start getting like the, the key is you have to get clever about how you digest the information.

Uh, the key is have an abstraction layer like John McLaren on your side.

Yes, it is that.

Well, the Clone ID is great then because this is really going to help. This is, this really solves the problem. Everybody can learn MCP and everybody can learn everything they need to know as long as they have a, oh, basically an AI John next to them. It's, it's perfect. Yep, absolutely.

Yeah. Did I mention I'm going on sabbatical? Good luck. You're gonna have to figure it out.

The clones came just in time.

Yeah, right.

Well, again, we want to thank you for coming on. We really do appreciate it. We love these conversations. And I will leave you with this little— Dennis loves these because, you know, they're very insightful. I kind of look at them as like the Jack Handys of the SNLs, like back in the— what was it, like 10 years ago or so? Yes, I love it. But, uh, but no, uh, you know, ChatGPT does support about 60 languages so far, but the one language it'll never master is the language of love.

Oh, oh, bye.

Like that? Did you like that?

Bye.