Data Quality @ the HubSpot Admin HUG 🐙

Um, so you are here at the, the HubSpot Admins Hug. It's a user group for HubSpot admins. We meet here every Tuesday at this time, and typically, like today, we have, uh, lovely folks from HubSpot's own product team, uh, giving you a, a deep look at, at one particular part of the product. And today we are talking about Data quality, the data quality tools that exist mostly in DataHub, but some of them have wandered a little bit outside of the confines of DataHub. Next week we'll be talking about campaigns and then we, we have events scheduled and we'll have more coming. So keep an eye on this page, bookmark it, love it. This is your source of truth for what we'll be talking about every Tuesday from now till there are no more Tuesdays. And we have a YouTube page. Today's session will be recorded. All of our sessions get recorded and end up here. So if you ever miss a session or want to rewatch one or share it with a friend, YouTube is the place to do that. We are getting very close to scheduling a show and tell. We do these every so often. You all are deep in the HubSpot product all the time. You're solving problems, building workarounds, constructing elegant solutions to things. Any time you do something that you're proud of or where you learn a lesson, either a good way or a hard way, please fill out this form and let us know. We would love for you to share that with the wider group so we can all learn from each other. We are hoping to get one of these scheduled in May. We could use one or two more presenters, so please fill out this form, submit it. We will reach out to you and coordinate a time. We have a space on HubSpot Community, community.hubspot.com. Check that out if you like forums. Deanna and I are both really super active on LinkedIn. There is a a private group here for, uh, the, the Admin Hug itself. You can join and share thoughts there. There's also a handle, uh, you, you can follow. You can connect with me and Deanna individually. Uh, we would love to, to know you better, to talk with you, and to hear your ideas for how we can make the hug better. Uh, now we get to the, the tab where I have a little bit of insecurity and wish I had done more preparation. HubSpot Academy runs boot camps. These are 6-week-long cohort-based You have a weekly webinar and you have take-home work on a variety of topics. They are mostly closed right now, but under the Super Admin Bootcamp and the RevOps Bootcamp, which are the two that are most relevant to you, there is a bootcamp sampler series you can register for. And I wish I had known that was a thing and I could have asked someone what it means. But anyway, if you want to check that out, could be good. I don't know what it is. But usually these just say close. They fill up very fast. And today they have a different CTA. So someone look into that and let me know what happens. Again, we are here every Tuesday. If you have an idea for a hug, again, we would love to hear from you directly. Feel free to connect with me, with Deanna on LinkedIn. If you prefer to fill out a form, you can fill out this form and we would love to just have your suggestions so we can make sure we're keeping this relevant and helpful to each one of you in your own journeys. As HubSpot admins. Um, that's enough for the icebreakers. Deanna has put together this great resource page. It has links to everything I just showed you plus some additional stuff. Um, check that page out, uh, love it, share it, uh, use it. Um, but that's enough for me. Super excited to have Alex and Yeha here from the, the Data Hub team talking about data quality. I'll let you two take it away.

Awesome, thanks Kyle. I'm going to share my screen. I'm going to go into slideshow mode, if I can. And then I'm going to pop open the chat because I want to talk to you all while I'm presenting. All right. How's it going, everyone? Thanks so much for having us. This is my third time, I think, presenting at the Admin Hug. And I love presenting to the Admin Hug and hearing from our admins. You guys give the best feedback. My name is Alex. I am a product manager. On the Data Quality team. I'm relatively new to the Data Quality team. So for the past 4 years or so, I've worked on our import-export tools. So if you've ever imported a file into HubSpot, that's me. That's what I worked on for 4 years. But recently, about a month and a half, 2 months ago, switched over to the Data Quality tools. So diving headfirst into this world, really excited to hear how you all leverage our tools today, what you want out of the tools, what we don't do well today, how we can improve them. So really excited to be here talking with you all today. Yeha, do you want to introduce yourself?

Yes. Hi, everyone. My name is Yeha. I'm a product designer on the data quality team. I've been a longtime listener for the Hug Gavin meetings, but first time presenting. So I'm excited to be with you all today. So yeah, thank you so much. I'll pass it back to you, Alex.

Cool. All right. So I want to start with two questions. So everyone, get the chat open. I first want to hear on a scale of 1 to 10, how would you rate your data quality situation in your portal today? Is it 1, awful, there's duplicates everywhere, it's an absolute mess in there? Or is it 10, that you have a good handle on it, everything's great? Okay, so I'm seeing some 1s, some 7s. I think 7 is the highest number I've seen so far.

Anyone beat it? Amy dropped in an 8 to 9.

Hell yeah.

That's pretty good.

I like the specificity with, uh, Florin giving it a 7.76. It's very specific. Perfect. Okay. So I think maybe if we averaged it all out, we'd be right down the middle, probably something like that. Okay, cool. That's someone that just started a month ago. We're on company data as we're talking. Awesome. Okay, great.

Cool.

All right, my second question for you, and then I promise I'll get into the talk. Uh, what tool are you primarily using to manage your duplicates today? We have our own tool in HubSpot, the Duplicate Management tool. It's one I manage, but it's not the only tool. Um, I'm curious if you use other tools. Um, are you using Deduplii, InCycle, Qualify? Um, awesome. Okay, good showing for InCycle. Good show for the HubSpot tools.

I saw one mention of Qualified, but yeah, seeing lots of, lots of HubSpot. I see Caitlin say, sadly, the HubSpot tool.

Sadly, the HubSpot tool. Okay, we'll need to dive into that. And hopefully we can turn that sad to, uh, to happy via some of our updates. Okay. If you use another tool, I'm going to share my LinkedIn profile at the end of this. We'd love a DM from you saying why you use that other tool versus the HubSpot tool. That'd be really helpful for me. Okay, great, cool. So this is the agenda that I have today. Thank you for humoring me and dropping in the chat, everyone. I really appreciate that. This is our agenda. So first, we're going to look at just a brief history of the duplicate management tool just to get us grounded in where we are today and how we got to where we are today. Then we're going to look at where we're immediately headed, what we're currently building, what we hope to roll out in the next couple months. And then we're going to look at what we want to do soon. So when I say where we're headed immediately, these are things we're actively building. We have a plan to roll them out soon. Where we want to head is more so ideas or like ideas that are being in their nascent phase. So hopefully at the end of this, you'll get a good overview of the general direction of the data quality tool in HubSpot and specifically duplicate management. That's our primary focus as a team. Um, after that, we'd love, uh, to do Q&A. Uh, feel free to ask any questions that you have, um, and we can go from there. Cool. Okay, so let's do a quick history of the duplicate management tool in HubSpot. So as far as tools, it's a relatively new tool. Um, it was launched originally in around October 2019, and since then we've added incremental additions to the tool to improve the quality of life as you're managing your duplicate situation in HubSpot. So in May 2020, we launched manual rescans. We then layered in duplicate rejects so you could reject specific duplicates that the tool was surfacing for you and say that weren't in fact duplicates. We added the ability to filter the table so you could filter it down to see specifically what you were hoping to see. And then in 2023, we layered on the data quality overview to give you high-level reporting on your duplicate situation. in the CRM. This all came to a head at Inbound 2025, uh, when we launched two features that we thought were going to be, uh, really game changers in how our customers manage duplicates, uh, in their portals. So the first of those was custom duplicate detection rules. Uh, this allowed you to specify, uh, your own rules for how you wanted to detect duplicates. You could set up to three properties in the CRM that you wanted to do exact match on and then manage your duplicates based off of those properties instead of the HubSpot default model, uh, which has those six properties, I believe, for contacts and companies that are predefined for you and you can't edit it. So you can create your own rule, edit it as you saw fit, and leverage exact match to surface duplicates. The second feature we launched was Auto Merge, which we were super excited about. This is our first foray into allowing you to automate the process of how you would handle the duplicates that were found in your portal. So you could set up rules and then say, okay, those duplicates that are detected, I want you to automatically merge them together. Now, I was at Inbound 2025, and I was there to talk about import and Data Studio. And every time someone would come to talk to me about Data Hub, they want to talk about auto-merge and duplicate management. So people were really excited about these two features. But there were problems. So quickly upon launching, we realized that the means for detecting duplicates just were not powerful enough right now. We didn't have strong enough duplicate detection rules. That enabled automerge to truly do what it needed to do. Compounding that, governance and control over the automerge process was kind of limited. And so you didn't have control over which records were enrolled into this automerge process, didn't have the ability to monitor it as it was going through this automerge process. And to make matters even worse, there was no viable course of action really, if something went wrong. And so we had situations where someone had 1,000 records that were merged into one record with no means to undo it. There was these data quality, data corruption issues that we just didn't have a good means of solving. So unfortunately, because of these data corruption issues, we had to roll back auto-merge. Okay, so quick history. That's just to level set us on where we are today. Now I want to talk about where we're headed. So our first priority as a team, maybe unsurprisingly given the preamble that I just gave, is to bring back auto-merge. But I want to caveat that we want to make sure that we are doing it responsibly. And so we're going to be taking great caution as we bring back automerge to ensure that as we do so, it meets the needs of our admins, doesn't lead to the same data corruption issues that we saw with the first rollout of automerge within the CRM. So how do we do that? We think there are 3 things that we need to do in order to bring back automerge in this responsible manner that we're looking for. So the first of those is to improve How we detect duplicates. So let's take a look at that quickly. So we have two means for detecting duplicates today. The first is the HubSpot default model that looks at a predefined set of properties within the CRM. It does fuzzy matching on those properties. You can't edit those fields. And so it's predefined, no means to edit it. And it's kind of, you know, somewhat of a black box. The other thing we have, we still have custom duplicate detection rules, which we launched at inbound, currently still in a beta. That allows you to choose up to 3 properties per rule, do exact matching on it, and it's completely configurable to how you want it. So, looking first at the HubSpot default model, we get feedback all the time that it's a bit of a black box, that customers don't necessarily know how we're surfacing duplicates, and that the duplicates we're surfacing via that model aren't necessarily helpful for our admins. These are direct quotes, uh, that it started flagging contacts with different names as duplicates, different last names. Created a ton more work for me and my team. And then this other quote: for every 5,000 potential dupes I go through, I find only a few hundred that are actual duplicates. And so the HubSpot default model clearly is leaving a lot to be desired when it comes to how admins can go through and manage duplicates in their portal. So the first thing we want to launch, which is coming soon, is this concept of a similarity score for the HubSpot default model. So we have this data already on the backend that shows across the properties that we're using to compare one record to another within that model, how similar are they for those 6 or so properties that the model is looking at. We have this data already on the backend, we're just not surfacing it to our customers. And so coming soon, we're going to start surfacing this data point to you so that you can see, ah, okay, based off of the 6 properties that the model is looking at, this duplicate pair is about, you know, 80% duplicate or 80% similar based off those properties. Properties, I might want to do merge those together. Even better, you can filter based off of it so you can only look at duplicates above a specific threshold and then ignore the rest. So hopefully this allows you to get more targeted with the HubSpot default model, take better action as you're managing duplicates with that similarity score. Okay, cool. The next thing we want to do is take custom duplicate detection rules, which is in beta right now, and bring them to general availability. So how are we gonna do that? The beta functionality right now, you can create 2 rules that are exact match on 3 properties, but it's limited in the scope properties that you can look at. Right now, I believe you can only look at single line text or number fields. In order to take that to general availability, to give it to all DataHub Pro Plus admins, we want to make it so that you can match on up to 9 properties in your portal. And then we want to expand to all property types. So not just single line text field or number fields, but also dropdown select fields, single checkbox, date fields. And so once we have that, we'll be able to expand to GA, give this tool to everyone. Hopefully by letting you define the rules by which you're identifying duplicates, you'll be able to be much more targeted in how you're handling duplicates in your portal. Yeah, the second thing we need to do in order to bring back auto merge is increase the transparency around the scope of your duplicate detection problem. I see this in portals all the time and it drives me absolutely insane. This is an example where on the contact index page, I've searched for Ronaldo Martinez. And I get, you know, 52 examples of this contact in my portal. But on the duplicate management page, we've limited the scope of the duplicate group that we're surfacing to just 5 records. And so as you go and start to reject specific ones, instead of you seeing that now there's 4 Ronaldo Martinez's, the next one in line just comes up and it makes it seem like you have kind of the situation where you can't properly see the full scope of your duplicates. So coming soon, we want to expand the ability to see the entire size of your duplicate group within the duplicate management table. So you can see truly, ah, okay, there are 52 Ronaldo Martinez in my portal, not just the 5 that are being surfaced here within the duplicate management table. Okay, the other thing we want to do is just increase the amounts of duplicates that are surfaced and remove limits in the tool. We shouldn't be handcuffing you based off of some arbitrary limit for the number of duplicates that we are surfacing. And so hopefully in the near future we'll be able to show you the complete scope of the duplicate detection problem in your portal. Whether you have 100 duplicates or you have 10,000 duplicates, 20,000 duplicates, 30,000 duplicates, you should be able to see that full scope and start to take action on it accordingly, which is especially important if you're starting to automate this process away. So hopefully in the near future we will be removing those limits that you see in the tool today so you get the full scope. Full scope.

Okay.

The third thing we need to do is we need to create tools that help you systematically resolve duplicates that are in your tool. So today in your portal, you have two means to manage duplicates. You can either do a single merge or you can do bulk merge. I want to take guesses in the chat for which do you think is more popular right now? Single merges or du— or bulk merges? Got one guess for bulk so far. Everyone else— okay, a couple more for bulk. Cool. I appreciate the engagement, folks. It makes it a lot easier to present. Overwhelmingly, single merges dominate. In the past 30 days, we've had 2.2 million single merges or rejects for duplicates that are found by our tool, compared to just 57,000 bulk merges. So people clearly do not feel confident taking a bulk action in their portal. And so we dig into the feedback to try to understand why is that the case. Say, I wish I could prioritize how deep, how to dedupe contacts. I need the ability to manually pick which property should stay on the merged contact. The most recent does not necessarily mean the most accurate. And then this final quote that I have here, I keep coming back to, it's one of my least favorite or most favorite points of feedback that we get, but I hate that it's a point of feedback where someone says that they had to merge 2,800. I'm just moving Zoom around so I can read it effectively. I had to merge 2,800 companies one by one. It took 56 painful hours to complete. So we clearly don't have a means for customers to systematically manage their duplicate detection and how they merge those duplicates today. And so this is on our roadmap to bring into the product merge rules. And so what merge rules will do is allow you to specify the winning record within a duplicate group. So you can say, okay, this record based off off of whatever rule you set up will be the record that wins out in the merge process and moves on and stays consistent after the merge happens. You can then specify property overrides. You can say, okay, for these specific properties, I want these specific values to be present on that winning record, regardless of whether or not they currently live on the winning record or if they're on another record that's surfaced within that duplicate group. And then finally, you can set exclusion criteria. So once we do have better bulk management, or we do have automation, you can set up rules that will say, okay, if, you know, the contact owner is different across these two groups, I want to make sure that they don't go through an automated process or a bulk process. I want to manage those manually. So this gives you a level of governance that you previously didn't have. This is something that we're currently working on building. It's coming soon. Okay. I had another slide here. Um, the slide is just to kick off Yeha's demo. So I'm going to kick it over to Yeha. He's going to show you what merge rules look like.

Thanks, Alex. So yeah, what I'm gonna be walking through right now is a prototype of what merge rules is going to look like. So let me get my Zoom ready. Alright. I'm guessing everyone can see the manage duplicate screen. Okay, great. Awesome. So, yeah, what I'll be walking through is what we're thinking for merge rules. I'm just going to make sure everything fits in here properly. So, what I'll share first is creating a custom duplicate rule in the ways that you would like to find duplicates, and then I'll walk through what it looks like to set up a merge rule and how that plays out as you're reviewing merge records and getting things ready for bulk. So, Let me first start here. We're on the Manage Duplicates page. I'm going to create a new custom rule. And let's say, this is a prototype, so some things will kind of like jump in the timeline. Like you might not see all the dropdowns or things like that, but I think you'll understand what I'm doing. So we're going to name this rule Name, Email, Phone Number, and I've used the properties first name, last name, email, and phone number. And so now I'm going to press See Duplicates to load up this duplicate table. Just some callouts here. Alex alluded to this. We are going to have full counts of the duplicates that belong in a group. So you'll see here there's 10 duplicates, and here we have 30 total duplicates. On the table, you still might only see 5, but we at least want to show a count of how many duplicates will be— will have been identified for that group. So now what I'll do is go to the rules section. So there's a there's gonna be a new tab called rules, and you can manage your duplicate detection rules and this new concept that we have called duplicate merge rules. So what this does is it allows you to have more control over how the resolution of duplicates takes place. So this first step here is that we're gonna choose, there's some basic rename the rule, and you choose the object type and then you also choose the winning record. So object type, we have these two objects for now. Winning record, some of these look familiar. We have oldest engagement, newest engagement, created first, created last, most recently updated. We also have this idea for highest fill rate, so choosing the record that has the most property values completed and the source type. For this demo, I'll just share I'll have the winning record be the most recently updated. And then next we have overrides. So select overrides for specific property values that should be included in the merge record. So I'm going to add just one override here. So let's say there's like a custom property that you want to have a specific override for, Zoom Info ID, you can choose You can make some rules for this. So for this override, you want the ZoomInfo property that is known, then you want it to be the most recently updated. So what this means is if there are multiple values because multiple are known, then we're going to look at the most recently updated. And if there is a tiebreaker for some reason, you can choose the winning record value to be the one that wins. These are some other examples. You can choose last activity date, is before a certain date or it's most recently updated. But basically just some granularity for specific properties that you want to make sure get to the merged record. And then lastly, we have merge exclusions. So what this means is, this is more applicable for bulk and automated merges because you can still see these duplicates on your table, but if for some reason This is an example. If a contact owner is different, what this will do is that it will exclude that duplicate from being resolved in that bulk or automated merge. This is an example that we heard from a customer we spoke with that said that if a contact owner is different, it's, it can affect sales and commission. And so people might lose out on commission if things are merged and they didn't see who all was working on it. So this prevents that from happening and you can take a closer look and adds a human touch. So let's create this merge rule. We named it PNW merge rule, so that's what it is here. And how this plays out in duplicate— the managed duplicates table, let's go back to the table where we had this custom rule created for duplicate detection. I'm going to expand a couple of these duplicates and review them. So another part of merge rules is we're going to have a new merge preview experience where you can see the winning records, so the values that are being chosen from the winning record. So hover state as well. And you can see like a nice preview of that, but it's basically all the values here. But you can see if I change the merge rule, right now it's using most recently updated. I can change it to the PNW merge rule that we just made. And you can see those values change based on that rule that you made. So this is being selected because of property overrides. And so you can continue to merge and next. So that one has been merged. This is an example where there's 5 records and you'll see that we're using the PNW merge rule and you can see why these values were selected. So we don't, we are working towards a world where there is auto merge. So we're expecting that you can review a couple of these and maybe if it's behaving the way that you're expecting, then you can automate the rest. And hopefully that makes the merging of records feel less manual, but also safer than just having a black box auto merge dropdown. I haven't been keeping up with the chat, so I'm gonna pop over.

Mostly it's praise. There have been a few questions that Alex has hopped on. Everyone else, if you've asked a question that hasn't been answered, make sure it gets in the Q&A because we will circle back later. I have a question. Yeah, huh? I'll exercise some executive authority here and ask my own question. Uh, when you were setting up the rule, um, you mentioned, um, if there's a tie, uh, the option you showed was just go with whatever the winning record has. Were there other options in that dropdown for tiebreaker?

Yes. Um, Alex, you might have to remind me what we landed on, but I think it was most—

it was the most recently updated. And I think we're also considering source of record.

Oh cool.

Yeah, the specific options that we have, I would need to circle back on, but I believe it's most recently updated. And there's also source and potentially fill rate as well.

Um, yes, I, I saw a mention, uh, so fill rate in, uh, I guess, uh, yeah, Ian mentioned highest fill rate because they, they're, they have some, uh, duplicates where one record is basically just a name and an email address and the other record has all the data. And if there was a way to just programmatically say like the one that has more stuff is the better one, uh, that would be cool for sure.

Totally.

Yeah.

And seeing lots of love for source in the chat.

Yeah. During the Q&A, we should dive into like how people would intend to use something like source as the winning rule. Um, hear more.

Yeah. Tell us more in the chat, everyone, about how you want source to work.

If you have, I guess, hopes and, you know, ways that you would use something like this, now is the time to tell us during this development process. Would love to hear from you all. How would you— you would set up a merge rule within your own portal to help you manage that merge process, knowing that in time the plan is to allow you to automate the process of duplicate management with these merge rules applied to it. Cool. Okay.

I just wanna say, Shell, you're great. Amazing.

Yeah. Can I steal, can I steal the screen share back from you?

Yes.

Amazing. Okay. Cool. I'm gonna share, I think, I think this is it. Does everyone see my slides now?

Yes.

Amazing.

Yeah, we see other considerations for automerge. Cool.

Okay. So those are the things that we want to be true before we consider bringing back automerge. We hypothesize that if we are able to give greater context into the duplicate scenario, launch custom rules to GA, and then launch this merge rules functionality that will be able to bring back automerge in a more sustainable, more responsible way. But there are some other things that we're thinking about here. So for that automerge process, when we initially rolled it out, once you turned it on, any duplicate that was being surfaced was just automatically enrolled, which is a very quick way for a bad situation to occur. And so we are considering how we might be able to limit the enrollment of records that go through this duplicate automerge process. So something like only enrolling records that are in a segment, for example, is something that we might consider. The other thing we're looking at is notification systems. So how can we notify an admin if something is going on within the auto-merge process so that they can intervene and take action if needed? And then the final thing, I know that I'm seeing in the chat people are really jazzed about undo merge. I want to caveat this final one. With it is an incredibly difficult problem to solve and technically very challenging. And so while we're looking into it, it is by far probably the hardest one that we would have if we're able to do it. But know that it's on our radar and something that we want to do if we're able to technically do so.

I would— you can move on if you want to move on, but I would just love to hear how you're thinking about that and what makes it technically difficult, difficult, because in my mind, Once a merge happens, the preexisting records no longer exist, right? They have been wiped out. So I don't even know what an undo merge would look like. And yet in my brain, every technological system has an undo button. That's so simple. We've had that since Windows 3.1, right? And Apple IIe. And so like, how are you thinking about it? Yeah.

So to maybe give a little bit of color as to why it's so technically complex, when you do a merge, let's say you have a duplicate group of 5. When you do a merge on that, you're going to merge record 1 with record 2, and that creates 1 record. And then you take record 3 and merge it with that, and it creates 1 record. And then you take record 4, merge it with that, and it creates 1 record, and so on. And so as you go through this merge process, if you say, ah, okay, I want to get the original 5 records from that merge, it's peeling back layer after layer of records that no longer exists to your point. And so a very tricky scenario. I'm seeing sandbox or like staging environment. That's exactly the kind of thing that we need to start to consider. But yes, a very technically challenging problem and one that we haven't been able to build yet in a way that we feel is reliable enough to then give to our customers. And so much like we want to make sure that we're being responsible in how we give automerge back to our customers, we want to make sure that we're being responsible in how we give undo merge to our customers for the first time.

That makes a lot of sense. And I know in other parts of the platform where we provide options to roll things back or undo them, it's often for a limited time, right? For the last 7 days or 28 days, because somewhere we have to store something and we can't just keep that forever, right? You can't say like, oh, remember that one time we used auto merge back in 2017? Can we undo that now? Right? Like too much has changed.

Exactly. The other thing to think about is like once you've, uh, merged records and create this new record, that record lives on and things continue to happen to that record. It gets enrolled into workflows. Properties change on the record. And so if things have changed with that record, what should we do with changes that have occurred to the record since the merge occurred?

And like associations and calls that have been logged and all of that, where would they go?

Totally, exactly. Yeah. So these are the things we're considering. Undo merge, super technically complex, something we're considering, but still trying to wrap our heads around how we do that in a responsible way. Would love to hear from all of you either in the Q&A after this or in my LinkedIn DMs. What would make you feel more comfortable automating the process of merging records together that are surfaced through the duplicate management tool? Let me know. Okay, so those are the things that we're working on actively right now as a team. What I'm going to talk about next is where we want to head. So things that we have either started the process of planning for, or they're simply ideas that we're starting to kick around and, you know, strategize around as a team. So the first of these is data quality across the entire data model. So currently we support contacts and companies. The HubSpot data model is far more vast than that and has everything from deals to tickets to custom objects and many, many, many more objects. And so we want to strive for a future where you can do duplicate management across the entire data model. Now, uh, the reason why we haven't as to this point is because the overwhelming majority of customers, uh, want contacts and companies. And so we've prioritized contacts and companies just because that's where we're seeing customers, uh, trying to do their duplicate management and data quality efforts. If you have a use case for how you want to manage duplicates for deals or whatever it might be, please let me know. Hearing feedback from you like that helps us prioritize initiatives like This. Cool. Uh, the second thing is expanding how we detect duplicate records. So today we have the HubSpot model, uh, which is a locked-down fuzzy match system. So you can't manage, uh, the properties that are included in it. Uh, it's locked down. And we have custom rules, which is exact match. Uh, in the future, we want to expand fuzzy matching and phonetic matching to custom rules so you can manage more precisely how you want fuzzy matching applied to those custom rules. So something that we're considering right now, something we're starting to strategize around, uh, if you would use something like that, or if you have another tool that you use to manage duplicates that executes fuzzy matching in a way that you really like, I would love to hear from you. Okay. And then the other thing we're starting to think about is agentic duplicate detection and resolution. So think about a world where you can work alongside Data Agent to find duplicates in your portal and then work with that agent. To merge those records. This is very early days in how we're starting to think about how we might do this for our customers, but we see where things are headed and we want to make sure that we are providing you with the most up-to-date means for finding duplicates and resolving duplicates. And if we can provide you with an agentic solution via Data Agent and Breeze, we would love to do so. So more to come on that as we continue to develop our plans there. The final thing I want to talk about is actually something that we are starting to work on a bit more, you know, in earnest in the immediate future. And that's data quality within context. So currently, if you're doing data quality, you go into the data quality tool in HubSpot and you work within this isolated area in the CRM. But I would love in the future to have a tool that is embedded within segments so you can apply duplicate management to the records that are involved in a segment, see which records in that segment are duplicates, and manage it there. I would love to be able to embed duplicate management within marketing campaigns so you can suppress duplicates from an email send that you're about to send, or remove them from your ads audience, or get more targeted with your campaign reporting so that you know how successful a specific campaign was. And then across the CRM, I want, you know, to embed duplicate detection in every area of the CRM. So whether that be an index page or on the contact record, you should be able to see whether or not a record that you're working on is a duplicate and be able to manage it there. Coming, I think, within the next couple of days, we will be launching a record card that will surface for you whether or not a record is going to be flagged as a duplicate. And so you can go and then click into the duplicate tool from there and manage change it from there. So that's our first foray into this space. Hopefully a lot more to come in that space to allow you to manage duplicates across the entire CRM. Cool. Okay. The final thing that we're thinking about is data quality at ingestion. So imagine a world where you are importing data or you're syncing data to the CRM and you can look at the source of data that you're bringing in and apply these duplicate detection rules as well as these merge rules that we've already discussed to that source of data and clean it up before it even gets into the CRM. So we are starting to explore concepts on this, starting to build this out. Very early days for that. But imagine a world where you can clean the data that is coming into the CRM before it even hits the CRM. That's the future that we envision. That's the future we want to give to you. If you have plans for how you would leverage that or how you want that to look, I would love to hear from you so we can take that into account as we build out this plan. Okay. That is my spiel for all of you. Would love to dive into a Q&A if there are questions that have been asked of us. I'll also just bring up this slide where me and Yehad included our LinkedIn URLs. If you want to connect with us, we'd love to hear from you if you want. to send us DMs about anything that we talked about today. Uh, we'd also love to hear from you.

Amazing. Uh, thanks so much. So yeah, everybody head over to the Q&A, add questions, upvote questions. Um, uh, and I'll just chat with Alex and Yeha a little bit here while we wait for that to happen. Uh, a couple of thoughts that have come up based on the chat. Uh, one, I don't know if this resonates with anyone else, but me and James, we had a nice conversation, uh, with deals specifically. The problem isn't necessarily duplicates. It's that. A sales rep will start to create a deal or they'll create a deal on a whim and it'll just kind of be an empty record. And so to programmatically identify like this is incomplete or partial or fake or in any other way not real or meaningful, and to clean those up would be interesting. And I think, I think for deals specifically, perhaps in other areas too, that, that, that's an interest. It does the same kind of thing as deduplicating, but there's not like two records being put together. Yeah.

So let me say it back to you. So a sales rep, like, will have interaction with a prospect and then as a precaution, they'll create a deal record just to be like, oh, like this could happen. And then that prospect goes nowhere and nothing happens with it.

And so then the deal just sits there.

Yeah. Okay. I'm wondering if there's like a workflow that could be created where you say like, if inactivity on this deal record, if like nothing's happened on this deal record in X number of days, delete the record. Um, because at that point we're not talking about merging records necessarily, we're more so talking—

it's, it's more of a cleanup thing.

Yeah.

Uh, people in the chat who know way more about workflows than I do, I think the only deletion workflow action we have is for contacts. I don't know if you can automate deal deletion, but maybe. I don't delete stuff with workflows, that stuff scares me. So, uh, you all pipe into the chat and let me know. But yeah, that— if that exists, that would the right solution. The other thing— oh, go ahead.

I was gonna say, I love proposing like workarounds or ideas.

Yeah.

How you might do something within the admin hug specifically, because the audience here will always know the workarounds infinitely better than I ever will.

Right. They'll be like, oh, we already tried that, Alex, and it didn't work because of this. And that's, that's what we need. That's what we're here for. Uh, the other thing that's come up a lot in the chat, the, the, this whole 39 minutes we've been together is integrations. Um, and you— what you just said, Alex, about preventing duplicate data from coming in in the first place, that got a lot of love. People are very excited about that idea. Um, but there seems to be a larger question of if we clean up duplicates in HubSpot but we're integrating with a system where the duplicates still exist, we now have a problem. And I'm wondering if that's something you're thinking about at all.

Yeah, it's definitely something we're thinking about and partially why we're exploring, uh, this I guess this, you know, action item to start to clean up duplicates at the source. I think the integration is, the point of integration is interesting and challenging. If anyone would like to have conversations with me about how integrations specifically are causing duplicates in their portal and how they would like to clean those up, I would love to have that conversation with all of you.

I will say another thing. Oh, sorry, Kyle.

No, no, you, you go.

One thing that came up too was record IDs. Something that has changed recently too is that there's actually a different team that— I'm just sharing some boring background, but they technically handle the merge part. And they've heard feedback that the creation of a new record ID has caused a lot of problems with integrations. And now what we're doing is we're going back. So like there's not going to be a new record ID. The, the record that you've chosen as a winning record, it will retain its record ID so that integrations will not be messed up after a merge.

Yeah, everybody sound off in the chat. Let us know if that is okay.

Great.

That's all they've ever wanted. Good job identifying that. Yeah, no, that makes a lot of sense and I wouldn't have thought of it. Yeah, actually, that's way better than what I was about to say. I'm glad you spoke up first. Yeah, the thing I was going to say to you, Alex, is this idea of records being merged in HubSpot but not being merged in, in systems where HubSpot has an integration, to me, is a similar flavor to the undo merge problem, right? We have one state where multiple records exist and another state where only one record exists, and they are somehow connected to each other. In a way that needs to be living and dynamic and usable. And that is, that's just, that my brain doesn't bend that direction. It's a, it's a hard thing for me to think about.

It's a tricky problem. Yeah. I'm learning as I dive into it again, this is like my second month on the team, but the problems to solve in duplicate management are, are extremely challenging, uh, and infinitely more challenging than the ones I was focused on in import. And so how we do this in a way that doesn't corrupt data and keep things clean and allows people to find success in the CRM. is foremost front of mind for me right now.

Yeah. That said, I imagine import work was a good preparation for you. When you say things like, we only do contacts and companies, we need to expand to other object types, the model is more complex than that. Like, that's the world you've been living in for 4 years. So makes sense to me. This is the longest I've ever, ever managed to wait for people to put in their Q&A and upvote. I feel I did a good job. I think the lesson we learned is don't sit in silence, just chat, Kyle, and you'll be okay. We have 22 questions currently. Feel free to keep putting them in, folks. We will get through as many as we can. It probably won't be all of them. The more helpful thing you can do right now is just skim through and upvote because we will go most upvoted, least upvoted. So right now, top with 7 upvotes, something I've encountered is that phone numbers that are formatted/unformatted won't be considered duplicates. Is that something that has been taken into account? So I think that's like if we have a phone number with like parentheses and hyphens and things and a phone number that's just a string of numbers, those two won't be seen as, as the same.

Interesting. So the exact match nature of the Custom Rules tool is not catching that two records are duplicates because the formatting applied to those records is different.

I think that's right.

Okay.

We have been focused on duplicates, but that is something that we've talked about is that the resolution of formatting issues could help with duplicate management. So, that is something that we've encountered that if the phone numbers are formatted all the same, then it would not, one, it won't create a new record, but two, it would just help find the records that already exist with that phone number. So we are working towards that world where we resolve formatting issues effectively and then also be able to dedupe from there.

Yeah.

The other thing I'll say is if you can apply the formatting rules to the properties themselves within property settings, So you set the property up to only, only accept rule, uh, property values that are formatted in a very specific way. Hopefully that will then downstream impact the duplicate detection rules. Um, but to Yeha's point, making sure that we are applying formatting rules within duplicate detection is something that's very front of mind for us.

Yeah, that makes sense. Does your team own the, uh, because what some of the first data quality tools we ever launched as part of Operations Hub, when it was called that, was the auto-like formatting of capitalization and stuff. Is there one for phone number and does it work for this sort of situation?

Yeah, I won't bore all of you with what teams do I own in HubSpot, what teams do I not own in HubSpot. Currently I own the duplicate management tools and it's something that I'm working on to then take back control over the formatting issues.

Got it.

Insights, all those things. So they're all interconnected. But the tool that we own as a team is the duplicate detection rules.

And then one other thing you mentioned, Alex, having validation on properties can, can protect from these problems. What happens if on, say, a phone number property, we require certain formatting in the context of like an import, which I assume, you know, a lot about, or like an integration when data tries to come in that doesn't follow those rules, is it just rejected or what, what happens?

Yep. So, um, via import, we will flag that for you on the mapping step where we'll say, yeah, okay, like this doesn't fit the formatting that's required, the custom validation rule that has been applied here, uh, and we'll give you the opportunity to fix it there in import. Uh, in DataSync, we still have, uh, work to do to establish a means for you to fix property validations in flight as you're integrating. But if you're entering a phone number directly into the CRM and it doesn't meet property validation rules, you'll get a little flag that says, hey, this is a valid value for this, you need to re-enter it, and you won't be able to save until you meet the property validation rule.

Great. Okay, thank you. Uh, next, Shell asks, our org has wholeheartedly rejected merging contacts when they move to a different company due to the logging on associated records rules. Is there any plan to narrow down associated record logging/prevent it at the association label level?

I'd love to learn more, Michelle. I would love to hear more on how associations are impacting your duplicate management process. I don't know if that's something we can do here or if we want to connect offline, but I candidly need to invest more time in understanding how we handle associations via the duplicate management process.

Yeah, and Shell, I'd let— go ahead and sound off in the chat. Oh, she's happy to connect. That's probably the best thing to do. But my understanding here is if, uh, someone is at a company, you have them associated a company, and then they move to a different company, um, and so you create a different record, uh, then logging things, and— or if you have them associated to both, right? I have this person's associated to this company with the association label former company and they're associated this company with the association label of current company. Um, is that anyway, something about that?

Yeah.

Yeah. No, I, I think as I've learned more about the duplicate management space, how associations impact a merge is coming up time and time again as something that's really important. And so I want to make sure I fully understand the scope of that problem and I'm able to address it effectively.

Cool. Charlene asks. Will any of these updates roll out to other hubs outside of Data Hub Pro?

Mm-hmm. So currently there is some duplicate management functionality that's available at the platform level. So if you are a starter customer of any of our hubs, then you would have access to those. For the functionality that we talked about today, custom rules, merge rules, when we merge, bring back auto merge, data quality at ingestion, Uh, all of those function pieces, functionality as we're building them today will be Data Hub features and Data Hub Pro Plus features.

Cool. Um, also I just want to call out way back at the beginning of this webinar, Debbie said something about, oh, it's so much simpler when there were just 3 hubs. Now there's so many hubs, I'm always confused. I don't know where they go. And I wrote a very nice answer, but I accidentally only sent it to panelists. So none of you got to see it. Um, this is, this is feedback we get a lot. HubSpot is very aware that we have become perhaps too hubby. And this is something that's being discussed by like our executive leadership team. I have no idea what the solution to that, if anything, will turn out to be. But that is feedback we are hearing. You are not alone feeling that way. Please continue to tell us that. Please continue to, to share that feedback with us so that we can continue to investigate a way to simplify that for you, because that's, that's how things get done at HubSpot when there's feedback behind it. So Nothing to share to fix it, but I want to let you know we've— you're not the first to say that. Uh, Amy asks, will there be a way to A, exclude some property values, for example an exclusion list for dupes, and B, filter duplicates by mismatch properties? For example, we cannot merge companies if they both have values for property X. Either one needs to be blank or property X needs to match.

So the exclusion criteria, yes, I think Yeha showed that. The ability to say, hey, if property X is equal to Y, exclude them from any bulk or automated process. I think the second thing you're getting at is saying if property— if phone number on this record equals mobile phone number on this other record, then I want to either merge or not merge them. Cross-property merging is not something we prioritize yet.

It's something that we're learning more I think what she's asking is for the same, if I have a, in phone number property, I have two different phone numbers. We don't want to lose one of those. We want to keep them separate records. If they are different, do not merge.

Thank you for clarifying. Yeah. So that's how it works today for email address. If I'm not mistaken, please sound off in the chat if I'm incorrect on that, but I believe that's how it works for email address that when you merge, if there's conflicting email addresses, we will add those other email addresses into the additional email address field. That's a conversation that I'm having with the properties PM about making the same functionality work for the phone number property as well. Not something that currently works that way, but something that we're actively aware of and having conversations about. Because of course, if there's multiple phone numbers, a home phone number, an office phone number, a mobile phone number, you'd want to preserve that on the winning record.

Cool. Charlene says, can we see a merge history in case we need to do.

Yeah, I believe you can see a merge history right now. Um, so if you go to the duplicate management tool, uh, there should be the ability to see, uh, which records you've rejected within custom rules. So it'll show you which ones have been rejected. Next up on the docket is adding the ability to see which ones have been merged as well.

Cool.

And there's also that property that maintains a list of like previous, like merged IDs. Right? If, if 5 different records have been merged together, you can see like, oh, that's where record 578239 went, or whatever.

Yep, I believe Lindsay's, uh, calling that out in the chat right now.

Cool. Oh yep, uh, Robin asks, in the merge review, rather than just seeing the properties you are reviewing, can we actually have the ability to add other properties to the view that are not being compared? As a safety, I would want to compare other properties in case there's a property I hadn't considered that needs to to be evaluated.

Yeah, I didn't call it out in the designs, but, um, what this would— I'm just sharing my screen real quick. I do have this ability here. You can set properties to review. So I defaulted to the ones that you made specifications for, but if there's other properties you want to review, a new column would populate and you can see the comparison of values there. I know that, um, we want to review all the properties just in case, so Yeah. Gotta keep it in there.

Uh, Scott says it would be nice to have more dates available to sort by. Can we be able to sort by dates the contact was created and sort by when the dupe was found?

Oh, interesting. So I would love to know what you mean by more dates to sort by. What specifically are you looking to sort by? Um, but the ability to surface when the dupe was found is an interesting one. Not something we'd consider necessarily, but definitely something something we could consider. I would just love to learn more about what your use case is there.

Cool.

We actually used to have like a discovered at property. I think it's just not popular on the table, but I think it's something that we could bring up.

So, okay, so Scott, you're looking to filter down the table because there's 10,000 plus, you want to prioritize the ones that are most recently surfaced. We do currently let you filter the table by timeframe, if I'm not mistaken, Yeha. So you should be able to filter the table down. It'll still show you all, you know, 10,000 duplicates or whatever. But you can filter it down to create a more manageable list to start to work through. And then once you've, you know, created— once you've resolved all the duplicates that have been surfaced from the past week, for example, you can then go to within the past month and then the past quarter, etc.

So Sabrina asks, when is the custom duplicate rule coming to us? And Kelly asks, When will this all become available? All the stuff you showed that is coming soon, does any of that have set timelines yet? What's the status there?

Yeah, yeah, yeah. So similarity score, within the next month or so, we should have a beta for that. I'm gonna add a heavy caveat that within the engineering process, timelines are variable and things can change, but similarity score within the next month or so. Custom deduplication rules, there's a public beta right now that you can all enroll into if you would like to, so long as you're DataHub Pro Plus, you can enroll in that beta. And then merge rules still being actively worked on. The timeline for that is very fuzzy. I would expect sometime this summer we'll have that, but I want to not give a more specific timeline because of the variability of engineering. And then just to throw it out there, Um, automerge, we launched it back at inbound. I want it back before unbound.

Great.

Uh, John put in the chat too, um, the actual dupes instead of capping at 5,000. We mentioned that earlier of like, um, that's what, um, the, the limits is what we're trying to say. So like, that's also something that we're actively working on. Um, Alex, do you have a timeline for, for that?

Yeah, I think for that, hopefully in the same timeframe as merge rules. So as we're giving you the tools to systematically handle the duplicates, uh, that are being surfaced, you want to expand the limits that we give for you so you have a better picture as you're creating those systems.

Cool. Um, Bill asks, can you show the fill rate for records in review your merge view?

Okay, so you want, uh, to— so we're currently, uh, exploring the option of allowing you to set, uh, either winning record rules, uh, or exclusion rules based off of the fill rate for a record. It sounds like— I'm sorry, I forget the name. Was it Phil?

Bill. Bill. Bill.

Bill wants to see that fill rate as like a property. So you can see, okay, this record is 100% complete, 90% complete, etc. We haven't considered creating a property like that. It's definitely feasible. We'd love to hear more about the use case.

Cool. Lindsay asks, if multiple contact records are merged but they have different subscription/communication preferences, How will the newly merged contact record treat these preferences?

It's a great question. Yeah. How do you know off the top of your head?

I would just say that's a really great question that we should consider.

It's an important thing to think about.

Yeah, we should definitely consider it. We should definitely look into that. We will follow up.

Cool. Um, I would love to hear from the admins. Uh, what, what's your preference? Should, should we go with the lowest subscription, the highest? Should we ask you? Should we ask them? Like, um, would love your, your, your input on that, everyone. Um, do the merges change the record IDs? What would this do to third-party API connection setup? So you mentioned that a little bit, but that is a, that is a coming soon thing.

So right now the answer is yes, it does change it, but in a near future, no, you can just choose which is the winning record and it would take the winning record ID.

Great. Um, okay, we've got a couple more minutes here. Uh, maybe it was already covered, but is there a way to look at which workflows dupes were enrolled in and whether the merged record will be re-enrolled in a workflow?

I think my hunch is that would be somewhat specific to the workflow rules that you have set up. Depending on whether or not you allow for re-enrollment or not. I don't have a good answer for that off the cuff, and I don't want to make something up. So I'm going to take that offline.

Yeah, I'm just trying to think through the implication. If you have a record that has— it matched the enrollment criteria and enrolled in a workflow, and then it gets merged with another record, and the resulting merged record matches the enrollment criteria for that workflow, would that count as a re-enrollment, or would it count as an enrollment for the first time? I think it's kind of like Yeah, the underlying question there.

I think, I think we have logic to say that, uh, merged records do not get re-enrolled in workflows that they've already been previously enrolled in unless you explicitly do so. But again, I don't want to misspeak.

Yeah, Charlene in the chat said that almost exactly the same time as you. Jinx, you too. Uh, so, um That is encouraging that you both have the same thought there. But yeah, definitely something to look into. Uh, so that, that question was by Tucker and upvoted by a few people. Tucker and all your upvoting friends, uh, something to investigate further. Um, we'll make this the last question. Uh, oh, uh, any plans for native email verification in data quality?

Yeah, we hear this question all the time about people wanting email verification. When we hear email verification, I'm reading that as will this email, if I add this to my portal, will it send or will it bounce? Um, it's something that we don't currently do today, something that we're investigating, looking into. We don't currently have a solution, don't have a timeline for when we might do it, but it's definitely on our radar and something we're investigating.

Yeah, it's interesting. For a long time, people used to ask, would we ever automatically calculate, uh, exchange rates? And I was always like, why would HubSpot build that? There are so many tools that do it. Like, you don't need a And now we do it right. And so this is one of those that for me it's like, well, you could get a tool to do that, but like, never say never. You never know what HubSpot's going to build. Alex and Yeha, thanks so much for being here. Diana has dropped your LinkedIn profiles into the chat so people can connect with you. Everyone, please connect with them. Send your questions that way. Feel free to give us feedback and, and, and ask any questions also. Either one of you have any last words before we shut it down?

No, just thanks so much for joining us today. Really appreciate the feedback and the ideas.

Love presenting to the Admin Hug. Thank you so much for having us.

We love having you. Thanks for coming and goodbye everyone. Have a good Tuesday. See you next week.

See you next week, guys.

Bye.

Bye, friends.

No solo in this game. It's the admin hub where legends lock in. From chaos to clarity, that's how we win. Dirty data challenge accepted. Portal perfection fully connected.

Connected.

Quick fix queens, workaround kings, custom coded everything. Tabs open, coffee strong. You've been solving all along. We've got the know-how, we've got the math. Even HubSpot brought a full staff to chat. You're not just heard, you help shape the tools. Admins rewriting all the rules. Thanks for building with the Hug. Thanks for powering the Hug behind every campaign, push, nudge. There's a whole crew home inside it. Admins too. Together, powered by insight.