Taming the Data Dumpster Fire: How to Make Marketing Metrics Make Sense with Eric Westerkamp

Hi, I'm Michael Hartman. I'm Naomi Liu.

And I'm Mike Rizzo. And this is OpsCast, a podcast for marketing ops pros and rev ops pros created by the MoPros, the number one community for marketing operations professionals.

Tune into each episode as we chat with real professionals to help elevate you and your marketing operations career. Hello everyone. Welcome to another episode of OpsCast brought to you by marketingops.com, powered by all the MoPros out there. I am your host, Michael Hartman, again flying solo. One of these days we'll get Mike and Naomi here. As I mentioned before, Spring Fling is around the corner in May of 2025. This is, we're recording this early April. So if you haven't checked that out and you're interested in joining that, you should go check that out. But joining me today is Eric Westerkamp. Eric has 20 years of executive leadership spanning sales and entrepreneurship. Eric understands what it takes to drive B2B growth. His foundational strength lies in building world-class enterprise sales teams, recruiting, developing, and motivating them to succeed. As CEO of CaliberMind for over 6 years, the leading B2B marketing data analytics platform, Eric now applies his deep go-to-market knowledge to empower Marketers with the data clarity needed to partner effectively with sales improve their impact. Building on prior executive roles at Frontsteps, OpenText, and EZ-Link, Eric is also a compelling public speaker. So we're glad to have you, Eric. Thanks for joining us.

Yeah, thanks. Glad to be here.

Yeah, so I don't know if I want to— I think what we're calling this, the title of this episode is going to be something like Taming the Data Dumpster Fire. So this ought to be a fun one and will probably resonate with most of our listeners. So what we're going to talk about, and most of our listeners are operations professionals, so we, you know, and they're out there supporting their CMO or a marketing leader of some sort, and they're trying to help with, you know, day-to-day operational stuff, getting stuff out to market, but also with helping to tell the story internally about what impact the that marketing is having on their business. So just from your perspective, I think both as one of those executives you're trying to convince as well as, you know, you're selling to CMOs and things like that, like what are you seeing as the major challenges facing CMOs and maybe the ops teams that support them in this area?

Yeah, I'd say there's 2 challenges. One is that the CMOs, are being asked harder and harder questions to justify what are you and your team doing. I'm giving you X millions of dollars to spend on, you know, ads and lead generation. Help me understand as a leader how that is really having an impact and affecting my business, right? And as organizations are— as organizations scale up bigger and bigger, there's a bigger gap between the C-level in the different individual units and what they're doing. And so it's more and more incumbent upon the CMO to really be able to figure out that story and just, not just justify, but to like really show the organization that we're having this major impact and here's how. The challenge is that many of the rest, a lot of the other business units are very data-driven.

Yeah.

Sales is actually a very data-driven organization typically, because it's very easy to get data on what's going on. I made so many calls, I booked so many meetings, I had so many X, Y, and Z, and I built— and at the end of the day, I booked so much new business, right?

Yep.

Very discrete, very easy to see that. Marketing, it's harder. But marketing is being asked by those other leaders to sort of tell the story about how their impact is grounded in data, right? And then being able to bring that back into the organization in a credible way. Now, the challenge the CMO has is that the data they get is very messy. Unlike sales has very good data, very discrete data, a lot of other operations, finance, things like that, they have very good data. Marketing has huge amounts of data, but it's often very messy.

Yeah.

So the second challenge is that they've been asked to tell this story. That's a challenge. And now they look at the data and information they're trying to tell a story from, and it's to some extent, it's often just a mess. You know, that's why we were talking about it's like a dumpster fire. It's like, here's all this stuff going on and I got to extract out of that a story. Then it falls on MOPS often to be the one, that bridge between that information and data and the CMO. And it's really their job to try to tame that and figure it out and get a story that's credible and then teach the CMO how to really tell that story.

Right. How to translate. It just occurred to me as you were describing this also that part of the challenge for CMOs might be that Because the data is a mess, it's incomplete, or it's inconsistent, you know, for a variety of reasons, right? And there's lots of valid reasons why that could be, is that they lack the confidence that the data can be trusted to tell the story.

Yeah.

I think there's a, like something that in there as well that maybe like, so hard for them to go in and go like with confidence to say, this is what we're doing, knowing that Yeah, there's maybe, maybe one of the legs on the 3-legged stool is gonna collapse at any given time.

Yeah. You know, a classic example that we've seen is the CMO walks into a, you know, a meeting, hopefully not a board meeting, and they say, we generated this much pipeline because that's what the reporting said.

Yep.

Then the salesperson stands up, the leader still says, yeah, but that's 50% more than the actual pipeline we started the quarter with or built that we showed in Salesforce. And then the CMO is sitting there kind of like, well, why is that? It's probably because of double counting. You know, they had some algorithm that was building up and trying to do it. It was double counting revenue, right? And it's often really hard. It's that data trust. That data trust issue is, from our perspective, the core really of what you need to start to set up with these organizations.

Yeah.

If you want any of that reporting to be believable, And for them to then be able to build those stories on top of it, right? It's super important.

So, I mean, one of my beliefs is that like, yeah, I agree with you. Like the trust in the data needs to be there. I think if one of the things I've tried to do as a marketing ops leader, whether it's talking to marketing leaders or sales leaders or sales ops or whatever, is like lower those expectations though too. is, you know, data quality is not great. And there's a lot of reasons why, again, some valid, some not. And I think to me, what's important is starting to do this reporting and just go like, I know that this is not— I hate to use the word right or wrong, right? It's the data's not right. Or it's, or, you know, I feel—

Yeah.

Maybe it's, you put a confidence level on it, right? I feel 75% confident that this is valid or directionally correct, whatever phrasing you to use, but I do think it's really important to start reporting even when you think the data's not great.

Yeah. Well, you, the way I would phrase it is like, yeah, if you don't start to address the data and start to try to figure out the reporting, you may not understand where you need to fix things and where you need to work on things. So like, so starting the project of doing reporting is super important. We often find organizations that are like, hey, we want to go and do something on reporting, but we know our data is a mess. And so, we want to go fix that first and then we'll come back. And what else— my response to that often is that almost never works because unless you start to couple the reporting with the data fixing, like, you often don't get it right because the reporting is sort of the barometer you use to understand if you're getting this stuff right.

Right.

So we said, instead, like, don't go and try to fix all your data and then come back to this because you're going to come back in a year and your data is going to be 20% fixed and you're still going to be in the same spot. Instead, start small. Think of a set of reports or information that you can start to generate from this data that are meaningful, but they're not trying to boil the ocean, right?

Yeah.

Let's start there and then let's get the data right. Let's fix the data to support that report. It may be as simple as I've got, you know, I'm moving my data into a data warehouse now and I want to rebuild some accurate pipeline and opportunity reporting over here. Now, this may really duplicate what you're getting out of Dynamics or Salesforce or something like that.

Sure.

But you start there and you build that and you're like, okay, these reports look right. So now you know a few things. Now I've got some reporting that marketing can use and often depending on the organization, they struggle to pull their reports right out of Salesforce.

Yeah.

You can do it over here in your marketing data warehouse. But you also know that the base numbers that you're working from reflect the numbers that the rest of the organization may be using as a source of truth out of Salesforce or some CRM. And then you start to kind of work out from there, right? Maybe you build a first-touch attribution model. Again, in my opinion, first touch, they have their usage, but they're not super impactful. But they do tell you some interesting stories about how things are coming in.

Sure.

But it's easy to debug them and see if they're right.

Yeah.

And you do that before you may move to a sophisticated MTA or something like that.

Yeah, it's interesting because I'm with you. I think we maybe I even shared this when we talked before is that I often would be asked as a marketing ops leader, like, I need dashboard for like a marketing dashboard. And I'm always like, do you really? Yeah, I think what you need is you need a handful of reports because if we go spend a bunch of time building a dashboard, Yeah. It's never going to answer all the questions. I mean, I think one of the things that I find is I'm with you, right? Do a little bit, learn from it, get that right, then go on to the next thing and so on. The other thing that I find, and I don't know if you see this too, is very often 2 things happen when you start to present this data, even if it's a single report. One is, again, especially at the beginning when maybe data is not as clean, will be, there'll be some sort of outlier. Instead of people focusing on the general trend, right? They see this one thing that looks, just stands out, right? It's above or below or off, like whatever the kind of, and then they start to question it. And if you don't have, that's when I coach people, it was like, you need to be prepared for those questions if they come up, right? So like being able to explain that the best you can is one. The second, if that's not the case, is typically we want to go, I see this, like, help me understand why they want to do a drill-down kind of thing. Are you seeing those 2 phenomena or any others when people start doing this reporting?

One of the things that when we're working with marketers is that we really try to help them understand what's going to kind of happen when they bring these reports out to the rest of the organization is that certain areas of the business, CEOs, finance, often What they're really good at, because they're inundated with data and reporting, what they figure out how to do very quickly is look for anomalies in reporting and data because that's where they really go focus, right? So, you know, but my own team will tell you that the worst thing is I'm sitting here looking at a report going like, well, how come that number and that number don't sum to that number? Because they're supposed to.

Right.

Right. And then the teams are like, I don't know. And You got to be prepared for that. Sometimes the answer from a marketer is as simple as like, listen, these numbers are not supposed to equal this number because I need to be able to tell that story.

Yep.

And be credible about it. But they need to know that that question is going to come because the worst thing is like, well, I don't know, I got to go, we'll go look into that. But then the rest of the things you say about the report, you know, are suspect because now people don't trust your data.

Yeah, your credibility.

So sometimes this data, especially marketing data, won't sum. And you want to call that out. These numbers are indicative and they're directional. They're not going to sum because they're coming from different locations and different data sources, and they're not meant to sum to always equal the number sitting over here in Salesforce. But here's what they do tell us, and here's the story about what this information is telling you.

Yeah, I find it, you know, it used to be, and maybe this is another thing I think people could learn from, it used to really, I used to take that kind of pushback or challenge personally. Right. And I think people need to realize like, this is not, these questions are reasonable when people are asking, unless, I mean, there are some people who are assholes, right. But in general, like they're just trying to understand. And so like, that's another reason.

Yeah. They're trying to understand what's going on.

Yeah. Um, it's, it's, um, okay. So let's, so one of the things that I think this does, right, is this starts to expose issues. Sometimes the issue with the data is, hey, in fact, I have a number of stories where I started doing reporting, whether it was Salesforce or database or the marketing automation system that showed there's some weirdness in the way that the sales team is operating, right?

Yeah.

And I don't want to get into details that led to reports. In this case, the ones I'm thinking of are attribution ones mostly. And I go, Like this doesn't make sense to me, but it led me to go not question, it didn't question the data. Like I wanted to understand why. So are you, I mean, is that like, to me, again, this is like exposing this data through reporting, right? Gives you an opportunity to go fix upstream problems, whether it's, I mean, if it's marketing, right? Are you always putting UTM codes on your, you know, the links in your ads? Or do you understand when sales is updating opportunities, right? Things like that. And why? Are you, like, are you, do you see that as something that when you're with CaliberMind, your customers, it sounds like you help them go through some of this process. Is that something you do as well?

Yeah, one of the things when we started building out our application, you know, we were coming at it from an assumption that the customer's data needed help, right? And I've seen a lot of other products out there in the market. And what they ask is they ask their customers to fix the data, then put it into the system, and then they'll have accurate reporting. And we decided to kind of take a different approach. I was like, listen, your data is probably going to be kind of messy. So how do we get you to good, accurate reporting when that's the case, right? So we built rules engines and data manipulation, deduplication, all this stuff right into our system so that we can actually build the rules that really help our marketers get to good reporting. And it may not be that their data is messy because of a, you know, a flaw in their process or anything. Imagine a situation where you were on one map, Eloqua, and you migrated to Marketo.

Yeah.

Happens all the time, or vice versa, right?

Sure.

The data from Eloqua looks different, and maybe you had a different way of naming channels that just changed. migrated, you've matured, and you name channels different, but you now want to run reporting that looks at sort of how things have changed over time. But all these names and things have changed, right?

Yes.

So you need a way of being able to say, I'm looking at that data, but I want to map all these channels to this new naming so I can now run reporting that looks like a continuum. The data doesn't actually match this, but you want to look at this continuum of what's really— like these channels have gotten better over time. But back here we named it like this, we managed it like this, and then it's changed here and changed here. If your system can't handle that type of transformation of data, you're going to end up with gaps in there. And what's going to happen is those data problems, those mismatches are going to start to surface up into your reporting because suddenly your channels aren't going to look like a consistent level of— they're not going to be the channels you're reporting on today. Suddenly you're going to see all these other channels in your report that came from previous And it's just gonna look like a kind of a jumble of data.

Yeah, it's gonna look like, it's like a bunch of noise when actually, if it was kind of refactored to be consistent, you would actually see something more useful.

Okay. Yeah, it's a great way to put it. Like, how do you refactor on the fly, right? And so, your system should be able to handle that if you wanna get to some accurate level of reporting and information.

So is it, I mean, this sounds like, I hate to call it lightweight, but it's a type of ETL kind of capability. Is that, that's built into CaliberMind? Is that what you're saying?

Yeah, it's ELT, right? So we extract, we load it, and then we transform the data.

Oh, okay. ELT, okay.

So that's kind of the difference between ETL and ELT. We're getting, we're basically taking all that raw data, putting into our system for our customers, and then we move it through these processes to actually help refine and fix and whatever. So that now you have data where the data is kind of all stitched together in a way that you can kind of report on, but you need to have rules engines in there, right? And we take it a step further where we'll actually allow our system to take some of those rules we've put in, deduplication, data normalization, things like that, and push that back out to these systems. So we'll say a great example is titles. We actually did an analysis where we took 1,800 titles And we threw them in and we said, how many different titles are in this batch of 1,800? And it was something like 80% of those titles are actually different, whether it's VP or vice president or V.P. Like it was just different.

Yeah.

So how do you take that and then create a segment where I'm like, I want all VPs of this, right? I want all executives of this, right? And so we have a rules engine that actually does that inside of our system. And what we found was that, and then I could report, I could have a dropdown of show me VPs of marketing. The rules engine is extracting sort of the level and the role in the department.

Normalizing, right? Normalizing, right?

And everyone was like, can we put that back into Salesforce so that I can do my Salesforce reporting on these same things? And that's where a tool that has these normalizations and these people that can potentially write this data back out to the source systems. Fix doing lead-to-account matching, doing account deduplication, things like that.

So do you think of that as— because we take— going back a little bit, we talked about like exposing data gives you opportunity to fix it. Is this like a part of that built into the tech ecosystem by doing that? Is that kind of what you're describing then?

Or at least optionally? Optionally, right. We found a lot of customers had a variety of tools to do this. Like, so they may have purchased a tool to do X fixing and this right here. And, and, but the challenge was that not everybody had it. The level of maturity of using those tools was, was highly variable across the customers.

Sure.

And so we said, we really need to kind of build this into our model directly. And so that's kind of really how we approached it, which was that we needed to have that capability. And Because we approached this as a— starting as a marketing data warehouse with things kind of built from it, our architecture lended itself to being able to create rules and do things to kind of get to this data, you know, and then kind of after the fact, we found everybody's like, we want you to take what you've done here and send it out. Like, fix these other systems too. That was sort of a follow-on that came after they saw what we were doing internally with all the data.

All right, this is maybe a little bit Do you— when you do that pushback, do you typically overwrite what was there, or do you typically find that they say push the cleaned up data into another field? And like—

It depends. Like, it depends on the customer, right? So usually what we do is we'll actually create reports that say, for instance, we're going to do lead-to-account matching, right?

Yeah.

We'll actually create a report that says, here's all the leads and here's all the accounts that we're going to do. Validate that. Like, so we'll do these steps where we actually don't actually run the final step and execute it. We kind of give them like, here's what our system thinks it's going to do. And we run through a bunch of steps to kind of validate that with customers, right?

Okay.

And then that will— and then once it's validated, we'll actually kind of turn it on. Other ones we'll actually write into, like our data normalization, we actually write them into probably new fields for customers. Right. So here's the, here's the CaliberMind title. Like, we actually take the titles and the first thing we do is we actually normalize them into a standardized format. Everything that says vice president or VP, like, they all get— then we run through a process that says now extract out kind of the level and the department, and then we'll write those into new fields potentially, right?

Yeah, yeah. Okay. I mean, I know that's— that would be my preference if I had had the decision on that. Like, I just— I don't like overwriting what's There, um, particularly if someone—

we do that, um, but for lead-to-account matching and deduplication, you'd actually modify the—

Yeah, yeah, yeah, that— I mean, that makes sense, right? I mean, those are 2 different, pretty significantly different things. But like, I can imagine with title, right, um, either, you know, salesperson got a business card— if anybody gets any more— they put in the person's title that's actually what they say they are. Yeah. And then we go and overwrite that as something different, and then we use that to Send it. We usually personalize email.

Yeah.

Yeah. Um, that makes sense to me. Yeah. Okay. So, um, the, the, the rules stuff, well, maybe we come back to that. Uh, there's a piece of this though that I still think we haven't really touched on that I'm, for longtime listeners, viewers would know for me, like I'm a big believer that storytelling, this, this quest to be data-driven. And you use that term for some of the other teams like finance or even sales.

Yeah.

I feel like many marketing teams, marketers, CMOs, that they've, with their quest to be data-driven or seen as data-driven, that they've given up on the idea of storytelling because I think there's still a component. I think there's a component of this, like, again, maybe it's because I believe that the data is just not going to be complete, accurate, high quality, whatever, that you need something else to stitch this— stitch it together that helps bridge that, right? Yeah. Uh, to make it believable. Like, are you seeing the same thing? And like, are you seeing it changing now? Or is there sort of a shift back to that that you're seeing? Um, just curious.

Yeah, no, we're seeing a lot going on kind of in that space. I'll say what we saw was First, marketers really, you know, years and years ago didn't report on a lot of data. And then as things moved toward digital websites and things, a lot— they would start to expose a lot more of that sort of raw data. Here's how many website hits we got.

Yes. Yep.

You know, okay. And at one point in time, there was a relatively strong correlation between website hits and downstream leads and things. But then that correlation really kind of fell away over time. So then they said, look, okay, I got these, but these now became MQLs, which people filled out form fields and things.

Or their lead scoring became an MQL.

Or their lead scoring.

Warm lead versus hot lead. Yeah. Okay.

And they kind of tended to move, gravitate towards just kind of reporting those raw numbers. Right. And I think what's happened over the last X years is that the rest of the C-suite, the rest of the organization, has started to believe less and less in those raw numbers and how does that impact the business, right? And that's kind of where I'll say the state of the business is right now.

Yeah.

And then you have compounding that, more and more systems pumping in more. I'm not just using the website, I've also got outreach firing and I've got, you know, emails going from my marketing. I mean, so more and more data, right?

Yep.

It's crazy to get lost in it. A lot of these teams brought in You know, to be honest, like data science, marketing analysts, people like that, those people are really good at analyzing the data.

Yes.

And they would say this data means boom. But that translation from the data to that, what the analyst is saying, to a story that the marketer, the CMO, or can tell to the rest of the organization, I'd say that's where the gap really is.

Hmm.

Yeah. And it's hard, right? Because you can see it like, you know, I've got a data analyst software background. But I've been in sales and marketing almost my whole career, right? And so even for me, it's really hard often to bridge the gap between the data and the story. And we've seen customers now start to really migrate to where they're really more interested in telling the story than the data because they feel that resonates better. I'll give you an example. We have a customer and we were providing them these buyer journeys. So here's accounts that you won.

Yep.

And then we were showing them, here's this giant— here's a buyer journey of all the things that happened for this account and why you kind of won it. They started turning that into literally a, they would pick out select accounts they want each quarter and they would create this graphic, an infographic of like the whole buyer journey literally on the site, you know, like thing like that.

Like Candyland.

Almost exactly like Candyland. And on it they would put like, you know, at this point in time, here's the roles and they interacted with content and then these things happen and then these things happen and these things happen. And then they would use that and give that to sales. And sales teams would then use this as sort of a playbook. Where is my account on this pathway? Like, what have they done? What have they not done? Right?

Yeah, I love that.

And that made us kind of step back and think like, well, how can I do that kind of in an automated fashion?

Right.

Right. And that's, you know, how do I tell, how do I take data and turn it into a story? And I'll be honest, that's when ChatGPT and Gemini and Claude all started to hit the market. And we realized, you know, along with others that Those systems can't do math. Like, don't even try. Like, we've tried. I'm not going to run an attribution model through Gemini. That's in the future. But can it take a buyer journey with 1,000 touchpoints and turn it into a credible story that a marketer can then pass to an SDR or BDR?

Yeah.

That's where it's really good. And that's— and what I'm seeing now is that these systems are helping bridge the gap between the data and the story. And that's where they're really powerful.

Yeah. I mean, what you just described, like, I think I shared with you like this, and it's really where my view, because I'm like you, I'm a, I'm an engineer by training, never really practiced it. So my, like, when I come into this, I think about data and I understand it sometimes too deeply. Right. And it's easy to go down a rabbit hole, but what hit me is when, and this is, I had marketing ops and I had an inbound team. And I started having my inbound team track leads that were handed off because I want— I was— I don't even really know what triggered me to do that. It's just sort of intuitively like, this would be a good thing to know how that resonated.

Right.

And it wasn't that I stopped doing reporting otherwise, but what I found was if I, like, if I just watched the body language in the rooms when I presented back to sales on what we'd been doing, the numbers were fine. Nobody like really, I didn't, I rarely got challenged on it. It was so, it was very new at this particular organization. Think about it. But when, like, I saw people leaning in, right? When I'd start, we started doing the stories about we got this deal and this is what happened. And, you know, um, we got it. This is what we did. We quickly got it to the right person. Yeah. Da da da da da da da. Right? And it became, I realized, like, that piece of it. was building more credibility than the raw numbers, especially when the number— the problem with the raw numbers is not the raw numbers, it's that they have no context.

Yeah.

Is it good? Is it bad? Is it like, how do we know? Right. What do our competitors— like the question, like what's happening with our competitors? Right.

Yeah.

Yeah.

I went up 100% or 200%. Is that from because I went from 1 to 2 or because I went from 1,000 to, you know, whatever. Right.

Or is it a technology error? Like, is it a bug? Right. I mean, that would be the more common question, I think, because they don't, they weren't trusting it. But yeah, I mean, website visits, like, I mean, it's been the blessing and the curse of all this digital stuff is that you get, you can quickly get insights.

Yeah.

But the downside is you get drowning in data and it's like, how do you, how do you figure out which ones are important and can help you be like inform What you do the same or different, or— that's the point, you know.

Are you going to change tactics? That actually brings up an interesting, um, example we just had from another one of our customers where they were seeing a shift, a drop in their organic search traffic, you know. And this was— and they saw this because of attribution, right? You know.

Sure.

And again, attribution plays a good role. It's not the end-all be-all, but it does give you indications of directionality and things and stuff that like stuff's going up, stuff's going down. And they saw this and they came back and said, well, why? They started doing investigations into that data, right? And what they found was that information, they were shifting to direct search from LLMs. And so then what they wanted to know was like, what LLMs?

Yep.

And so we actually ran a report for them that's super interesting. We actually have a report for that customer that shows what this increase in traffic coming from LLMs by LLM, right? And probably no surprise, it was primarily ChatGPT and Google for this type of company, right? And that then, so the answer is like, so I saw some change in data. I went and investigated. I found that something really had happened and we're getting a change. And then what they did is they started looking at how do I optimize content and things on my websites to help those engines do a better job of getting our message out, right? So then at the end of the day, they changed tactics. And then because they had good reporting, they can come back and see like, is that working? Is that increasing? Is it continuing to increase? You know, and then by understanding which engines, they could kind of go, okay, so I really need to optimize for these 2. I don't really need to worry about these.

It's so interesting. I was sitting here smiling because this is not the first time I've heard somebody give that same story that they saw it. They've seen a drop in organic search and a pickup in, I'll call it LLM-based traffic. And actually earlier today I was working on something and I have I have definitely— I've not totally gone over to just using LLMs for search, but I— this is a total tangent, but I have found that probably more often than not I am using them because typically when I'm doing something where I'm going out and searching, unless it's like a clear, pretty clear fact kind of thing.

Yeah.

You know, I'm typically— it's like a— like I've got a longer question to ask or I need to give it more context. I'm finding that I get much better results. You're right, though, not so good on math.

We're still, I mean, you know, in the background, we're constantly running experiments with these systems. And what we found is that when we need the LLM to do math, what we do is we tell it what the data is and what we want the output to be like, and to write a Python program that does that and then run the program. Does that pretty well. So, it's like, I wanna do a projection, I wanna do a quick predictive analysis of X, Y, and Z. You can't ask it and give it the data to do that, but you can say, here's the type of data, write a program in Python that does that, and then just run the program and see the output. And that actually works pretty well.

That's really interesting. Yeah. And I've heard it does a really good job of generating code.

Yeah. We just did a training internal here for what we— all hands is on Thursdays. We just did an internal training. And we actually did a training by business unit internally on how they're using LLMs in different ways. And what we found some really interesting use cases. Sales is of course using it to do research on companies and things like that before they do outreach. Very, very typical. But our customer success organization is doing it to write responses to customers as they come in. They're using it a lot for doing data debug. So you've got a whole managed services team.

Sure.

And they'll have a customer come in and be like, I'm seeing something weird, blah, blah, blah. And they'll actually have our— have the LLM look at the data and compare it to other things and highlight anomalies. So they— so it helps them figure out where to go faster.

Oh, okay. Yeah, that makes— that's, that's actually really, really good. Uh, it just occurred to me because I, somewhere along in my career, I, I helped a company, um, come up with an approach to how to, to enable the organization to do more customer support without growing Yeah, their staff at the same rate was typically, it was in parallel with the revenue growth. And it came down to a methodology that involved a knowledge base and, you know, getting that and made it, exposing it quickly. I could imagine that LLM would be a really good thing to put on top of it.

There are amazing tools for some of this stuff, right? Where you can, we can, we're actually building it directly into our configuration stuff. So a customer can come in and say like, How is my configuration different than standard? What have I done to this? How do I set up rules in your system? Because maybe I'm seeing something weird, and, you know, these systems get complicated.

Sure.

How— what have we done to it? What does it look like? By the way, I'm having this type of issue. And, oh, well, we're noticing that in this section, you're mapping your account object from this field, but it probably should be this field. You know, and so we're seeing really big usage in that area.

So exposing the butterfly effect, right?

Yeah.

Interesting. So I just want— so the— this, this, um, AI— I think it's AI-based, this customer journey dream mapping stuff that's built into it, and it like, it actually generates a visual out of the tool. Is that what it does?

Um, we can generate a visual right out of a reporting. What we're actually doing for a lot of customers now is We'll go in and we'll set up a rule in our system that says like, I don't know, every time an MQL, you MQL, you create a lead, right? Or every time an account gets to this level of engagement, go look at their buyer journey, right? And that buyer journey could be 10, 100, 1,000 touchpoints depending on your organization.

Sure.

And now figure out who the most impactful people are in there, who are the core buyers, who are the core people you're interacting with.

Right.

What are the core types of content and marketing events and things like that they've been interested in, they've done? And then give me a quick summary of the timeline of things that happened, right? And we'll actually push that right into, for instance, a field right on the lead in Salesforce.

So that's— Yeah.

We're also using it to say a lot of our customers, um, you know, they've got, you know, Apollo or Outreach or Salesloft or something like that, you know, plugged into their sales system. So all this email content is in there.

Yeah.

So we'll be seeing all that and we'll say, by the way, pick out the, like, are there any next steps that I should be aware of? It'll actually look at that stuff and say, and I'm going to say like, oh, by the way, they've asked for a demo, make sure you do this. This person asked for a security review, blah, blah, blah. And we actually push all that into Salesforce or we make it available right in our app for customers.

That's awesome.

Okay.

Yeah, I love that. Okay. So I don't want our listeners to walk away from this thinking that we should stop trying to be data-driven or data-informed is the way I like to think of it. And that like marketing shouldn't do any more reporting. Like, so I'm just curious, like, you know, I think marketing still should be held accountable and have metrics of some sort. Like, do you have any thoughts on, especially in your role as a CEO, like what do you look for from a marketing team or what are you hearing your customers and clients say? Like, this is what we want marketing to be reporting on.

Yeah, you know, I'd say that more and more marketing is being held to the same— to a pipeline number. Now, whether it's closed won or created is different, but I'd say that things are shifting to where the organization is really holding marketing accountable for metrics that tie directly to the business that are measurable.

Okay.

Revenue.

Like marketing sourced versus—

Marketing sourced revenue. Yeah, marketing source, right, or marketing influencer, right, depending on where you're looking. And so we're seeing that a lot. Now, the challenge for marketers is that that may be the metric they're being held to from the organization, they need to tie things to, but they really do need to look at a lot of other metrics and indicators to be able to get to influencing that number.

Right.

Right, so just knowing how much you do, but, you know, again, website hits, May not be as impactful if it was, but if your website hits are dropping, you want to know that.

Yes. Right?

If they're going up, where are they coming from? What's my organic search volume look like? What is, you know, so you really do need to put, I would say you instrument your funnel and at your different stages, you're really looking at different metrics that impact that, right? At the top of the funnel, it's really, am I gaining awareness? Well, how do I know that?

Yeah.

Boom, boom, boom, right? Am I showing up on, you know, if you're a software company, am I showing up on G2? How many website hits? What, you know, how many MQLs or MQAs am I creating?

Okay.

What are the metrics? Usually those are more engagement metrics that it's like, are people downloading content? Are they looking at things on the website? Are they responding to outbound emails? Are they— so you look at those different metrics, but it's really driven by stage to the point where the metric that you're probably going to try to tie your story to is the pipeline metric.

Yeah, I mean, I've— the way I think about it is, um, I'll skip the lowest level. Lowest level to me is like, is the system working as expected, right? Data consistent, complete, um, yada yada, uh, which really doesn't go to anybody typically unless it identifies there's a sync problem with another system or something.

Sure.

Um, but what I think a lot of marketers miss in this goal to kind of go to the storytelling or even attribution or contribution, where they really have impact is on the day-to-day activities they're doing, right?

Yeah.

So, which they historically would've, you know, we published X number of emails, we did a number of blog posts, we did this many ads, you know, and I think people have gone like, oh, that's seen as vanity metrics, but I still think there's value. And it's about like, to me, you brought up website visits, I agree. Like, You should be watching that. Now, do you need to report that to the C-suite? Probably not, right? Do you need to make, like, use it as a barometer? Like, oh, hey, there's a problem that we need to get in front of because 3 months, 6 months, 9 months, whatever it is down the road, like, this could have an impact.

Yeah.

And if you're doing a marketing tactic, an email ad campaign, maybe it's, you know, using multiple channels, Yeah. You should, you should be monitoring that in the early stages, like a hawk to go like, is it working? Because if it's not, you need to cut it, adjust, whatever. And I see a lot of teams, um, they're so, they're moving so fast that they're not doing that, that stuff on the tactical stuff, because I think they've been told like, it's just vanity stuff. But to me, part of it is that Who they're, who the recipient of that reporting matters. And to me, like the gap is that yes, you should be doing that and you should have expectations about it. Now, again, it's like, do you share that with everybody? No. Do you need to pay attention if you're the demand gen person? Absolutely.

Yeah. Yeah, absolutely. I mean, you know, if you're the CMO, you want to know how you're probably reporting on here's my, here's my core channels that we work with and their efficiency. And if they're— and how much— and are they working? And how we're investing in these channels to increase. And, you know, because usually the CMO talks a little bit about what happened.

Yep.

And a lot about what they're doing for the next quarter. How am I going to help get to X? Well, here's the things that we're doing here. I got these channels and I'm making some changes. We've noticed that we're adding here, but that's a super high-level report, right? You don't run your business off that. Right. You were like, cause underneath that you're like, okay, inside this channel are all these tactics and things and stuff. And you need to know what's happening there so you can actually have an effect on that channel.

Yeah. Right. I mean, and that's to me, like when you start getting into the attribution or contribution thing, that's where you start exposing stuff to others. But this is what I used to be a really big, huge fan of attribution. And I've really cooled on that. I don't think it should stop because I think it can be, it's useful to identify What channels, what audiences, what types of tactics are most effective and or least effective so that it can inform how you, you know, what bets you place, right?

Yeah, absolutely.

To some degree, it's a bit of a—

Yeah, it's a tactic. It tells you interesting things from the data that can really help you with your business. But as a marketer, that shouldn't be the only thing you're doing. You should have some level of engagement reporting, isn't it? Because Engagement tells you something different, and engagement can often give you signals earlier than attribution. Attribution is great, but the problem is that it's kind of a retroactive look at it.

Absolutely.

It's almost, you know, boom. But it gives you— without attribution, it's hard to create that final story that goes to the CMOs because that's how you tie this stuff back to revenue. That's something that you need to do. But prior to that, I'm kicking off all these tactics. They may not, depending on your sales cycle, they may not end up showing up in attribution for 30 days, 45, 60 days, right?

Well, I have an example of a company I worked for had a deal that closed 2 years after the first interaction, which was at an event.

Yeah, yeah, right. But you want to know if those tactics are generating those early metrics, right? Are they I'm running an event. Did I get enough meetings? Like, did we get real meetings out of it?

Yep.

Right. You know, and then if I have meetings, did those meetings have a follow-on? And okay, now I'm seeing— because if those early things are no, I'm not seeing any of that, you're never going to see it in attribution.

Right.

You want to know that early because you're like, okay, this isn't working. I want to kill this and start something else because these early metrics— and get the early metrics right, then they'll start to show up in attribution later on.

Well, I mean, just like an event's a good one, right? If the goal is to get meetings, right, that should be what you're focused on. And rather than say, like attribution might come back and say, we shouldn't do that event again.

Yeah.

What it may be telling you though, is that the way you, you were present at that event was wrong. You didn't have the right people in the booth. You didn't have like, you know, that kind of stuff. And so—

You didn't prep for it and have people try to set up meetings before they got there or whatever it was.

Yeah. Yeah. So I think that's why you need not just one, right? There's a basket of things anyway. Um, okay. We've covered a ton of ground in a shorter period of time, but is there anything else that we haven't covered that we wanted to, you think we wanted to share with our audience?

No, other than I think that, um, there's a lot of that. What's happening with AI is causing a huge amount of, um, I wouldn't say concern, but confusion in the market.

Yeah.

What's real? What's not real? What can I do with it? What can't I do with it? Right? And then you also have, this is almost a completely different topic, but you also have marketing teams that are running, they wanted to leverage these technologies, they're running smack into their own legal departments. You can't touch this stuff because boom, boom, boom. Right? So I think the only thing that's out there is that I'd like to say is that the AI stuff is real. It has some very strong power in certain areas. You need to make sure that you're focusing in the right areas, right? And the organization— what we're seeing is that the organizations that are starting to leverage it early are starting to gain advantage over organizations that don't. Their teams are getting more efficient. They're getting faster. Like if I have an SDR that can cover 20% more accounts per weak, that's an advantage, right? I either have to spend less on SDRs or I get more out, you know, whatever that is. And so companies really need to start thinking about that stuff now, but they also need to have a very critical eye on like, what can it do? Where is it good? And really pick projects that really nail where it can have the most impact on the business.

Yeah, I mean, I think we've got somebody potentially coming on too in the near future. We're going to be sort of talking about the difference between what is like AI, machine learning, automation. Like these are all terms are out there, but they sort of have different meanings, right? But they get conflated a lot. So I think that's— and I know like from my own journey, like I was a relatively slow adopter, but now I'm like, I'm finding daily uses for it, you know, in different ways. So, Eric, lots of fun. If folks want to— hear more about you, what you're doing, what's going on at CaliberMind, what's the way for them to do that?

Well, check out our website, www.calibermind.com. You can reach out to me directly. I'm on LinkedIn. I think I'm /ericwesterkamp on LinkedIn. Reach out to me directly. It's probably the 2 best ways to get information on what we're doing.

I appreciate it. Well, Eric, again, thank you. It's been a lot of fun. I get excited about this stuff, so hard for me to control myself. Thanks again to our audience for always continuing to support us. If you have suggestions for topics or guests or wanna be a guest, feel free to reach out to Naomi, Mike, or me in the usual channels. Till next time, bye everybody. Bye.

Michael, thank you so much for having me.

Absolutely.