The Data Disconnect: Restoring Common Sense and Context to Marketing Analytics

Hi, I'm Michael Hartman.

I'm Naomi Liu.

And I'm Mike Rizzo. And this is OpsCast, a podcast for marketing ops pros and rev ops pros created by the MoPros, the number one community for marketing operations professionals.

Tune into each episode as we chat with real professionals to help elevate you and your marketing operations career.

Hello everyone. Welcome to another episode of OpsCast brought to you by marketingops.com and powered by all the MoPros. I'm your host, Michael Hartmann. Today, flying solo, Mike and Naomi are off doing whatever they're doing. Today, I'm joined by 2 guests, both from CaliberMind, who live at the intersection of data marketing and business strategy. First, Nadia Davis, VP of Marketing, and Misha Selkander, VP of Technical Delivery. Nadia and Misha have been vocal about something many of us in marketing ops have felt for a while, that we've gone from being data-driven to being data-drowned. A sentiment I agree with. We'll talk about what's been lost in the process, how to bring context and explainability back to analytics, and how teams can focus on the fundamentals that truly move the needle. So, Nadia and Misha, welcome to the show. Thanks for joining.

Thanks for having us.

Yeah, looking forward to it.

Yeah, it'll be fun. I feel like I have this kind of conversation too often these days, but let's get started, Nadia. I think when we talked before, you mentioned that marketing is one of the most, as a dynamic, diverse function, Agreed. I think we've had several guests on talk about that. You know, it's an unusual combination of creative, technical, but also kind of accountable to numbers. So, how do you see that complexity creating challenges in how data is used and interpreted for marketing teams?

It's actually a packed question. We just got back from B2B Marketing Forum, B2B Marketing Profs Forum in Boston. And this very topic was front and center, right? And it's true that under marketing leadership, you have the full gamut of creative, you know, creative beginnings. You have people in comms, you have writers, you have graphic designers, you have people who enjoy campaigns and like that puzzle-solving creativity, right? And then you have people who are more down the lines of STEM and math and measurement. and they really enjoy bringing data together, you know, graphs, charts, everything. And then you have the technologists bringing all of the marketing tools together. Let me tell you, the martech community, like I've always been on the outside of it, them selling to me. Now I'm on the inside of it, you know, sharing with others what we got. There is a lot of noise going on there, right? Well, a lot of noise. And sometimes even vendors don't know whether, you know, the sales teams within vendors don't You don't know what it is that they got and whether what they're selling will match the needs of the client on the other side. So, I don't think, you know, they're selling you something knowing that you don't need it. They truly believe it would work. And maybe the client is not sophisticated or savvy enough on the data side of things to understand how it would all come together, right? So, the challenge becomes you have all of these different marketing minds. Some are more data savvy, some have more data affinities. While others are, they're really, you know, their talent is not in the numbers. Their talent is eliciting certain emotions, engaging people within the prospect pool, right? And they're the ones starting the journey of delivering something to the client. So, your brand is memorable. So, you know, you drive recall and then the other people measure it. But everybody's held accountable to some kind of a metric. Coming up with a metric is a challenge in itself. A lot of us have, you know, investors and VCs and Ps behind us, and I understand those people too, because they, when they give you money and we are the spending function, they wanna make sure the money's working really hard, every single dollar. Yeah.

It's a reasonable request, right?

You invest in your 401, do you wanna ask Vanguard what return you got this quarter? Is it up or down, right? So, like, you kind of get it, but I think all of this combined compounds to the notion that there is this a little bit of running in place, trying to measure everything while being able to measure nothing. And this confusion and the noise that pushes you to go fast while really keeping you running in place, because you don't even know if you can measure it all within the setup that you have.

Yeah. And I think there's some of it is also driven by the need for seeing short, measuring short-term impacts versus long-term impacts.

Oh, that could be another podcast.

Yeah, I know. I know. Okay. I won't do, I won't go there. So, so what I think one of the other challenges, and Misha, you talked about it, you said you see a lot of times that there are smart people out there doing advanced analytics even, right? Let alone basic ones, but they're sort of disconnected from business impacts. It's just like, first, maybe break down what you mean by that, but like, why do you think that's happening? And maybe more, are you seeing it more today than in the past?

Yeah, you know, I think it's like with many things in life, it's a question of incentives and for whom and who's involved and which teams and We're seeing more and more, particularly with this notion of, yeah, we know analytics is important, we want to make data-driven decisions, and that's all great. But everybody wants to show that their decisions are the correct ones, that their metrics are— that what they did worked, that it has returns. So, right, I mean, there's a question, right, of incentive in getting the next budgetary cycle.

Yeah.

Why this exists kind of financially, you know, I can probably speak to that, but there is some really unexpected consequences from this. And there's some really funny examples, you know, one that just came up the other day, an organization says, well, we have a function that does outbound interactions, right? So something like—

Sure.

BDRs, and they need to be part of an equation. of attribution so we can figure out how, what returns we can get from this team, what returns we're getting from this team. But what happens is the inclusion of outbound touches in an attribution model can be really adverse, right? You say, well, the more emails we send, the higher the attribution for this channel will become, right?

Right.

So we remove a little bit of the kind of the common sense of what you're introducing in the model because of the incentive To, right? So to show returns for this one team. So it's funny how these things happen. And I guess what I meant by this is we need to take a step back and say, does this make sense? Or are we just like trying to, you know, fill in all the needs of all these teams, ignoring what the data tells us piece?

Yeah. I mean, I've seen, I think I've seen examples in other kind of variations of similar thing, right? You put in place a metric. that you think is going to be beneficial, but people figure out how to game the system a bit, right? So I think as things like customer support, where I've done some work in the past where you're like, the goal is to close cases quickly. So cases get closed, but they don't get resolved. Like, they actually don't resolve the problem, right?

Yeah.

So it's like, it's interesting. And there's obvious ones in our domain, right? Where sales teams, like, I've seen this before. Marketing generates a lead, hands it off, sales declines it, then goes opens a new opportunity, right? I'm sure there's all kinds of variations of this kind of stuff so that you get control over it. So you mentioned common sense though, Misha. And so like, what do you mean by that? I mean, in my head, like one of the things I remember distinctly, like I'm a numbers person, I'm an engineer by training, and I remember talking to a CMO at a really large organization who happened to be a friend. years ago in the early days when like search marketing was a new thing and you're like, I was like, so much data, we can do all this great stuff. And she's like, yeah, but they can't measure everything and I don't want to like just go by the numbers, right? Are you talking about that difference or is there something else you mean by that?

Yeah, intuition certainly plays a big part in this. There's no doubt. I think, you know, having looked at, spent most of my days looking at different types of models and what makes sense. And I will probably give nowadays as much weight to an explainable model as I will to a very accurate one. Because if it's a very black box model, and yet you can't explain it to the end user in your organization, trust me, this account is very engaged. But, but, but they just visited the same, you know, careers page 60 times. Well, you know, so like, like, you have to, we, you know, we all have to Take some time to think about, does this even make sense in the model? And then there's also the piece of like, what business question are you trying to answer with this one metric? And sometimes, I know we like to have very unified KPIs, number of MQAs or something like that. And I've just, come across these situations where the answer to how many of a certain type of activity or a certain signal exists could be one question, but the what happened in between these signals, what actually helps me move from this level of engagement to this one, is a totally different type of question, right? So for the former, you might want to count the behavior every time it takes place. But for the latter, you might say, you know what? No, if it happened one time, even if it continues to happen, let's consider that they're already in this stage. They already showed this level of engagement. Let's not give this signal over and over again, right? So, and I've seen, I see organizations say, oh no, no, we have to have one definition. And like, and so we're gonna try to fit every type of measurement into this one, into this one entity. And that it, it just simply doesn't work. So, Um, I guess maybe that's a little bit of common sense is, yeah, it's, it's still data-driven, but, uh, I need to think about what it means. What is this data telling us?

I wish I would— go ahead.

What comes to mind when Misha was saying that, you know, there's this saying, if you torture data long enough, it will tell you exactly what you want to hear. So that makes sense, right?

Yeah. My, my version, my version, which is maybe more on the spin side, is the, the Mark Twain quote, there are lies, damn lies, and statistics.

Yeah, yeah, yeah. That's another side of it. Yeah.

No, I think there's a lot of that that I see too, that where, well, in the pursuit of being data-driven, right? People kind of like, they feel like they have to bring every, like anything that's possible, right? If I go back to my early days in marketing, it was in database marketing. And one of the things I built up in the early days, built a big consumer marketing database and we had third-party data. And it was like, when we go to the vendor for the data, it was like, here's the whatever, hundreds of fields that we could bring in and it was really hard to choose. So what the default is like, we'll just bring everything in, even if we don't ever actually use it, which means the costs go, like, it feels like we're at that times some multiple, right? And it's easy to get caught up in all the data that's out there, but not actually thinking about how is it, how can you use it?

Yeah, it's almost like, I feel like the, on any dashboard or on any You know, we have sets of widgets. I would almost nowadays recommend to have the business question as the title of, or that specific widget. So it, it, you, you don't like decouple this. You don't just have MQAs. You have a chart of MQAs over time. Well, why? Who, who does this matter for? I mean, sure, there is an, you know, a C-level. Sure, you might have a C-suite, you know, dashboard, but tactically, like, what are you gonna do differently about this? Maybe the, maybe the question is, which campaigns last month were more effective in driving MQAs? And so you have a very specific distribution of campaign touches focused on very specific touches, right? And, and, but the, the, but the behavioral action is much, it's much closer to the end user. Maybe takes away the noise.

Yeah. Now, you know, it's like you were about to say something. Yeah.

Yeah. The other side of that, and actually this is not my piece of wisdom. This is from Evan Hanscott, who is our head of data science. He presents kind of the idea of common sense this way. He goes, when you get a stat, right? Something that you derive from your data, if there's nothing that you can do with it after that, once you got it, meaning you cannot answer the question, and so what? What do I do with this? What's my next step that makes an improvement within the business? Then that's not common sense. You just got a stat. You just did a math sentence and got an answer, right? So, in business, it has to make sense, meaning it informs you of either what the current situation is so you can make a decision of what to do next, or it helps you pick the next step and where you're moving forward. So, I think that's the, you know, Misha was talking about the front end of it, kind of what questions are we asking and what are we presenting in front of our users? And then the back end of this is like, once they see that, Yeah. Do they have enough to make a decision, or is the data the way it's shown, you know, gives them enough to make that decision?

Yeah, no, I can think of many times I've seen that too. Mishi, you mentioned the word explainability versus sort of the black box. Like, could you break that down a little bit more? Like, what do you mean by that?

You can have Different. So there are several vendors in this space that attempt to rank kind of engagement levels or whether first-party or third-party levels of organizations. The simply ranking or simply saying this organization has 17 hot peppers of engagement, And this one has 14 hot peppers. Certainly useful for a sales rep or for whoever's gonna take the next action on this account. But without speaking as to why or without showing the why, one is it's really hard to take the next action 'cause there's no context.

Right.

But also, as soon as something falls apart, as soon as you call an organization and they say, well, we, I have no idea who, who are you? Where, where are you calling from? Like, as soon as that happens one time, right? and there is no explainable element behind it, it kind of all falls apart. And so, you know, this can be applied to an attribution model as well. If you're a campaign manager and you say, well, your campaign is doing, you know, doing this well and last month it did worse, so you're doing well. How, right? How is this being calculated? Well, it's actually pretty straightforward where, you know, we're doing this model. But if you say, well, look, we have this really sophisticated model, it's absolutely accurate. We all need evidence, and I think sometimes the simpler the evidence, the better.

Yeah, I think that's right. I think people want to— it kind of falls into this, like the trap that people get into. Was it the Pareto principle, the 80/20 rule, right? If you get a model that's 80% accurate, predictive, whatever it is you're trying to get out of, and it's easy to explain, It's probably better than when it's 100% predictable but impossible to explain.

And I would love to add here that the one thing that we all have at our disposal now that prevents, in a lot of cases, that use case of explainability is the AI analytics, right? Where just trust me because the model within AI, the black box within AI came up with this number. Great. Can I click into that number? That takes us to data lineage, right? Can I see how— what the number derived from? And when you can, to Misha's point, people that live in the world of numbers who kind of on the Venn diagram overlapping marketing, finance, sales, right? They need to understand that data lineage. But if you cannot show it, your explainability is out of the window. Yeah.

Yeah, it's like— go ahead.

You'll just have more users at the end of the day. You know, like we have to think of the end user and you can have more people engage with your data. Like I see this so many times that there's, there's a champion who creates this amazingly over-engineered model that has so many, I mean, it's so clever. I mean, and it's probably really, really good. The concern is, well, one, if the individual leaves, right? And somebody else is going to adopt it.

Yeah.

But also the, you know, any other user of the same model, any questions they have, only one person can answer, which is why I always, even though we have, you know, very sophisticated models, the journey to get to those models for me will always go by something like an even weighted attribution model. It's like, let's first validate that all your touches make sense. And only then can we say, well, do you want to really, like, are they the same level of effort? And then you start tuning from there.

But yeah. Well, I think the, like the, the thought that's going through my head is like all models are flawed, right? And if you add in that you're trying to model human behavior and sentiment, right? It's nearly impossible to get it right. You can get lucky, but I think what happens is, like the scenario you described, Misha, is that downstream, if people are relying on that and it's not explainable and something comes out of it that doesn't match what their expectation is, they start to lose trust.

That's right.

At which point then the model doesn't matter. They don't believe it and it doesn't, it gets really hard to regain that. And that's like, it feels like that's the real impact is that, right? So as good as your model is, if it can't, people can't understand it and they see things that are quote wrong with it, then they're going to start to trust it less and less over time. Okay. So maybe this is the same or it's the other side of the coin, but Nadia, you also talked about like, This like complexity is sometimes like, and that like level of detail and data is seen as a badge of honor, right? Like, what do you, would you like any examples of that, or what do you mean by that?

So, if we think about the go-to-market team at large with all the different players, right? You have sales, you have finance, you have CS, you have marketing, you have data science or BI, you know, where operations potentially sit in the same fence. If we think about the personalities that make up successful players on those teams, they have a whole range of personalities, and those people are motivated by different things.

Yes.

To assume that everyone's motivation and assumption of when they do a good job is the same, it's a very flawed assumption. So when you come, you know, marketing usually is the most collaborative function, but so if the marketing player does not have that collaboration ingrained in them, being able to facilitate cross you know, pollination of ideas, of processes and everything. You're not a good player because there are many of us and we tend to hand off things from one person to the next to get it to market, right? Then you take more technical roles, and the mindset and internal kind of motivation and priding oneself on a job well done is different. It's more like, I am challenged by complexities. That's what drives me. That's what feeds my curiosity. And I am valuable to the business because of my curiosity.

Right.

So, naturally, I'm more inclined to create very complex things and over-engineer just because I want to see if I can put the man on the moon from my keyboard at the computer, right? And this African proverb comes to mind, if you want to run fast, run alone. If you want to run far, run together.

Hmm.

So, you will run slower, but you will, you know, business is a team sport. However, You know, you cannot forget that these people that are so good at what they do, they are that good because they run fast, because they have that curiosity. So, I've seen, kind of getting closer to the example, we just did a virtual event series, and we had all sorts of guests coming talking about, you know, different things. Hitchhiker's Guide to Marketing Analytics, people can watch it on demand.

Yes.

We had some brilliant people, brilliant people talking about the use cases of what they built. And let me tell you, some of them said that if they were no longer in their seat, whatever they created would probably be an abandoned castle with the key lost, nobody being able to get in there. Just because what got created has so much of the know-how and, you know, level of attention that is specific for this individual who knows how it's all done. It's a custom build, you know, and that's where the legacy kind of tribal knowledge is important. But if you don't have that, that's the complexity that really prevents the business from going forward faster if that individual is no longer there. I don't know, Misha, if you can relate to that. You're like, you're more kind of a technical person. You probably see that side of the fence way more.

Well, I will just say that my earlier comments don't mean that robust models aren't necessary. I still think that models need to be robust, but it's just, there's a, At the layer as close to the end user as possible, you should be able to answer their question as specifically and as clearly as possible without much noise. But under the hood, you're absolutely right. If you have a lens of looking at a campaign performance, is my campaign doing well or not well really depends on who the user, you know, what is their lens? Yeah, is their lens maybe more financial and is a return on, you know, to the bottom line of the business? Or is it a matter of like, am I getting the right job titles into this campaign? Or is there an easy way that I can gauge that and its change over time? So under the hood, I still think there's room for a robust— or I think that the comment was maybe a sophisticated or kind of, you know, a lot of data. It should be there. But oftentimes, we created this new metric, so we should also put it on the dashboard for this end user.

Right.

And I think that this view level, like what the lens is, the lens, I would think over time should be maybe fewer metrics, but very specific to what's going to make the end user's job easier.

Do you find, um, one of the things I see is you said like, what did the campaign perform well or not? Right. And it feels like it's a little bit of a loaded question. And I think that I see people go like, what is the one metric? that tells me that. And my take on this is all like, there's usually like a basket of metrics, but it comes in 2 flavors, right? One are like absolutes or, yeah, did I, how many, say it's for a webinar, how many people did I open my email for advice? How many registered? How many showed up, et cetera, right? But also like, and did I have a goal for those, right? So that's one. The other is how did it perform compared to others that are comparable with the same audience, same type of thing, same channels? And I don't see a lot of people thinking about it that way. And there's not like one, right? If you had a webinar that you said, the most important thing is to get, if we got 5 of the right people who then moved on, that's great. But if it was, if we got 500 people, but none of them were any of those 5 people, then was it a success? Like, I don't think people think about it that way.

Yeah.

Yeah.

That's exactly it. I like the earlier comment about, can you do something about this? And if you can't, then maybe don't include this in your models. Like, I often see demo campaigns having really, really high attribution percentages or numbers. Like, well, yeah, because every single sale you have is going to go through this. Like, are you going to— what's the action? The action? Oh, we should do more demos.

Yeah. Well, oh, I would like more demos.

Yeah, yeah, exactly, exactly.

Yeah, right.

So maybe a different question would be, okay, how do we, should we not use the metric necessarily of dollars, but what happens before demos? What if we look at the period of time in buyer journeys before that point in time? What happened there? And what campaigns did we run that we can try to replicate? Is it physical events or is it, you know, is it webinars or is it more emails? And that question can become, can have an actionable answer potentially.

Yeah. Interesting. So this kind of brings me to a little bit of like, it feels like there's an opportunity or maybe a challenge, maybe it's both for people to kind of go back, either go back to fundamentals and basics a little bit. But I feel like there's still, my strong feeling is that there's still a gap in We're drowning in data. It's like one of you said that earlier, right? But at the same time, we don't have a lot of people. And if I think about marketing ops specifically, but marketing in general, I don't think there's a lot of people who have the skills or capability to even have a conversation like we're having, right? Let alone take it and adapt it to their organization. So how, I think in one of our conversations, you guys talked about mentorship, like mentorship or things like that? Like, how do we help people in the space get better at doing this and thinking about it and communicating it with their teams?

I can take this one and then Misha, I would love for your commentary on what I have to say. I think what I see in a lot of cases, and, you know, regardless of what size of organization you are, whether, if you have marketing operations team that sits separately, like that would be the case there, right? You have campaign people, demand gen people, performance marketing people on one side, revenue marketing people on one side, right? They are the closest to what is happening before the numbers start showing. So they have the context, right? Audiences, you know, intent, seasonality, you know, the nuances, the creative, like they have all of that. The marketing operations people do not have this context. They're just ask in a way, in a lot of cases, a layman's way, can you pull me the numbers of, I don't know, opportunities created in the 3rd quarter?

Yeah.

So, that's all they know. That's what the ticket says, right? So, you just go and do exactly what the ticket says, literally present that data to the other side. They look at it, and that's not what they wanted at all. They maybe wanted to see, you know, the touches that happened over the 3rd quarter to opportunities that got opened over, you know, 1st and 2nd quarter. But they did not articulate that because they don't see the data. They don't see, they don't understand the data structure. They don't live in the world of, you know, rows and columns like you would if you were dealing with a CRM or database, right? So, there's this disconnect. And, you know, one thing that comes to mind, like in, you know, for us specifically, it's relevant. One of the recent releases that CaliberMind did is the built-to-scale architecture. And built-to-scale architecture, it's exact— it was ideated as a framework to prevent exactly that, this disconnect. Of someone taking a ticket and executing on the ticket, delivering it with all the time that elapses between the two, just to get the question back and rework, and then another question and another question. I think marketing operations gets inundated with like, can you do this? Can you rework that? Can you put a filter on that? And, you know, that's what we're addressing with built-to-scale architecture, where everything is modular, where you can get together, which people should do more often, Look at the dashboard and say, what questions do we wanna ask? And as marketing operations person puts these graphs and charts and dashboards out of the library of templates where the data is already connected, the data is already modeled, they don't have to go and spend hours trying to bridge this gap of pulling the data together before they even do anything on the fly in front of the demand gen person, right? All of that is done. So you start looking at it together in real time. And people's eyes illuminate really when they see how all this changes. They get inspired, they have more questions. And you deliver the final product way quicker with people getting exactly what they want because they had the context and they brought it to the person who has the knowledge of putting the data together. And you give them the connective tissue, which is the tool that allows us to model all of this in real time and put it together and visualize it so that both parties kind of leave happy. I think that's what it is. It's 3 things. It's context, technical skill and the toolings, you know, the platform that allows people to do that in real time without too much, like, you know, working behind the scenes. So that's kind of my take and a little bit of a shameless plug.

Yeah, I see here.

Yeah, I will just say that maybe this repeats a little bit of my sentiment from earlier, but I would just highly recommend going to the end user, the one, you know, your campaign manager or whoever leads your physical events and say, what kind of, what would help you make better decisions? Like, you tell me, because I just, I find that a lot of this happens because, oh, I have access to the data and they don't even know that I have access to this, so I'm just gonna illuminate everything. And so like a very basic example of this, kind of a little bit in the context of Calm Mind is, You know, we used to have these engagement metrics. Your account will have a score of 97.5.

Sure.

Sales reps, what do I do with this? And what they really wanted is they wanted in a split second to know that this account is better per— if I were to turn on my computer today, help me create my order of operations. Like, that's really what they want. And so they know that this account is probably, you should call first before this account. And the other thing they want is context and show me in a very easy way, very easy to understand way, what should I talk to them about? If I were to remind them of who we are, how do I do this in the easiest way?

Right.

So, you know, yes, it might, it takes digging. I'm not sure that they might give you the clearest answer right away, but through, I think, a brief conversation, you could probably get to this notion of what would make your day be more efficient? Right? To really make better decisions where, what types of invitations would work for your next in-person event, for example. Like that might be more useful than knowing attribution for a specific event potentially.

Yeah. So 2 questions. One is pretty specific. So to what you just talked about, which is you brought up like a score thing, right? Whatever, 0 to 100 or something. And, you know, people get stuck like, what's the difference between 97.5 and, you 94, right? And this person we had on as a guest was like, really what helps get away from that is doing something relatively simple, right? Gold, you know, bronze, silver, gold, right? And just kind of break it out, like make it simpler so they're not caught up in the raw number. Like, sound about right to help?

Yeah, yeah, absolutely. Yeah.

So the other thing that came to mind here, and this is maybe a personal, maybe I'm hearing what I want to hear a little bit, which is You keep coming back to like asking specific questions or working collaboratively the way Nadia, you talked about like to come up with metrics. But it feels like what I hear more often than not is like, we need a dashboard, right? The dashboard's not really well defined. And my pushback generally when I get asked to build a dashboard is what are, like, let's start with a handful of specific things. And over time, if we build a dashboard, we build a dashboard, but maybe we don't really need a dashboard. So like, is like, do you have a, do you have a preference, right? Do you start with a dashboard in mind of what everything we want, or would you rather start with let's tackle individual kinds of questions that we want to answer first?

Well, you know, a dashboard, I'm afraid that, you know, or maybe it's what I hear when I hear dashboards, I hear we need a dashboard that will answer everything for as many people as possible. And I would rather say, hey, Michael, Like for, you know, I don't know, for the next event, how do you, like, what types of things will help you make a decision for your promotions for next event or something along these lines? Those questions will become more specific. It does mean that on the information on this dashboard, you might filter out a lot of the noise and you might have fewer, there might just be even less information, but the information will be very, very specific. So I think when I say yes, Yeah. Dashboard. So yes, metrics, but also as specific and as easy to understand for the viewer of that dashboard. And it does probably mean making assumptions there and like not to— and excluding elements even though they exist in your universe of data.

I actually have a comment on that. You know how we all talk about Democratizing the data. Oh, yes.

Democratization is a good thing.

I mean, democracy is a good thing. Democratizing data is a good thing. But when, like Misha nailed it, when you think about irresponsible democratization, that means you open your data to people who don't really know what to do with this, but feel compelled to take action because they have access to this. And everybody has imagination, some people more than others. Some people have a quick, trigger finger, so they want to pull it and go do something, right?

Yeah.

So, it's almost like giving people less, but what matters, that's where, you know, the gold of it all is. So, it's like the Goldilocks of reporting. If, to Misha's point, you have your campaigns manager and this person needs to see this, this, and that, but nothing more, give them that because that creates balance on the team. They don't start getting into things that belong to someone else. They don't start making assumptions and that person now has to defend their stuff. Like, It's very important to keep those guardrails so that you have that level of, you know, I pay attention to what matters for my role to move the needle for the business, right? And, you know, another kind of metaphoric example of what came to mind, what you guys were talking about. So, my husband has been in the world of intralogistics engineering his entire life. So, they service conveyor lines, they service robotic equipment, You know, forklifts, like anything that you have in an Amazon warehouse, a Williams-Sonoma warehouse, like those large consumer packaged goods warehouses, right? So sometimes they would get a service call and the person who is on the ground, the warehouse manager goes, I need you to do XYZ to this robot. He's not qualified to make that statement, but he thinks he knows, right? So he, in his best intentions, he tells them what to do. My husband would always, he would come home and he would be so like aggravated. I just wish you would tell me what it does. He doesn't need to tell me how to do our job. I know what needs to happen. Just tell me what it does. To Misha's point, right? Like, explain to me what you're observing so I can make a recommendation from my expertise what it is that you need. But I know it takes a little bit for people to arrive there. All things are done out of good intentions, but sometimes good intentions take the road to hell.

Yeah. That's funny. Yeah. So, it's interesting, Misha, the way you described that. Like, one of the things I believe also is like, there's this huge volume of data now in all kinds of things we could do. This is part of why I resist the urge to build dashboards, because to your point, right, it's the idea behind a lot of them is it answers all the questions for everybody. And I think there are certain things that metrics and reporting that is appropriate for different audiences, right? So I don't think I would ever want, like, I don't say never, but I think it'd be less important to show executives, here's all the email opens and click-throughs and web activity. Like, they don't, I still think there are people who should be paying attention to that.

Sure.

But it's not that group. And I think what happens is that we get the right metrics to the wrong people sometimes. And I think then it causes more confusion. So.

Yeah. Yeah. I'm, you know, the data, having Curated, clean, normalized, standardized data. Like if you were to ask me, you know, what did Caliber mind, or in this universe of analytics, what's the most important thing? It's that because by you having data that you can go to, you have this kind of very clean, you know, shelves with it all existing. Then you can go and you can, it's easier to actually answer, curate via models or dashboards, however, answers to different questions. But I think what happens a lot of the time is, and we see this happen a lot, is, you know, oh, we need to prove value and we need the model that's gonna give us. And so this model, you know, for this touch on the 3rd time this happens and not on a Sunday, give credit, but otherwise give it to this one. Like you get to such level of over-engineering because of this pull. Again, the question of incentives of this. of this pull of like, no, we need to show that what we do matters. And you think at this level and you really forget the campaign manager that just wants to do incremental betterings of their campaign. They just want to do, make sure the next campaign they do is slightly better than the last one, maybe a little bit more of the right audience. But you almost, you don't enable them because what you're thinking about is this kind of like, well, how am I You know, what is our attribution versus then? And maybe what the right thing to do is a really simple model, whether attribution or otherwise, that focuses on that type of campaign and only on those touches. And is it doing better over time?

Right? So, um, so this is— this brings up another point that I run into a lot, which is, I think what drives this idea for dashboards or more data is There's a lot of people I think get like, they're, especially if they're actually using it for decision-making, right? They want more and more data. Like they're uncomfortable with either what they feel is incomplete or missing data that's maybe hard to get. Or because, and I think the nature of this, like, I like your point about like, I like clean data, but I think the reality is like there's a level of effort that requires for sales and marketing data in particular and complex B2B, like it's just not clean. Like, I don't know anybody that thinks their data's great. So if your assumption is it's not going to be perfect, right, you're now dealing with a level of maybe incomplete data and data that you don't totally trust, right? Or your confidence level is, I don't know, pick a range. How do you get people past that desire to have complete and accurate data when the level of effort to get from where they are to that may be I think that the best analysts we work with know that it's a journey.

Like, it's, again, it's not about getting to perfection quickly, but it's about just having a better dataset this quarter than you had last quarter. You know, maybe change the level of granularity. So, for example, you know, job titles in a B2B environment could be a very messy entity. There's just too many variations of them. So, you might say, well, I want to bucket them in some way so that it becomes more meaningful for analytics. Buckets might not be, you know, there might be steps in how you bucket. You can do something more crude like keyword bucketing, or you can do something much more robust like introducing an AI. Like, there are steps. You can take this one step.

Sure.

And then saying, you know what, I think next quarter I'm going to want to revisit this. It doesn't mean that this is futile, but it will, yeah, it might not be as accurate per se, but it could still be very useful. And I think the best customers, our best customers in general, people I work with in the industry who I think are really good about this, understand that perfection won't exist. It's like trying to make the models be useful. And yeah, it's a big part of this is the curation of data. And the other part of it is speaking probably to more end users and seeing whether we even consider the questions that will help make their day more efficient.

I actually have a counterargument to this one, kind of being empathetic to the world of business and seeing different sides of it. What I see on the analyst side in a lot of cases, there is, and this may sound harsh, I don't mean it that way, there is this luxury of time to wait and see that incremental improvement, right? The same luxury is not extended to the other side of the house, performance marketers, demand gen marketers, sales, who live and die by the quarter.

Yes.

So in this specific scenario, this desire to bring more data stems from the notion that we're Spinning our wheels, going 100 miles an hour over here. We did all of this stuff, and once it trickles through the pipelines, we don't see the output. What if we could have more? What if something else could be feeding into this? What if we are missing out? Or what if there's some kind of other tool that would surface whatever we're missing so we could take credit for more? We could articulate that all this stuff that we did till 7 o'clock on Friday night for 4 weeks in a row actually pans out to be something. So, I mean, I don't have an answer to this one necessarily, but I am mindful of difference in like the pace of roles and different of goals and sense of urgency. And they say that pressure cuts diamonds, but it's really hard being under pressure.

Yeah, it is. All right. So we're kind of ready to kind of run up against our time here, but maybe let's, like, we've talked about a lot of different things and some ideas here. If you were, maybe each of you take a shot at this, like if you were to give our audience, I think marketing ops in particular, maybe like one nugget, like go back and like, this is the thing you could do to help move the needle on your ability to be effective with reporting analytics, attribution, pick like whatever, what would that be?

I can get us started and Michelle, I'll give you time to think.

Thank you.

Um, the one thing I would say, talk to your demand gen people more, get the context of what they're doing, because then when they start asking you questions, at least you will have the background against which they're operating. So when you start bringing them data and reports and start being that translator or interpreter of what the dashboards mean, you'll be able to tie that to what actually happened on the ground. And the more you talk to them, the better. Don't be the kind of the ticket taker, submit an Asana task and we'll never talk again type of person because you will see that incremental improvement in, you know, quality of your output and probably less rework too.

Yeah. Yeah.

And for me, it would probably be something along the lines of familiarity with the data or being so comfortable, I guess, with the underlying or the raw data that exists or that you have access to. that you can connect the dots between questions and what's available because I think that's the best thing, you know, you can do for your organization. There are many, of course, flavors of questions, but knowing what's available and, you know, and then also like, I know it seems silly, but validate it. You know, we always try to look at things in aggregate, And so, oh yeah, these numbers kind of look, they make sense to me in aggregate, but drilling into a specific journey or a specific opportunity and what do I have the right touches? Does this make sense? That can go such a long way in gaining confidence, you know, and understanding, right? What can be answered with the data? So I guess getting the hands dirty in terms of what's available so that then you can be a better resource for your business.

It feels like those— would you, the one that I would say I think also is if you're concerned about your data quality is like, don't wait to start reporting. I think to your point, Misha, earlier, like if your goal is to continue to improve, right? If you wait till it's good enough, right? You'll never get to it because it'll always be flawed. And so I always go like, start reporting on it. That's going to expose the issues and you can then fix them and Like, it can be a flywheel effect. And I will say an idea, like, I tell people all the time, go spend time, sit down, like, whether it's virtual or in person, right? Spend time with those people you're working with, understand what they're doing and what drives them. Because you both said it multiple times, right? People's incentives are going to drive the kinds of questions they ask and their behavior. And if you don't Understand that, then it's going to be a challenge no matter what you do. Yeah. Any final thoughts before we wrap up here?

It's 2026 around the corner. Everybody's trying to— I know it's scary. Everybody's trying to come up with their metrics and their frameworks and their goals. It's going to be a fun season.

Yeah. Someone asked me to do something and I still need to do it to to do like a little quick video of predicting 2026. And I'm like, I'm so hesitant about doing something like that because like, I just, I don't want to look back a year later and go like, I was so off. That will happen. But everyone needs to do it. Well, hey, it was so much fun, y'all. Nadia, Misha, glad to do it. I'm glad we were able to make this work. I know it was a little bit of a journey for us, but if folks want to kind of— keep the conversation going and learn more about what y'all are up to, what Calbermind's up to, what's the best way for them to do that?

Probably LinkedIn. Yeah, I'm on LinkedIn all the time. Happy to be on.

Yes, you are. Yes, you are.

Yeah, happy to answer. Happy to answer any data questions. Just brainstorm. Always happy to do it. I like to geek out on this stuff all the time. So Calbermind or otherwise, LinkedIn is great.

Fantastic. Well, again, thank you so much. Appreciate it. Thanks to all of our Longtime and new supporters, we appreciate that. And as always, if you have ideas for topics or guests or want to be a guest, you can reach out to Naomi, Mike, or me. We'd be happy to talk to you about it. Till next time, bye everybody. Bye.