S2 E5 | Benchmarking in Business: Types and Benefits

The benchmarking that we're doing here is it's not about bringing the best product. This is not concept testing of finding which one do they like the most. It's actually more like a mirror and forcing them to actually say which would you pick and why. And in some cases it's asking them why they didn't pick these other things. And so when we go to them with, with different things, it's with a range of experiences, it's with a range of different things that, that no one of them is going to be perfect. But it helps us frame kind of what do they mean by things and ultimately how do they measure the progress and at the same time, what are they eliminating? And so it helps us with that higher fire criteria. So most people want to go to the customer with like here, here's our three best concepts. What we would say is, no, I want to go back with these eight really different concepts so I can actually then learn what they like and don't like and why it fits and why it doesn't fit. So we can describe product market fit.

Welcome to the Circuit Breaker podcast where we challenge the status quo of innovation.

And new product development.

We'll talk about tools and skills and methodologies used to build better products and make you a better consumer. I'm Bob Mesta and I'm the co founder of the Rewired Group and I'm one of your co hosts and we're joined by Greg Engel who is my co founder and chief, Bob interpreter. Join us now as we trip the circuit and give you time to reset, reorganize and recharge your brain to build better products.

Hey, welcome to the Circuit Breaker. Greg and I this time are talking about benchmarking and we talk about both technical benchmarking as well as consumer customer benchmarking and kind of the different spins we have on it and the different goals and objectives behind it and how it plays out as kind of a way of prototyping to learn so we can get deeper language, so we can understand the cause and effect, so we can understand the right metrics, so we can understand and have a common language for marketing and sales and product. And so ultimately we use benchmarking in two ways to help us kind of move, move the ball along to build better product. Enjoy.

Hey Bob.

Hey Greg.

So today we're going to talk about benchmarking and I want to, I want.

To make sure my favorite topics, I.

Want to make sure we really level set ourselves though.

Yeah, yeah. Because this could be deep and it could take too long. So I want to make sure we have boundaries on this one, and I.

Want to make sure we talk about the different types of benchmarking.

Yeah.

When we say the word benchmarking, there's gonna be a lot of people out there that think technical benchmarking, which is tear down a product, figure out what does. What does it do, blah, blah, blah.

I'd love to do that too.

And that's a very powerful benchmarking. It's one version, and then the other version that we often do is. We can call it context benchmarking, consumer benchmarking. It's really. Why do people choose what they choose? And unpacking, language, unpacking situations. Unpacking those types of things of use.

Yeah. And getting to higher and fire criteria.

And both are very powerful.

Both are. Both are needed as you.

And they're used at different times, and they're used for different things, but they're both. They. They can be married together very easily.

Right. So we. We would. We always refer to them as, if you will, technical benchmarking and customer benchmarking. And they're two very, very different processes.

So I want to talk first about technical.

Yeah.

Explain to people. And I think people know what technical benchmarking marking is. But from your perspective of an engineer, of. Of a product developer, what is technical benchmarking to you?

Part of technical benchmarking is it has two roles. One. One role is to actually understand kind of how you stack up against other product and. Or, you know, services in terms of technically fulfilling those requirements. Right. So like at Ford, one of the things we did was we would take. We take an exhaust system and we would tear it down. We'd basically get it to all its components, we put it on a big board, and then we take. We take our exhaust system, we take Toyotas, we take General Motors, we take all the different exhausts, and then we compare them side by side and look at them and understand.

Why is that important?

It's important for a couple reasons. One is to understand if they're technologically doing something different or not. And then two, the overall performance of the system and making sure that as we end up understanding what consumers say, how does that actually relate back to the technical requirements that we have to put in place to build the specifications? So it's the translation back.

So the first one. And I want. Unpack that because you actually said a bunch of stuff in one thing, blah, blah, blah. So when I'm. When I'm looking at it from a Comparison of my product to somebody else's product. I'm trying to actually figure out, do they know something? I don't know one.

Yes.

Are they doing something. Are they doing something slightly differently than that? I may or might want to.

From a cost perspective. Boy, why are they doing it this way? Why would they do it that way?

And so I can go back and think of different ways to do it. And then the second one you kind of talked about was then trying to put it to some consumer metric, connecting it to like.

So there's a methodology I learned very early on called QFD where it's like, okay, the consumer says this, what's the technical way? Which we know that we've met that requirement. And so it's the translation into the technical realm. So think of it as like a transform from the consumer world to the technical world. Let's just make sure that what we're measuring relates back to what the consumer is saying they want.

If we're making. I'm trying to think of something that's non. For doing a school. Can we do technical benchmarking on a school? And what types of things would we be benchmarking in a school?

So we could. We could benchmark kind of, for example, the. The.

For technical.

Yeah, yeah. On the technical. The size of the room, how many kids in the class teaching. We can talk about the. The time they spend on each subject. We could talk about the different curriculum that they use. We can talk about kind of the sequence at which they do that curriculum. We can talk about how they handle exceptions, kids who are a little bit behind or kids that are a little bit ahead. All of those mechanisms we can actually start to benchmark to understand, like, how do they manage their school versus how do we manage our school?

And one of the reasons we might want to do it is let's say we were in a school and we started getting a bunch of kids that had learning differences. We might go and study a learning different school from the technical aspect of what do they do? How do they teach? How are they getting this done so that we can bring it to our school?

Right. So, for example, one of the things we found was that they might break them into smaller groups. And as they have different groups, then they had rules. And the rules were like the first question if you don't get something is to ask another student as opposed to the rule of, hey, go ask the teacher. And so you start to realize that that simple mechanism changes the dynamic of the room. It changes the role of the teacher. It changes, actually. It builds confidence for the K. And so understanding that kind of subtle mechanism of when you need help, ask a peer first actually has a very large impact. And we learned that through benchmarking.

And I think it leads us to the next thing, which is consumer benchmarking. I often call it contextual as well, because I'm trying to put people in context. But I will often bring in the things we learn from the technical benchmarking into this because it will be what products we choose to bring to the table. Because in the consumer benchmarking, and this is something that we do with customer, we do with customers and our clients, which is we actually put them in a context. We actually bring product with us and we ask them to basically shop in the moment of what have they used? What would they use? They pick up product and we talk about how they use the product. What is that benchmarking? What does that benchmarking do for us?

So this all goes back to the benchmarking that we're doing here is it's not about bringing the best product. This is not concept testing of finding which one do they like the most. It's actually more like a mirror and forcing them to actually say, which would you pick and why? And in some cases it's asking them why they didn't pick these other things. And so when we go to them with different things, it's with a range of experiences, it's with a range of different things that no one of them is going to be perfect. But it helps us frame kind of what do they mean by things and ultimately how do they measure the progress and at the same time, what are they eliminating? And so it helps us with that higher fire criteria. So most people want to go to the customer with like, here, here's our three best concepts. What we would say is, no, I want to go back with these eight really different concepts so I can actually then learn what they like and don't like and why it fits and why it doesn't fit. So we can describe product market fit much better.

I want to make sure we're saying the right things because we're using words that we might give them concepts we're going to do at the end. But the first thing we do is we give them things that are real in the marketplace and we put them in a job to be done. And we say in have you ever had this experience? And we lay out what the job is and they say yes. Well, we know they're going to say yes because we picked them because they're going to say yes. And then we have them look at the table that we have with all these different products, and there will be products that you wouldn't think would fit for that job to be done. And we're actually using, I don't know if we're completely using an RDOE method, but we're using some orthogonal array, we're.

Using some designed experiment to actually help us understand the set of things to bring to them.

So we're using that to figure that out. We're putting that on the table and we're letting them choose things that they've used in that situation before. And we find out why did they choose those things, what about it, how do they pick it off the shelf? We talk to them about their journey of using it. We talked about their failures of using it. Then we'll also go back and we'll look at things that they didn't pick and say, well, why didn't you pick this one? And it's all an attempt to get that job to be done more full, more defined so that our, our partners that do the research, the R and D, the marketing, can actually have more language and more flexibility in that job to be done to actually mine. And that's why we're doing that. And the technical. We're trying to figure out, how can we do something? How are other people solving this problem?

Yeah. And it also brings into the cost aspect of it, like how much does it cost us to do this versus so it's cost and performance is on the supply side and it's price and value on this side in terms of how much am I have to pay, how much am I willing to pay to make this kind of progress? And so ultimately it's trying to make sure we understand kind of their ability to switch and what are the features that are resonating and the benefits that are resonating with it.

And I think you hit on it. There is technical benchmarking is usually done on the supply side. And consumer benchmarking is done on the demand side, as consumer would lead you to believe. Right. But I think we have to. When you do consumer benchmarking, you want to bring very technical people into that as well. Because you want them to hear the words.

Yeah, because there's also this other problem of the technical measure that we have doesn't necessarily relate to what the customer actually means. So somebody will say, God, this is really slow. And like, oh, they want faster speed. And it's like, no, the outcome that they Want is actually more, not speed. And so you start to realize, like at some point we got to make sure we're connecting what the consumer says and what they mean to basically the technical measures that we can measure that actually assure that we can deliver on their job. Right. And so the other thing I would say is that this is so in the book we just launched into five skills of innovators. This is prototyping to learn. This is a set of things that help us at the boundary of life. So the first round of job interviews are usually around. They're either phone interviews or video conferences, but it's all about the language they use. But ultimately they get to this edge of language that they can't, they can't. They don't have words for it anymore. And so part of it is this is that next level of now how do we actually articulate experiences and what are the words that they can prompt by actually interacting with the product?

Well, I think the first interviews we do the jobs you don't interviews to set the boundaries. Yes. Are the interviews of why someone switched a behavior.

Correct. That's. That's a big hire.

And the consumer interviews or the benchmarking interviews are about how people use.

That's correct. The little higher. And. And it's like when. When do you buy the bottle of Windex versus when do you spray the bottle of Windex kind of thing.

Right. And. And we need both to give us language into the job you've done.

That's correct. And you got to satisfy both.

So they're very important to do. Now the hard part with consumer technical. I think most companies are very good at the technical benchmarking.

Well, sometimes they over. They're over good at it and they end up kind of beat, trying to beat each other and they literally outrace the customer and the customer has no idea what they're doing.

So but I think the technical how people do it that I think that's kind of ingrained in most, in most companies, not in all companies, but in most companies, the consumer interviews or the consumer benchmarking is a little bit trickier because it's, it comes down to the same things that we run into problems with any kind of qualitative interviews is what the customer says and they mean can be different than what the, the company employees hear. And it's very important when you do this type of research that you actually again, try to divorce yourself of what your definitions are and accept the definitions of what the consumer is telling you. So a consumer might say something's creamy. When you're like, no, that's not creamy. Because creamy means this.

That's right. Technically it means that.

And no, you have to understand what the. What the person is. Or bitter is another one we run into a lot. Right. Bitter is a very technical thing, but consumers will use bitter as a bunch of different things or the number of steps, because somebody will say, well, I don't care how many steps. But then when you talk to them.

It'S like, this looks like it's too many steps.

It's really only two steps, or whatever it might be.

Or they'll say things like, you know, well, I trust this one. And you gotta. You gotta unpack that to say, like, what does trust mean? And what are the characteristics that make that more trustworthy than something else? And so you start to realize that trust is an effect, not a cause. And so how do we actually unpack that down to kind of what do we do that causes trust? And the technical side. Right. And so all of those kinds of things are part of this. And being able to understand what do people mean and getting it down to the technical requirements, which, by the way, this is. This benchmarking is really more for the product people and the technical side, so they can actually engineer and build better. But it can be used with marketing and with sales. But the fact is, usually when sales or marketing lead it, they don't go deep enough. They don't actually get to the underlying causal mechanisms.

Well, I think having someone lead it that it's divorced from the company is vital. Whether it's us, whether it's somebody else, I don't care who it is. But anybody trying to lead it from.

They bias it a little too much.

I debias it. So I'm gonna call a little bit of BS on you just said, though, because it's actually important for everybody because it. What it. What the main purpose of it is, and this is what we struggle with, our customers, our clients. What the main purpose of it is, is to actually make the job to be done more full. And that helps everybody.

It does help everybody. And my only point is when. If it's led by other people besides the technical, they never usually get deep enough. That was my problem of why it was actually built, was because they would only get to some surface definition of.

So I want to fix what you.

Said there because then make it better.

Because what. What we're saying is, you weren't saying it's not useful for the marketing of people. What you're saying is if they lead it, it could be problematic if, if technical people lead it, it could be problematic.

Right.

So getting that, that, that impartial person that's divorced of your product is very important and, and helps you make more progress. Because as we know, and we've discussed in multiple different podcasts across the now two years we're doing this, our two seasons, is we have biases, Humans have biases, and we need to know them. And trying to get over those biases are hard. And even as a third party, I have biases, but I have less because I'm not paid by the company, I'm not on the team that lives and breathes this thing over and over again. And I actually have more biases to the consumer because I'm trying to live through them. So that's why you kind of want a third party doing this, because of those things you're just saying. But I want to make sure we're saying this completely so people understand it. Marketing learns more because they learn the.

Words people use and the actions that go with those words.

If you actually help people understand the struggle they're in, people will realize they're in more jobs we've done than they actually are. Especially when you're talking about consumer packaged goods, because we are in multiple jobs multiple times a day, all the time. Technical people learn because we actually can get down to some hiring, firing criteria.

Thresholds, targets, things like that, though that.

Helps marketing and sales, it's more important or helps more the technical people. That piece of information, language helps everybody.

Language helps everybody. And I think that's usually the number one reason why people, you know, like this whole approach is because it actually allows them to have one source, and then they all have to actually learn that source and translate it from that source, as opposed to kind of like almost aggregate all the information. And kind of like we both. We all use the same words, but we have different definitions of them.

And in watching people go through this struggle, when we put all this product in front of them, which is not as overwhelming as a grocery store, but.

As close, or the app store or.

But it's close, right? Watching them through that helps everybody with a little bit of empathetic perspective. And that's what we're really trying to understand here, is we're trying to have empathy of when people are in this context, because what we know is people act differently based on the context they're in. I could take that same sort of product, put them in a different context, and I'm going to get different answers so we want to make sure we stay in one context. So the important things with consumer benchmarking is you understand what you're trying to understand. So you unpack which job you're going after, what set of customers you're going.

After, what context, what outcome and then.

You put the product in place that.

Stretches the boundaries, things that you might not think compete with you, you want to put in there. And we've even put, if we've done food, we've even put non food things in there sometimes, right. So we want to put that gamut of things that force people to actually make real choices. We want to put them in that context and let them tell us how they shop, how they pick what's happened, when they picked what's worked and what's not worked so that we can then take that information, put it back to our job to be done and put more information in our job that we want, put more forces of progress in there, put more trade offs because you're here a ton of trade offs.

Trade offs is trade offs and higher and fire criteria are kind of the most important thing here. Like when what's the threshold by which they go, no, not going to do that.

Where I've seen consumer benchmarking gone wrong is when the technical team, or it could be others, but I'm going to pick on technical team here, try to get so concrete in what that design requirements are, when they try to get too concrete, it actually goes off the rails. And what I mean by that is when they start saying, well, in this job or in this context, then it means this food has to be this certain size. Well no, because I have different, I could have be feeding adults that can have a bigger size or kids that have smaller size or bigger dogs or smaller dogs or whatever it could be. That's, that's actually going too far.

Well, and it's, and it's different than the, the, the, the notion of like most of the time when I say technical people, they end up varying things in such a small degree that they actually have no impact.

Well, they want to get too concrete, right. So I think what we find in the consumer benchmarking is as humans, we want to find the answer as quickly as possible. And finding the answer is a really long process. People can tell us, our consumers can tell us sets of things, they can't tell us what to go build. So we have to then take all that information and figure it out. And we, and what I find is everybody wants the answer, not a piece of the answer. And all of our benchmarkings, be it technical, be it consumer, whatever it is, it gives you pieces of the answer. It's still your job to piece all that together, to say, with the customer constraints, with my business constraints, what can I build?

That's right. That's right. And ultimately the customer is an expert in their problem and they're an expert in their outcomes, but they actually have no real expertise usually in how to get there. And if they do, it's typically with technology or things from the past, not that they know what's possible. And so part of it is, is to realize, like, they can help you understand where, when and why. Right. So you can figure out how, what, how and how much. And so ultimately it's those kind of separation of those two worlds that you actually have to kind of figure out.

All right, so we normally do homework.

Yeah. What's the homework? And I have a couple.

Go for it.

So, so what one homework would be is, is what I would say is think of something that you've been wanting to buy, something you're stuck. And what I would say is, can you actually look at multiple things and see how multiple things actually help you decide which one to do? Because a lot of times we end up kind of realizing like, we can't make a decision because we can't compare and contrast. And you start to realize the role that having choices actually helps you eliminate. So you can choose. So the homework is think about something you've wanted to buy and instead of trying to just focus on it, try to add two or three, three other things that you might purchase in place of that, or as a half step and understand how to benchmark. How would you like compare and contrast those three or four other things to that one thing you're saying?

So you want people to build a competitive set.

Correct. And benchmark to help them see why that, what, what they're really trying to.

Achieve, what are, what are they benchmarking on?

So based on something, there's a struggling moment that they have.

Okay, so what I, what I would say is I want you to build that set of options, competitive set. And I want you to start thinking about from a consumer aspect, how does that fit your context? How does it fit your desired outcomes? What trade offs would you have to make for each one? And those are the things that, from a consumer perspective, we want you to think of from a technology perspective or a technical benchmarking we want you to look at. How are they solving the problems? How are each one of those things solving it differently from a technology Is one you know if it's food, is one doing it in small bites? Is one doing it in a big bar? Is one doing like what? What are those technical things they're doing? Are they putting multiple ingredients? And did one use a sweetener? Another we use sugar. Those are the types of things we want you to think of from a technical aspect.

Great, thanks.

See you next time.

Bye.

Thanks for listening to the Circuit Breaker podcast. If you haven't already, please subscribe so you won't miss an episode. If you know somebody who's stuck on the innovation treadmill, please share it. If you'd like to learn more information, Visit us@therewiregroup.com to find out how we work, how we can help some resources, some books, some software. Join us next time as we trip the Circuit Breaker to help you recharge, re energize and refocus your new product development.