33. Solving the Problem of Fit: Todd Rose and Bob Moesta

Hi, this is Clay Christensen and I want to welcome you to a podcast series we call the Disruptive Voice. In this podcast, we explore the theories that are featured in our course here at hbs, Building and Sustaining a Successful Enterprise. In each episode, we'll talk to alumni of our course and others who are trying to put these theories to use in their lives and in their organizations. It's great fun to hear from them and I hope that you find these conversations inspiring and useful. If you, if you have an idea about a topic or a speaker that you'd like to hear more about, or if you'd like to comment on our work, please reach out to us here at the school.

Hi, I'm Shea Roseman and you're listening to the Disruptive Voice. I'm so excited for our conversation today. We have Todd Rose and Bob Mesta in the studio. Todd is a faculty member at Harvard's Graduate School of Education and the director of its Laboratory for the Science of Individuality. He also co founded the public think tank Populace and is the best selling author of the End of Average and most recently of Dark Horse, both of which we'll talk more about in a minute. Bob Mesta is, along with Clay Christiansen, one of the principal architects of the Jobs to Be Done theory. He co founded the Rewired Group, an end to end innovation consultancy where he continues to advance and apply the theory to real business challenges every day. He's also the reason we're all here as the one who recommended the End of Average to me and the team, which just blew me away. So, Bob, Todd, we're all huge fans of your work here at the Forum. Thank you so much for joining us in the studio.

Thanks for having me.

Yeah, thanks for having me. I'm as excited, I think, as you guys are. This is like I've been working through Todd's books a lot, so this is going to be. This is a pleasure.

Great. So, Todd, you published a book called the End of Average. Tell us what you mean by the end of average.

It's funny because it can sound like almost like a bumper sticker, right? No one's average or whatever, but actually it's a pretty important scientific insight that I think has implications for most of society. It's this. For about 150 years, we've basically viewed human beings through, through the lens of the average person. And that's true in even how we think about ourselves, but how we design education, workplace products. And what we found out over the last couple of decades is that kind of Shockingly, there actually isn't an average person. And that, it turns out individuality matters quite a bit. And that doesn't mean individualism. It doesn't mean selfishness. It just means the reality that people are distinct and those distinctions actually matter quite a bit.

Yeah.

What are some ways that people, our listeners, may be used to thinking about themselves in terms of the average person?

Wow. I mean, it might even be easier to think about the ways they don't. Right. It's so embedded in our society. So almost all of standardization is based on averages at some level. If you're a parent and you take your kid to the pediatrician, and they say, well, how are they doing? Right? And they say, well, let me show you these growth charts. Well, those growth charts are literally just the average at each age stitched together. And it turns out that not a single kid that has ever been looked at actually follows them. Those growth charts, like they say they should, we do it in the way we think about school. We give kids standardized tests. And did anyone ever wonder, where did we come up with the amount of time kids get to take those tests? It's quite literally the time it takes the average of a representative sample of people to finish. So basically, the way we've designed our classrooms, the way we've built our products, the way we think about employees, just about every aspect of our life is dictated by this average person. That turns out to be a myth.

I always like to talk about it as in terms of healthcare. What's your blood pressure? What's your average max heart rate? Like? All of that comes from exercise, all those things, all from averages. And when you look at my body type and other people's body type, they're all very different. And so the reality is. But yep, 166 is my max heart rate. How do we know that? And am I really working out enough if it's 166 and it's really 187. So this is where, again, being able to understand the details is really, really important.

You know, to that point, there was literally a person who invented the idea of the average man. It was a Belgian astronomer named Adolph Ketollay. He also invented the body mass index. And so to this day, we're stuck with this thing that, like, works well in the population. If, you know, the body mass index of a country, it's a pretty good indicator of health for the country. It is a miserable predictor of individual health because it confounds fat and muscle and bone into one number that Just doesn't represent any particular person.

And why is that so problematic?

Almost everything starts with an accurate understanding of yourself. And if you don't have that, you get in big trouble. Right. So if we have built everything from our science to our society on an idea that the average is a good approximation for most people, then you're just going to be off in almost everything you do. Now, the implications, some of them are more just about on the commerce side, which is still important in a market economy to be accurate in terms of who people are and what they want. But it can become profound too. Right. So most of medicine until recently was based on average based research. Right. And so you take something like cancer progressions. So colon cancer is the second most diagnosed and lethal cancer in the world last time I checked. And for 35 years we had a theory that was based on average progression. There was one pathway, one precursor lesion, a series of genetic mutations and manifestation. Turns out there's actually three. You know, and that average progression only accounts for less than 10% of people diagnosed with colon cancer. So you're literally condemning people to death by treating them as averages. Now, in medicine, we've blown past that. You are never ever going to accept average based medicine anymore. Like we have personalized medicine. You get molecular fingerprinting. You want to know, treat me as an individual, because this is about life and death. We want that same urgency brought to the rest of society where we may not have the sort of life or death scenario, but it's our human potential, it's our happiness, it's our productivity, and it matters.

Yeah. So, Bob, you have a very particular view of the way in which that matters. Can you talk about why this idea that the average is not necessarily representative of a set is so important in your work as it relates to jobs.

So for me, part of it goes back to this notion that nothing is random, everything is caused in what I call the n of 1. And so this is the work I did with Taguchi and Deming really early in my career. And the aspect here is that underlying the causality of what causes people to do what they do is what's really, really important in Todd's work. He's really been able to kind of talk about this notion of pathways where what we would tend to do is average all that data together and kind of say, well, here's the best way we should do this. The reality is that there's probably four ways or five ways, and it depends on where you start and what your outcome is. And so jobs Is really based on this notion of the progress people are trying to make lives. And that as we look at markets, there's a concept called Personas, where they look at people and they'll average people out and say, here's a Persona and here's a different Persona. But the reality is that to me is it's really about the context and about the individuality of one situation or a set of situations that then talk about the progress they want to make with the outcome. And so to me, it's helped me build way better products by not thinking about what's the average, because I end up being wrong. But if I can actually understand these pathways, I can actually figure out how to actually build way better products that have way more scale.

And you know, one of the things that I love about the jobs theory is that it shows that sometimes when you think about individuality, you think of like either chaos or just like almost intractable. Right. But in truth, it's almost never that there's a million pathways. That's right. Like systems don't sustain themselves that way, but it is a guarantee. It's always more than one. Right. In complex systems, it's called equifinality. It's one of the only rules we have. And what's cool about job series, like, you start to see, look, when you start by taking individuality seriously, you can arrive at a set of patterns that have broad applicability that you would not have arrived at had you started in the aggregate and tried to work your way down.

That's right. And you end up with the trade offs. And so ultimately, it's about understanding the trade offs consumers are willing to make to make that progress. What are they willing to give up to get? And it's typically a combination of that context, outcome, and trade offs that allow us to design better, more valuable products for people.

You mentioned the idea of Personas, Bob. How does that fit into this larger idea of product design and the progress that people are trying to make?

I think Personas are what end up is. Again, I think it's pattern recognition in the demographics. So people will basically build patterns around combinations of. They're this old, they have these many kids, they have this kind of income, and it's looking at the data set they have. And so it's better than averaging it all. But to me, a Persona is kind of like a consumer without a soul. Like, we know all the things about them, but we don't know what they value and how they make decisions. And so to me, just because I'm 54 and I have this income and I live in the zip code. It doesn't cause me to buy this new car. And so Jobs is actually trying to strip away those other kind of larger, higher kind of demographic, I'll say, variables and literally just look at it and say, what in the world was going on in your life today that said, boy, I need to buy a new car. And what were you hoping for? The moment you said you picked that one. And if I can see those things irrelevant of, to be honest, the solution I picked and irrelevant of my demographics, I now have a way clearer way to design better products.

And I mean, to me it's getting at the why, right? Like that simple. I mean, like you said, demographics, it's better than nothing, probably, but it is. It's soulless, right? It doesn't tell you anything about the motive forces behind the choices people make. And the reality is, is that until recently, we didn't have the tools and even the capabilities to get at this at scale, but now we can. Right? And the truth is, is that anytime you care about the motive forces, the why, it's always an individual level phenomenon.

That's right. And the only way it can really be leveraged is through aggregation as opposed to segmentation. So to me, it's pulling it or aggregating together as opposed to trying to pull it apart.

Once you've identified the why, how does that change how you think about a product or a job spec, let's say, Todd, I know you talk a lot about education and sort of matching skills to future employment opportunities or in schooling in particular. What sort of opportunities does that open up?

So it gives you a couple of things. One is it starts to help you understand Clay has a theory called modularity interdependence. Right? And the notion is sometimes you have things that are very integrated and you can only do one thing with it, or you can actually create something that's very modular and they have trade offs between the two. And so part of this is once I understand a set of jobs, I can actually see where if I could build a platform to do something and then have customization to do the different jobs in a different way, but have an underlying kind of fundamental system to do that. And so part of this, it allows you to actually get scale faster. But the other part is it allows you to actually drive satisfaction in completely different variables and completely different situations for in some cases, the same customer. The other part, and Todd hit on it, is systems thinking is at the core of this. So it's really thinking about the market as a core or what are the systems that enable consumers to make progress. And so if I can actually surround myself with context, outcome, trade offs, higher fire criteria, I then can actually design a whole bunch of different systems to do that job. So it allows me flexibility in cost and delivery and a whole bunch of other things.

And if you imagine on the human side too, like, I think the getting at and understanding the why behind things and having people be able to understand that for themselves is the way forward to solving what I think is one of the biggest problems in corporate America today, which is just this profound lack of engagement by people. Like, it's unreal that like a majority of people are disengaged at their jobs.

So I was just listening to a little bit of the book on the way over this morning when I was flying over, and I love the notion of. And it just reminded me of just healthcare, which is the difference between choice and picking. Right. And that so many people think that they have a choice in healthcare, but the reality is they just pick the program that fits best for their, you know, their family and their budget and everything else. But they actually don't engage with it. They have no idea what it does. They have no idea. There's no way that the language alone is so complicated that they can. They can't even figure out how to engage with it. And so the lack of engagement, it's not having them do stupid things to kind of know about it. It's literally making sure it's relevant to their lives and they know how to make choices. My wife had a medical emergency back in November and we literally could not decide where to go. Do we go to urgent care? Do we go to the emergency room? Do we wait for the. And all back and forth. And next thing we know, she's in the ICU for five days. I'm a fairly educated person. We should know what we have, we know what our insurance is, but we actually don't even know how to engage in the right way. And part of it is the language. The notion of fit is really, really. It's an important concept that I think is kind of the next thing of how do we actually describe fit? Because now it talks about a relationship between two things as opposed to average, which can be just basically one thing.

And so this idea might help us get to a better fit between a spec, let's say, and an individual.

Yeah, yeah.

And you think about the power that this unlocks in the sense that, like, there's this other variable that we're not considering, which is the why behind what people do. And even like when we hire people, it's pretty remarkable to me that we'll spend a lot of time looking at your rank and grades and test scores and your strengths and. And yet rarely ask why. Like, why do you care about this?

Yeah, so I've been doing job interviews around people who have switched careers. It was like, why would you leave this company and why would you join this company? And when you understand the causal mechanisms, you start to realize there's some patterns here, which is, I don't feel like I can grow anymore. I feel like there's no sense of progress for me where I'm staying. And to be honest, I'm more valued if I go out and come back. All of a sudden you sort of realize like, oh, we have no way of which actually they feel like they're making progress and like, oh, you want more money? No, money has nothing to do with any of this. And so you start to realize that the variable set that I call it the supply side has, which is hr, is money and benefits and vacation time. It's like, no, no, no. I want challenge. I want a great team. I want, like, these are other variables that aren't even in the equation yet.

And I think that as we surface these patterns and they're actionable, I think you've seen companies, especially in talent scarce industries, trying to fumble around figuring out, well, we got to do something to keep talent, right? So now we have Korean barbecue and seven other kinds of food. And you're like, sure, that's great, but like, you're actually missing the point about what most people really want. And our ability to surface those patterns and say, look, address these and you're going to have a very engaged, motivated workforce. And it probably cost you less than all the food.

Well, the other part is it runs out. And so this is one of those things is like, okay, it's all about food. And you end up spending so much time and effort and all the wrong things that you're trying to do as opposed to digging why, you know, and so when I was in Japan, they had this thing, the five whys. You had to ask why five times. And if it's food, you gotta ask why, why, why? And part of it is, I want a sense of community. I want to eat with other people. Like all of a sudden it's not the food, it's the environment that the food's served in that's sometimes more important.

Yeah. So as A way of getting at that.

Why?

You've both talked in different ways about designing toward the edges rather than to the average. Talk about what that means.

When you're in systems that are optimized around average, we often confuse muddling through with actually thriving. Right. And the people that can muddle through. It masks a whole bunch of things that could be optimized.

What do you mean muddling through?

Okay, so let's take education. So just because I get straight A's does not mean you're actually developing my full potential. It doesn't mean I actually care. But that's what we use right now, right? Like, we don't know. It's like, well, wait a minute. This system more or less works because not everyone's failing. Right. And some people get good grades. But if you're trying to develop people, how would we understand the pain points? How do we understand where the need really is? And what we find is when you go to the edges of the distribution and you actually play there and you figure out how to make things work there, those are not different people. They're not like those people over there. They're actually revealing the thing that is usually true in general.

The people who may be failing the class.

Yeah. Or like you think about, like, I mean, you can name the products, right? Like you think about closed captioning. And we said, well, it's going to be for a specific disabled community. Except for they're like, I think, third or fourth on the list of who actually uses the Moses. That's right. It's me in bed watching TV with my spouse because I don't want to get divorced after having the TV on till one o' clock in the morning.

That's right. I think part of it also is that we, I think, I know you have. And I, we both kind of grew up on the fringes. Right. And so I think the notion to me is that what I've learned is that when things are in the middle, when they're in the muddle, people have no language to articulate it. So if I ask if you use Tide every week, and I ask you, how do you use Tide? You know, why do you buy Tide? To be honest, you just make it up. You have absolutely no idea. And what happens is we just get a larger sample size to help us get common language of, oh, it smells good and it, you know, and it gets the clothes clean, but they don't know how. But if I ask somebody who switches from cheer to Tide or Tide to cheer, I can learn really, really quickly, like, what was going on. Why? Because we're creatures of habit. The moment that we struggle with something and we make a change, there's motive behind it, there's outcome, there's intent, there's energy. And so part of it is being able to dig through that and figure that out. Clay and I talk about this a lot. We end up measuring all the wrong things. We measure the things that are easy to measure, that aren't meaningful, but we very rarely measure the meaningful things that are hard to measure. So, like, you go to the doctor, they measure my weight, my blood pressure, and my temperature. How is that meaningful to my body and what I need to do? And so I have a different doctor who basically helps me with all my thinking patterns because of my dyslexia. And so she literally has tests that are way harder to measure, but they're so much more important to helping me be a better me. And so, to be honest, I very rarely go to my primary care physician. I. Almost every 90 days, I'm in my neurologist's office.

Yeah. So I want to talk about data and this idea of measuring things that we're able to measure that may not always be the most useful things to measure in a minute, but before we get there, Bob, this idea of designing to the edges and sort of going towards the end of the distribution to find out really. To find really meaningful insights strikes me as related to something you've talked a lot about, which is this idea of hacks and even, in some cases, petty law breaking as a way of studying those behaviors, as a way of identifying people's struggling moments or sort of the causal drivers of behavior. Can you talk about that?

Yeah. So in terms of trying to find where to innovate, first of all, consumers don't know. And so if you actually go to ask them directly, they have no notion of what's possible. They're only going to repeat things they already know. And typically, they have no idea about any of the realities of trying to make any of that work. And so. And to be honest, they lie. And they don't lie on purpose, but they lie for a whole host of different reasons. So part of it is being able to understand what causes those things and taking a step back. And so, to me, the struggling moment is that seed. And if I can find where people struggle and when they struggle, we can actually. We can actually figure out where and when and how and who and the five whys behind all those things. You start to realize, like, you know, Uber is just an extension of all those people hanging out at the airport trying to give you a ride when they couldn't give you a ride and say like, well, how do I make this legitimate? There's points of how many people want to want a better cab but can't have it? How many people want a better class or better teacher but can't find. My favorite question here is, how do you shop for a doctor? All these things where we struggle and we just kind of assume like, well, you just get the doctor you're assigned. It's like, no, that's just not acceptable. So how do we actually build a system where people have choice and not picking, but choice? What's the difference between picking and choice?

Yeah, I mean, picking is when you're given stuff and usually you don't actually care too much about the difference, but it's like someone else serves it up to you and you're like, I guess A, right instead of B. Choice is very active. It's driven by an understanding of your own motives, which I think is a fundamental part of all this. And we've already kind of touched on that. And what's so fascinating is when you see choice rich environments genuinely choice rich environments, it's not just serving up options for people. It's allowing them to carve off, like, options that you didn't know existed. And so we've spent a lot of time studying people who do this in spades. And I think this kind of ability to genuinely understand how to make choices on your own behalf, I think is fundamental to both good consumer markets, but also the ability to live fulfilling lives.

And can you convert from one to the other? Can you take an environment in which people are picking and convert it to one in which they're choosing to.

Look to Bob's point, like, if you want to start to see how you convert those, go look on these margins of stuff that we would have looked at as behavior would say, well, let's not encourage that. The way we think about it is, you know, in economics, we think about things like preference signaling. Right. Being very important to markets, people have got to be able to signal their preferences. Well, when in systems that don't have a lot of authentic choice, you can actually, you can mistake having to go along as if you agree with it. Right? That's right. But so you go to these margins, realize it's often not something unique about those margins. They're. They're revealing the preference of the broader population who doesn't either can't signal or.

Don'T have the ability, don't have the language. So you're finding the anomaly. And that anomaly becomes the n of 1. And then the question is, how many people are there behind that N of one that actually want to do the same thing but can't, which is what we call non consumption. And so ultimately, we're looking for places where people want to make progress but they can't. So the other part about choice is that they're engaged and that there's an assumption of a preference. And so the fact is I can pick one, but that doesn't mean I actually prefer it. And so part of this is really digging way past almost where the boundaries are. So to me, the notion of a standardized system is like, how do I get outside the standardized system? So I can actually see free will at its best. And free will will say it's about fit, not about. Not about kind of failure.

Yeah. And, you know, so what's so powerful about this is when you think about all the social systems that we're trying to like, disrupt right now that don't work. So you take something like education. Well, if you follow this to its conclusion, you realize the place I start to look is things like homeschooling or where parents are supplementing their education because they're making choices. They're either spending their own time or money to get their kids something. And sometimes they can't really articulate it. You have to help them figure out. But behind it is a motive or a set of motives that turns out to be way more prevalent in folks that are just like, almost just captive. Like, I don't know, my kid has to go to this school down the street because. Because that's what it is.

That's right, though. They're hating school, they're getting bullied. This is happening. That's happening. It's like, how do I actually decide to take them somewhere else? Or how did you decide to homeschool your kid? And then they'll tell you the story and you'll be like, oh my gosh, we've heard that story about 300 times before. But they just didn't come up with homeschooling. And so once you understand these voids where people are trying to make progress and they can't, now it's actually time to say, hey, some people went to a new school, some people moved, some people actually homeschooled, some people. And you start to see all the. What do these solutions have in common and how do I actually put them together and weave it for those People who can't afford it or can't don't have access to it. How do I actually design access?

And you know what I love about this is that to me, the distinction between this and the system we have is it's respect for individuals. Either you respect people and you believe that they have a right to be able to, like, exercise choice, and you think that if we provide a supportive environment that actually it becomes mutually beneficial for everyone, or you believe that you should have a few people on top dictating the sort of efficiency of the system. And we've played that game out now. It doesn't work.

It doesn't work. So Michael Horn has done a great book called Blended, where he walks through and basically says, you know, if you do whole classroom teaching, you're teaching to the middle of the class. But now I've got these kids who are on the top end who literally move through it so fast they're bored. And if I don't give them something, they get in trouble. And then I've got the kids who are a little bit slower in this subject and they need some extra help. I'm trying to manage the variation in the room. And the reality is there's a whole set of new ways to do this, which is how do we actually get kids engaged in understanding what are their learning objectives? How do we actually have them be held responsible for actually making progress? How do we get them to help each other? And so the role of the teacher goes from being the sage on the stage to a guide on the side of literally orchestrating the process of learning as opposed to being the knowledge. And I think those kinds of things, when you start to see people doing amazing progress in math and in science and some of these subjects where, you know, and I'm talking as young as kindergarteners, where they're learning colors and shapes and stuff, and they're doing it with guidance as opposed to basically memorization and standards.

Does it ever work? I know, Todd, you talk about some of the quantitative basis for this idea of the average in your book. Is it ever okay to use those kinds of methods? Is it always harmful?

So first of all, you're a sociologist, go crazy. Like, averages are awesome, right? Like, if you are, if you care about populations, that is your go to thing. So when it comes to making statements or about individuals, this is what I would say. Can averages sometimes work? Yeah, sometimes. But we know we even have a mathematical theorem, it's called ergodic theorem, that tells us the conditions that have to be true for you to be able to use group averages for individuals, and we violate them every time. So if it works, it's sort of like lucky that it did. So what I would say is there's no reason to go ahead and keep assuming it'll work when you can start at the individual level and work your way up. And guess what? If it turns out to work, there you go, right. You're not out anything. You will arrive at that same pattern through individuals. If you started an average, but almost, almost always what you find is at a minimum, it wasn't enough. There's always more than just that exists through averages. Sometimes you realize that the average literally was an artifact and it doesn't represent any. Anybody at all. We found this out in neuroscience and in brain imaging, where we spent so much time looking at averaged brains. And my colleagues, like Mike Miller, found that the patterns we found for memory retrieval on average, like, literally didn't apply to any single person.

Wow.

So we had whole theories of how your brain retrieves memory that relate to absolutely nobody.

So we're violating these. These conditions left and right.

What are the conditions?

Yeah, what are those circumstances?

You have to be a frozen clone is what we call it. So quite literally, like, no kidding. This is the mathematical requirements for it to work. You have to be identical to the other people in the group on all the important dimensions. Identical. And you cannot change over time. So let's say the very definition of learning would be change over time. The definition of development be change over time. Right. And the mere fact that even if we don't change over time, we are wildly individual right now in this room. And so it's one of those things where if we were having this conversation 100 years ago, we didn't have the ability to do anything else. And so what we had to do is say, well, how could we use aggregate data and like, what has to be true? And we're like, okay, it's probably close enough. And you never had a way to really validate whether these statements were actually true. And now we do. And so I think that what's really interesting is in the fields where there's moral imperative, we've moved on so fast. If you're in medicine and you know that relying on averages is killing people, it's just immoral. So that's already changing. It's not for nothing that's gone first, but I think in these other places, it's not surprising that you're seeing in with consumer behavior and consumer areas that that's going quick because there's an obvious incentive. What we're after, I think we both share this thing, is like we have to bring that to the rest of our systems, right? It's just. It's not okay to put people in a standardized education system that views them as an interchangeable part.

The way I would say it is average is actually really important on the supply side for me to build standards. So if I want to build an engine, I need to make sure that I'm manufacturing the cylinders at this diameter, at this consistency on these things. And so what we did is we've kind of overused the notion of average from the physical things and things that we use from an engineering perspective into the social sciences. And I just, I think the social sciences, it's a different system. And so the reality is that because there's about motivation and there is about intent and there is about fit and there's choice, it's very different. An assembly line that builds engine doesn't have choice. There's people who make settings. They have to basically make sure that they start at the end. They have certain requirements they have to do. And in those kinds of things to get scale, I need the average. I need to know variation. I need to know when something's outside of variation. I need it to improve quality. But this is not a quality problem. This is actually a market problem or basically a human problem, not a machine problem.

And I think. I think it's really well said. And I think that one of the big distinctions that I think for most listeners is like, when you take this approach that we're talking about here, it changes the way you think about individuality. So historically, we either see it as a nuisance variable or we see it as something to select on. So it's about classification and selection. This shifts to. Is. It's a design challenge. When you realize that this individuality matters in a motive way, right? It actually matters, then you design for it. And when you design for it, what you find is a lot of people are capable of doing really great things in the right context. But when you've designed it away, it's not surprising. You get a pretty narrow talent pool and a narrow sense of satisfaction, engagement.

So if we have the analytical tools now, we have the ability to design for these kinds of complex systems. Why are we all still so wedded to this idea of the average?

I think part of it is that we have to get people to step back from the problem. Because again, they confuse, for example, the notion of a root cause. What's the One thing that caused this to happen. And for me, it's sets, sets of things in the past, sets of things in the future or imagined future. This is a whole bunch of sets of motivation, right? And so what happens is because we're thinking about average, we're looking for the outlying cause that caused you to buy your sunglasses. And there's not one. I always think of it this way, in a sense, spreadsheet. I have a column that's got a thousand rows in it, right? And that one column, I'm going to average a thousand pieces of data in that one column. But the reality is each data point is actually dependent upon every other column in that data. And so when I actually average it, I actually negate all the other things that are wrapped around it that make it isolating and make it basically wrong because there's dependencies between the columns that actually make it there. That's why cluster is so much more powerful than segmenting and averaging.

And I think that like one of the big challenges here is we're dealing with a fundamental assumption, right? And assumptions exist. You're not, they're meant to get you off the ground. You're not supposed to think about them. And then you, you're trying to uproot a deeply held assumption. And yet everywhere you look in your lived world, most of it's telling you actually averages are pretty good, right? So you, it takes a lot to get past that. And it's not surprising, short of stronger pain points for you as a person, broader counter, examples of what else is possible, it's not surprising that people struggle. And by the way, we just have to give them, we have to give them another way of thinking, right? It's why, it's why the job stuff is so important.

The thing I go back to is if you watch football and you show the stats that they have, the stats are everything in the past. It has nothing to do with predictive, predictive in the future. And so the notion is, if you actually understand that the quarterback is in a completely new situation, seeing a completely new defense in a completely new way, but he has an 83% completion rate, it actually has nothing to do with what he's going to do next.

It drives me nuts.

So this whole notion is there's a huge misuse of stats over and over and over again. One of my daughters is a math major and she calls me, she goes, dad, people are confusing statistically significant with important. And it's like if we just increase the sample size, we can make something statistically Significant. Which makes it now important. It's like, no, that's not how it works. She goes, I know, but I don't know how to tell them. And so there's all these kinds of misnomers of how people want to use stats and data and average to literally tell their story.

And garbage in, garbage out.

Exactly.

It doesn't matter if you have 10 times the garbage.

If I increase the sample size, it doesn't actually make a bad question better.

So how can we be sure that we're using that information responsibly and that as we've been talking about, you know, we're understanding the difference between the why versus just the what?

Yeah, I mean, look, I think there's a couple of things with the data. Like, there's an issue of quality, which is to say that you got to measure the right things first of all. And I think we've already talked about.

Like, but I think the notion is how do we come up with the right things? And I think that's one, number one on your list, is we aren't having the discussion about what to measure. We're just not doing it. And it's, and it's, oh, I can measure. The people are literally offering up 50 things to measure and they're like, oh, that's great, and we'll just amass big data and we'll figure it out from there. And it's like, my experience is that never happens.

It never happens. And by the way, what's really dangerous is we have to resist the temptation to be like, well, in the meantime, until we figure out something better. No, because what happens is that it captures. And suddenly this thing that we, we all knew was kind of crummy and not a good idea now becomes the way everyone talks about things. Right? It becomes the metric and things start to drive against it. And now you're stuck with this thing that is really hard to displace.

Do you have an example of where that's happening?

Oh, my goodness.

Like, net promoter score is my favorite. Oh my God, kill me with that one. Right?

I mean, think about in education, our reliance on norm referenced standardized assessments. By definition, norm referenced means we don't actually care what you know, we care how you compare to other people. It's reference to the average and we give you a score. So when parents see their kids stuff, come home, they're like, well, you scored in this percentile. I'm like, yeah. It does not say what they know. It doesn't matter. You cannot act on that. If I Said, okay, how do I help my kid get better? It cannot tell you those things. And I don't think anyone would start the system over and say, you know, it's a good idea. Norm reference standardized assessments. We have other ways of doing this. Right. We can do individual growth scores, we can do all kinds of stuff. But you have an incumbent advantage with this particular kind of assessment and you have lots of people who have a vested interest in preserving it. So for me, it's like, let's have the conversation about what's worth measuring and don't feel like we have to have this rush to get crummy outcomes and metrics just to have something just because they're there. Yeah, and by the way, we'll get, I mean, Bob's archon, we'll get to the patterns pretty quick. We'll get to the insights. It is not hard.

It doesn't take long either. It's not meant like I can do 10, 12 interviews and start to see patterns very, very quickly.

So we're very lucky to have both of you to learn from. I think in our last few minutes here, I'm just curious whether you have any final words or parting advice for our listeners.

Yeah, look, our society is moving towards something that's more personalized, whether we want it to go there or not. And I think the open question is, what is it going to be and who is it going to benefit? And you can imagine that we double down on an average base view of people and we use our big data to actually lock in this antiquated idea that we've known is just not true. That's one way, right? We could just do it on steroids and it's just like, what a wasted opportunity. Or the systems we create actually do go the way of individuality. And by the way, most of these things like Amazon and Facebook have, or they're trying to, they're trying, they're trying and suddenly our systems know us better than we know ourselves. And that asymmetry in knowledge is frightening to me. I think that is the nightmare scenario where we lose agency, we lose our ability to honestly be able to make choices on our own behalf. And unlike the sort of standardized model that where it was just brute force, you knew it was being done to you in this world, you will thank them for it and you'll call it personalized. And so I see this as a really important moment in time when if we can get rid of the shackles of average based thinking and we can adopt this approach to individuality that Bob and I've been talking about that we can actually get somewhere where we have a society of people who honestly feel like they have a shot at living fulfilling lives. And I think it's vital to the health of our democracy and it's vital to our economy.

I agree. The way I would add is I believe that we grew up and a supply side economy of people making products and people trying to figure out how they fit into people's lives. I think the future holds a more demand side economy, which is basically as people become more and more aware of what progress they want to make and as we become smarter and smarter about ourselves and what we can do. The supply side, which is mostly where average comes from to build average products, average systems, standardized things. The reality is that it's like the jet fighters where they told us it was impossible to actually make the C seat and the cockpit and everything adjustable and what, six, eight months later they did it. I think it's a two way street here because I think consumers actually have to get more engaged. I believe that we're actually unengaged in very important things. And the reality is that they're making them kind of, they're picking things as opposed to choosing things. And I think getting people to be more mindful about the choices that they're making. And so for example, I'm working with somebody on an election right now where it's like not voting is, they're not picking and they need to actually choose a new mayor. And by, by using that as the campaign vote of like your, your, your city is struggling. It's because you're not voting, you're not choosing to be engaged and basically making sure we have it as a demand side issue.

And so my news feed, at the end of the day, even if it's ultimately personalized in a sense, it's still not, I'm still not picking.

Yeah, this is, this is about who, who gets to decide the kind of life you're going to live, who gets to define what success is for you and how you get to achieve it. And what we're talking about is empowering on the demand side individuals to know enough about themselves, the things that truly matter, the things that have been masked in our standardized system and allowing them to put that into play in our democracy and in our market economies. And I think that everybody wins in that scenario. And for me, the barrier right now is a mindset and we've got to surface this. We've got to get people to realize what's actually holding us back and that we have a way forward. This is not rocket science. Like, we can do it.

Well, hopefully our conversation today can help folks realize just that. I think that's an awesome place to leave it.

Great.

Thanks so much for being here.

Thanks, Shay.

Thank you.

Thank you for listening to us at Disruptive Voice. If you like our show and want to learn more more, please visit us at our website or leave us a review on itunes. Until next time, good luck, everybody.