The inevitable weakness of metrics

Welcome to MIT Technology Review Narrated. My name is Matt Honan. I'm our editor-in-chief. Every week, we'll bring you a fascinating, new, in-depth story from the leading edge of science and technology, covering topics like AI, biotech, climate, energy, robotics, and more. Here's this week's story. I hope you enjoy it.

Narrated by NOAH. Listen to more of the best articles from the world's biggest publishers on the NOAH app. Or at newsoveraudio.com.

Brian Gardner writes, The Inevitable Weakness of Metrics. There are plenty of useful things a metric can reveal. There are even more it can obscure or corrupt. It took me well over a decade of tracking my own life in ever greater detail to fully appreciate this duality, which probably reveals something about both me and the nature of measurement. Like a lot of people bitten by the self-quantifying bug, I initially started gathering personal data to pursue a nebulous collection of goals and desires. As a sedentary technology journalist, I wanted to feel better physically and emotionally, to get outside more, and, where possible, to bring order to some of the messiness and uncertainty of my daily existence. These all seemed to be things that could be improved with the cool clarity of numbers. Self-quantifiers often get stereotyped as obsessive self-optimizers, and many of them are, but my reasons for producing and collecting personal data were less about life-maxing and more about life-meaning, at least at first. As most people who know me will attest, I do not have now, nor have I ever possessed, a productivity mindset. I'm also not at all that interested in life hacks, shortcuts, or new ways to compare myself with other people. Instead, what I wanted out of metrics—what I hoped I could divine from a never-ending stream of numbers about my health, work, and social life—was something more elusive: self-knowledge. This was my first mistake. The idea that "the more we know, the better" is so profoundly embedded in our culture that it feels weird to even point it out. Since at least as far back as the Enlightenment, the primary way we've all agreed to go about knowing more has been through measurement and quantification. After all, more knowledge—more data—leads to better decisions, which leads to happier, more fulfilled people. Or so we're told—and with increasing frequency in the era of AI. When two Wired magazine editors, Gary Wolf and Kevin Kelly, coined the term "quantified self" in 2007 and helped launch the movement we are all now helplessly a part of, they were essentially selling this very idea. 'Unless something can be measured, it cannot be improved,' wrote Kelly in an early blog post, doing his best impression of Lord Kelvin. So we are on a quest to collect as many personal tools that will assist us in quantifiable measurement of ourselves. Almost 20 years later, that quest is easier than ever thanks to a flood of devices, apps, and websites, all designed to help us build our self-knowledge through numbers. My first "tool" was a small plastic clip-on Fitbit I started using in 2011. It did one thing: count the number of steps I took in a day. As a lifelong video game player, I was already well acquainted with the motivational power of simple scoring systems, and I hoped my new gadget would offer the gentle numerical nudge I thought I needed to step away from my Twitter feed and, if not touch grass, at least walk next to some. Walking also seemed to be one of the few times I had what could charitably be called intelligent ideas, which seemed like another promising byproduct of doing more of it. Alas, that was short-lived. I can't tell you precisely when getting out into nature more or thinking smarter thoughts stopped mattering to me as goals, but I suspect it took no more than a few weeks. What I can say with certainty is that my initial goal of 6,000 daily steps quickly turned into 10,000, which then jumped to 15,000 and eventually settled at 20,000 for years. Stories about becoming a steps guy are clichéd at this point, and they've earned that status for a reason. It didn't take long for me to trade in pedometers for heart rate monitors. I also started running. Smartwatches, sleep tracking rings, and an embarrassing number of macronutrient tabulating apps. Outside the health and fitness realm, my early career as a journalist also happened to coincide with the rise of social media and web analytics tools like Chartbeat, which promised to further quantify difficult-to-measure aspects of my life like job success and impact by tracking things like page views, followers, retweets, likes, and all sorts of other attentional metrics that now carry great weight. Ultimately, during the 10+ years I diligently tracked my heart rate, steps, active calories, sleep, story engagement time, stress levels, and other metrics, I gained virtually nothing in terms of greater self-knowledge. I suppose I did learn that I like to make numbers go up and down, but who doesn't? The swirl of data that followed me everywhere did not lend additional meaning or insight to the way I relate to myself, my work, or the important people in my life. In fact, the more I used numerical proxies, the worse I felt about pretty much everything. What I did learn were two important lessons about what happens when you try to quantify the minutia of your life. First and foremost, whatever the amount of data you're currently collecting about yourself, it will never feel sufficient. There's always a new metric around the corner, a better way for a tracker to remix its readings and more accurately measure what's important. Heart rate variability, daily stress, exercise readiness, cardiovascular or fitness ages. Measurement begets more measurement. You can count on it. The second lesson was less obvious but no less significant. The more personal or nuanced your goals are when you set off on your self-quantifying journey, the more likely it is you will ultimately replace them with some simplified metric or ranking. Want to become a better journalist? Why not use page views and leaderboards as a proxy for success? Enjoy cooking and want to improve? Foodie metrics dictate that more complicated recipes with longer ingredient lists are the answer. Even when we know that the value of good journalism isn't reflected in how many people read a given story, or that the joys of cooking are as much about improvisation and experimentation about successfully following some complex recipe, it's hard to resist the allure of a simple score or stat. Metrics inevitably redefine your core sense of what's important, whether you're aware of the trap or not. Over the years, people have invented various terms to describe this phenomenon. In his recent book The Score: How to Stop Playing Somebody Else's Game, the philosopher C.T. Nguyen calls it value capture. Value capture happens, he says, when you adopt external sources of measurement and then let them rule you without adapting them to suit your life. In value capture, you're essentially outsourcing your values, Nguyen writes. You're letting an external metric or ranking set what's important for you. Crucially, you're also outsourcing the process of figuring out your own sense of meaning. It's why my walks quickly shifted from feeling meditative to prioritizing miles. Individuals, institutions, and indeed entire societies can fall prey to value capture. In fact, once you start noticing it, you start seeing it everywhere—in journalism, education, and business, but also in our food, our hobbies, and yes, the way we measure our health and happiness. Here's how Nguyen puts it: Value capture happens when a restaurant stops caring about making good food and starts caring about maximizing its Yelp ratings. It happens when students stop caring about education and start caring about their GPA. It happens when scientists stop caring about finding truth and start caring about getting the biggest grants. It even happens in religion. A pastor recently told me that his church had become completely obsessed with baptism rates. The higher-ups had established an internal leaderboard in which the pastors competed on monthly baptism rates, and it was starting to dominate everybody's attention. He'd found himself caring less about the long-term spiritual development of his flock and focusing more on trying to deliver popular sermons that would up his baptism rates and move him up that leaderboard. At its core, The Score is trying to untangle a mystery that Nguyen, a specialist in the philosophy of games at the University of Utah, has been thinking about for a long time: Why is it that numbers and scoring systems in games can be the source of so much joy and fluidity and play, But public measures and institutional metrics, i.e., scores that apply to the real world, seem to drain the life out of everything and thrust us all into a bleak mindset of grinding optimization. To begin to answer this question, he turns to one of the foundational inquiries into the limits of data and quantification: Theodore M. Porter's 1995 book Trust in Numbers: The Pursuit of Objectivity in Science and Public Life. Porter, a historian of science who specializes in the social power of numbers, has spent his career looking at why quantification has become so dominant, not just in political and bureaucratic life, but everywhere. One of his key insights about the inherent attractiveness of quantification, which he calls a technology of distance, is that it minimizes the need for intimate knowledge and personal trust. Put another way, metrics travel extremely well between different contexts and are easy to grasp and aggregate. Whether it's a student's GPA or a country's GDP, these measures are understood by pretty much everyone. But that understanding comes at a price, Porter reminds us. To arrive at a clear metric, you inevitably need to simplify what you're attempting to measure, often jettisoning heaps of nuanced, qualitative, or open-ended information so that others can find the resulting number legible. No one, hopefully, believes that a GPA captures in any meaningful way a student's entire educational experience or aptitude for learning, but we've agreed to use it because more qualitative assessments are onerous to wade through and require expertise to decipher and compare. Ditto for the economic metric of GDP, which politicians and societies are now compelled to drive higher and higher because a group of economists once concluded that that this figure correlates with general economic well-being. This is the essential tension at the heart of all data, argues Nguyen. Any institutional quantification, he says, requires that the evaluation procedure and its product be comprehensible across contexts. That profoundly limits what the metric can actually measure. In value capture, you're ultimately taking that decontextualized nugget and internalizing it, he writes. You're guiding your life using an evaluative technology that has been engineered to travel between contexts by stripping it of nuance. Every so often, I'll find myself in friendly debate with a numbers person—a statistician, an economist, or a friend who's still a committed self-quantifier. After patiently listening to my measurement-gone-awry examples—the disastrous attempt to quantify pain as the fifth vital sign in the mid-1990s, which exacerbated the opioid epidemic, or any of the countless examples of the McNamara fallacy, where decisions in academia, medicine, and politics are based solely on what's easily measured. Many will insist that I'm misunderstanding or misinterpreting the whole point of measuring. Metrics, they'll say, are simply a means, and the important questions concern the ends for which they are used. In other words, these unfortunate outcomes amount to user error, —not something inherently dangerous or misleading about the nature of measurement. At some point during these conversations, Goodhart's Law will invariably come up, usually as an explanation the metrics-minded deploy for why the ends get all mucked up. The principle, which is attributed to the British economist Charles Goodhart, is often expressed as the following: When a measure becomes a target, it ceases to be a good measure. I have a profound dislike for Goodhart's Law, not because I think it's untrue, but rather for the way it gets interpreted. As Nguyen notes, Goodhart's Law says very little about why metrics fail to capture what's important or what to do about it. Find better measures, some will conclude. Don't let metrics become targets, others will insist. These are not helpful takeaways. All measurements, I would argue, are in fact targets, whether you intend them to be or not. Metrics inevitably present one direction or option as better, Newian writes in The Score. Longer lifespans, faster student graduation rates, more page views, higher customer satisfaction scores. What people are talking about when they bring up Goodhart's Law isn't human error. It's actually a fundamental problem with measurement itself. I want to be clear here. Measurement can and does serve a number of vital functions. It has, in a very literal sense, made the modern world possible, with all its life-saving, suffering-reducing, and awe-inspiring scientific breakthroughs. When used with care and diligence, metrics can make our progress—or lack of it—clearer and more transparent. Are we decreasing carbon dioxide emissions or not? They can also introduce accountability into formerly opaque systems, such as by measuring whether a company is complying with state and federal regulations. They can even make us more objective, reduce biases, and galvanize us to act. But as Nguyen points out throughout The Score, the fundamental weakness of metrics comes when we use them to pursue subtler, more personal goals. What I think many of us miss— what I know I certainly missed— is that there are always trade-offs when you try to distill something important down to a data point. When we turn to metrics to understand ourselves, our social world, and culture as a whole, they will never come close to capturing what matters. Even worse, they'll often actively obscure it. Today, I find that numbers have very little to offer when it comes to my daily work, my physical or mental fitness, my relationships, or any other part of my life I consider important. Granted, I'm lucky enough to be in relatively good health at the moment. I don't have to track my glucose levels or monitor my blood pressure. As a freelance writer, I also have the luxury of not having numbers foisted on me in the form of key performance indicators , objectives and key results , or any of the endless quantitative evaluations that come baked into pretty much every corporate and gig economy job. Still, in a very real sense, there is no escaping metrics, or especially the logic that accompanies them. Knowing has become numeric, and we all live in a world that increasingly sees us as a collection of numbers, as data subjects. The first and most urgent challenge I'd suggest is finding a way to keep us from seeing ourselves and each other that way. This won't be easy. As Porter, Nguyen, and countless other philosophers, anthropologists, and historians have already observed, the language of numbers is largely how we ascribe value today, as well as how we digest and metabolize our relationships to ourselves, to others, and to the world around us. Indeed, many of us have accepted not only that metrics have a natural existence in human affairs, but that there are in fact no aspects of human life that cannot be somehow translated into data. So how do we push back? Nguyen's book offers a useful first step. As he notes again and again in The Score, believing that numbers say something real or useful about human needs and desires gives them power. We can, at the very least, start to seriously question that belief. To ask what meaning and pleasure we might be giving up in pursuit of a metric. Doing so will hopefully lead to another realization: that playing the numbers game is ultimately a losing proposition for humans. If we insist on expressing our worth through attentional metrics and productivity scores, if we continue to turn intelligence and creativity into a series of benchmarks for AI to surpass, we've already lost. Of course machines will surpass us in a world built around metrics. That is literally what we create them to do. The answer is not to turn ourselves into machines too. If there's one thing that keeps me up at night, it is that we've become so accustomed to seeing and understanding the larger world and ourselves through numbers that it has deprived us of the language to express what's fundamental and valuable about our own humanity. We need this ability now more than ever, especially if we're going to adequately answer two of the most important questions of our era: What are humans for? And what is AI for? As part of my own attempts to disentangle myself from a life of numbers—efforts that started shortly before COVID—I've abandoned most of the tools of measurement I spent a decade collecting. I've largely given up on social media. I stopped using apps to track my health and well-being. The watch I currently wear tells me the time and the date and nothing else. In fact, the only holdover from my days of obsessive self-quantification is a dogmatic devotion to walking—without all the step counting, of course. These days I walk when I'm feeling disillusioned or overwhelmed. I walk when I can't figure out how to finish an essay. I also walk because I enjoy spending time outdoors with my dog and catching up on the details of my neighbors' lives. The benefits of pursuing this daily activity are as clear and obvious to me as anything could be in life. I just can't express them in a number. You are listening to MIT Technology Review, where Brian Gardner writes The Inevitable Weakness of Metrics. This article was published on June 19, 2026, and was read by Michael Saitow for NOAA.