Raw AI usage metrics like prompt counts can hide inefficiency and declining quality. More prompting often means more time spent fixing AI outputs, not more productivity. Track actual value by combining usage data with direct questions to users about what changed, what became possible, and whether AI work actually saves time.
just because you have lots of prompts doesn't mean your work output has changed, right? So sometimes it could even be worse. This famous word from last year, work— AI work slop, right? Where you actually spend more time fixing AI output from either your colleagues or yourself than getting what you need out of AI. That's counterproductive. But where does that show up? Well, it's actually the opposite. If you just measure the usage. It could look like everyone is using AI, or a lot of people are using AI, or they are prompting more, perhaps because they're terrible at prompting. And like, who knows? There's a lot of things that underneath that we need to start measuring. So I also advocate for things like a combination of both quantitative and qualitative approach to measuring and actually asking people, what are you doing with AI? What do you get out of it? What has changed because of AI? What could you not have done without AI?
