In this YouTube monologue, Andy Campbell critiques conventional workplace quizzes that rely on fact‑recall or easily gamed multiple‑choice items. He argues that such assessments can be passed without improving actual job performance. Campbell proposes a shift to scenario‑based questions that mirror real‑world tasks, using answer options that reflect plausible mistakes rather than obvious wrong choices. He emphasizes the need for meaningful distractors that reveal judgment errors, detailed explanatory feedback that helps learners understand their gaps, and an iterative design process that involves subject‑matter experts testing the items. Finally, he recommends re‑testing learners with new scenarios to ensure knowledge transfers to real customer interactions, warning that a perfect quiz score is meaningless if employees cannot handle authentic situations.
Chapters
Do you ever get to a quiz question where you can tell which answer they want before you've even finished reading it? 3 answers are about 4 words long and the 4th is a beautifully written paragraph that sounds suspiciously like the policy. Okay, I think I've cracked this one. That's the thing that bothers me about some of the quizzes we build. People can get really good at passing them without getting much better. At the thing we're supposed to be helping them do. I'm not saying every question needs to be some elaborate simulation. Sometimes you really do need to check whether somebody knows a particular fact. But I want to think about what we're asking them to show us. Say somebody's job involves handling refund requests. We could ask them— What the refund limit is. Fair enough if they need to know that, but what if the limit is right there in the system? What I really want to know is whether they can look at a request and figure out what to do with it. So give them one. A customer bought something a while ago, they've already had a replacement, and now they want their money back. What would you check before you approve it? That feels a lot closer to a conversation they'll be having at work. They have to notice something, maybe look something up, and decide what happens next. But then we have to give them answers worth thinking about. If one option is check the policy and another is hang up on the customer, we're probably not learning much about their judgment. I'm hoping most people can work that one out. I'd rather use options that reflect mistakes somebody could reasonably make. Maybe they approve it because the amount is small. Maybe they send every request to their manager because they're worried about getting it wrong. Maybe they overlook the replacement because they don't realize it changes the decision. Now, when somebody picks an answer, we've got something useful to talk about. And please give them more than Incorrect. I already know you didn't like my answer. Help me understand what I missed. For example, you checked the amount, but this customer has already received a replacement. That means you need to check the exception guides before you approve the refund. Okay, now I understand why you wanted me to slow down. Then let them have another go with a different request. See whether they notice that detail this time. I'd also have somebody who knows the job try the question, because if they look at it and say, well, it depends, we need to hear what it depends on. We might have left out the very thing somebody needs to make a reasonable choice. A quiz score can look reassuring. 100% is a lovely number to put in a report. I just don't want to find out later that we've taught people how to spot our correct answers and they're stuck when a real customer shows up.
