A short instructional video (Innovation Minute Ep. 25) by Stephen Shapiro explaining how to run better experiments and reduce confirmation bias. He first notes that humans are wired to seek confirming evidence (seen in politics and innovation). Practical techniques recommended: form a team of devil's advocates and separate roles—seekers (those who want the problem solved), solvers (independent creators of solutions), and experimenters (objective testers who prove or disprove hypotheses). He argues that role separation improves objectivity and reframes failure as part of rigorous experimentation.
Chapters
Today I'm going to give you some specific techniques you can use to run better experiments and overcome the brain's confirmation bias. In the past videos I've talked about why the brain wants to prove our beliefs to be true. We run experiments designed to confirm or disprove our beliefs, but most of the time we prove them to be true. Because if we believe it's a great idea, we're only going to find evidence to support that idea. However, there are some things you can do in order to overcome this natural tendency. The first thing is to recognize it's how we're wired as human beings. We're wired for confirmation bias. When we have a strongly held belief, we only find evidence to support it. This is what we see in our political systems. If I have a strongly held belief about a particular political issue, I'm only going to find evidence to support my beliefs. The same thing is true when it comes to innovation. So when we really can understand that this is how the brain is wired, we can at least take the first step in saying, hey, maybe everything I believe to be true isn't true. The second step is to create a team of the devil's advocates. These are the people who are there to help disprove your hypotheses. In fact, one of the things that I like to do is make sure that we separate the seeker from the solver from the experimenter. Let me say a bit more about that. The seeker is the person who wants the problem solved. I have a problem, I need to find a solution, so I'm going to seek for a solution. Now, I'm passionate about this. So the problem is my passion is going to get in the way of my being able to conduct good experiments. It's also going to get in the way of my being able to find good solutions. So the people I look to to solve the problem, the people who are going to develop the breakthroughs for me, probably don't come from my area of expertise. We talked about this in earlier videos. So what I need to do is make sure that my solvers are separate from my seekers. The solvers have to be an independent group of people designed to help find breakthrough solutions. But we have to go a step further. We have to separate our solvers from our experimenters. If I'm the person who came up with a great solution, I'm going to be so excited about it that I'm going to want to see it implemented. But this means that I'm going to find faulty reasoning. There's going to be experiments that we conduct that should disprove my hypothesis and my beliefs, but it won't. So we need to have a group of independent experimenters who really are designed to objectively prove or disprove the hypothesis. When you change the way you think about experimentation, you will fundamentally change the way you think about failure. Because again, failure is not a good thing. Experimentation is good, and experiments are only good when they actually prove or disprove a hypothesis in an accurate way. The next round of videos we will tackle some other topics related to innovation. I look forward to seeing you soon.
