Jimmy KimCEO & Founder, Commerce Roundtable · Marketer of 17+ Years, 4x Founder. Former DTC/Retailer & SaaS Founder. Newsletter. Host of ASOM & Send it! Podcast. DTC Event: Commerce Roundtable

The Golden Rule of A/B Testing: One Variable at a Time

23 Apr 2026 · 1 min

Watch on YouTube

0:52

In this brief conversation, Jimmy Kim and a co‑speaker outline the core principle of effective A/B testing: maintain singular focus by testing only one hypothesis at a time. They stress the need to isolate a single variable so that any observed difference can be confidently attributed to that change. To illustrate, they walk through a concrete example—testing whether adding a customer‑review quote (social proof) above the call‑to‑action in an email boosts click‑through rates. Version A is the standard email with no review; Version B is identical except for the added review quote. All other elements remain constant. If Version B outperforms Version A, the increase is credited to the social‑proof element, confirming the hypothesis.

So what's your golden rule? What's the right way to test?
Dude, singular focus. One test, one hypothesis. You have to isolate a variable and then test that, you know, by itself.
All right, let's dig into that. Give me an example for the audience.
Yeah, sure. So let's say you want to test whether social proof increases clicks or not. Your hypothesis is going to be that adding customer reviews above the fold is going to lead to an increase in click-through rates. What's going to happen, right, is you'll have 2 versions. Version A will be kind of your typical email that you send with no reviews, and then version B is going to be the exact same email, but all you're going to do is Add like review type quote above the CTA.
Everything else stays the same.
The only difference is the addition of the review quote.
And then if version B wins, then you know it was the social proof, right?
And now you have learnings you can apply, right? Social proof above the fold in this particular email or in these multiple emails increased our clicks by X percent.