476. Setting Up Experiments for Your Business

Welcome to episode 476 of the Brainy Business Understanding the Psychology of why People Buy. Today's episode is all about setting up experiments.

Ready?

Let's get started.

You are listening to the Brainy Business Podcast where we dig into the psychology of why people buy and help you incorporate behavioral economics into your business, making it more brain friendly. Now, here's your host, Melina Palmer.

Hello.

Hello everyone.

My name is Melina Palmer and I want to welcome you to the Brainy Business Podcast. I share a lot of tests and experiments on this show, some of which have been done in academia, some in industry, and hopefully they have inspired you to want to do your own experiments. Of course, for the academics listening, you are potentially already doing lots of these, which is awesome. But in business, real experiments are not as common as we all might like. Experiments matter because they can help you to see if what you're trying is working without having to rely on just instincts or gut feelings. It's one thing to say I think that did better than we would have before, and it's another to say we saw a 38% increase in sales when we changed the word them to the number 18. Right? Feels totally different. To be able to have evidence like that to prove that your work is working, or to find areas where it isn't worth the resources you're investing, a type of content that isn't paying off or anything else. To be able to know that to have that evidence, you need to run some tests. If you haven't done this before, it can feel a bit overwhelming. But it doesn't have to be in this episode, which originally aired way back in August of 2019. I share my top tips for conducting your own experiments at work. So get ready, we're gonna jump right in. But before we get to it, just one last thing. Don't forget, links for my top related past episodes and books are waiting for you in the show notes for this episode, which are found within the app you're listening to and@thebrainybusiness.com 476 all right, let's talk about experiments.

Experimenting is so important for any organization, but I know it can feel intimidating if you've never done it before. The truth is you probably have done experiments and not realized it because they don't need to be big. In reality, good experiments are incredibly narrow in their focus because if you test too many things at once, you don't know what contributed to the result or why it came about. I remember my first research paper, which was published in the association for Consumer research back in 2009. Interestingly, as I was looking back for doing this episode, I believe this is actually the 10 year anniversary of that publication as it came out in volume eight. Amazing how time flies and that it ended up being exact down pretty much to the month. There's of course a link for you in the show notes if you want to check that out. That paper was the result of my senior thesis as an undergrad at the University of Washington, which was a requirement for me to graduate from the Global Honors Program. So I was actually the only undergraduate business student on the whole campus who did a thesis like this. It was uncharted territory for me, and in working with my advisor I had to present ideas for what I wanted to research. This probably isn't a big surprise to anyone that's listening, but my style and inclination in that process was to solve gigantic problems, and so all my ideas were far too grandiose for what could be achieved in a project like this. As I was required to write a 15 to 20 page paper on the subject, I figured it needed to be a big topic because how else could you have enough content for that many pages? Boy was I wrong. So every time I presented ideas, he would send me away saying to narrow them down, make them smaller and more specific. Truth be told, it was a little bit frustrating over time, but eventually got to the final paper and the title ended up being Global Advertising Standardization in Japan and the United States. A Closer Examination of High Involvement Products yes, it's a mouthful. The basics of what I did for that study was obtaining magazines from both the US and Japan from the same month and year to ensure the ads would be the same and separated them into categories based on emotional processing and mental or cognitive processing to see when the advertisers were more likely to standardize and use the same ads in these two very different countries, and when they would be more likely to use different ads. My sister, who obtained her undergraduate degree in Japanese linguistics, was living in Japan at the time of my research project and was kind enough to translate a giant pile of ads for me. Thank you, sis. I'm sure I said thanks then, but I'm going to say thanks now again. The study found that advertisements for low cognition products, so ones that you have to think less about, were twice as likely to be standardized as high cognition products like cars. Ads using pictures were more than twice as likely to be standardized than those using a lot of text. At the time, I didn't realize how useful this information was or how impressive the results were I remember when my advisor called to let me know that the paper had been accepted by ACR for publication and I said, well, they must pretty much accept everything, right? Is it pretty easy to get in? He politely said no. How naive I was in looking back and reading my paper today. There were so many variables and items cross referenced just for a study that looked at one month of magazine ads. In my original pitches to my advisor, I'm sure I would have recommended all types of advertisements and over a longer span of time, I also believe I recommend including a third country to make it extra interesting. As it was, I had to break everything into categories and I remember having hundreds of ads strewn around my apartment looking for brand matches across the countries and then categorizing them and using a 100 point scoring rubric for each ad which I modified from an existing model. I'm so glad I did this and I learned so much from the experience and I can see why my advisor kept telling me to think smaller. The study I ended up with, which felt incredibly small to me at the time, was actually a huge undertaking with important results and it was a true academic experiment. An experiment like this is probably something you would want to bring in outside help for, especially if you haven't been trained in the space. When there's a lot weighing on the outcome of the findings of say, you're deciding whether to launch a new product or want to test out different names in a rebrand, it's worth bringing in experts. However, there are tests you can do on your own fairly easily, which can still have a great impact on your business. And the nice thing about doing small tests? It means you can be more agile and adapt quickly, which is very helpful in business these days. My general tips, which I will break down throughout the episode, are to keep these three things in mind when setting up your Be thoughtful, keep it small, and test as often as you can. I've already told you a little about keeping it small. In addition to making it so you can actually do the test on your own, it also allows for you to understand what contributed to the result you're seeing. As I talked about in the Color Theory episode, there are a lot of sites out there claiming to know what the best color for buttons and calls to action are. And yet when you dig into them, they all claim to have a different perfect color. One says red, buttons are always better, another says orange, and this one says green. In some of those cases, they went from a link in the text as their call to action to to Adding a big red or orange button with the callout and then claimed that the color is what made the difference. Was it the color or the fact that there was a button at all? Or the placement of the button or the font or the verbiage? Those likely all changed when the button was added. So it isn't just color that impacted the result. It was a whole myriad of things. This should instead be separated into multiple mini tests to determine what is really best for piece A B. Testing in emails or on websites are really great for this and many systems are set up to track all that for you. Really, really simply. Let's say you send a weekly email to your list. In the first week you would test the link versus the button and that's it. All other text and layout and colors and everything are exactly the same. Whatever the link says, the button says. In the next week's email, you can look at the color of the button. And don't be tempted to also integrate things like different fonts or placement or size. Everything including the subject line and imagery and verbiage must be exactly the same. The only difference between the versions is the color of the buttons themselves. If you have a large list, you could try multiple versions, say four different color versions, as long as they all send at the same time. Because time of day or day of week could impact performance as well. But if you have a relatively small list, say under 1000 people, just stick with two options. Now, you may spend a few months going through color tweaks to see what's the best performing button. And that's okay as long as you're tracking what you do and what the results were. You're learning at every step of the way. Use my tips from that Color Theory episode, which is linked for you in the Show Notes, to narrow it down to contrasting colors that will work in your specific situation and branding. And if it feels like a waste of time, remember that Google tested out more than 40 different shades of blue to determine the perfect color for their links. They've publicly said that that the shade of blue not the difference from red to blue or orange to blue, but between greenish blue, yellowish blue, or a purplish blue, results in an extra $200 million in ad revenue for the company every year. While your business may not be as big as Google and the results may be comparatively smaller, what if finding the right color combination on your emails generated an extra 25% in clicks? Or if you could tweak one item at a time and convert 10 more people a month on your website. These small tweaks don't need to take a ton of your time, but they uncover small changes that can make a big difference. One of the studies I share most often is the one with the end cap displays for Snickers bars. This study used anchoring and adjustment and found when they said buy 18 for your freezer, there was a 38% increase in sales over saying buy them for your freezer. This is a huge difference and most everyone is impressed when they hear about the results. The article I wrote for CU Insight on this concept, called One Word, that increased sales by 38% resulted in tons of credit unions and others reaching out to me to ask questions or see about working together. But if it wasn't tracked, no one would have known the real difference. We all get hunches all the time, but behavioral economics shows us those are often wrong because they're based on logic, not the true rules of the subconscious brain. A perfectly rational being would not be impacted by color or font size or framing, but we humans are. So this all matters, and testing can help you to figure out what really makes a difference for your audience. Speaking of framing, some other things you could test would be how your ads or emails or direct mailers or website pages do when you change a number frame. Say in one you put 78% of clients buy from me again and the next says 4 out of 5 clients buy again and another simply says most clients buy from me again. I know I've mentioned before when I worked with a credit union on their advertising for their checking account and we changed from focusing on the APR that you would get to asking the question, did your checking account pay you $315 last year that their month over month checking account openings went up 60% again? They wouldn't know this if they hadn't been testing and tracking the information on that project we worked on together. You can also do tests on blog post headers or copy on social media posts images you use on ads. Really, the options are almost limitless. And this leads us to the second important thing to focus on when you're doing experiments, and that is to be thoughtful. You know, this is one of my favorite things since I close all my emails and podcast episodes with this phrase. But what does it mean for experiments? For one thing, being thoughtful means looking outside of what you always do or what you know to be true. Often the things we take for granted are those that present the biggest opportunity for learning. Like in that previous credit union example, it's really common to advertise on APYS or APRS rates of any kind, and that's what everyone else does. So you think it's the best way to go. But behavioral economics teaches us that humans do not always act rationally or with much forethought. Make sure you fight the tendency to make assumptions about people's behavior as you look for opportunities to learn in your organization. It also means being thoughtful and taking the time to plan before you jump into a test or start testing absolutely everything. While these are all small items being tested in bulk, they could add up to a lot of time and just a volume of data that you don't want to deal with. Doing multiple versions of every email, post, website page and mailer could quickly become a full time job for a few people and you also have to analyze the data that comes in. In some cases it could be easy the number of clicks on a button, but in others it can get more complex and take a lot more time. Instead of testing everything, test the Right things Before you start building a test, know what problem you're trying to solve and why it matters to solve it. What are you trying to achieve and why does it matter? For your business, this is useful for a couple of reasons. First, it can narrow your focus so you aren't scattered. That means you can be more efficient with your time and dedicate enough resources to implement what you learn and continually get better. Anything can be worth testing, but everything can be a waste of time if you don't have a clear focus and goal. Second, it helps communicate the why behind studies and your organization in general. If your company is about driving value, then all your tests should be about creating more value for your customers. What allows you to spend less on advertising so you can give discounts to customers? How can you showcase the new products most effectively if your company is instead focused on getting additional products in the hands of existing customers? Increasing the efficiency of your emails and getting people to notice them is important as well as getting them to click. If you have an application process, do you know where people get stuck or why? What would get them all the way through? Does a certain type of customer get stuck or is it everyone? It is important, of course, to focus on items that are driving revenue and value to your company. Make sure the juice is worth the squeeze so you aren't putting a huge amount of time and effort into something that will never pay for itself. Being thoughtful allows you to pick your battles and if you're thoughtful up front, you're building things out with intention and hopefully putting together a schedule of what you're wanting to learn and why it matters to the business. When you test things, they are inevitably going to be findings that seem as though they can impact other areas. It's possible that they do, but it's important to understand generalizability. To put it simply, the results of one test will not necessarily hold true in every situation or for every business. Think back to the red buttons, because they have a lot of contrast. Red buttons would probably do really well on Facebook, where they're surrounded by blue, but on Target's website, maybe not. Being thoughtful when you build your experiments will mean you understand what you're looking for and the parameters, so you can know how you might be able to extend the findings reasonably beyond the single test. In addition to being generalizable, it's important to know whether the data you're collecting is qualitative or quantitative, and how each can be applied. Conversations with people are qualitative, number of clicks are quantitative. They're both important and should be used in tandem in businesses, but you should have at least a basic understanding of the benefits and limitations of each. I've linked to a short YouTube video that outlines some of those differences, and there's a ton of information at your disposal if you want to learn more about these two categories of research. And when you do find something, dig a little deeper to see what else you can learn. To use a financial institution example, say you sent an email to 1000 people who are prequalified for an auto loan. 500 got one version and 500 got another version where you were testing button color. Let's say 50 clicked on the button in test A and 150 clicked in test B.

Great.

You know, this color seemed to work well, but what else can you learn from the test? 200 out of a thousand people clicked on the button, how many ended up getting a loan? What is similar and different about them? How long have they been using the financial institution? How many products do they have already? What type of products are they in certain age or income brackets? How does this all differ from the 800 people who did not click? If you follow up with the 800 people who didn't click, what do they have in common with each other and different from the 200 clickers that you could call out and feature in a second email as another test, this can help you segment better in future campaigns. Say all people with at least two products and who have been a member or customer for over 5 years get this message and those who have one product get a different message which is still different from those who've only been with you for six months or less. Digging a little deeper and asking questions is really useful for ongoing experiments and becoming more efficient and knowledgeable in your business. But you can't dig deeper on information you didn't set up for in advance. If you know you'll want to dig into demographics and other details, you probably need to build that into your data pool up front. So think and talk through what you may want to know and what could matter before building out your tests. Unless you want an analyst who is definitely not your friend. Of course, it's important to combine this with keeping it small so you find the right balance for your business. Don't get so bogged down with all the things you could do that you don't do anything. But don't scale back so much that the information isn't usable. And my final tip is to test early and often. Smaller experiments like I've been talking about here allow for frequent nimble testing and ongoing quick improvements. My research study that I get to tell you about in a couple of weeks took six months of data collection with much more time to plan, analyze and then write out the results. It took two years from pitching the idea until publication. Actually almost exactly another one of these to the day. I guess August is a really big time for me. I don't know, it's kind of funny how that works out. Not every academic test takes this long, and it was more like a year from final pitch to paper completion. But it still takes a while. Small tests let you act quickly, so the more you test, the more you learn. As you get into a groove with experimenting, it does get easier. And as you learn a little more each time and combine results and ideas, the bigger, more exciting findings start to emerge and come together. You get a little smarter each time, learning more about your business and customers. Oh, and don't get discouraged when you have a non finding. Say if you test a bunch of button colors and nothing gets people to click more, or if you change up subject lines and don't have a difference, no result is still telling you something very important. Maybe it means that you don't have to worry so much about what you put in that picture in your email blast because no matter what you do, it doesn't seem to matter to people. Maybe you don't need a picture at all and you can reduce staff time searching for the perfect photos. Findings tell you what attracts attention or what matters to your customers, but non findings tell you what they don't pay attention to or care about. This can help you further focus your attention and it's very valuable information as well so you can run leaner and smarter than you would have without having taken the time to test and experiment. Now you have some simple tips and guidelines for running those tests and experiments that will make your business stronger and more efficient. Remember to keep it small, be thoughtful and test often and don't get bogged down by analysis paralysis. It's better to just start and figure it out than to hold up and wait until everything is perfect and never get going. You'll get better as you go.

So what got your brain buzzing as you learned about setting up experiments today? For me, I really love experiments and think they're so valuable in business. And while I know that we all get busy and it can be easy to let this slip, I encourage you to invest in doing small tests throughout the year. You don't have to test everything. And as you heard from my tip to be thoughtful, you really shouldn't test everything because it's a waste of resources and it doesn't have to be huge. And again, they typically shouldn't be so that you can keep it small and test often. Invest in building a testing habit in your organization. Learn a little bit on a regular basis and you can continually be improving, which is a way to help you always stay ahead of your competition. Emails and website pages are a great place to start because they're easy to track, but there are countless areas where you can test on internal communication and opt ins. From employees to sales language to the way you present packages, there are so many options to test. It's awesome. So what are you going to test first? Or what have you tested already? I'd love to hear about it. Please come share it with me on social media. You'll find me as the brainy biz pretty much everywhere and as Melina Palmer on LinkedIn. And of course, if it is a more sensitive test, feel free to email me melinathebrainybusiness.com I can't wait to hear from you and learn how you're using testing in your work. And who knows, maybe it'll get featured on an upcoming episode of the podcast. If you have really cool stuff that.

You'Re doing that would make a great.

Case study to share here or on LinkedIn or other places where I write articles.

You know, let me know.

I'd love to hear about it. As we close out the show. Don't forget about those show notes with links to my top related past episodes and books and more. It's all waiting for you in the app you're listening to and@the brainybusiness.com 476.

And.

Just like that, episode 476 on setting up experiments is done. Join me Friday for a brand new episode with Dr. David Daniels to discuss whether or not investors value gender diversity. It's going to be a lot of fun. You don't want to miss it. Until then, thanks again for watching, for listening and learning with me. And remember to be thoughtful.

Thank you for listening to the Brainy Business Podcast. Molina offers virtual strategy sessions, workshops and other services to help businesses be more brain friendly. For more free resources, visit thebrainybusiness.com.