#48 – Conversion Optimization Plan for Peak Seasons

Let's put aside the fact should I be doing conversion optimization experimentation during holiday season or not, or peak season or not? I'm just going to put that aside because if you're still thinking through that, you probably want to listen to our episode last week and hopefully we answered that. So you've convinced me. I am sold. I am going to optimize for conversion rate optimization. What are some of the mistakes those who have made the decision? I can think of five mistakes that people do.

My name is Khalil Sladeh and I love to talk about conversion rate optimization.

I'm Simba and I love asking questions about conversion optimization.

This is CRO Live hour, a show all about a B testing, experimentation and conversion rate optimization.

Each episode, Khaled amazes me, answering some of the difficult CRO questions, dropping insights like it's no big deal, while pretty.

Much every episode will take on a new set of conversion rate optimization questions that are not easy. We will talk strategies, we will talk process, and we will talk tactics. Simba will be bringing all the questions.

Oh man, I bring tough questions like do a B testing results fade over time? How do you go from low to high testing velocity? How do you measure the success of a conversion funnel? And how do you align your CRO program with a growth strategy? Yeah, Khalid, these are very, very tough questions.

Yes, they are, but we always answer them here. And if you love conversion optimization like we do, and certainly like Simba does, subscribe to the CRO Live hour podcast today, wherever you listen to your podcasts.

Before we jump in, we have a few questions that came through. I'll start kind of with a typical question. How was your week, Simba? Are you noticing a little bit of change in the way we run our marketing? At least from my direction or no.

I think we are because, yeah, I'm noticing that because there are certain things that the way we used to do marketing in the last three, four years, it's kind of like different from the way that we are now packaging. There are so many things that we are doing that we never used to do before. Called maybe that I never used to. Now we had like we have been focusing like on content marketing for quite a long time, but there are different channels we have been getting into. For instance, we've been also working on video on YouTube and we are also doing some other channels different from content as like text or article. I'm learning a lot of things and as I did it. So yeah, I can see like a lot of changes. How about you? Have you Seen like any differences?

Yeah, I mean, I do think that in the last few months we're like, you know, becoming more and more strategic. We're not there. Correct. We know where we're at and we know where we're heading. So kind of focusing on the strategy. And I think one of the bigger things for us, we've always done marketing. That's how we grew the agency. But now we're starting to think through aligning sales and marketing, which is an interesting challenge because, you know, James runs our sales, you run our marketing as like, hey, really? Yeah, you guys support each other, but really we're not working very closely. How do we actually get to that point? So I think it's fascinating looking through that journey and understand that a little bit more.

For me, like, I think my goal as a person, like foundation marketing, I want to do like the same thing that James like, when it comes to sales, we are not like 100%. So I'm just like trying to also like touch a lot of you together. But up from living, say, oh, we're doing this and this, and then you want to do like hands on, like audits.

I am less hands on when it comes to marketing. Definitely. You know, how the good old days or the old days, I don't know if I would, you know, it's just fascinating. Everything is good. But with sales at this point, my involvement maybe in any point in time is about half an hour meeting with James every week to say, but also that's kind of like running our typical sales program as we work on new initiatives. When it comes to marketing and sales, I am more involved because they're new. It's like, hey, yeah, we need to there. So it's just fascinating and it's just an amazing journey as well. You know, one thing that I've been doing which is so different than anything else that I've done, but I felt so disconnected from the CRO work that we do. I talk CRO all the time, but it's been almost. And I'm like, you know, the teams give me reports and I talk to the teams and I look at experiments, but I have not done it myself. And there's one thing to guide, there's a whole other thing to kind of do some work. So I've picked up two clients now. I'm not doing the whole work, but there are some pieces that I'm like, you know what? For example, I love analytics and I'm like really big into Google Analytics, but I have not looked at GA4 previously. So I'm like, you know, they're like, oh, yeah, there's training here. I'm like, no, no, no, guys, I want to do like an analytics assessment for a site using GA4. And they're like, really? You want to put that presentation together? I'm like, yes, I do want to put the presentation together. So it's fascinating because I spent a week kind of looking through that, thinking through that, and now, like, you know, the other piece, which I absolutely love but I haven't done in also some time, is actual customer interviews. And it's funny because it's so muscle. So you have the muscle memory there. But if you haven't done it, this is a little rusty. Just like playing some sort of a sport after just sitting around doing nothing. So I just, as we speak, I see like three emails. They're like, oh, you want customer interviews? Come here are some customer interviews for you. So it will be fascinating.

It will be interesting to see how that plays. So today we are talking about a topic that kind of like links to the one that we had last week. Today we are going to talk about peak season, holiday season, for E Commerce.

I am ready.

Okay, cool. So I'll give you the first one, which is what are some mistakes businesses make when trying to optimize their conversion rates during thick seasons?

What are some of the mistakes that people make? So let's put aside the fact, should I be doing conversion optimization experimentation during holiday season or not, or peak season or not? I'm just gonna put that aside because if you're still thinking through that, you probably want to listen to our episode last week and hopefully we answered that. So you've convinced me. I am sold. I am going to optimize for conversion rate optimization. What are some of the mistakes? Those who have made the decision? I can think of five mistakes that people do and I'm sure our audience can think of even more. So if you are listening, if you're watching, maybe sure that you drop in a comment to tell us what are some of the common mistakes. But here's at least in my experience, what I see. Number one, is starting too late. Yes, we are going to run experiments. Yes, we are going to try and optimize for conversion. But we're going to leave the planning for that until a week or two weeks before the holiday, before the peak season. Don't do that. Do yourself, do your team a favor. The agency, if you're working for an agency or your internal team, do them a favor and do the work early. On peak season is peak season. That's where you make a good chunk of the money. Mistakes during the peak season are magnified. Some mistake during peak season can cost you a ton of money. So we want to avoid that and we want to walk into the holiday season and most of the decisions are already made. So it's really not about strategy, more about execution. I don't want to sit there and think about like, oh, well, here are the campaigns I'm going to be running, here are the creatives I'm going to do, and here's the messaging. No, no, no. All of that should be predetermined prior to going into the holiday season. We're already starting those conversations with most of the companies that we work with. Say, hey, so Black Fridays every Monday are coming up. You know, it's two and a half months. We need to finalize most of the offers that we have by, you know, first week of October. And we know that some of the companies that we work with are not going to finalize by beginning of October. But at least we're pushing to have those conversations early on, listing the decisions, the next steps to have those decisions. So don't wait until the end. Number two, let's say you've decided on the campaigns. You know, everything is set up. Do not shorthand the QA quality control, quality assurance process. Again, if you are running experiments during the holidays, your goal is to increase conversion rates. If there's a bug in an AB experiment that you're directing 20%, 25% of the traffic to it, and that bug is stopping people from converting, you sort of just shot yourself in the foot. So we want to make sure that we are doing very thorough QA of everything that we are going to launch on the website. Now, mind you, this applies at any point in time. Correct? The last thing you want when you're deploying an AP experiment is not to do good, solid quality control process on it, to make sure that there are no bugs in it. But away from the holidays, non holiday, non peak season are forgiven. They'll forgive you if you make a mistake here and there. The holidays don't. So you need to make sure that the quality control process is in place. Everything has gone thoroughly through that quality control process. Number three, there are times where you focus on the brand. And I would say most of the year, whenever we are doing experimentation, we're always thinking about the brand and how do we enhance the brands through our experimentation. The holidays and the peak seasons are a little bit different. You have a Highly motivated visitor. Especially if we're talking about Black Friday, Cyber Monday, or the holiday season after that. Brands matter, but they matter maybe a little less than the offers that you have. So focus on your offers. Think about the offers. I know one of the mistakes that we've made early on whenever we're testing is we try to introduce versions, variations that challenge the control that focused solely on the brand. And those didn't do well. Now, brand plus offer probably will do better than offer by itself. Again, that's the reason you want to think through all these different things that you are going to be testing, making sure that you have that variety there. But think that's really when the visitors there, they are looking for those special offers that you have. So you want to make sure that you serve them with those offers right away. One of the things that I see some companies do is they say, oh, well, we have our Black Friday campaigns ready, our Cyber Monday campaigns ready. Then, oh, gosh, we're done. We ran the experiments, we're done. I'm like, no, no, no, Black Friday, Cyber Monday, that's the holiday season. As a matter of fact, we usually have our campaign set up for Black Friday and then we have a little bit variation of those running for Cyber Monday. But we don't stop there. We're probably taking the learnings from that and moving really fast. So if our typical speed is a hundred miles an hour during normal business operations during the holidays, we're going 200, 300 miles an hour the minute we finish Cyber Monday, you know, we shut down typically our offers on that Tuesday night. So Cyber Monday throughout Tuesday, maybe Wednesday morning, there are meetings that are happening on Tuesday and Wednesday. Say, what did we learn from those experiments that we ran, which offers were successful? How do you build on those? And we start prepping to launch those beginning of December, throughout Christmas and New Year's. And there's, of course, different periods where you focus on different things. There's the cutoff times correct for shipping to be able to make the Christmas holiday. So everything needs to be planned out. If there's one thing, there's a lot of experimentation that's going to happen and all of that needs to be planned out. Now, the fifth thing that I'll mention, and people sometimes don't pay attention to, but, you know, there is a reason why I mentioned because we probably got hurt because of not paying close attention to it. Think about your team rotation and your team hours and the off time. This might sound like a bit strange, but for us, most people in the US Take Thanksgiving Day off. That's kind of the Thursday.

Correct.

We don't take it off. We're working through Thanksgiving because there's just a lot that is happening. Even on those weekends. We are working, there are people working and there's team rotation that's in place to make sure that we provide whatever support to the sites that we are working with. So think about that structure. Sometimes I have team members who ask for time off from that last week of November and I tell them, I'm sorry, like, no, that's a week where hands on deck. Even if you have everything prepared, we need to have you working, working there. One thing that we're going to be experimenting with this year, we'll see how that goes, is really things taper off after Christmas. Most things slow down. Most of the companies I work with are taking time off. So we are thinking, since our team has pushed so hard in the month before, is to have a company wide time off where we say, you know what, we're shutting down for this whole week as a company. We just need to recoup and re energize everybody. So we come at the start of the new year, we're refreshed and we're able to focus on work again. But those are the top five things that come to my mind. I'm sure there are a ton of other mistakes that people do, but I try to at least say, you know what, okay, so everybody's going to mention the typical things. Maybe here are a few things that are different that we've learned over the years to be weary of. It goes back to, I think the third point sometimes try and figure out like, oh, here's some usability issue that I'm going to be running a test for during the holidays, not the time for it. Holidays are highly motivated visitors that are coming to the site for a specific reason. They're looking for special offers. There is time to test features. There is time to test usability. There is time to test user experience. Kind of pure user experience. Hot is probably not the best time to do that because you have those visitors who are highly motivated. Fundamental to any experimentation that you do is I need to understand my visitors and I need to cater to them. And if I'm not aware of that, then I'm hurting myself. So forget about all this stuff. You can do it pre holiday, you can do it post holiday. During the holidays we're very focused on things to, I would say around the offers or things that will ease people into buying more hey, Mr. Or Ms. Visitor here. Here's more offers. Here's quick things for you to be able to add items to the cart, to place an order. How do I get you to get your credit card out and place an order with us?

So, moving on to the next one, are there any specific technologies that you recommend for tracking and analyzing user behavior during exit?

There are sites most companies I work with will get during the holiday anywhere between 5x, sometimes 20x the amount of traffic that you're getting to the website now, those visitors are not your typical website visitors during the rest of the year. Things that apply to typical visitors, let's say in February, March, let's say you don't have anything peak, no special offers during that period are not going to be similar to the visitors who are coming to the website towards the end of November or December or whatever holiday season depends on the site and whatnot. Understanding that user behavior, understanding the barriers that stop somebody from converting is very critical in order for you to convert those people. Now, there are two sides to this, by the way. There is the tools and then there is the people who are going to run the tools. And I think that's kind of one of the biggest challenges with user insights and user behavior tools because sometimes they provide you with a ton of data. But if you don't have the person who's sitting there at the helm kind of looking at all this data and making the right decisions. So, okay, you have a ton of data that no one is doing anything with. So that's maybe kind of like what I would preface everything that I say, make sure we talked about the team and the team availability, that somebody's sitting there looking at the data and helping you make decisions based on the data that we have now, we're looking for trends. None of the tools are a hundred percent accurate. I mean, even if you're looking at analytics, you're looking for trends, you're making assumptions and then you're trying to make adjustments to your marketing based on those trends. The three most important things, three most important categories of tools that matter the most to me during the holidays. I'll start with heatmaps. Session recordings are great, but heat maps during the holidays are a lot more helpful because you have the influx of traffic aggregates the data. It shows where people are clicking, how further down the scroll, how further down the page they are scrolling. If I have a whole bunch of offers but only 20% of my visitors are seeing them, guess what? I am making changes to the page right away to My landing pages to my product pages, whatever pages I have, those offer. Because an offer, a great offer that people don't see is a great offer that doesn't help me generate money, make more money, make more conversions. So heat maps are really helpful, especially again with the influx of traffic. You can generate those heat maps really quickly, analyze them, where are people clicking, how further down the page they're scrolling, what changes? And this matters, asking this question, what changes we need to make to a particular page or set of pages based on that insight from the heap. So that's number one. Number two, I love to use on site polling, asking our visitors questions when they come to the website. And for us, the most important questions are what we call an exit intent. Let's say somebody got to our product pages or maybe got to the collection pages and then they left. We want to understand from them what stopped them from converting, what is it? And then the next is like, okay, well here's what stopped people from converting. What can we do about it? What changes can we do? Or is it just a fact of life? I don't like, by the way, the answer is just a fact of life is the way it is. I'm like, no, no, you can always do something about that. We're working with one company and they launch an exit intent poll where they ask people, hey, what stopped you from making a purchase? And they gave different options. 75 and I'm rounding up the number but about 75% of the visitors said, price, you know, your prices are too expensive. And then so there's a follow up question. If our prices are too expensive, what's specifically about the price that stopped you from buying? One was I'm holding out, waiting for a special promotion or offer. I'm like, okay, great. So we got our users used to that, so maybe that's a good thing, maybe that's a bad thing. One said, well, you don't have enough payments options for me. And one said, yeah, so price is too expensive. Holding out for payment options. Prices are too expensive with the majority of the people. And they're like, oh well, our price is our price. We can't do anything about it. I'm like, really? Of course you can do something about it. They're like, oh, we're not going to discount. I'm like, no, no, no, we're not telling you to discount. Price is a story. What is the story that you are telling your visitors? Now this might not be a test that I'm going to be running through the holiday because Again, there's lots of thinking that needs to go into that but this is definitely would be on top of my list post holiday to think about how do I justify, how do I persuade people that actually the price that you're paying you think is expensive. You're actually getting such an amazing deal by paying this price. So sitting down and thinking of an A B experiment that solves that problem, it's not easy but it can be very powerful. What's fascinating is about the St. Paul is zero percent. Zero percent. Well, we're not buying because you don't have enough payment options. Although the brand was thinking oh, we need to add other payment options and we people were not concerned about that. They were not thinking about it. You could have spent the time and the development agency that you're working with would have been more than glad to take your money to implement different payments methods. But people are not interested. That's not the concern that they have. So polls to me are very powerful. And then the final thing that I mentioned if you're going to do any type of testing during the holiday is multi army. That's AB testing software. So multi arm banded tests. So typical a B test. We've mentioned this before. I have version A, version B control versus version 1, splitting the traffic between them, running the test for a couple of weeks and then looking at the results. A map Multi arm bandits doesn't work that way. What it says, oh, so you have the control and you have version one. I'm going to start with 5050 and then it's constantly adjusting the traffic allocation. Say hey, the variation one that you have is doing a little bit better. Let's give it a little bit more traffic. Oh, the control is not doing so well. Let's take some traffic from it. So it's always optimizing the idea there is, let me help you maximize your revenue. If something's not doing well, let me take away some of the traffic that's getting, give it to the version that is doing better. So Multi Arm Bandit is very powerful software. I highly recommend using it during the holidays.

That makes a lot of sense. But I feel like you have launched a test during the primary goal is doing very well. Like maybe all the company is getting higher but the second variation or maybe let's just say the control is doing good when it comes to other secondary goals. What then do you do in that case?

Simple answer. I go by the primary control by the primary conversion goal. That's the metric that matters the most to me. Now sometimes people Say this is my primary conversion goal. And they look at all these other metrics and I tell them, okay, doesn't mean that we ignore the other metrics. But when we say that this is the primary conversion goal, that's going to be our North Star, that's really what's going to help us determine which variation that we're deploying, which variation we direct more traffic to. It doesn't mean that I ignore the other metrics. I'm probably going to be asking questions like, hey, why do you do the other variations do better? What is happening in other variations that drive people better across the website? And why this particular variation is getting us more orders. Although it's doing a lot worse than like, you know, the other metrics. I wish that there's like clear cut and dry answers for this stuff. But I mean, I think that's the beauty of what we do, where you're always analyzing, you're always sitting back and thinking. It's like, hey, so what's happening here? Why is this happening? What do we learn? And let's say the control is doing horrible when it comes to order confirmation, number of conversions, but it's doing really better in driving people to the website. What is there that we can learn? Because again, it's not a single experiment. I'm probably going to have four or five other experiments running on the homepage in the next five to six weeks. So what can I learn from that to actually enhance my next experiments and.

Make them even better? That makes a lot of sense. So how do you balance the need for agency and scarcity in your CRO efforts during Pixies without alienating customer?

And if you balance it or not, customer experience comes first. Customer experience really just needs to come first. You're not going to trick people into conversions. You might trick them once, but guess what, it leaves a bad taste in their mouth. And that is the end of that relationship. And especially we live in a time with user generated content all over the place. Social media all over the place. One mistake can hurt you so much. Now here's the thing that doesn't mean I don't use urgency. But first, I am very honest about the urgency that we have and I'm very honest about the scarcity that we have. It's not manufactured urgency and scarcity. Because I've also seen companies that are like, oh, we're not going to use any urgency as scarcity. I'm like, why do you have like in a warehouse full of those items? They're like, no, actually we only have like seven more items and selling so quickly. I'm like, you already have that built in scarcity into your product. Just let people know about it. It's a lot different than somebody who has a warehouse full. Like, you know, he has 2,000 items sitting in his warehouse and he's showing, oh, only five items left. I'm like, okay, now you're not being genuine and you're not being honest, so you're not going to trick people. We've all have seen those counters. Oh, 10 minutes, like, you know, until to count down. But then you exit the website, you come back and it's like, no, the counter resets. Like, do you think I'm going to buy from you? It's just leaving a very bad taste in my mouth. And it's like, I'm going to just avoid you. So think customer experience first. Balance that with urgency and scarcity. But really, the overarching theme is customer experience. We want to make sure that we are providing the best customer experience.

That looks amazing. I think, like, those are the questions that we had. Yes. Maybe you have anything.

Yeah. So super excited about this upcoming holiday season. I think more and more brands are focused on it. It's funny I mentioned, like, you know, that we're starting to have conversations with some brands now. This is about the time that we start having those conversations. And I would say, like, three, four years ago, people would look at us like, really now? Especially with the D2C brands, they're like, yeah, too early. Don't even talk to us about that. But I would say the majority, we still have some laggards who are still saying it's like, too early. But the majority of the companies that we work with are already saying, yeah, here's the creative, here's what we're working on. Here's. What are your thoughts on that? So it's fascinating. So I think it's going to be a very good holiday season, but let's see what the numbers say.

It'll be interesting. By the way, we also have, like, an online event, like in the next few weeks.

So, yeah, we are going to be making announcements about that. With that, thank you, everybody, for listening. I have one request. If you enjoy the podcast, wherever you listen to it, share it with a friend who think who would actually enjoy it and benefit from it. But until next time, happy testing, everyone.

Sam.