#1 – JTBD, Customer Journey, Prospect Theory, Emotional Messaging, Server side A/B testing vs Client-Side A/B testing

My name is Khalil Slade, and I love to talk about conversion rate optimization.

I'm Simba, and I love asking questions about conversion optimization.

This is CRO Live Hour, a show all about a B testing, experimentation and conversion rate optimization.

Each episode, Khaled amazes me, answering some of the difficult CRO questions, dropping insights like, it's no big deal, while pretty.

Much every episode will take on a new set of conversion rate optimization questions that are not easy. We will talk strategies, we will talk process, and we will talk tactics. Simba will be bringing all the questions.

Oh, man. I bring tough questions like, do a B testing results fade over time? How do you go from low to high testing velocity? How do you measure the success of a conversion funnel? And how do you align your CRO program with a growth strategy? Yeah, Khalid, these are very, very tough questions.

Yes, they are. But we always answer them here. And if you love conversion optimization like we do, and certainly like Simba does, subscribe to the CRO Live Hour podcast today, wherever you listen to your podcasts. Hello, everyone. This is Simba Carlos Sade.

Simba.

And we are live again with the Seattle Live Hour. How are you doing, Simba?

I'm good. I'm good. How are you?

I'm tired. It's been. It's been quite a week. Fun. I'm not complaining, you know, but sometimes it gets busier than I like. But, yeah, it's been. It's been interesting. We should probably talk about some of the changes that we've made in our marketing team at invesp figpi. So that's. That's been interesting. Kind of the reorganization there. But before we get into that, talk to me. How was your week?

Busy as always. And I think because of the changes that have just happened. So I'm going to be much more busier than I was. But it's always like a good kind of a busy. So, yeah. Any interesting things that I did this week? I can't remember, but it was a good week, though.

I am not sure. I was so busy, I can't remember all the stuff that I've done. So let's talk about the changes that we've made. So we have two companies. We have investment, we have figpi. Two different. It used to be one company. Now they're two separate entities. There's a marketing team here, there's a marketing team there. And things were so intertwined between them and there was so much dependency between them that it's made sense. And I was a bottleneck. I've Always known that I'm bottleneck and you are taking on more of really kind of the role of managing the team, giving them feedback. So we did the restructure and now you are officially managing the team and I'm trying to step away to let you do your thing. Much deserved role. But talk to me, what do you think is going to be the most challenging things? Because you've been playing this role now for a while, by the way. I know that, you know that, but what do you think? Kind of like moving on to this next stage. What are some of the biggest challenges that you think you might face?

I think, okay, let me just like, maybe give a brief intro to like a context to those that don't know me, that has seen me maybe probably for the first time, like I was hired like@Invest three years ago as a content writer. My main focus was to write articles. You know, I was only focusing on articles and three years down the line, okay, maybe about two and a half years, I kind of like evolved from just focusing on articles and getting into other different things like emails, webinars, social media and all that kind of different tasks. So I've learned a lot in the past three years. It's crazy that, because when I came here for the first time, I didn't know what conversion rate optimization was. And I always like, tell even like people who join us that I, I really didn't know what it was. People think that maybe I'm now like very, very good at it, but I'm not. I'm still learning a lot of things, you know, and, and the learning never stops. So right now we recently made some changes and I'm going to be focusing much more on managing the team because we have like a good team, maybe about seven to eight people on our marketing team. And I'm going to be kind of like managing what they do and seeing if we can do good work. But you spoke about some of the challenges that I would make that I will face maybe along the way. I spoke with the team today one by one, and I took some time to speak with everyone, just like to get everyone's personal goal working here at invesp. What are they trying to achieve? You know, because yet invesp, I know like the goal that we have, I know what we are trying to achieve. So if I can try to align each and every individual's goal to the company go, then it will make like everyone work in a like, happy environment, you know, so that's one of the things that I try to do today, talk to everyone, kind of understand what they they are trying to achieve. Like maybe by being here and what are they trying to learn, where do they see themselves in the next two years here at Invest or that kind of thing so that I can better help them get there. Because I was also like once in their shoes and I was also helped and to get where I am today. So in terms of challenges, I think that the first thing that I had to do was to focus on the internal alignment within us so that we see that we are chasing the one goal everyone, whether you're on social media, whether you only focus on videos, whether you only focus on articles, we all have one goal that you are trying to achieve. So I think it was a good thing and it was a good way to start, to kick start my new role. Yeah, that's. I'm not sure if I answered your.

Question, but you know, you semi answered it as usual. You didn't give me the answer. But you know what, I've learned this by the way, when I used to do media, it's like you never answer the question that was asked. You answer the question that you want to answer correct. So you answered it well. And I'm actually excited to see you in that role because I think, I mean the marketing team is at the core of what we do. That's really what generates the business, the brand recognition. And at the same time, typically I manage the team, but in all honesty I manage the team and then I look at the technical team as well, development team at Figpi and then I look at the sales side of things, which means that trying to manage three things, I don't give it the right amount of time and energy. Correct. So it's kind of like the ups and downs. So it's exciting just to like, you know, see that. Okay, now you're focused on this, let's make it happen. So I'm super excited about this, you know, no pressure, Simba. So we just want quadruple our sales in the next like you know, 12 months.

Yeah, yeah, that's the plan. That's the goal. Hopefully like we will get there and we'll come up with different strategies because one thing that I know about the team that we have, we are good like at production, whatever everyone is doing, everyone can produce something. So we have to see, to try and maybe take it to another level in terms of whatever that we produce. Because the landscape, like the competitive landscape is very, very tough. So we have to try and compete with whatever kind of marketing that we do.

That's awesome. I'll tell you, by the way, one of the things about like, you know, because everybody was running in different directions, I was looking at some of the articles that we produced in the last couple months and some of them, although they're really great articles, but we didn't even publish them. And I'm like, oh man, I need to talk to Simba about this to look at some of the articles that we've written, because they're just sitting there. Let's just make sure that we publish some of those. So that will be exciting to see how things tick and start moving. So super exciting with that. I saw some of the questions that came through. Man, some of them are tough, but.

We always answer them here, like, right.

Hey, I mean, at least. Oh, and that's, that's the other exciting news is that we're turning the CR Live hour into a podcast. So that is super exciting. Do you know when we might actually be launching the podcast?

Sometime next week probably. So we just like finalizing everything right now.

Okay, very cool, very cool. Awesome. Okey doke, my friend. Well, let's talk CRO. Let's talk questions. You know, how many questions did we get today?

Today we have about six questions and interesting mixture, so you'll see. Okay, okay, so I'll start with the first question that we got. How do you use jobs to be done to map out your customer journey?

How do you use jobs to be done to map out the customer journey? So let me talk historically. In the past, I used to look at Customer Journey as an exercise that big brands can invest in. They produce these nice looking documents that everybody gets excited about and they go through the presentations and they think it's great, and then they put them aside and never use them again. And I've seen this with some of the big ginormous enterprises that we work with, where we come in and they're like, oh yeah, yeah, we've done a project to define our customer journey. Um, let us give you the documents. And then they start searching for them and no one knows where they are. And maybe a week later somebody finds those documents and then they send it to us. And the next question that I usually ask them, okay, great, that you've done all this mapping and sometimes you've paid 200, $300,000 for it. Love it. But what were the actionable steps that you took based on that mapping? Most of the time. Not all. Most of the times, like, well, we didn't do anything. So it's Similar, by the way, to Personas. People talk about Personas, the importance of Personas. And I'm guilty of this because if you look through our book, Conversion Optimization, the Art and Science of Converting Prospects into Customers, we have a whole chapter on Personas. The problem is Personas, Customer journey, all of these are marketing concepts and they're great, but they're not useful unless you translate them into something actionable on the website. So I sort of left customer journey on the side until about five years ago, four years ago, when we got into introduce the concept of jobs to be done. And at a very high level. Jobs to be done says that a person hires a product or a service or a human to accomplish a job in their life. Correct. And that job could be a functional. It has three sides to it. A functional side, an emotional side, and a social side. Great. That's also theoretical. The implementation of the jobs to be done framework, how we figure out the job that actually somebody hires a product or service to accomplish is through customer interview. Most customer research, if you think about whether it's usability testing, user testing, the surveys, the focus groups, most of it is what I call forward looking. How is it dealing with our product? What do you like about it? What features should we add? It's all forward looking jobs to be done. And the way we implement jobs to be done is kind of looking in the past, or at least we start looking in the past. So the first question that we do in a jobs to be done interview is asking. The first question that we do is asking, hey, when did you buy this product? Great, I bought it like a week ago, a month ago. And then we go back as opposed to going forward, hey, so when was the first time you thought to yourself, oh, I need to buy this product? We hear the answer and then we start drilling a bit deeper. Correct. Because them thinking about that they need the product is different than the person recognizing that there is a need in their life, whatever that need might be. So we start drilling a bit deeper and we go further. It's like, hey, so what was going on the first in your life that first time you thought, oh, I should really buy this product? Oh, this was happening. Oh, when was the first time you recognize that there's an issue there? So we're always kind of going backwards until we find the very first time somebody recognizes that there is a need in their life, there is a pain in their life, or they have a desire to have a gain. And then we start building that journey to understand, okay, so who did you talk to what are the factors that influences influenced you? What kind of search and evaluation of alternatives did you go through after the first time you heard about the product? And then what was the second event in your life that made you say, you know what, I gotta just buy this thing. And what was the search criteria and the evaluation criteria? Going through that exercise really helps us build a very powerful customer journey. Now, again, this is a customer journey, correct? And now we use the jobs to be done to build an idea of the customer's journey that's not actionable insights yet. However, listening to somebody build that map or historical journey and understand how they came to the decision of actually buying a product, what influenced them, what are the factors and the criteria that they've used is very powerful because all of a sudden you're hearing the words from your customer, very powerful words, how they describe your product, how they describe the need or the job that they've hired the product to fulfill or do. And the minute you do that and any smart marketer looks at that and they're like, oh, gosh, guess what? I have very powerful copy that I can use in my email campaigns and my ads on my landing pages. That's number one. Number two, I can even use some of that information from that customer journey to think through what features I need to focus on on my different pages on the website. So I think utilizing the jobs to be done to build the customer journey can be a very powerful tool and perhaps one of the better tools that I find out there to build a full customer journey and a full view to the customer journey of coming to that decision of buying a product or service.

I've listened to a couple of like, jobs to be done interviews that you have done. And one of the questions that of course you have already mentioned that one of the first questions you ask is when did you buy this product? Right. And in listening to all those interviews, I have never heard you ask, like the customer, why did you buy? Why don't you ask the why question?

Because if you ask the customer the why question you're asking top of mind question, you get the top of mind answer. Top of mind answers when you're interviewing customers are extremely dangerous. People most of the time give you quick answers. They haven't thought through them. They didn't drill deep into the emotional side of making a decision. The Japanese have a rule. If you're going to ask why, you need to ask why five times. Why did you decide on this? And again after the answer. So why is this And Then the third and the fourth and the fifth time they tell you, after you ask why five times, you get to the actual reason. When we do jobs to be done interviews, I avoid the why right away and I ask when. So I'm trying to understand the context of somebody making a decision, and then I drill deeper into that. Lots of times we make decisions and we as humans, those decisions are motivated by emotion, but we justify logically. And when we ask, hey, why did you buy this product? Why did you choose this company? Why did you buy this house? You get the logical answer. Logical answers are fine. But if I'm trying to persuade somebody to convert, if I'm trying to influence somebody to take an action, I need to get to the emotion, get to the emotion, play on the emotion, and then justify logically. That's the reason I always like to focus on the when. And, you know, or sometimes I say, hey, so what was, what was going on in your life that day? Oh, well, you know, I got the promotion and. Oh, how long you've been thinking about the promotion? Well, like, you know, it's been like two months. And like, tell me the story of the promotion. Them. And this was a specific example about somebody buying a purse. It sounds like a really quick decision, but when you drill a bit deeper, you find out it was about the promotion. It's about the competitive nature of the job that she. She was taking. It was about presenting herself in a certain way. It's about now being independent and strong and like, you know, this is manager who now finally got the chance to prove herself. If we had asked, hey, why do you buy the purse? While it looked nice, you know, and I wanted to reward myself, you drill deeper and you go through that process of discovering why somebody, what were the factors to that person making a decision. You get a very rich context that can help you as a marketer do some amazing things.

Okay, just before we move on to other questions, I just want to do, like, a bit deeper on jobs to be done. Like, I know that suppose I'm a client, right? I'm an E Commerce client. I have an E Commerce shop selling hundreds of products, right? And I reach out to you. I hire invesp to help us with conversion optimization. And one of the things that you do which is within the process is also conduct the jobs to be done interviews. So as a client, will you need me to have, like, maybe to kind of like reach out to some of the customers that bought from me so that you can interview them for the jobs to be done? So Suppose I have 100 products. So does it mean that you need more than 100 clients for each product?

No. Yeah. No. This is a really good question. So it depends on the job to be done. Interview that we are doing. We work with companies that carry 50, 60,000 items on their sites. We're not concerned about those particular items. We're trying to understand why somebody chooses that particular website to accomplish a particular job in their life. It's not about the particular product in this case, it is about the particular company and the, the job that company and its site are fulfilling in the life of their customers. So that's number one. Especially for larger websites. There are instances, there are instances where we focus on the product. But that depends that that comes into play with companies that sell a smaller set, very well defined set of products. For example, we worked with a company that sold bed trucks, single products, rather expensive, about $2,000. And in the mind of the customers, the company and the bed truck were just one item. Correct. Because that's the single item that the company sold. Worked with another company that sold skincare products, subscription skincare products for men. Again, now it's a single product. You're focused on that product and the benefit of that. Of that product. Correct. Not the company. We work with a company that sells really beautiful bags, designer bags, and they're well known as a brand. Again, in this case, the company and its products are merged into the mind of the customer. That's it's tremendously different than a company that is a marketplace. Ebay is a perfect example of a marketplace that we work with. Well, in this case, I can't really do millions of interviews. I'm focused on, well, you know, why do you choose to do work? Like, why do you choose to come to ebay to buy a product as opposed to going to an Amazon or a best Buy or there is specific product for a specific set of customers that ebay fulfills. And we have other marketplaces. And the interviews for those marketplace companies are interesting. Completely different. Correct. From the interviews for companies that sell a smaller set of products where the interviews are very product focused. Nice.

So one thing like about E Commerce, right. The E commerce customers, they are different. A customer who comes through social and the customer who comes through maybe an affiliate website or someone who just like typed in the name of the website and then like they get into the website to buy something. They all differ sometimes in terms of behavior. Right. So how do you know that the customers we interviewed are going to represent like the whole other customers because like on jobs to be done, I'm sure you don't interview about hundreds of customers, just a few.

We typically interview anywhere between five to 10, maybe 12 customers. Sometimes we expand based on the data set. So let's talk about the interview process. Let's talk about the interview process for jobs to be done. A company starts working with us. They say, hey, we tell them we have to do jobs to be done interviews. We usually train them on the jobs to be done interviews so they know exactly what they're walking into. Last thing you want is for you to be conducting an interview. Somebody from your client side sitting there listening to the interview, and they're thinking to themselves, what kind of questions are these guys asking? Because some of the questions that we ask when we're doing Jobs to be Done interviews are a bit goofy, funny, different, definitely. Second thing that we do is we ask the company that we're working with to send out, to pull them, to pull a list of people who bought from them recently. Anywhere between, I would say, a couple of weeks, up to three months. I personally like to pull people who have converted a couple of weeks to about a month. They pull a list. If it's a large company, they might pull a list of tens of thousands of people. And we tell them, okay, we don't need that. Let's try, based on whatever information that we have, to pick different people that represent different groups. We send out an email initially to 25 to 50 people. In that email, we tell the people, the customers who receive the email, to say, hey, we're doing customer research about how people buy Product X. Would you like to participate? If they say yes, they click on a link that takes them to a survey or a questionnaire. The goal from the questionnaire is to actually get an idea about the person and whether they are a good candidate to be interviewed. So part of the questions we're asking about the products, how they bought it, who spent the money. Part of the questionnaire, we're asking about their demographics. So as we collect data, we start saying, you know what? Okay, we've got somebody from this demographic group, this from demographic group. And we try and vary the interviews that we will, that we do for those different groups. As you conduct interviews, you start seeing very clear lines that get drawn. Sometimes two, three different demographic groups are giving you the same answer. Because guess what? Demographics in this case don't really matter. It's really about, again, the job that we're hiring the product or the service to do, correct? So the lines between the Demographic groups starts becoming more blurry, and it becomes. What becomes a lot more clear or more obvious is the jobs that people hire a product or service to do. You see one group hiring the product to do this job, another group hiring to do, like, you know, the product to do a different job. A product could be hired to do three, four different jobs. And your mission as a marketer is to figure out those jobs and to say, you know what? I've gotten enough interviews. I have enough data. But you don't stop there, because you use the jobs to be done to build customer journey. Use the jobs to be done interviews to build customer journey. Use the jobs to be done interviews to make marketing decisions. Now imagine making marketing decisions based on the insights from 3, 4, 5, 10 people. Scary. I wouldn't do that. So what we do is we take those insights and then we validate them through surveys that get sent to a much larger audience to validate the insights. So while jobs to be done interviews give us a lot of data points from few from a small set of people. Surveys that are used to validate the insights from job to be done interviews give us small data points from a very wide set of people. So use the combination of both to come up to the final marketing decisions.

Yeah, that's a good one. I remember, like, this other example that was used by Ayat talking about a client who used to sell, like, dance videos. And then she was saying that she interviewed, like, this pastor, and then she said that, you know, say the story.

I love. I love story.

Yeah, yeah. And then she interviewed, like, a pastor who once bought, like, one of those videos or two. I don't know. And she said, like, she wasn't buying video, but she was. She was using that. Those dance videos for therapy because she was. He was about to go through divorce. Like, the pastor was about to go through divorce. So, yeah, it's always interesting how people hire a certain product for.

Yeah, I mean, imagine this. The pastor loved those dance videos that he actually gave a whole sermon on a Sunday about the dance videos and how they helped him in marriage counseling. And he actually said this at recording of that sermon, by the way, like, you know, of his. And I'm like, whoa. So we. We called the customer. We're like, do you recognize what your product's being used? He's like, yeah, yeah, like, not to learn dance. Like, you know, and people. That's the reason they do it. Well, here's another reason. Here's the job that people are hiring your product to do marriage counseling. But you didn't know that. So it was really fascinating. I see Nazish Waseem is saying, yes, it does help us in analysis of customer behavior. And it's about psychology. I do agree. Jobs to be done help us understand the human psychology without coming out and asking the obvious questions. Correct.

Yeah. So one of the follow up questions that we got is how important is it to map out the entire customer journey?

Good question. We map it out completely. So usually, and this is influenced by the work from our friend Bob Mista, the rewired group in Detroit. And by the way, his training on jobs that we done is just absolutely amazing. I highly encourage it. If you get to go through his training in a live audience, highly recommend that. But then he has it also online. According to Bob, people go through what he calls two events. There's the first event when somebody recognizes that they have a particular need and then they, after that first event and they go through passive, passive search. And then there's a second event that they go that the person goes through where they say, you know what? Gotta make the decision. And then they go more into an active search. Think of this as almost the customer journey that the person goes through. Bob insists that there's typically two events. Sometimes I find that I go through multiple events. Correct. But maybe there's two major events that trigger, oh yes, I have a need, yes, I need to make the decision. We like to map that completely. And with each stage we're asking, okay, what's the criteria? What was the event? What's the criteria? Where did you search? Who did you talk to? What influenced your decision? It really gives us a really full and beautiful view of that full customer journey. So that's what we do. That's how we implement. We take the full customer journey and we map it out into actual specific actions that the person takes while they're making the purchase decision.

So earlier on you spoke about how the jobs to be done help you to understand the emotional aspects or the emotional motivators that pushes or pulls customer customers to make a decision. So can you say that the jobs to be done help you to connect with customers emotionally? Or there's also another way that you can use.

Different companies will use different ways to connect with their customers emotionally. Every marketer you talk to will emphasize the importance of talking to customers. You see it in marketing books. You go to conferences and people talk about it. But the real question is how often do marketers actually talk about customer interviews as opposed to preach about talking to customer interviews. What you will find out is most marketers don't talk enough, if at all, to their customers. Sales guys talk to their customers because of the nature of the job. Marketers don't. They make lots of assumptions about their customers or if they do it, they don't do it frequently enough. And if they do it frequently enough, they don't know what questions to ask. All of these are hurdles that marketers need to go through first. I think it starts with making the decision that I am going to commit to talk to my customers and I will be talking to five, ten customers a month. Nothing less than that. Less than that, you're really not connecting with your customers. Number two, knowing what customers I should be talking to. Sometimes you want to talk to customers who bought from you recently. You want to sometimes talk to customers who thought about buying from you, but they chose otherwise. Sometimes people who are using your product and they decided, you know what, I'm done with this product, I'm going to switch to a competitor. There's different market segments that you want to be talking to and you can segment the market even more. The third thing is you want to know what questions you want to ask. Different customer segments have different questions. And to me, Jobs to be Done framework gives you answers to many of these questions and allow you to drill deeper, deeper to the emotional side of understanding a customer, what motivates them. And at the end, by the way, Jobs to be Done is just another framework of analyzing customer needs and motivation. We use it at invesp. We find it very powerful. Jobs to be Done has always been used with SaaS companies to improve conversion rates. I think the innovation that we did is we took this framework that's been there for a long time and we say, we said, you know what, let's go ahead and use it for, let's go ahead and use it for E Commerce. And we see how powerful it is for E Commerce. We see how powerful it is in connecting with customers emotionally when we're creating landing pages, by the way, Jobs to be Done. I mean, it's powerful insights. But on landing pages and in ads and emails is a whole other level of emotional connection with potential customers or with actual customers.

It's funny that you say marketers, they don't like, frequently talk to customers. Right. Of which 90% of the message that gets preached on LinkedIn is talk to your customers. Talk to your customers. But the funny part is like, not everyone is doing it.

Yeah, talking to customers is not easy, by the way. It's just like best practices. I always Say they're easy in theory, difficult in practice. Oh, what am I going to talk to? How do I talk to. I'll give you a perfect example of talking to customers. Remember you when you first started doing interviews for the different articles? It makes sense to talk to authorities when you're writing about an article, correct? Yeah, it makes, it makes perfect sense. Everybody will say, yeah, yeah, of course we should do that. How many times did you do it in the first couple of years?

Maybe in the first year? Maybe I did it like thrice. Just I spoke to maybe three or four people. That's it for.

And how many articles did you write by the way, that first year?

I would say maybe more than, more than 15.

More than 50 or 15.

15.

I think more, my friend, because you're producing an article per week.

I'm just trying being like, yeah, you.

Produce about 50 articles. I can tell you, I know exactly how much you, you wrote 50 articles. You wrote. You talk to three people. Although you know the idea that you should be talking to experts. Yeah. And how was it the first few times? Because that was a hurdle. I need to make the decision to talk to experts. It's not easy. So that's the first hurdle. How easy was it talking to experts the first time you started doing it? How was that?

It was difficult for me because maybe the first interview would last for about 10 minutes because I didn't have like follow up questions to ask. So it was a bit challenging.

It was very mechanical. Correct. Here's the question that I have. 1, 2, 3, I'm done. Thank you, goodbye and good luck. Yeah, that's the second concern about like, I know I'm talking about like, you know, you talking to experts, interviewing them for articles. Well, the same thing with customers, marketers, struggles, like what kind of questions am I going to ask? And then they run out of things to say nowadays. How often, like percentage wise, how often do you talk to somebody when you're writing an article? Is it's your first rate was like, you know, like three out of like 50, about 10%. How about now? How often do you talk to people when you write an article?

I talk to maybe per article. Suppose like I'm writing in, let me just say an interesting article, I talk to about 4, 5 people. 3, 3 to 5 people per article, Per article.

So we go From I wrote 50 articles, I talked to three people into now every article I'm talking to three to five people. And how long are those conversations by the way? Now?

Yeah, about, I can say maybe between Four or five minutes somewhere right around it.

Now you're comfortable talking to people. Correct. And now you're comfortable actually having a conversation going back and forth. That's the same, by the way. Same concept. Same concept. Now you enjoy it. It was difficult at the beginning. Now you can't imagine, like, hey, this is core to what I do. Same thing with marketers. We understand the idea. Implementing it is the difficult part.

Yeah, that was a good example.

There you go. See, I can always give good examples. At least I try.

Okay, so let's talk about industry conversion rate averages. How important are they and can people rely on them, like as a benchmark to. Yeah. To improve their conversions?

This is a question that I get asked in almost every conference I go to speak at and every webinar that we give to in every sales call. Hey, what is the industry average for my industry? If I'm in apparel, If I'm in SaaS, somebody yesterday sent me a message saying, hey, we sell different products and we want to switch to subscription. What is the average breakdown between that we should expect between subscription sales versus one off sales? And I thought it was funny. And I'll take that particular example because the way I answered her, I told her, we actually have quite a few clients who are subscription D2C customers. And there is not a single breakdown between how much subscriptions constitute of their revenue versus one off products. We have companies where 100% of their revenue comes from their subscription. They don't. They wouldn't even consider selling one off products. We also have the opposite of that. We have companies that have that subscription constitutes less than 5% of their revenue and 95% come from the one off products. Now, I can tell you about that group that's trying to move to a location similar to the first group. They're trying to increase their subscription, but at this point, subscription only counts for 5%. We also have companies that have actually felt very comfortable where subscription constitutes about 70% of their products. 30% come from one off. So what's the average? It's all over the place. What is the average website conversion rate? We get asked that question. And what's funny is if you search for that question, probably a blog post that we've written many, many years ago in which we update on annual basis based on the data that we pull. Because we worked with hundreds of companies, we track data from even more sites. We update the data to say, okay, here's the average conversion rate for different industries. Here's the abandonment rates, here's bounce Rates, so we update those. But the reality of it is the average website conversion rate doesn't really matter. Most people like to have a benchmark to compare themselves to. But let's say that you're in an industry where the average conversion rate is 4% and you're doing 8%. Does that mean that's oh, well, you know, we're good. We're not going to not going to try and do better. Try and tell that to your CEO. He or she probably not going to like that. All that matters to you is you want to take your site where you're at and you want to aim at a certain increase in conversion rates or revenue per visitor or revenue per visit. We've talked about conversion rates and how fickle they can be. But you want aim to increase your revenue about 10, 15% quarter after quarter. That's the goal. That's what we are trying to achieve and that's a lot more important than the average industry conversion rate. I'll mention the last point. And how do we actually come up with the number of an average conversion rate for a particular industry? Well, what is the initial data set that we use to come up with that average? I can speak comfortably about apparel size. We probably track 150 to 100 apparel size because many apparel sites know us really well. We have some big name clients and companies that work there and we look at the average conversion rates. I have clients in the apparel, in the apparel industry that have a conversion rate of a half a percent and I have clients with six and a half percent. I probably have even clients that I can't think of at this point with even higher conversion rates. When I average it out. Okay, maybe Apparel is around 2 and a half, 3% average. Depends on the population. Maybe if I, I'm looking at a Data set of 200 size, what if I expand it to 2000 size? What if I expand it to 20,000 sites? How is that average going to matter and how accurate is it? That's the reason I'm weary of average conversion rates. They're there. As a human, I like to look at them. I don't pay them much attention. Sometimes I shrug my head and I say, okay, well how can we improve whatever conversion rates we have on our site?

Yeah, I've also read about this kind of this topic a lot and I think the general consensus among most experts like CRO expects is a good conversion rate is the one that is always improving. If you do 2% this month, next time just try to aim higher.

You know, so it's funny because every company that comes to talk to us is frustrated by their conversion rates. Everybody, okay, right now, I'll give you an example. We're working with a company, small company. I discovered by the way, that sometimes some competitors track like, you know, who is working with us. So they try and like, you know, glean things from me. Those like, oh, so this is the size conversion rate. But this company is small. They're about less than 10 people. They're doing about $10 million, which is not bad for a small company. About a million dollar per person. They're frustrated with a conversion rate that is around 0.2%, 0.2%. So for every thousand people that come to their side, they get two orders, very little conversion rates, but think they're $10 million. So you can imagine, you can do the math quickly about how much traffic they're getting and what the average org value really high tickets item. And they're frustrated by that conversion rate. Understandably so. But we're also working with a company out of Texas where the owner there is frustrated with a 28% conversion rate. 28% conversion rate. And I talk to him and I tell him, you know, most of my clients would die to have even half of that conversion rate. And when you talk to him, I think he exemplifies the CEO who says, you know what, we can do better, we want to do better. 28% is not good enough. How can we actually improve that conversion rate? As a matter of fact, let me say this. When they started working with us, they were at 25% conversion rates. And our teams are always fascinated when I bring them a new company. We go through a very lengthy selection process and we like working with companies that have a 0.2% conversion rate because guess what, you can come in and you can just do some magic and like the conversion rates just increase. The higher the conversion rate is, the more challenging it is to increase it.

Correct.

If you think about success rate for a website like an Amazon or Google, sorry, Amazon or booking.com, they always mention that their A B testing success rate is around 10%. Correct. Because these are highly optimized websites with very high conversion rates. When you come to a website that has a conversion rate of 0.2, well, guess what? It's not highly optimized websites. You can actually probably come in, hit the low hanging fruit, do a whole bunch of things and you can show the pendulum will swing. It's a lot different when you're dealing with the website at 25%, it's like, oh gosh, they probably have all the basics done. All the low hanging fruits are fixed and it is, it's a lot more challenging. So these projects are more challenging. These projects are a lot more rewarding as well. Because when you increase conversion rates, there is like of lot A. Oh my God, I can't believe I actually moved the needle. It is harder, but as hard as it is, as more pleasing it is to see that, that increase in conversion.

Rates, you know, when you say that this other client that has like 28%, at first they were at about 25%. Now that they are at 28%, it seems like a low number until maybe when you speak about in terms of like, that's about 300,000 increase.

Yeah, yeah, that's three. It's like, you know, that's actually. No, that's a 10%, by the way, increase for them from 25% to 28% is a 10% increase. But it depends on what kind of revenue. Correct. Because if you're doing about like, you know, $80 million. Correct. And this particular website is very interesting because it's not a humongous website. Correct. Let's not give away the website, but this is a website that has about four or five pages. That's it. So you can actually track the conversion and revenue, actually release the conversion rates very closely. When you come in and you deliver and there are 25% and there are $8 million, you deliver 10% increase in conversions from 25%. 20%. That is about $8 million. You know what's even more fascinating? It took us four months to do that. Oh, four months. And yeah. And I think I had to be told to leave the team alone only twice during that period because initially me coming and telling the team, here's the funnel that you should focus on. I talked with the CEO of that company and like, okay, here's the funnel that we're going to focus on. And Ayat and her team decided after they've done some testing that, oh no, no, there's a different side of the funnel we should focus on. And I still recall that I was very frustrated with them when they decided to switch. I'm like, what? We agreed on this particular funnel. And I always get the look. And the look tells me how did leave us alone. We know what we're doing. So I think I only got that twice. But again, I think the numbers speak for themselves.

Yeah. How many like a B test do you think, like you launched in that Period of four months.

Oh, gosh, I don't know how many a B test, I mean, at a minimum, by the way, for us, we do an A B test per week. At a minimum, four months. You're talking about 16 A B tests. Knowing our team, we're probably done close anywhere between 20 to 30 A B tests. That's the thing about us. We're high quality research and high velocity.

Yeah.

And by the way, this being high velocity is something that we've added in the last probably three or four years. We used to do a couple of a B tests a month. It was a slow process. And then we've decided that, you know, what if we're only doing two a B test a month and this is the value, what will happen if we're able to double or quadruple the number of experiments that we're launching in a month? And we saw the results of that. It's a lot more exhausting because it requires a lot more coordination, a lot more focus, but it's highly, highly rewarding as well.

Okay, so I'm left with about two questions then. Okay, yeah, I'll let you go. So one of the questions that we got is how does prospect theory factor into a conversion rate optimization program?

Oh, gosh, how? You know, we probably should should change the question. How wouldn't prospect theory fit into conversion rate optimization? Most people who do conversion rate optimization and focus on the persuasiveness of a website are applying some concepts or some implementation or some aspects of prospect theory on the sites, whether they know it or not. So let's talk about the prospect theory. So it was introduced back in 1979. I always struggle with the name. So it's Daniel Kahneman, I think. I'm very sure. I always butcher names. So they published a paper which was basically prospect theory and analysis of decisions under risk. And this tells you that people make decisions based on potential value or loss. Sorry, potential value, gain or loss, rather than the actual final outcome. So what you perceive is the value versus what the actual final result. That's how you make a decision. And under that, there are lots of biases that influence us. So the very famous bias that everybody thinks about is the loss of version. Correct. We want to avoid loss aversion. That motivates us a lot more than the gain. Correct. So that has so many implications. But we'll come to it. Let's talk about certainty. Certainty affects the certain effect. Happens when people outweigh the outcomes that are considered certain over outcomes that are merely possible. So let me give you an example. Let me tell you that Simba, you have two options. In one option, I will give you a chance. 100 chance to win $900. 100% chance to win 900. Or alternatively, I'm gonna do a coin flip and not the coin flip. I'm going to give you another set of possibilities where you have a 90% chance of winning a thousand and a 10% chance of winning $0. So the two options, the first one is 10% chance to win 900. That's one, you know, guaranteed 900. Or if you'd like, you know, and if you're a betting man, 90 chance to win a thousand. Correct. And you know, there is also 10 chance that you win a zero. Which one would you go with?

I don't want to lose anything, so I'd go with the 100%.

$900. Okay. Yeah. Although, by the way, the expected value if you run statistics is similar between the two. But 80% of the people will choose, hey, I want. Certainly. So let's make this a bit interesting. 100% chance to win 900. 90 chance. 90% chance to win $2,000 and a 10% chance to lose to. To win nothing. Which one in this case, guaranteed 900 or that 90 chance sticking off.

Personally, I'm not like a betting person, so I would still stick to my 100% chance.

Even so, I wonder if I increase that more and more. Correct. But again, in this case you went with certainty. But this is about a potential gain. Now let's give you another example. Kind of we're gonna flip this. What if you have a hundred percent chance of losing 900, you're guaranteed to lose 900 or 90 chance of losing a thousand or a ten. Sorry, a ten chance of losing nothing. You saw how guaranteed. Give me 900. Or you know, there's a 90 chance that you might lose a thousand, even a bit more. Or there's a 10 chance that you might lose nothing. Which one would you go with?

I would go with that. That's a trick one.

So hey, you can give me the 900 right now or we can do like, you know, we can roll it and see, there's a 90 chance that you'll have to pay like, you know, the, the thousand. Or maybe there is a 10 chance that you don't. You don't pay me anything.

So yeah, for me personally, I would try to go with the 10 chance that I don't pay you anything. I'll take my chances.

You're gonna take Your chances. So because over here, you know, there's loss, 80% of the people want to avoid the risk, the guaranteed risk of paying 900 and they will choose the, the other, the other option. So people think about this now. Okay, so this is the theory. Okay, great. So how do I implement it on websites, right? People always want certainty. A clear offer of a 30% discount, 25% discount is a lot better than sweepstakes where, hey, like, you know, like enter the sweeps. No, give them something certain people like certainty of, of winning. Assure users very clearly in your copy of what they will get. Correct. Buy one, get one free. Again, like, you know, you need clarity. Now there are also times I always say people like certainty in their lives. But what you need to do as a conversion optimizer, also using prospect theory and the concept of, of certainty, sprinkle a little bit of uncertainty. Correct. There's certainty, for example, for joining our membership club or like in a kind of like loyalty program. And every once in a while, unplanned, we surprise people by discount. We actually surprise a smaller set of people with a discount. People appreciate that uncertainty. So you sprinkle a little bit of uncertainty in your, in your marketing. There is under the prospect theory, the isolation effects, where something isolated can stand out. And we see this, by the way, in Designs. Correct. Your CTAs need to be clear. People want to focus on something because if everything is homogeneous in our mind, we have to work quite a bit to remember everything. But if something stands out, we remember that very, very clearly part of that, and it's kind of a different concept. By the way, what is really fascinating to me, imagine you go on a trip, everything is absolutely amazing in the trip. And then towards the end of the trip, something happens, it annoys you. You've spent a whole like, you know, two weeks in that trip enjoying yourself, then something happens that annoys you. So you've had that last experience that might have lasted for a couple of hours. Then we ask people, so, hey, how was your experience? Guess what they remember. They forget those two weeks that are absolutely amazing. Most people, and they focus on that last event that happened just the way humans act. And for us as marketers, we have to examine that very clearly because a single event in the interaction of a customer with our company can ruin the whole experience. Lots of times marketers perfect the campaigns, do the ad copy, but then they forget about how the actual customer interaction and that causes correct a bad taste and somebody's in somebody's mouth part of the prospect Theory is the frame correct? How do you frame somebody? To somebody, some, some concept to somebody. Let's say you have a drug, you know, this is a new drug. It has. And this is, this was tested. You present the same drug results and the test results to a scientific group of scientific doctors. And we say this drug has less than 0.001 chance of causing deformities when children are born versus saying that 10 in 100,000 children then will be born with deformities. Which one do you think people will actually like, say, with which, which two statements will impact people to say, you know what, we are going to actually approve this drug or, or not approve it? Is it the 0 0.1 or is like, well, there's 10 out of 10 or 10 out of 100,000 children will be born with deformities. Which one, like if you hear two, two sentences, which one scares you more?

I always like try to go with the safer one. Because.

I'm saying, by the way, it's the same thing. Which is fascinating. Yeah. The way you put it, what we find is that when you give the exact like not 10 out of a hundred thousand, the chances of getting the drug approved is much lower. We're saying the same information, but the way we presented it to people actually matters on how they actually perceive the danger or in this case the message of the particular marketing message, other concepts. And again, we can talk about prospect theory forever. The bandwagon effect. Everybody wants to jump on the bandwagon. Correct. I mean, it's funny because we say that, but that's what social proof is about. That is what's really telling people, like, hey, everybody's doing this. Or like, you know, influencers are doing this. Everybody wants to copy everybody else, although they may or may not want to admit that. So I think we have some amazing articles we probably should link to them on our site that discuss all these different aspects of the prospect theory. And I think we have also an article about loss of version because loss aversion itself and how you implement conversion optimization has 13 different ways to be implemented on a website. So I think anybody who does conversion optimization should really understand the prospect really well and should think about how they can apply it in their, in their marketing.

Yeah, those are good, good examples. So one last question. I know that we're about to hit like the one, but I have just one more question before we go. Server side a B testing or client side a B testing, which one do you think is better?

Oh man, that's not A quick question, that's a lengthier question. Let's talk about server side A B testing versus client side A B testing. Most companies nowadays, whenever they do AB testing, they do what we call client side. In client side AB testing, you have a particular page, you put a JavaScript on it. That JavaScript allows you to use JavaScript and CSS manipulation to change the layout of the page, change how the page looks. The development of that JavaScript to manipulate the page to create the different designs is very quick. You're really hacking, if I may say, you're hacking this page just to see if people would actually like those designs. Usually you're writing the script to change the page from the original state it's in using the A B testing software. So somebody, a front end developer, goes into the A B testing software and then they start saying, oh, find the headline. Switch this headline with this other headline. Oh, change the location of this image or change the layout of the page, nothing changes. And the actual code base for that, for that company, you run the test and if you have a winner, you say, you know what? Okay, now we have a winner. We know that people actually like this new layout better than our original layout. You take this new design, the winner design, and you go to your actual developers who are responsible for the site and you tell them, hey, can you go ahead and implement this new design? They don't actually use the JavaScript code that was used to implement implement an A B test. They actually come in and they say, oh well now we need to code this properly. Remember when you're developing an A B test and most people hate it when I say this, but you are hacking the code, you're trying to write very quick code that execute efficiently. I must say this, but to prove an idea, whether people like this design or not, on a server side, what you do is you say, okay, well I have this design for this particular page, I have three other designs. Instead of using JavaScript to manipulate the page, I actually go to my team and I tell them, hey team, hey developers, I know you're busy, but guess what? I want you to implement those three different designs on our site. I will have v1v2v3v1html, v2html, v3html. My team actually in this case has to develop those three different versions. Then I split the traffic, the visitors, between those different versions. The first case of client side, a visitor comes to the page, the page loads and then since we're using client side, the A B testing software says, oh, here's A new visitor, which variation? We're going to show them the page just loaded, I'm going to execute the JavaScript to change the page. And the server side, what happens? The visitor comes and the AB testing software says, oh, I already have those four different versions. Where am I going to send the visitor to? Send the Visitor to version 1, 2, 3, 4. What are the advantages? Disadvantages? Client side AB testing is really quick. You're hacking the code until you prove an idea and then you implement on the website. So that's an advantage. You don't have to come to your developers who are probably busy to ask them to implement something. You just need a front end developer. They do their thing, they prove an idea and then you go to your developers to implement something. The disadvantage of a client side code is what we call the flicker effect. So the original page loads and then we run the JavaScript and we change the design to one of the variations so there's what we call the flicker. So the original design of the page flickers in front in the eye of the visitor. On the server side, the advantage is there's no flicker effects because when a visitor comes, the pages are already developed and the server says, oh, give me page one, give me page two, give me page three. There's nothing that flickers in front of the visitor. The disadvantage is, guess what? Your developers who are already busy are developing all those variations, knowing that they have to code them properly. Correct. It's different kind of coding than typical AB test. And then they'll have to throw away good chunk of that code. Most companies do. Client side. Some companies struggle tremendously with the Flickr effect. It becomes an issue for them and they're like, oh, this is absolutely jarring. The truth is it goes back to the mindset of experimentation. There's. You think to yourself that people are very focused on your website and they're very focused on the Flickr. Listen, you go to sites like, you go to sites like ebay, you go to sites like cnn, you go to sites all over the place and the flicker effect happens over there and most people don't even notice it nowadays because you're arranging page elements. And the idea over here is that most technical teams, most designers, always think that everything has to be perfect. Most experimentation teams say to themselves, we're trying to present people with the best option that we have available to us at this point, knowing that we're going to change it in a week from now. That's really the biggest difference between the two. I personally go with the client side, I can live easily with the Flickr effect, but nowadays every time a company comes to talk to us, I show them actually the Flickr effect. I tell them, here's a Flickr effect. Are you okay to live with that? Yesterday we had a call, absolutely amazing company. We had the VP of product, VP of engineering, cto, actually VP of marketing and a product owner. And I actually showed them a page. The marketing product didn't have an issue. The CTO says, oh, that was jarring. In all honesty, maybe I'm just used to the site. I don't even see the difference. You know, it was just, I'm like, oh, okay, but notice the CTO noticed it and I can appreciate that because he looks at the different set, he looks from a different perspective than the perspective that an optimizer and experimenter is going to look at. So things, what works for one company might not work for another. So you, but it's important that you walk into this understanding the difference between the two and what you, what works best for your company and for your culture.

Now that you, as you were explaining those, those two, I now have like a clear understanding of why you used to say, like, there's a difference between a, a developer who knows how to, who codes for a B testing and one who just like a developer with no experience in coding, AB test.

There's, yeah, there's an assumption that many make that a good solid developer, somebody who writes really good, well structured code that is reusable, that has abstraction that is easy to maintain, will make for a good A B test developer. And they are two different universes that are completely different. A typical developer wants to write the most perfect code, a code that I can look at a year from now, five years from now, and say to myself, this is beautiful code. This is a code that I can reuse again and again and again. That's the mindset of your typical good developer. A good A B test developer is not thinking that a good A B test developer is thinking to themselves, how can I implement this in the fastest way? It doesn't matter whether it's dirty or not, because anyways, even if I have a winner, I'm not going to implement it on. I'm not going to use this code to implement on the website. So it needs to be implemented in the fastest way and the most efficient way. And I know that I'm going to throw 90% of my code away within a week after the experiment runs. One developer is thinking, oh, this code is going to live Here, maybe for five years, the other developer is going to think to themselves, oh, this code is going to live here for maybe a week or a couple of weeks at the most. And because they start from different starting points, they arrive at different endpoints. Correct. And that's the reason sometimes we used to bring an amazing developer, somebody who has lots of experience, and we ask them to develop a B test. And we noticed that, hey, man, it's taking you a long time to develop an A B test. Why is it taking you 40, 50 hours to write an A B test? Well, because they're writing well structured code and they. No, it doesn't really matter now. You don't want to do something silly with the code. But at the same time, it takes average nowadays about 15 hours to develop an A B test, no matter how complicated, how sophisticated it is. Lots of times a lot less than that.

Those are the questions that we had for today. Unless maybe you have something to add.

I do not have anything to add, man. How long we've been going on? I can't even tell. For a while.

Yeah. One hour, nine minutes.

One hour, nine minutes. Enjoyed this as usual. You know, it's funny because we've been broadcasting and I don't know how, how well this works because we've been broadcasting on my Facebook, so I know my family members are getting to see me. So that's just sort of very funny. Family and friends. Some of my SEO friends are like, I was like, hey, you know, they're commenting. But also we've been broadcasting it on YouTube. So I want to see how that experiment goes. I know on LinkedIn it's working well and we get a ton of messages. So with that, thank you everybody for listening. If you have a CRO question, if you have an E Commerce question, shoot it to either to Simba or me and we'll be glad to pick it up next week. Thank you, everyone.

Bye.