#43 – How To Approach Customer Review Mining In CRO

Customer reviews are very powerful in helping you build trust. You have enough reviews, you have enough stars next to your name. That is a great way to tell people, hey, I am trustworthy. Yes, I am well established in the industry, and here are what my actual customers are saying about me.

My name is Kalislade, and I love to talk about conversion rate optimization.

I'm Simba, and I love asking questions about conversion optimization.

This is CRO Live hour, a show all about a B testing experimentation and conversion rate optimization.

Each episode, Khaled amazes me, answering some of the difficult CRO questions, dropping insights like, it's no big deal.

Well, pretty much every episode will take on a new set of conversion rate optimization questions that are not easy. We will talk strategies, we'll talk process, and we will talk tactics. Simba will be bringing all the questions.

Oh, man. I bring tough questions like, do a B testing results fade over time? How do you go from low to high testing velocity? How do you measure the success of a conversion funnel? And how do you align your CRO program with a growth strategy? Yeah, Khalid, these are very, very tough questions.

Yes, they are. But we always answer them here. And if you love conversion optimization like we do, and certainly like Simba does, subscribe to the CRO Live hour podcast today wherever you listen to your podcasts.

Awesome. Another week, another CRO Live hour. Simba, how are you doing?

I'm good, very much good. How about you?

I am doing well. Summer has been a busier than I'd like, and it's amazing to me because I was talking to the kids and they're going to go back to school in three weeks. So summer officially is almost done. And it feels strange. I feel bad for the kids. I'm like, oh, man, you're going to be back to school. And I remember when I was school, I never liked going back to school. I mean, get excited because you're going to see your friends, but summer is fun.

The funny part is, like, I'm not actually, like a big fan of summer. I prefer winter than some, obviously, because, like, where I come from, it's super hot all the year. So I would rather prefer, like, going to December.

Yeah, it's funny. So I come from a very hot place and I used to love summer. I hate winters. I hate the cold weather. I hate this. But this summer, and maybe it's just kind of a transformation. I am changing. I'm looking forward to winter now. Talk to me four months from now when it's below zero and I'm like struggling With Chicago weather, maybe I'll have a different opinion. So we shall see.

I think like the past two years this summer has been kind of like different. Especially like in Europe, they have held fires everywhere and it's been super hot heat waves everywhere. So I don't like summer. I live here for those reasons. And they also think that I work better when it's a bit like cold, not hot.

I agree. So today we're talking customer reviews. But before we talk customer reviews, I know we've spent quite a bit of time now in the last two weeks kind of doing lots of analysis for YouTube because we're investing tremendously in YouTube as a platform. Talk to different people, listen to some interviews, and I'm going to just put you on the spot. Before we talk conversion optimization, let's talk YouTube. What are some of the insights, like, based on all the research that we've.

Done about YouTube, I think one that comes into mind and one that we've been talking about a lot is about packaging of your YouTube video, which is like the thumbnail as well as the headlines. Because for people to actually like click on the video, it's usually like the packaging of it, the thumbnails, for people to actually like maybe buy a certain project. It's the same thing with the packaging, right? Because we actually packaging, no matter how good like your video is, but if it's not really like packaged well in terms of like the headline and the thumbnail, most people are also going to not see it or maybe rather ignore it. I think that's kind of like so basic or it sounds so cliche, but it's one of the mistakes that people tend to make and one of the mistakes that we've been making. Like because some of the videos that you have been creating, the content is actually good. You can go through like the videos, right? Like the comments, people really enjoy it, but sometimes the packaging of it. So that's something that comes into mind. How about you?

I like you put it and it's very similar to conversion optimization, except that's what you're trying to optimize for in this case, multiple things. First, you need to get people to click on the video. Like you said, most amazing content. If people are not aware of it, who cares? And it's almost by the way, if you have a website, if you don't drive visitors to your website, oh, it's really very well optimized website has the best conversion rate optimization and persuasion. But if people are not coming, sorry, no one really cares. And you can't even run any A B testing. So you gotta get people to click. You gotta get that traffic. So completely agree with you on the packaging. Now I'm gonna do the second portion. You got somebody in your packaging, makes a promise to people. Here is what the video is about. And I think you have to deliver that value right away. So we've made some tweaks to our videos before where usually I used to do an intro. Hey, this is Khalid with Envel. People couldn't care less about Kala than invest and what the channel is about. I clicked on your video because you made me a promise. Give me that promise. So we've improved that a little bit by saying, hey, this video covers this, this, this. I think that intro is important, but I do think, and it's interesting because you look at the audience retention, how much people are sticking. We have a message that's about 20 seconds about the channel right after we tell them what the video is about. And people just drop off. It's like, dude, I don't really care about you or your channel. Give me me the content. I was going to tell our video editor, kill that completely. Here's what the video is all about. Let's jump right in without talking about invest. Now, we want people to subscribe and we want people to like. So maybe we need to figure out a more a smart way of doing that after people have watched maybe two, three, five minutes of the video to say, hey, by the way, if you liked it. And just one ask subscribe or another ask, you know, like one quick ask that you mention to people, that's something else. And then finally, in all honesty, and it's very similar to every channel, in order for you to be effective on YouTube, you have to almost 10x your output, which is just the thought of that. It's rough. Correct. Because right now, what were we doing in terms of video production at this point?

You mean like the video that we put out? I think we do like five shots every week. Starting from next week, we'll be doing like two long videos and then five shots every week. So that's a total of about seven videos per week.

There are some channels, by the way, and this has been interesting because I think I was looking at creator science, and they do one video almost every two or three weeks, but they're able to make that work really well. Some channels that puts a ton. I mean, they're putting like, you know, five, six shorts per day. And I'm like, oh, my God, just the thought of that is A little scary. But I can see how if you're putting that many shorts, you're gonna improve your game and you're gonna attract more people. But you have to be willing to sit down and record that many videos. So the process of producing more videos will be interesting and just kind of an experiment where we're at near 1500 subscribers.

We've actually crossed 1500 subscribers. I think, like, that's a good point. Like, it depends from one channel to another. Especially, like, for us, we just, like, starting out, like, right on YouTube, like, taking YouTube seriously. I think it's important, like, to focus on the input, to have like, as many videos as we can per week. Some channels, they're big, like, Mr. Beast post, like one.

Don't compare me to Mr. Yeah, you.

Can'T talk of YouTube without really.

Yeah.

So it does, like, one video per month. Right? So it was like, they can afford. They're that big. So I think you also have to look at the size of your channel. But otherwise I totally agree with what you're saying. I'm not really sure about the idea, right. Of starting out the intro. Maybe we might play with it, but not really, like, to take it off. I have, like, some theories about that, but we'll talk about them later. Yeah, yeah.

I mean, it's fascinating to try. I mean, we can talk about the intro, but also, by the way, another big question in YouTube. And yeah, I know this is conversion CRO live hour, but everything relates to conversion optimization, whether we know it or not. Because conversion optimization is about getting people to do what you want them to do from a specific channel. So, okay, YouTube. Usually we talk E commerce, sometimes SaaS, sometimes lead gen. Now we're talking about the platform such as YouTube. I think it's also interesting to think about the audience that we are building. The audience that we have is very narrow. We're talking conversion rate optimization. We're talking about a B testing, and YouTube evaluates our videos. And I think I forgot, like, maybe we've crossed 350 or 500,000 views on our videos, but that sends enough signals to YouTube that, okay, here are the topics that people are interested in. This channel relates to that. This reminds me 10 years ago, no more now, 12 years ago, when it comes to SEO. So Invesp was publishing articles about conversion rate optimization, about AB testing. And then I had this great idea and it was done purely to get links where I said, oh, we're going to publish the top blogs in different industries. So initially I did top blogs for online Marketing. That's exactly how the idea started. Drove a ton of traffic, a ton of links. I'm like, oh, I wonder if I publish the top blogs in any like, about books and about history and about religion. Has nothing to do with Invest. Got us a ton of links from all these. I mean, literally probably generated 200,000 links. And I was happy with the links. But guess what it did. It confused Google because at some point Google thought, oh, this site is about conversion rate optimization. Now it starts thinking that actually no, no, this site is not about conversion optimization. This site is about blogs and blogging. And yeah, we ranked. I mean, you search for top blogs on books. Invest was the first one you search for. Conversion rate optimization did not appear there anymore because we send the wrong signal to Google. So sometimes you do something without thinking about the audience that you're building and what's exactly the niche that you're going after. You need to think about that and goes back to again, conversion optimization, thinking about the particular audience, how do I target the audience and my content needs to be around that. So that's all interesting.

Yeah, that makes a lot of sense. So today we'll be talking about customer review mining. It's an interesting topic and I can't wait to hear you talk about it to name some stuff about it.

So yeah, let's jump in. Customer reviews. And I think we've got quite questions about customer reviews. We've been building them up for a little while by the way. It's funny, one thing that I did is I like some of the questions that we get are quite wordy and I'm like, wow. Sometimes like, you know, some of the questions are so short, I'm like, I don't understand the question. But sometimes when it's so wordy, I'm like, I don't think anybody asked that question this way. So yeah, we've cut them down, we've kind of changed them a little bit just to make sure that they are spot on and very quick. Communicate the message that the person who's asking wants to ask.

So the first question is, how can businesses utilize customer reviews to improve conversions?

Customer reviews help you in two different ways. One that is very quick and one that requires quite a bit of effort. Customer reviews are very powerful in helping you build trust. You have enough reviews, you have enough stars next to your name. That is a great way to tell people, hey, I am trustworthy. Yes, I am well established in the industry and here are what my actual customers are saying about me. So that's One, which is easy, correct? If you are on the right site, you have enough customers and you're collecting enough reviews from them, it's a fast way to build that trust with your visitors. At some point, you know, there's lots of debate how many reviews are enough? I'm going to leave that aside. But I can tell you when I see that a site or a service or a product has only three reviews versus a product that has, I don't know, 2,000 reviews, guess which product I'm most likely to go after? Probably the 2000 review product. In my mind as well. I'm thinking that maybe 10% of the people who actually have used the products have actually bothered enough to leave a review for the product. We purchased quite a bit from Amazon, and Amazon is always sending me that email like, oh, how was your experience with the product? But most of the time I'm not leaving that review. I leave a review in one of two cases if I'm really, really impressed with the product or if I'm really, really annoyed with the product. So using reviews to build trust is very powerful. Here's the other area that I utilize reviews to help improve conversions. Figuring out the fears and the uncertainties and the doubts that people have when they bought the product, figuring out the issues that they've dealt with after buying the product and actually starting to use the products. This process of mining the reviews, figuring out the FUDs pre purchase, post purchase, is very powerful for any business. And I think it helps you improve conversions. Because if you understand that people have always voiced three or four concerns about your products or about buying any product from your category, it is very powerful in utilizing that in your sales process, in your copy, in your ads, on your landing pages. Because you always should tackle the fuds that visitors have that the potential customer has. You tackle them head on as opposed to try and avoid them completely. Because people, consumers are smart, they'll figure them one way or another. And if you tackle them, you have a better chance of converting the visitor as opposed to pretending that those fuds are not there. As a matter of fact, one of the best approaches I have seen is, okay, here are the top three fuds that people have. And here's maybe the biggest concerns that people have with my product, with my service, with like, you know, my SaaS. And I would give a guarantee around that FUD that is very powerful. That's a surefire way to tell somebody, convert. And guess what? We know that you have a concern around that. And I'm going to go ahead and address that concern with a warranty. So that just removes the concern that somebody has. The only caveat to that is that your warranty or your guarantee needs to be meaningful to the visitor. Unbelievable. Because sometimes people give guarantees and your visitors are like, nah, don't believe it, don't care for it. So there's a lot to talk about when it comes to that.

Yeah, that makes a lot of sense. I think like maybe on the other side. Right. Of fads, of course, like you can look at customer reviews to see like the concerns that people have. I think also like the other thing is too, you can also like get some ideas maybe in terms of use cases, how people use like your product. You can maybe use them like on your ads or in scoping around your site.

Yeah, that's fascinating, by the way, about kind of looking at reviews from the other angle from customers and how they leave reviews. So usually when I buy a product, I don't really care for the four, five star reviews. I'm like, okay, those people are happy with it and whatnot. I'm looking at the one star, the two star and the three star reviews. I want to see what those are. The one star, I'm always asking, is it because of logistics? Amazing to me how many times companies get one star review because, oh, the product was not delivered on the right time. Oh, it was broke. It's things has nothing to do with the actual use of the product. So I'm like, I'm reading them and I'm like, okay, are they real issues or are they around the delivery of the products? Oh, like I ordered this size, the wrong size came. I'm like, okay, so it's poor customer service and I'd like to be aware of those issues. But those are not actual concerns around the product itself. And for that I look at the two star and the three star. So I'm looking at those and analyzing them to kind of understand, oh, people are complaining about this. Sometimes I don't do a good job in reading those reviews and I end up regretting it. But more powerful is what you mentioned as well for me. Now we'll switch back to the business. Reading those reviews and seeing, oh, there's different ways that people are utilizing my product. I never thought about that. A perfect example of that and kind of thinking about the reviews. So there's Tylenol pm, which is Tylenol Actimifene, that's available everywhere pm. It has a little bit of sleeping ingredients in it and people were taking it because they Want something to sleep. Now it had Ectomafene, which is a medicine. And people were saying, oh, no big deal, I'll take a Tylenol. And I'm actually aiming. And the component, the ingredient that matters to me is the ingredient that helps me fall asleep. When Tylenol was reading the customer reviews, they're like, oh, most people are looking just for the sleeping ingredient. Yeah, they're sometimes using it because of Tylenol plus the actin, like the actin effect plus a sleeping ingredient. But there is a good number of people who are just looking for the sleeping ingredients. So they create a separate product that just helps you sleep, which is, I think Z's or something like that. That's the name of the product. So we utilize customer reviews to say, oh, people are using our product in a different way. How do we actually spin this off or focus on that in our marketing? So that's very powerful.

You actually touched on the question that I was going to ask you next, which is how do you approach customer review? Mainly, but I'm thinking of maybe I'll ask it in a different way. How often should you be doing customer review mining?

So we do customer review mining very regularly. I would say every five or six months. We're doing very detailed customer review mining. Now of course, this starts with the assumption that we are collecting enough customer reviews. If we start working with a company and they don't have enough customer reviews, guess what? We're pushing hard to get more customer reviews. If that means like, you know, giving coupons for people, that means sending more emails. If that means running contests to get more reviews. Reviews matter not only in helping us build our social profile and the social proof that we are trustworthy product, but also that customer feedback that we are getting is very powerful. So we'd like to do the reviews. Usually, I would say five to six months we're running. It's like, okay, bring me all the reviews that came back in the last period and let's do detailed analysis around those. And the analysis, by the way, for the reviews can be painful. Sometimes it's becoming easier. Maybe we'll talk about how we do the mining. It's becoming, I would say, easier. It's a tedious process. It depends on the number of reviews. Having to do review, like having to do analysis on some products that we work on, they get thousands of reviews. That's a painful process, but it is very rewarding process as well.

That's very interesting. Suppose like let's say maybe I'm part of the marketing team of an E commerce brand and we don't get like as many reviews as maybe other companies. Is there like a certain place where I can go and get those reviews? I'm asking like this question because like we might be having competitors but they have maybe reviews on Amazon. Is it also a good idea for me as a marketer to go and read the competitors reviews?

Yeah. There's two sides to this. If you're an E commerce website, the reviews for your product should be on your product correctly on your product pages. You need to do whatever you need to do to figure out how to get reviews on your product pages. Some companies will work with, they're like, oh, it's so difficult, you know, people are not leaving reviews. No, no, no. The marketing team did not put the effort to figure out how to get those reviews. Somebody placed an order with me, guess what, you know, leave a review. It doesn't matter to me whether it's one star, three star, five star, but get you know, a $10 discount gift card discount code from my store on your next purchase. Now of course this starts with the fundamental assumption that people buy your product regularly. If you're selling once in a lifetime product. Yeah, you're not going to, here's a $10 coupon. Don't really care for the $10 coupon because I'm buying one product from you and we are done. So you need to understand your product, you need to understand your markets and then you go about finding those reviews again. Figure out a way to get those reviews on your site. That's number one. Number two, if you're selling the same product that other competitors are selling, sometimes some E commerce companies are selling looking at the same product. It's available on my competitors websites, it's available on Amazon, it's available in other different places. Yeah, go ahead and read those reviews. It doesn't matter to you at that point. We want to get insights about selling those products. Now let's say we're talking to a company about a couple of weeks ago and they're like oh yeah, our products are available everywhere. You can buy them from, you can buy them from Amazon, you can buy them from Home Depot, you can buy them from Walmart. Understand? Okay, so this product and you are in a very tough spot because guess what, people will most likely start their shopping journey on Amazon or Walmart and then they might come to your website. Well the good thing about them is they're spending a lot of money on Google Ads and they have SEO rankings. So People are coming. Like, okay, so people are coming. We're bringing them. Now we have two things that we need to do. One of them is figure out using those reviews, because there's sometimes hundreds of those reviews on Amazon. Let's figure out what are the biggest concerns that people have. Amazon sells tens of millions of products. They're not going to spend the time to analyze the reviews and address all those issues. You, because you've taken a specific niche and you're selling a very well defined category of products, you have the luxury of analyzing the reviews. Figure out are the concerns that people have and addressing those issues through videos, through, you know, kind of your product information. Really utilize what Amazon has and the reviews that they have over there to create specific information that you can use on your product pages. So now you have an edge that gives you an edge over Amazon over anywhere else. But that's not enough, by the way, because if you just utilize the reviews that you have on Amazon, you figure out the issues, you address them. You need to figure out a good conversion funnel because the last thing that you want is like, oh, I've addressed all the concerns that people have and then people go ahead and place an order on Amazon. So I've done the analysis on Amazon, gave the users the right information, but then they went and converted on a competing website. I see that the best example, which was sort of sad, Seattle Coffee is Gear Coffee company, something like that. They put on the most amazing reviews for coffee machines and grinders and whatnot. They have like a really good blog, they have a really good YouTube channel. Their checkouts process absolutely stinks. Now, the products that they sell are sold at the same price on Amazon. But I was trying to find a coffee machine. I went and I evaluated and I saw their videos and I'm like, oh, I can go ahead and place the order with their site, try to place order with their site. They had a bug. I end up going and converting on Amazon. Now, some people love wine. I tell people, I don't drink wine. I love coffee. If there is something that I do, it's the coffee. And it's a $200 machine. So for me, I'm like, oh, I'm going to spend. And it's funny because I had loved their reviews enough to the point that I found their VP of E Commerce and I connected with him on LinkedIn, sent him a message like, I'm like, hey, there's these issues. Never bothered to respond to me. I'm like, I'm not really interested in selling your service, but at least you have those issues on your sites. Didn't hear back from him. Since then I've ordered six or seven coffee machines of the same. Because every time we have the Christmas gifts that we send to our clients, I like to send them the coffee machine. My brother just moved to a new house, so I just bought another coffee machine. Guess what? I bought the coffee machine from Amazon. This is sad. The fulfillment was by the Seattle Gear coffee company. So they're actually now listing their products through Amazon and Amazon takes a huge cut from them. You should have done the job correctly in the first place place because people are coming to your site instead of having to say, you know, Amazon's another channel. No, it's a channel that is expensive and it's a channel that yeah, helps you stay alive, but it never makes you enough money. So long thoughts around like, yeah, that.

Was a good story. We've talked about the place of customer reviews and conversion optimization and what to look for in those reviews or when you're doing customer review mining. I'm just wondering, are there like any tools that you use when it comes to analyzing customer reviews? Because at some point I think I remember reading about people using ChatGPT to analyze like a huge chunk of customer reviews. What's your take on that? Are there like any other tools that you think are best?

All honesty. We do two things when it comes to analysis. When it comes to customer review is kind of sentiment analysis. There are tools even before ChatGPT that existed. We never really relied on them because they were more. More pricey. And when we tell our clients, hey, you want to get this particular jewel, most clients, they're like, hey, nice, we're getting reviews, but we don't want to spend 10, $15,000 a month. So what we do is we actually, we used to rely on a very manual process to do the reviews. And I thought, although it is a slow process, I thought it was very insightful when somebody actually sits down and reads the reviews. It's just the same when you are reading the logs for chat support.

Correct.

With the SaaS company. Extremely insightful. If you are trying to improve a website conversion rate. So there is a manual process where you pull the reviews and then you are looking for specific insights from those reviews. Now one of the new things that we're doing is we are utilizing ChatGPT, but the manual process doesn't go away. It's kind of a mix between the two where you're using ChatGPT and you're giving it A specific, specific prompt, looking for specific insights from those reviews. We're pulling all the reviews and you're comparing that to the manual process. I would say that utilizing ChatGPT probably cut down the review time by 70%. Let's talk about the specific things that we are looking for when we're doing the review mining. We are looking for motivators. What are the trigger. We're initially looking for the trigger events. What happened when somebody says, you know what, that's it, I need to deal with this problem. I have to solve this issue. Some reviews are great because they provide you that. Some reviews are not so great, but you want to look for those trigger events because that can really help you understand how you can target your markets, how you can create specific landing pages for those specific trigger events. So that's number one, you look for motivators. Okay, so you've identified as a potential customer that you have a problem. Great, we figured out the trigger events. But what's motivated you to choose this particular product, this particular service, this particular line of business to solve your problem? Because there's other competitors. So there are specific things that triggered you to say, you know what, I am going with you, as opposed to all the other alternatives that are out there. You try and find what we call the barriers or the fuds that could have stopped you. And some of the reviews will also provide that. Those, again, the FUDs are really important. Here are the concerns that I've had. Now you can find some of those concerns pre purchase. You can find them post purchase, correct? Okay, here are the concerns. Here are the issues that came up prior to making a decision. Here are the concerns that happen or the issues that people are dealing with. Post making the decision, actually starting to use the product. So that's another category. And then the final thing we look for in the reviews is what I call the surprise. Things that people did not really expect, but they were surprised by where they're like, oh man, I didn't know that. It does that. And it's absolutely wonderful and it makes life tremendously easier. You find a surprise you can use, sprinkle that in your copy and your product pages and your landing pages. And that is very, very powerful. So those are the tools, those are the things that we look for, whether it's a manual review process or using chatgpt GPT, kind of giving specific prompts. And one last thing I would say about using ChatGPT to do customer review analysis and data mining, ChatGPT will give you the analysis and sentiment that it can find in the text. But I always ask it to give me those 200 reviews that I just put. Yes, give me the sentiment analysis, but with each one of tones that you found or sentiment analysis that you find, give me five or six specific examples of where you found the that. So that can help you. It's not like, oh, just give me something generic. No, give me specific examples.

You have done right the analysis and you've got like just a few insights. Is there like a way that you can prioritize the insights that you get from after doing a customer review mining? Because I think of it this way for conversion rate optimization as a whole. Like there are different prioritization frameworks. They say like you should focus on maybe issues that require less effort, but which brings high impact or high value. Is it like the same thing when it comes also to customer review mining?

Yeah, I think when you do the customer review analysis, you will see how frequently people are mentioning an issue. 70% of our reviews are always concerned about this one thing versus this other issue is only mentioned about 20% of our the time. So that naturally gives a higher priority to that concern. Like hey, if you know that 70, 80% of your customers are always concerned about this issue, it's an issue that you want to address, okay, that becomes then our top concern. But that's just one side of it because the other concern is like how do we address the issue? Sometimes addressing the issue is by saying, well, we don't have this problem, we don't have that problem. That may or may not be enough. Sometimes you have to do more work in addressing the issue because it requires a lot more work. I'll give you an example from conversion rate optimization since that's what we do. Hiring a digital marketing agency is tough, right? Because you don't know how good they are. We have lots of reviews and we have reviews from very well established, you know, VP of E Commerce at eBay, VP of E Commerce at C Gallery. Those are very well established companies and we make sure that every review that we put, here's the LinkedIn profile, you can actually reach out to the people, ask them questions about working with us. We record videos with those people so we build lots of credibility where people are not questioning the reviews that we have. Great. One concern that you hear from people when they hire digital marketing agency, is it going to work or it's not going to work? They're concerned with that. And I see that consistently. Regardless whether you're hiring a CRO agency, an SEO agency, PPC agency, how good are they and whatnot? There are different ways that you can address that. And it's fascinating to me. I saw the same problem. Our competitors saw the same problem. One of our competitors said, you know what? We guarantee a lift in your conversion rates. I thought that was really good. We guarantee it, otherwise your money back. Okay? I'm like, wow, that's very powerful. I know we're really good, but I refuse to put a guarantee. So we'll put that aside. Another agency said, we guarantee a lift within six months, otherwise we'll do two months for free. That was an interesting approach. So I looked at this and I'm trying to. I think for me, I'm like, I don't want to guarantee. So I started reading the fine print. So the first company that guaranteed the results, they're like, well, if we run in six months, maybe we'll do 20 experiments for you. And if any of them had even a 0.001% lifted conversions on the A B testing software, we fulfilled the guarantee. So it was really not the guarantee. It was just kind of trickery because I'll do 20 experiments. Yeah, I will achieve like a 0.00. So they offered a guarantee, but you're selling also to business owners who are smart enough who are going to see, oh, this is not a real guarantee. Give me a break. So that just fell flat on its face. The second approach of, hey, we'll give a guarantee, and otherwise we'll give you two months. I'm like, oh, that's actually sounds like a good guarantee. You'll give me two months. I took a different approach where I said, I'm like, really? The concern is not about the results. The concern is trying to drill a little bit deeper into the concerns that people have. The concern that people have is I spend money with you, I'm stuck in a relationship with you, and you're not delivering results. Correct. That's the problem. It's not whether you guarantee results or not. Because even in, like, you know, regardless whether you give me six months and then two months for free or I'm with you, I said, you know, let's take a different approach where instead of guaranteeing the service, we'll go month to month. If you're not happy with this at any point in time, just walk away. I'm not locking you in for a long time. Two different ways to like somebody who's observing, they say, oh, the first company actually provided more of a guarantee. You're not providing a guarantee. I said, Well, I am addressing the issue just in a different way. So I went to the Facebook group that had lots of our customers there. I mean, there's thousands of them. And I said, hey, which option would you go with? The six months plus two, three months or hey, month to month? And what people said, oh, we'll go with the month to month. And what's even surprised me more, they're like, yeah, we'd even go with three or four months contracts as opposed to the six months plus two months. And I'm like, why? They're guaranteeing. And they're like, listen, if within six months they didn't deliver, we don't need to continue working with them. We don't care to continue working with that company. I'm like, wow, that's interesting.

Correct.

Because you see a concern and you're trying to address it, but you have to address it in a way that makes sense to your target customers. Otherwise you're not benefiting from that. So, long story. You figure out the issues, you figure out how important they are, and then you need to come up with the solutions. And some of those solutions are business solutions that you need to think about. And based on that, you prioritize. If it's just a matter of like, oh, people are concerned about this and I need to address it. It's in my copy. Do it right away. Correct. Because that's the least amount of effort. If there's a business discussion of fundamental change in your business, yeah, that's going to require a little bit of more effort. So that's really how the prioritization happens around those issues.

Yeah, I like the idea of looking for patents. I think that makes a lot of sense. Like if you see like maybe a certain issue being repeated by different customers, I think that's something that you have to address. Are there like any other mistakes that people make when it comes to customer review mining? Because, like, talking about it right now, it sounds like an easy thing to do, but it actually takes like a certain skill set or a certain skill to actually do the customer review mining.

There are a few mistakes that people do when it comes to customer reviews. We'll do the reviews once and we're done. It's a mistake. Reviews are a gold mine. But if you're not doing analysis, it's a gold mine that's sitting on the side and you're not doing anything useful about them. So you set regular intervals to do analysis of customer reviews. Even better, you share those customer reviews with your top management. So they're set seeing feedback, actual feedback from customers who are using the product. Set a regular schedule for doing the customer review analysis, for taking the insights, for sharing it with everybody. Otherwise it is not very useful. Sometimes people think, oh, there's just so many customer reviews. So they take a small sample of 50 people and then they're making marketing decisions based on a small number of customers reviews. That's a disaster that's waiting to happen. Oh, we've analyzed 50 reviews and everybody's saying we have this problem. Well, not everybody. Maybe just seven people. Yeah, they're loud and they're leaving those reviews. You have to be careful. We need to have enough reviews. We need to do real meaningful analysis. You need to dig deeper. If you don't have enough reviews, do whatever you need to do to capture more reviews. And don't make big marketing decisions based on the feedback of 2, 3 people. Correct. Validate those reviews by talking to actual customers, by conducting on site polling. Correct to say, oh, are these concerns that people have? So validate those before you make that decision of saying, oh, we're going to go and here's how our marketing is going to focus on no matter how motivated you are about, oh my God, people are having concern around this, let's address it, take a chill pill, do more analysis. Make sure that you're actually making the right marketing decision. And I think that's kind of like the validation process around reviews. So I love to do customer review analysis, come up with the top insights and then do two ways of what we call validation. One is through one on one customer interviews where we see like, you know, we ask questions of recent customers who bought and was like, hey, we're trying to understand their journey to making the decisions. Like was this even a consistent, did you even think about that? Now this is a very small sample set because we might be interviewing five to 10 people. Even better on site polling where we say, hey, do you have any of these concerns or what stopped you from shopping? And then we use the issues that came from customer reviews and we list them. Issue one, issue two, issue three, we took those issues from doing the analysis on our customer interviews. But issue four or the fourth option is other text and allow people to get free feedback. If we find out that 50% of our customers are selecting other, that means our analysis of customer reviews was not done correctly. We need to go back to the drawing board.

That makes a lot of sense. And I also think like that validation using other conversion research techniques makes it like all the more interesting. So I Guess those are the questions that we had for today about customer review linings. Unless maybe you want to change some insights.

No, no, I'm always done, man. By the time we finish talking about the topic, I'm amazed sometimes. And talking about those topics is a lot easier than actually doing them. It's like, oh, yeah, customer reviews. Here's analysis. Until you bunker down, you actually have to do that. It's a whole other process. Like, yeah, great, you talked about for half an hour. But every time I have to do the actual activity of analyzing reviews, that's a week from my calendar. I understand, but the insight's there. And that's the thing about talking about all these topics. There's the theory behind doing them, and that's what we do in the CRO Live hour. But then there is the actual doing, where you start saying, it's like, oh, Kala did not address this issue. How do we actually address it?

So you actually make it sound easy. So.

Yeah, Yeah. I don't know if it's easy, you know, so sometimes it's funny because people tell me when you talk about conversion optimization, a, B, testing in a conference, like, you make it sound easy, but then when people start doing it, they're like, oh, there's is so much there. I'm like, I don't know. Maybe like, because when I'm talking, I'm like, taking on the teacher role and sometimes you oversimplify certain things. So I don't know.

Yeah, I guess that's what we had for today, right? Awesome.

That's all. So, one ask from our podcast listeners. We want reviews for our podcast so we have consistent followship every week. Or if you are listening to the podcast, we'd love to. We would ask you to leave us a review wherever you listen to this podcast. Until next time. Happy testing.

Sure, thanks.