#33 – Website Survey Questions You Should Ask Your Customers

Every time I think there is somebody who's coming to the website, there is a chance for me to learn something from them. I just need to understand what type of questions I should be asking those people. Now, I look at this in two different broad categories. There are very specific questions that will help you improve conversion rates, and then there is very general questions that can give you general feedback. My name is Khalil Sladeh and I love to talk about conversion rate optimization.

I'm Simba and I love asking questions about conversion optimization.

This is CRO Live hour, a show all about a B testing, experimentation and conversion rate optimization.

Each episode, Khaled, amazes me, answering some of the difficult CRO questions, dropping insights like, it's no big deal, While pretty.

Much every episode will take on a new set of conversion rate optimization questions that are not easy. We will talk strategies, we will talk process, and we will talk tactics. Simba will be bringing all the questions.

Oh, man. I bring tough questions like, do a B testing results fade over time? How do you go from low to high testing velocity? How do you measure the success of a conversion funnel? And how do you align your CRO program with a growth strategy? Yeah, Khalid, these are very, very tough questions.

Yes, they are, but we always answer them here. And if you love conversion optimization like we do, and certainly like Simba does, subscribe to the CRO Live hour podcast today, wherever you listen to your podcasts. So, Simba, how was your week?

Busy as usual. But last week we mentioned that we were affected by the Google Core update that happened in March, right? Yeah, and it's been encouraging because we're starting to see some improvements. We're starting to recover some of the traffic that we lost. The thing is, we haven't like reached our full potential because at some point we went over a hundred thousand visitors per month. We are still far, far, far below that. But the fact that we are starting back to see some increase in traffic, that's good. So I'm sure, like in the next few weeks, after we are done working on some of the updates on previous articles and stuff, I'm sure that we'll be doing far, much better than we are today.

Very cool. Very cool. It's just fascinating to me because I think we have talked about hiring somebody to come full time as an SEO and I think we're doing that. But now, just kind of a side note, I don't work really well with budgets. And remember we had a meeting earlier today where I'm like, simba, I think we need to Come up with a budget here, a budget there, because we are bringing on board a fractional cfo. Can't really afford a full time cfo. I saw their salaries. I'm like, wow. But we're bringing somebody who's fractional, and I think he's going to start helping us set up the budgets. And part of that I'm like, okay, so we need to bring in an SEO person. We're hiring. We just added a couple of new team members. It was so much easier to manage. I mean, you've joined like, how has it been four years now with invesp?

Yes, four years, one month. Yeah.

I mean, we were smaller. Correct. It's much larger. And now finding that the decisions that I used to make so quickly when we were small need to be replaced with a little bit more thoughts so we don't get ourselves in trouble.

And also, the team is growing. I wonder how you keep up with everything happening.

I don't. That's kind of my philosophy. Because if I have to keep up with everybody, that will be a tough one. I'll notice different things here and there. So, for example, with client delivery, I had to run stats. I always have ideas. I'm always kind of fascinating. I'm always reading something new. I'm like, hey, how about if we improve this area a little bit better? How about we, you know, so that's what I focus on quite a bit. I am involved in sales and you see me with marketing as well. So sales and marketing. I haven't had one on one with different marketing team members for a while. But also, we're fasting. I'm fasting. By the way, today is the last day. I don't know when we're going to release this episode. This is my last fast day. Back to my coffee and water. But it's funny because I was talking to Anya. She's my assistant. She's a project manager. She's like, you're tired. I'm like, no, no, I'm good. She's like, yeah, you're tired. You're not getting enough sleep. You're a little snappy. I'm like, no, just sometimes I get annoyed when things happen and sometimes things take a little too long.

So I don't know that happens. So how are things on the fake pseud?

Fascinating. Revenue is growing month after month. I'm having to jump in and do demos, which I actually enjoy. I didn't think I would enjoy it, but I actually enjoy giving demos. So I just had a demo with a Fairly large website, and I love it. I mean, invesp sometimes takes four to eight weeks to close a deal, sometimes even longer. Actually, it's funny because a couple of the new clients that we've just signed up this year, one of them, I talked to them in 2020, and in 2023, they came back and they're like, hey, we're ready. And I'm like, forgot having a conversation with you guys. And I guess in 2020, I told them, you're too small, doesn't make sense and whatnot. But then they remembered us, and they said literally the email was like, hey, we're ready to move on. And then the other deal that we closed, we closed it in January of this year. My first conversation with them was in January 2022. So it took almost a year. And again, we had a conversation. They had issues with their site, and then they disappeared and they came back and they're like, hey, we're ready to move forward with. ThickFi is a completely different animal, even with larger enterprise clients, which I'm surprised. And by the way, the number of visitors doesn't tell you. I used to say, oh, the number of visitors don't. Because I've dealt with larger websites with lots of visitors, but they're very small, really, when it comes to the organization. But this company that I just talked to, fairly large. And he's like, yeah, they've been playing with FIGPI on a free account. They want an extension for a week. He asked a couple of questions. He's like, hey, we're ready to move forward. We want to do the subscription. So I actually pinged Rizal. I said, hey, can you just help him do the setup? So it's fascinating to see how that works. I think the challenge that we have right now is we're doing a whole bunch of marketing activities. And I cannot exactly attribute something. They're like, oh, this is exactly helping. So there's a lot of organic, this paid. You've mentioned we're weak on LinkedIn. I think we're weak on Twitter as well. By the way, lots of the people who use fake buy and invesp clients, by the way, they're split between LinkedIn and Twitter. So we need to up our game there. As in, I need to up my game there. Because you cannot fake social media. If you really want to be successful, you got to do it.

I'm not sure if it's really a deal of mine or something that I see, but I believe that, especially when it comes to B2B or a SaaS platform like FigPi, you have to have like a strong presence on social media, especially LinkedIn, because that's where like most of the people who use figpi are. And that's how I think about it. I might be wrong, but word of mouth as a channel is very powerful and once you miss that, you spend a lot of money like on ads and stuff. That's how I think about it.

Fascinating, Stan, but you cannot imagine how many demos I've done. And this is embarrassing for me as somebody who does marketing. So many demos and they're like, really? So this is fake pie. They're like kind of walking them through them. They're like, where have you guys been? We don't understand. Amazing platform, really fast, really like, you know, combining so many different things. He's like, well we've never heard of you guys. And I'm like, I'm like, take a dagger to my heart and just kind of twist it. That's the weak side of FEC Buy. We did not follow our own advice where we focus a lot more on the technology, did not pay close attention to the marketing. We're fixing that and it's a painful lesson by the way. But sometimes I guess you have to make the mistake and learn from it. So here we are.

I've seen some of the companies who actually hire those people evangelize a product on LinkedIn or people who have a huge following. They just hire someone like who just comes in and just talk about the product on LinkedIn and social media. Because of their audience and stuff, people start jumping in. But I'm sure like for figpie is a good tool and if you're listening to us via podcast and you're just hearing about figpi, just go and try it out and you'll be amazed.

And by the way, coming up with product names is not exactly my strongest suit. So that's the two very strange names. Investors and figpi. So figpi F I G fig the fruits and PI. Pii. And then people always tell me the personal identifiable information. I'm like, no, no, no. Plan implements improve. That's what the PII stands for. And because we were using PII for a long, long time, even before like no PII became famous. It is right now. But yeah, definitely try figpi. So it's tool that gives you the a B testing the heat mask, session recording on site polling, which is what we are going to be talking about today. And even more powerful by the way, is the recommendation engine that comes with Think by Where it goes to your A B test buds. And we are talking what we're all about visitor and customer surveys today. What to do, what not to do, how do you draw conclusions? So let's jump right in. Simba.

Yeah, sure. So I'll just shoot the first question. How do you determine the best survey questions to ask your visitors and customers? Where do those questions come from?

Every time I think there is somebody who's coming to the website, there is a chance for me to learn something from them. I just need to understand what type of questions I should be asking those people. Now I look at this in two different broad categories. There are very specific questions that will help you improve conversion rates and then there is very general questions that can give you general feedback. So let's start with the specifics. The type of questions that we usually launch to ask visitors coming to the website are designed to help us improve the site usability and ultimately help us improve the site conversion rate and helps us increase our revenue. One of the first areas that I focus on is conversion blockers. What are the things that are stopping those people coming to the website from buying an item? Now let's say you are running an e commerce website. You are getting 10,000 visitors, you are getting 100,000 visitors coming to the website. You hope that a good percentage of those visitors will actually convert. But the truth that you and I know is that most visitors unfortunately do not convert. Most e commerce websites are averaging 2 to 3 to 4% conversion rate, which means that 98, 96 to 98% of the visitors are leaving the website without converting. So this is where on site surveys, on site polls are very powerful. That's when you ask people as they are exiting the website, hey, what is stopping you from converting today? That's an important question that you can ask. And what I like to do is I like to come up with maybe five or six different categories reasons that I think people might fall into as they are leaving the website without actually buying. So it might be that one of the options might be, well, I'm just browsing another option, maybe I could not find what I'm looking for. Another option might be I'm too expensive. Or we as an e commerce website, we are too expensive. Or maybe the items that I'm looking for are out of stock. You can come up with an initial set of questions and then you always add the magical option of other just in case you did not cover all the options. People can select other and then they can do free form text. This can Be very powerful in informing your marketing of what you need to do. Now think about this. We were launched a on site survey for one of the companies that we work with and we asked a simple question and we noticed that 42% of the people who are leaving the website, they said that the prices are too expensive. Well, the price is the price. So you can just say, well, it is what it is and move on. No, you should not do that. If you are doing experimentation, if you are somebody who is focused on improving the site conversion rates, remember a price. In this case, every price is a story. What is the story that you are telling people? So if I know that 40, 42% of my visitors are complaining about the price and they think it's too expensive, I need to show them, I need to justify why this is actually the right price. I need to actually explain to them why you are actually getting a good deal although you are paying something more expensive compared to other sites. So I do that on my collection pages, I do that on my product pages, I do that on my cart, and I do that during the checkout process. Do that. So figure out what we call the conversion blockers. So that's one category of questions that I like to ask. Another category that I like to ask that will inform also my marketing and my design decisions around the website is motivators. What are people trying to accomplish by visiting our website? And while conversion blockers, usually we like to design them, where they are triggered when somebody is about to leave the website. Motivators are designed to pop to the visitors anywhere between 45 seconds to 60 seconds after the visitor has seen the website navigate, maybe between couple of pages. Sometimes I do see some of those pop ups appear way too quickly, like within five to 10 seconds. And it's like, dude, I'm still trying to figure out even if I'm in the right place. Slow down, slow down. So what we need to do is we can again launch those on site surveys, polls, asking people, hey, what are we trying to accomplish? This can be very powerful. I'll give you an example from Our own site, invespcro.com we would love to convert most people into hiring us to do conversion rate optimization services. So we asked people, what are you trying to do? Today we've discovered that about 75%, 77% of the people coming to our website are there to learn about conversion rates optimization. We had no chance of converting those people into actual clients. But what we said, you know what if the majority of our visitors are coming to Learn about CRO, then we need to emphasize the blog a lot more because we have amazing content over there to learn about conversion optimization. We've launched the CRO Academy, the conversion optimization mastery course. We've launched many courses. We said, hey, here it is. If you want to learn about CRO, here is how we can help you achieve that. And then finally, I like to ask hooks for those who converted. So this is on order confirmation on thank you pages. That's why I ask, people say, hey, so what actually persuaded you to convert? I want to understand from those visitors who got to the point of conversion, we did things right. What is it that we did right? This can be very powerful because you can take that information, you can emphasize throughout your site. So all of these are within the category of very specific questions. Now there's another category of general questions that you can ask on the website and those are the type of questions on site polling surveys that you can do where you're trying to solicit general feedback about the website. Hey, is there anything missing from the page? Any information that you're looking for that we are not providing? You could even ask about competitors that people are considering. People sometimes feel very, very strange about asking about competitors. But I say, just ask the question, it's not going to hurt. Hey, who else are you considering to purchase? But of course, you need to structure your survey and on site poll in a way where it's meaningful questions to our attendees, those who are watching us live, or if you're just watching the recording of this video on YouTube. Have you launched on site polls? Have you done surveys? Share with us the best questions that you ask. We'd love to see what people are seeing and what works for them and what doesn't work for them.

Nice. That makes a lot of sense. Are there like any mistakes that people usually do when it comes to these kind of surveys that you think, oh, they're just doing these silly mistakes, but they won't gain any insights from them?

Yeah, some of the silly mistakes, Perhaps the silliest mistake that almost so many marketers fall into when it comes to creating surveys is being trigger happy, launching those surveys before somebody is ready to answer the survey. The funniest survey is I was on a product page for an E commerce website and a survey pops up and it's like, hey, what is stopping you from converting today? And I look at it and I say to myself, actually I was ready to convert. This is a dumb question. Why are you assuming that I'm not ready to convert now that the software that they're using has misconfiguration or they're just asking this question to everybody that is dumb. So that's number one, make sure if you are using those surveys, you are triggering them at the right time with the right conditions. Number two, people marketers set up those surveys and they give you a set number of predetermined answers and they don't allow you to use some sort of free form in their surveys. What are you trying to accomplish today? 1, 2, 3, 4. Well actually I'm trying to accomplish five and my other visitor friend is trying to accomplish number six. But because you are limiting us in the options that you provided, guess what? You are going to get skewed results or people are going to say, you know what, I'm not going to fill this out. Your visitors are actually being nice by filling out those surveys. Number three, some marketers and some sites owners launch surveys and they don't do anything with the data that they've collected. Absolute waste of time. Data for the sake of data is just pure story. Data for the sake of data is a story that you will forget a week later. What you are always looking for is for something actionable. As we collect the data we need to come up with at least three or four different things, different changes that we are going to make to our website. That we are going to a B test because that's really is powerful where on site polling, on site surveys inform your A B tests that can take your A B testing to the next level.

That's good. Those were really good points. I just want to take you back to the number two where you said usually marketers they ask closed ended questions, right? Like where they give people some ideas on what to say or something like that. But if you think of it, I'll just try to play the devil's advocate just for me. Now if you think about it right, it makes sense also to give them like those closed ended questions because it becomes easier for people to respond because not everyone is willing to kind of like write a paragraph and tell you something.

Yeah, no, I agree with that. But what I'm saying is give options and categories based on your analysis. I like to do it in two stages. I'll say, you know what, here are the four or five reasons people don't buy from us. Here are the four or five reasons people are coming to our website. But I always want to include an other option. You allow free form text and then do analysis and you say, you know what, really maybe like 30% of our visitors have actually filled out this other option and they give us this text. And here are some other options that we did not consider based on the free form that people had filled out. I take some of that. Maybe I'll knock off both the options that I initially had in my first poll and I come back and I launch the poll for a second time and collect more results to see. Oh, how is the data getting skewed now? Based on the feedback that I got from people, I'm going for a second stage correct. A second poll that's going to help me really get more accurate view of my visitors and what they like and what they don't like. What you want to avoid is kind of the famous politicians. Correct. It's like, tell us why you will not be voting for XYZ today. Is it because they are dumb or because they're not physically responsible or because they're actually I will be voting for him. And here's other reasons. Correct. The way the question is worded is incorrect. The options are incorrect. And in this case, you know, I guess in this case customers don't lie. But the poll and you've structured it lies, you know, and if you're doing marketing, you want to avoid that as much as you can.

I know that when it comes to things like a B testing, there is sort of like a sample size that you need in order to maybe determine whether the results are conclusive or not. Is it something similar when it comes to surveys? Because for me, let's say I have a website that gets about million visitors, right. How many do I need to collect in order to maybe draw some insights?

Yeah, that's a really good question. It's a tough question. So I think there is an actual statistical answer to this that I do not know in all honesty what I usually like to do. And when it comes to a B testing, you can determine sample size and everybody's already in a variation. Correct. They don't have to do anything. They're not even aware that they're part of an AB test. When it comes to on site polls, when it comes to surveys, people have to choose to participate. You might have a million visitors, but the participation rate of your survey and poll could be at 0.01. Correct. Or even less. I'm usually happy if I collect about anything more than half a percent of the website visitors. Half a percent. And again, I'm not just taking those insights and just making marketing decisions. Correct. I will be AB testing any insight that I'm getting from those to say, you know what, this is really interesting. I have a million visitors. Of Those million visitors, 5,000 people have participated in my on site poll. I ran it for a couple of weeks. I'm going to take some insights from there and I'm going to test them and I'm going to validate those insights that I collected from the poll. I'm going to validate that through an A B test. And there's a huge difference between what you get from a poll where people are actually failing and thinking through their answers versus an A B test where you're actually presenting a solution to people and subconsciously they're making the decision. People sometimes tell you that, oh, I like xyz. That's what they're saying in their survey. Correct. And there's the answer to the survey or to the poll. And then you test XYZ and you find out that the person doesn't actually like it or the way it's presented to them. They think that they like it. So there is the subjectivity of the poll versus the objectivity of the A B test that you have to balance between the two.

That makes a lot of sense. You mentioned that when you collect those answers from people who participated, you then a B test them. Right. For someone who has a website that doesn't have enough numbers for AB testing, what's next? What should they do with the insights that they got from the poll?

So we need to talk about. You don't have enough visitors coming to your website to a B test. You can always AB test. You might have to relax your standards tremendously instead of saying, you know what, in order for me to achieve statistical significance, I need to run this test for two weeks and I need to achieve 1,000 conversions. Well, you don't have 1,000 conversions. You might get five or six conversions a month. You can still AB test, look at other metrics. Correct. Instead of just actually somebody submitting an order on E commerce website or filling out a contact form, you might say, you know what, we're going to just see the navigation from one page to the next. Let's give an example from Invess. That's Invesp. We're happy if we get 30 to 40 conversions a month. For us as a B2B company that sells services that are more on the expensive side, 40 conversions actually fairly impressive for the target traffic, correct? Not for necessarily the blog traffic. We still run A B tests and it's absolutely incredible. They take a little bit longer. We relax the standards but between one landing page and the next, we are talking about anywhere between 20 to 25% lift in conversions between an original landing page versus three, four iterations later. The landing page that you guys came up with last. It's impressive, the conversion rate and the difference in conversion rates. So that's one thing. Now, you can also run a usability test if you really don't have the traffic and you don't have the conversions to run AB test, You can run a usability test between the old design, the new design, but you need to really understand what you are measuring with the usability test, and you need to know how to recruit people for that usability test. One of the biggest mistakes when it comes to usability testing is using sites such as. No, we're not going to mention sites. There are sites that specialize in usability testing. They just basically have people who make a living out of usability testing. They're not your actual users. They're people sitting, saying, oh, I'm just going to do another usability test and I'm going to take $20 for it. Not so great. Try and figure out how to recruit people who are very similar to your actual visitor base and then give them a very clear scenario and then do the analysis to say, well, here's the response. Here's the time it took in previous design, here's the time it took to complete this task in the new design. Here's the satisfaction rate. There's so many different measurements that you can do around the usability test. It's not the same as an A B test, but it's definitely much better than just saying, well, yeah, we're gonna take things and implement them. Because again, you need to make sure that whatever changes you're making, you're able to measure to some extent the impact of those changes on your bottom line.

So how do you balance the need for quantitative data with qualitative data when conducting surveys?

So, to me, surveys are very qualitative. They're user research. Pure user research. Yeah. There is numbers involved, correcting to how people you've collected, how people are responding and whatnot. That is pure user research. It's funny because recently I saw somebody says, oh, actually, surveys and polls are quantitative. And I'm like. And then they said, heat maps are qualitative. And I'm like, dude, you got the definition of those completely flipped out. You know, heat maps and session recording clicks on a screen, actual grid location. They're actual data points that can feed into a machine to do a Machine learning analysis model against them. But besides the point, I like to do user research. I like to get the insight using those on site surveys and polls. But I know it is still user research. And one of the biggest problem with user research, if you ask dumb questions, you get dumb answers. Ask top of mind questions, you get top of mind answers. Top of mind answers are not going to be very powerful. They can lead you astray. You need to be careful with them. That's where quantitative methods become very useful. That's where a B testing comes in really handy. Where you say, you know what, let's validate. Now let's take this to another extreme. We love to do one on one customer interview. So if we were worried with on site polls and surveys and the fact that we're only collecting 400, 500 answers, imagine us doing a jobs to be done interview for a company where we only conduct five to seven interviews. That's it. We're doing five to seven interviews, talking to five or seven customers, asking them questions, pulling insights from their answers. Can I make marketing decisions based on those answers? No. What we do is we take insights from those jobs to be done interviews from those one on one customer interviews. We've interviewed seven people, we pulled some insights and we say, you know what second step, validate those insights from the five or seven people or the 10 people. Validate them by launching an on site survey. So now we've expanded the universe. We started with five and seven, now we're doing on site surveys. Now we're getting answers from 300 to 500 people. Great. We validated the initial set of answers. Is that enough? No. Then we go to the next stage. That's when we're looking at more data based decisions, quantitative decisions. That's when we do a B testing. So it's kind of a cycle. Correct. Each step feeds into the next step to give you a full picture of your site visitors, what they like and what they hate.

Yeah, that's a good answer. One last question, then I let you go. How often should you launch these kind of surveys? Suppose like we launched one last week, we collected like maybe more than enough answers. When will be the next time we'll have to do this again.

So I like to tie this into what I'm going to do with the answers with the surveys. How often should I launch surveys? You need to think about launching one or two surveys per quarter. But that's not set in stone. We like to launch the blocker surveys at least once a quarter and we launch them on different pages on the website. So we'll say, okay, we'll get some blocker surveys to understand what's stopping people from converting on different types of pages. Categorize those, say, oh, there's a problem with the category page. Here's what people are complaining about. Here's a different set of problems that we are dealing with on the product pages and different problems that we are dealing with on the cart or the checkout. Again, this applies to any type of business. I always like to give E commerce. Everybody can relate to E commerce but one blocker survey and if you really think about it, it's almost like three different surveys, correct, based on the type of page, but that's one. And then based on that I need to do my analysis. I need to launch some AB tests, I need to measure the impact. Next quarter I will again launch the blocker survey and see if there's an actual change in the numbers based on the fixes that I've done. At the same time, I like to launch a motivator survey what is bringing you to our website today and then figure out again different type of pages where people are landing and figure out the changes that I need to be making on those pages. That can be very helpful. And then finally the hooks. So you can be doing three types of surveys per quarter looking for the insight. But remember, we're not launching surveys just for the sake of surveys. We're launching surveys for the insights and for the changes that we are going to make in our marketing approach to our site and the user experience around our website.

That makes a lot of sense. Those are the questions that I had for you today. Unless you have anything to add or say.

No, I think that is it. I'm actually super excited about next week as well. Man, I'm going to be back now if somebody back as in with caffeine and water and like the whole thing. If you have a question about conversion rate optimization, about E commerce. Me and Simba live in the land of LinkedIn. I'm starting to slowly move into the land of Twitter. We'll see how that works. I just need to stop watching Liverpool's news on Twitter because that's what I waste my time on and I'm trying to avoid wasting time as much as I can. But yeah, just shoot us your question. We'll be glad to answer it.

Awesome.

Until next time. Thank you, Simba.

Sure. Thanks. Bye.