How can an organization evaluate what their customers think about them, and why does that matter? We discuss how a survey could help to win awards and give your business a boost.

Hi and welcome back. In fact, it is a welcome back to Chris Robinson, who's come to talk to us about evaluation. Chris, welcome back. Nice to see you again.

Thank you very much. Great to be back. Thanks, Jim.

Now, you were saying that you were helping companies with their awards, but also you've got another part of your business, which is evaluation. What problem are you solving, Chris, for business owners, with this other business that you've got?

Well, based on the part that does, awards began in 2006. And what happened is we found that to win awards, you had to do surveys and research, and you couldn't win an award just with lovely purple prose. You had to back up your claims with evidence. No better quick way to get evidence of happy customers, happy staff, happy suppliers, happy community than a survey. And a well designed survey won awards. So we found that nothing won awards like a survey. But we also found when especially working with L and D and customer contact teams, nothing made them do a survey, which they'd be meaning to do for ages, but suddenly they had to do it because they wanted an award. So the two things worked really well. But because we started up just before the recession, we found that a lot of people didn't want to be seen to hire a marketing agency. And so we set up a new brand, a new legal entity called Boost Evaluation Limited. And what was great is it actually grew legs. And there's 22 people at Boost, and a chunk of them work on surveys and research, and a chunk of them write award entries. And the two things marry up. But wanted to talk to you with my evaluation hat on today in particular.

Love to hear that. So tell us, how does a company go about doing evaluation? So it's meaningful because there's no point in having an evaluation that just says we're great, that doesn't manifest itself as a press release or anything. So how do you get it so that it's well balanced and also somehow giving insight into what could be better? Because that's also another function, isn't it, of evaluation?

Yeah, absolutely. And it's very interesting you phrase it like that because often, because of our heritage in awards, people would say, I want to win an award. So how do we prove that we're amazing? And you do a survey, and if you cherry pick the best news and strip out the not so good news and pick out all the really glowing testimonials, then you can package that up and without distorting the truth, you just focus in on the good news. And often what we found is that people Loved the fact that you put all the good news into a narrative, whether it's a case study or an award entry, and it felt great. But what they often forgot to do is what I think is more important with an evaluation and why this is relevant on a podcast that focuses on marketing and PR is the fact that you need to listen and that will improve your strategy, improve your future, and make some important decisions. And that's often forgotten because they're so busy reflecting on how awesome the past was, they forgot to inform their future. So what I was when I was listening to your fantastic podcast and I was walking the dog and I thought, you know, this is something I'd love to share. We as a business found the entire awards industry was rubbish at doing this. So we ran a survey, got 330 people who enter awards to tell the awards collectively what they wanted, because the whole industry was doing totally different things, guessing, firing at the hip. And we got a bunch of wars organizers on a call and we said, this is what people want. And for them, it was an epiphany. And for us, too, because we're in this industry. And it made me think, God, if we hadn't have done that, no one would have done it and the whole industry would continue firing from the hip. And we're doing it. We're working with the recruitment company, but said we want to enter an awards. And I was. And they've stopped entering awards and they're focusing all their energy on a customer survey which is literally about to go out. And we're working with some training departments who are all about. We have had to reinvent training for the last year. What did we do? Well, what do people want to go back to? But what do they want to stick with? Now we've done it. Do they like digital learning? And a lot of companies have had to reinvent themselves and come up with entirely new propositions. And they need to learn from the process and not just go back to the status quo and find it, but actually find that the future is probably different. But in what way? And we've seen that you need to reinvent yourself. But how? And, you know, I had these. I had a light bulb experience and I've watched others have it and I wanted to share through your fantastic podcast, encourage other people to do the same thing.

Chris, thank you for the compliments. And I was listening to a podcast about the daily Brewer podcast and newsletter, and they were talking about digital metrics and how people are looking at all the insights from Google, AdWords and all the trackability, but that's only given them historical data and that they are even suggesting get on the phone to a sample of customers and talk to people. So can you just share with us, Chris, Evaluation, how do you do it?

Well, telephone is one mechanism and sometimes people default to that. It's very difficult to schedule a phone call or get through to someone. It's quite an intrusive mechanism. It's often easier to begin with a digital survey. There's loads of platforms. SurveyMonkey is the most popular one, there's plenty of others. There's more expensive ones like Qualtrics, some various small niche providers as well. But SurveyMonkey is a good starting point and often you start with a digital survey, but you can invite people at the end to say, can we contact you by phone? And so it's less of an intrusion. Is also as a mechanism it is very effective. But I would always encourage you to put in an incentive. So a big part of the how to answer your question there is to get as many responses as possible. So incentivizing a survey will get you a higher response rate, but it also gets you a balanced response because they're responding because they want the incentive. Whereas if you don't put in an incentive, you end up with what we call the Marmite effect, where people respond because they have a strong view. Either they love you or they hate you, and that's more likely to make them want to tell you their view, so you get this imbalanced response. So digital surveys, yes. Forums, panel conversations are great as well. Zoom meetings are brilliant for that, individual telephone calls. But in terms of the how, always bear in mind you've got two aspects. You've got qualitative, which is the stories, the anecdotes, adding color to your research. And you have a quantitative, which is where the survey. You've got to have numbers. So a bit of both, a bit of numbers and a bit of colour added to that through qual and quant. There's also two other bits of jargon I'd like to share with you. One is objective measurement and the other is subjective measurement. Now, subjective measurement are questions that you can only measure through a conversation through a survey. And so obviously, if you're going to make a phone call and a survey, you need to focus on them. If you have a discussion with your colleagues about, begin the whole research exercise in terms of your question of how, right at the front of this is a hypothesis discussion. What is it we believe to be true? What is it we don't know what is it we know exists, but we don't know how much or how little. And you just brainstorm all the questions you want answered, the hypotheses you want tested, and then you come up with a list of questions and those are the things that you split into. This is a subjective question. It's about feeling and emotion and their sense of the future, their votes, their experience of the past. These are sense things, they're subjective. Stick them in a survey or a phone call, then you get objective data. These are things that defy interpretation. And that's what you were describing there. These are data that you can get from Google Analytics or from, you know, you can get objective, which is somewhere between the two, like trustpilot scores. But it's a reflection of the past and it is tracking data points. So when you are coming up with your evaluation strategy, start off with a discussion about what are we trying to prove, what are we testing? What are our hypotheses? What do we not know that we need them to vote on? Come up with that list. Divide it into qualitative and quantitative and subjective and objective. So it's four different bits of jargon, but it is a process. What you must never do in the how is just dive in and just write a survey and chuck it out there because you'll come back to it and think, God, I wish I asked a different question. Or I should have phrased it differently. It shouldn't have been an agree to disagree like question. It should have been a vote or a scale. So that's the process.

Okay. I love the idea that you have to have a hypothesis to kind of prove or not prove first rather than just generally going out. What about sort of the duration? Is there kind of some sort of magic number or. Of either questions or duration?

That's a very good point. And yes, dropout is a major problem. Bain and co published a book called the Ultimate Question. And their philosophy is that it should be one question. And that question is the most popular question. And everyone's seen it a million times. It's on a scale of 0:10. Would you recommend us to your friends? And so that's called the Net Promoter Score. It's the most popular question out there and it's one question. And the whole philosophy is you just ask one question. However, I think that's overdoing it a bit. If we've done a. Facilities management did a massive exercise, huge, huge survey. It was. They were doing loads of thought leadership off the back of it. So it was getting them to say whether they've done it or haven't done it. If they haven't done it, then they got questions about what they thought were the barriers and excitement, the roi. If they have done it, they reflected on what works and what didn't work. So it's a massive, complicated multipath survey and yes it was. We would time it and we'd say that's a 10 minute exercise on average. So make sure you set expectations. This is a 10 minute survey. But in return you'll be in with a chance of winning an iPad. So you've got a bit of a. What's in it for me? People will give you a lot more time if there's a possible iPad or instead of a small chance of a big prize, you have a high chance of a small price. So you can have we've got 10 pound 30Amazon vouchers or even smaller prizes. Everyone who answers gets a free cup of coffee through Uber Eats.

The other aspect or strategy seems to be to have multiple surveys, a bit like a funnel. So have you experience in having maybe sort of a short survey with a. Would you be willing to answer more questions? What about that, Chris, this sort of ongoing survey where you're funneling people into becoming almost sort of a short list focus group, is that a strategy that you've experienced with as well?

Yeah, there's actually two paths with surveying and companies often do both. If you're in business to business, which I think a lot of your listeners are, then you tend to fall into the once a year or once every half year staff surveying exercise where you survey all your customers at one instance in time. And that is a different approach to the other strategy, which is the continuous customer monitoring. If you work in a call center and you got one of these massive call centers in Dublin, for example, you want to track daily by call center agent by department, the net promoter score on a call by call basis. So you do those classic text messages that you get and that's. And there you would do your one or two instant questions that the vital questions and then you'd say can I ask you a few more? And you get like a 90% drop off rate but you've captured the critical 10%. And those are two approaches and they both work. But it goes back to the what are we trying to prove question at the start.

You've also raised an issue there of the sample size. I read somewhere that the sample sizes as small as 30 respondents can give you an indication of what the overall universe is going to think and one notices. Like for example, some of the American election surveys were kind of 100 people. Is there some experience of what creates enough of a sample size to make it valid?

There are mathematical models you can use. It's one site. You put in the variables and it tells you the degree of accuracy off the back of it and or the required sample size to hit your target degree of accuracy. And now the very often in B2B you have a small universe, you have 50 odd people. So how do you make that more accurate? The trick that we recommend is having a 10 point scale because we find that statistics, whatever surveying technique you use, Whether you're surveying 10,000 people or 10 people, you'll find that you get a wobble of the statistic. You keep refreshing the survey and you're monitoring it and it wobbles and wobbles and wobbles and the wobble becomes smaller and smaller and smaller over time until it becomes rock steady. And no matter how many more responses you get, it's really not going to change much. If you have a binary question, would you recommend it to a friend? Yes or no? The wobble lasts a lot longer because one person can have a big influence on it. If you have a 10 point scale, the wobble settles down a lot faster. So if you're doing a smaller group using a 10 point scale like the net promoter score, but using it throughout, the math suggests you get, it stabilizes much quicker in terms of the number of people. It depends on your deep dive into the data. You might be that you are looking into splitting the data into new customers, long term customers and sort of slice and dice for data like that, in which case you've got to look at what is the smallest group. And I would suggest that a statistic isn't sound unless you get at least 10 responses. But if I was doing a survey in PR Principles, a magazine or a newspaper wouldn't be interested in it if you wanted to publish it unless it was 500 responses. In terms of your customer survey, if you get an 80% response rate, doesn't matter how big your customer survey is, that is representative if you've only got 50 customers and 40 of them took part in a survey. That's the truth. So I dodged your question a little bit there. But there are techniques to make it more accurate and equally it depends on the context.

Yeah, what about the survey with your suppliers for example, and your partners and also your staff? Because in Speak PR we talk about three different audience groups. Chris, we talked about your external customers, but also your staff and your suppliers, your partners can make a big difference. What's your experience of sort of evaluation there?

Yeah, employee surveys. There are a lot of debates. One is about anonymity. We're dealing with a Swiss telecoms company and they're quite upfront, say, right, it's not an anonymous survey. We know exactly who you are and that's just our culture and that's just how it is. Whereas a lot of organizations, particularly organizations where there might be a union involved, anonymity is essential. Also, if, like we're doing a compliance related survey, you know, asking people about their behaviors to do with compliance, we're doing that with one client. You really don't want give the person the impression that it'll get back to their boss that they've admitted that they're likely to discuss the company at a dinner party. So you're asking sets of questions. Anonymity is a big thing there. And staff surveys, you want to give anonymity. With customer surveys, you want to know who says what. And you don't need to worry about anonymity that much. You don't have to say, don't worry, it's completely anonymous. Say what you like. So they're different in that respect. And with customers, I would also encourage people to have a box saying, if we can quote you, tick this box here, because they might give you a fantastic testimonial which you desperately want on your homepage, rather than go back to them and say, can I have a sign off? If they just tick a box saying, can we quote you? You can actually harvest loads of snappy, spontaneous quotes. And I mean, you're a PR professional. You know how hard it is to get testimonial out of people. And then when you do, it's. It feels like a PR testimonial. Whereas sometimes your dream snappy, the sort of funky sound bite you might get on a film poster, you're more likely to get out of a survey where people are actually being really brief and say, oh, wonderful company, love working with you. Oh, yeah, I don't mind you quoting me. And so it's very different. But staff surveys, I would encourage you be anonymous, truly anonymous. That's why, you know, this is partly why we're kept in business, because we can say we're using a third party. Your, your responses will not be shared with anyone within the business. But yeah, anonymity is a debate. It's worth having make decisions because if you do anonymity, you lose ability to send out reminders accurately and also to attribute problems. And there is a gray area. Sometimes you want to know which department they're from, but you just don't want to know their name so that you can go back to the manager and say you got a bit of an issue, mate.

Chris, what sort of costs are we looking at? Can you give us some idea even just what does it cost for Mailchimp?

Well, SurveyMonkey, there are free licenses so that you can just sign up, it'll be peppered with adverts and very limited in the number of people you can send it to. But equally there's a whole grading scale of how much you want to pay. So if you're happy to pay 2, 300 pounds, you can get decent license. Decent license and lose the branding and have lots of questions. You do. It's an American system, so you do have to adhere to American can spam laws and you can't use the mailing system to spam thousands of people. But SurveyMonkey is a great option, very, very cheap. There's loads of equivalents out there, but in terms of hiring a consultancy to do it, we just charge consultants day rates and so sometimes you can knock out a survey for 500 quid but then on the other side of a spectrum there's often a five figure fee for a two year long evaluation strategy. So it is a bit of a piece of string.

Great. Well it sounds like the measure is whatever you have as your original hypothesis that sort of creates the scale of the survey. Chris, if you want to find out about you, where can they find you?

Our website is boost-evancing.co.uk and his services are listed there.

Thank you so much, Chris Robinson for joining me today on the Speak PR podcast.

Lovely talking to you Jim. Thanks for having me back.

Thanks for listening to this episode. And so we wish you the best of health, a sustainable business or a profitable one. And if you're in business, just listen and do some evaluation of your customers. But I also encourage of your team and your partners so that you can get some guidance into what they they need from you going forward as we get out of lockdown. Thanks so much for listening to this episode of Speak.

Prior to.