He analyzed 3,000 quiz funnels and found what actually works

An interactive conversation using the JAG principles is you bring that previous answer into the next question in a fun way.

Most marketers collect zero-party data like this stuff for no good reason except to shove it in their ESP and sit on it.

If you ask 100 questions and you're going to actually use that data, people have no problem taking it. When you give people too many choices or open-ended, they don't know what they want and they want to be led by somebody who's helping with the quiz.

From the open to the click, they're the masters of the flow. Just watch your list grow and your leads turn to go. Did we retain you so far or do we have to extend it? If you want results, you know you gotta send it. Yeah, you know you gotta send it. Yeah, you know you gotta send it. Yeah, you know you gotta send it.

What is up everyone? Welcome back to Send It where we discuss anything and everything retention marketing, specifically in email and SMS marketing and e-commerce. My name is Jimmy Kim and I'm here with my co-host.

Chase Diamond.

All right, Chase. So today, Chase, I've got a little special treat. As the title suggested of the pod, as you're coming into this, we've got somebody, it's an old Twitter friend. That's the best way I'll explain it. And he reached out today, actually today as we're about to shoot this pod. And he told me about some really cool things that he'd done. The man had taken over 3,000 quizzes. So he can bring them all together, checkpoint them, check them all out, and then actually rank them and come up with different things. And I thought this was really important because a lot of people are talking about quizzes these days and adding them to their, You know, front end, their pop-ups, their different acquisition and retention style of things. So I wanted to bring him on. So his name is Jonathan Drake. He comes from a company called, and I think this is the company, howtomovetheneedle.com is his website that you can check out. But Jonathan, you there?

I am.

Awesome, dude. Well, dude, welcome to today's show. I'm excited to talk about this because We need to know about what you discovered by doing 3,000 quizzes because I know Chase and I, we would never do 3,000 quizzes ourselves. And that's just a lot of quizzes. And I bet you've seen a lot of cool things. And I think we had to talk about it just because it was so cool. But before we get into that, let's— quick intro, man. Tell us about yourself a little bit. Tell us who you are. And you've been doing this thing for 15+ years. You're like me. We're dinosaurs in this space. I'd love to talk about your little history a little bit before we jump into it.

Sure. Yeah, I've been in the online world for a long time. I look back, so it's almost 18 years because our son was 3 months and he's now 18. So it's a good way to kind of measure. Started off like a lot of people just building websites, which is kind of funny to think how easy it is now. It was not that way back in the day. WordPress, WordPress MU sites.

Yeah.

And really got into, started doing product launches. People know Jeff Walker. If they don't, they haven't been around a long enough time.

Yes.

If they do know Jeff Walker, you can kind of get a sense of, you know, how long they've been in the internet marketing world. So he did product launches. I became one of the product launch managers, ran several 7-figure launches. This was in 2009, 2010. And, you know, kind of from there I realized all the product launches were all based on email, you know, that how many emails you send and like the sequence and, you know, all the segmentation. So I realized like that was one of the key drivers, if not the key driver, obviously the content and those kinds of things, but the key driver was email, right? Is your email getting delivered? And then so I realized like, okay, that is a great part of, you know, a key part of any business online at the time. And still is today, right? Like, I don't think email's going away anytime soon. So that kind of became the front door, but then it really got into lots of different things. So one of the things I signed up for an enterprise account with Oyala. People remember they were the first to offer analytics for email. And with that's what it was like. I wanted to get to the data, and you can kind of see the thread kind of through my professional history is data, right? Like, what can data provide? Where can you gain some insights? And then I started a company called Heistit. Heistit was an early competitor to Wistia or to AppSumo. I think AppSumo had been around for 6 months. And so I started Heistit with a buddy of mine and we were able to partner with It was early day Shopify. So Shopify gave us 10,000 codes. Shopify gave us 10,000 codes. Squarespace gave us 10,000 codes. And Wistia, Squarespace, Wistia, VWO, right? Several billion dollar businesses and several hundred million dollar plus now. But at the time, people weren't used to spending and doing SaaS. SaaS really wasn't a thing. It was you purchase one time, whether it was Apple products or Adobe products or something like that. So it was really the early parts of SaaS coming in. So I ran that, it did not succeed and kind of beat myself over, beat myself up over that for a while. But I realized in hindsight it was really timing, right? Like if we would've come out with that 3 years later, people were ready for SaaS, right? Like we gave 'em 6 months of free access to all these top-tier tools. And so it kind of went from there. I ended up going into and continued doing email. I worked with Perry Marshall internally, again, kind of the direct response guys, you know, a lot of the info-based products, and worked with him internally 3 years. He was kind of the early gurus of the 80/20 method, you know, focusing on 80/20 with marketing. Had a huge email list. I actually helped him to cut it in half almost. because, you know, working on deliverability, uh, really came up with, you know, the size of list he had allowed me to see into, you know, the data and make some, uh, not only assumptions but then test them out over time. Um, and, and it's not just email, right? So I helped rebuild the website. We talked about, you know, the different segmentation in his business, you know, how to identify his whales and his minnows and, you know, when to place phone calls. And the list goes on and on. You know, the other is page speed. So anyway, one of those— and going back to how we got to— how we'll get to quizzes now— is there was a company called Jellyfish Vision, uh, or Jelly Vision was the name of it, and they had a thing called interactive conversations. Um, they had— so when I was working on Heist It, we asked them to set it up with us. I had an Eloqua. I don't know if you remember Eloqua, was kind of this enterprise-based— there you guys, I knew you guys would Not many people know Eloqua, right? They ended up getting bought for, I think, for a billion dollars. But we had an—

they were one of the first public ESPs at one point or another, if I remember correctly.

Them and then Marketo came on board. And so we had an Eloqua account and we were running Eloqua because they were the only ones who could do what I wanted to do on the backend, right, with building out the sequences and the flows and that kind of thing. So I had an Eloqua account and they did a video, Jellyfish. Jellyvision for Eloqua, and it is awesome. And so they had a thing called interactive conversations, and I can show you, I have saved the PDF, I think since 2012 at the time, and I share it with people all the time, right, that it's, that's what, you know, for me, that's what a quiz is, right? A quiz is a conversation, it can be, let me say it that way, it can be, it's not always is, can be that. And so I'd say maybe about a month ago, there was a guy online runs Bamboo Earth, Dave at Bamboo Earth, and he posted about his quiz. And so I looked at it and I said, hey, you know, like, I think you could do better, right? Like, this is— have you ever heard of the Jack Principles? And he was like, no, I haven't. Okay. I didn't expect him to. Most people haven't. So I shared it with him and I actually rewrote his quiz for him. And I realized like, okay, who else is doing anything about quizzes? And I couldn't find it, right? Like a dedicated— and I see them all the time, right? And the other is, I know we live in a bubble in the marketing world, so I talked to my wife about it and I said, hey, you know, have you done any quizzes recently? Oh my goodness, she said, I do them all the time, right? Like almost anything where there's any sort of selection type product when you're trying to get into it.

Sure.

And so I said, okay, let's dig into it a little bit more. And so I looked at his and then I realized, okay, he was using a company called Octane AI.

Yeah.

And then there's another one called QuizKit. And so this is where I have a BuiltWith account. I've had it for 10+ years. And so again, you can kind of dig into the data like, oh, I can go find, you know, the 2,000 customers that they have and it would be great. Can I go and see their quizzes? Yes. Okay, then can I go— what's the other one? Then it's QuizKit, and there's a few others out there. So I kind of went to what I considered some of the top-tier quiz platforms and was able to scrape them, scrape the quizzes, and I came up with a checklist, right? Like if I had a perfect— in a perfect world, you know, how would I build a quiz or an interactive conversation? And in doing that, I learned a lot of different things. So I know it's kind of a little bit longer, but we'll We got there.

Yeah, we got there. So, all right. So you decided to go take 3,000 quizzes basically, right? And scrape them and do everything. Before we get into the 3,000, let's talk about the principles because I'm looking at this sheet here and you're going after a lot of different checkpoints, but let's go over like the top 3, 4 points. Like obviously a lot of these checkpoints as I'm looking at this sheet here in front of me, like a lot of these are yes and no, they all matter and everything, but What do you think are the things that really made surveys stand out or quizzes stand out? Basically, Chase, you've been on Omnisend for a while now, right? You moved over from Klaviyo now a few years ago.

Yeah. And honestly, I put it off for the longest time because migrating sounded like an absolute nightmare. Email flows, SMS segments, forms, templates. Dude, it was a lot. And it's not exactly something you just knock out in an afternoon these days.

Dude, that's exactly right, because that's the part that people keep getting stuck on, even when they know they want to switch. Nobody wants to rebuild.

Yeah, and it's a lot, right? And that's why I love what Omnisend's doing now with their new migration. Switching over takes just 5 business days and their team does a ton of the heavy lifting, which is fantastic. They help rebuild automation, segments, forms, templates, so you're not starting from zero. Plus, for my account, and I've seen other brands, they're saving up to 35 or so percent. It is really incredible.

I've experienced this myself. 5 days is an incredible fee. That's fast. And I know that they really got a system down because listen, the hardest part about signing up for a new platform isn't the price or anything else, though that's a nice benefit. It's that moving that makes it really hard and making sure that stuff doesn't break, right?

Yeah, 100%. That was my big concern.

And you know what? What's really cool about Omnisend is that they got a bunch of new AI stuff now too, right?

Yeah, we've been geeking out. pretty hard with it. So they have their Omnisend MCP, which essentially connects Omnisend with ChatGPT or Claude. So instead of having to dig through dashboards and trying to piece together everything yourself, you literally can just ask questions, pull reports, check performance, and move way faster directly within Claude or ChatGPT.

Yeah, which is basically how people wanna work now.

I think it's really gonna be the only way to work now and in the future. And if you've been thinking about switching, especially if you're on Klaviyo, You do need to check out Omnisend.

Big shout out to our sponsor here. Go to omnisend.com and check out their migration page. We've got a link. If you sign up through our link, you'll get 30% additional off for the first 3 months on top of the savings that you'll have. But putting this off because it sounds painful, migration's hard, that's what they build for, and you should go check them out. Now back to your pod.

It's interesting, there's a difference of a survey and quiz, right? There's terminology, but a quiz is a little bit different, right? A survey is trying to gather information just for the company, right, is how I see it.

Sure.

A quiz is more gathering, and this is where I realized early on the split was there's a business case, right? Like, what is the business trying to obtain? And then what is the consumer trying to obtain? And there's always that push-pull, and you can see it in the quizzes. So that's why I broke up the benchmarks. One of the things I realized is the first question, right? And it sounds so simple. The first question is pretty important, right? You can shoot yourself in the foot. First of all, you know, I would even go back is, you know, did you have a strategy coming up with the quiz? What's the business use case? You know, what are you trying to accomplish? Those kinds of things first and really try to figure those things out. Then, you know, as far as like the actual tactical, I would say the first question is pretty important.

Sure.

You know, a lot of people put up a— it's almost like a barrier, right? Like, do you want to you know, how old are you, right? It's like people don't want to say like that. There's not really a lot of benefit. And you can kind of, you know, with copy, you can help with those questions and really figure out how to get that question in there. So I'd say the first question is a pretty important thing. And there's actually— I did a post about it. A lot of people have what's considered like an intro screen, right? It's like a page. Why are they— know you're on your website. Why do you put something up like, hey, Do you want to take our quiz? No, like you can do that inside and have the quiz question right underneath it, right? You don't need to duplicate it. So anyway, it's kind of like things, just don't shoot yourself in the foot, especially early on, first question, right? Get that first question quickly. I would say then figure out where the drop-off is in your quiz, right? Like, and some of them are okay, right? And this is where, you know, when you go to the site, you can see there's a lot of push-pull and when, you know, With the 3,000 quizzes, you can, you know, people can opt in, they can see their exact quiz, or I will go and, you know, go and get it. I pretty much have anybody on Octane Lane or Octane AI, and the majority of Quiz Pit are already in the system. And there's several others, one's called like Blueberry, there's a few others out there. I can go get any of them, I, you know, through the tools that I have set up. And, you know, you can really start to see the fall-off, right? And so like, what is the goal of the company? A lot of times it's to make us, it should be to make a sale, to gather their info, right? And it's like, okay, but even in the info, there's a couple things, you know, you can have a quiz where you just give them— and this is again the customer side— they just want the results, right? They don't want to give you their email unless you give them a reason to, right? Like, and that's where it's important, like, can you give them a reason why giving you their email is important? So it's kind of the front end and back end of the quizzes where I think it's important. you know, to really focus on. And, you know, that's where the major drop-offs are too, the first question and the last question, which is usually, you know, show me your— give me your email. And then it's like, give me your email. And then it turned into quickly people are like, I want the SMS, which is great for the business, right? And you have to really then explain to the people at that moment why giving the email is gonna be important, why giving the SMS is gonna be important. And you can do that, right? You can say, hey, Give me your email and I'll send you, you know, a discount, right? And I don't— I'm not a huge believer in just giving discounts away, but this time it could be worth it, right? And something to test and figure out what it's worth, you know, how much is that email worth? And then giving them the actual result, right? I'm a huge believer in giving them the result that they came for, not just the business case. And so if you have 2 products, it's like, okay, you know, do you need a quiz? Probably not, right? Like there's other ways to get conversions and get their email and those kinds of things, but If you know, you know, Bamboo Earth is is one of the ones who does a quiz, and same with Jones Road, which is interesting because he just went all custom with his quiz. He dropped, you know, and I think he's been pretty adamant on Twitter that or on X that he's done that. He's gotten rid of a lot of tools. He was using Octane before. Now he uses his own custom. He rolls his own, and you know, so but I'm so I'm able to still go, and I that was part of the thing. Can I tested? Anyway, so that's where I'd say, like, just figuring out the business case, what those are, and go from there.

And I think this is no different than any principle that, you know, Chase and I talk about. I mean, people spend time on their pop-up, for example, right? But the amount of time that people— if we went and audited every pop-up in the world, I'm sure we'd come up with the same result where there would be a ton of pop-ups that were pretty terrible. And those aren't because to your point, I feel like people set it up in like a passive kind of situation where they're like, yeah, I probably should ask some questions so I can get them warmed up and get some first-party data, you know, get the personalization or the zero-party data and add personalizations for later, yada, yada. There's like this full intent. Like I would say that nobody who starts a quiz or pop-up thinks about like a small picture. Like I'm just trying to get the email address. They're usually trying to get like information out of them so they could personalize their email and different things. Just like you're kind of saying about quizzes, I would say it's the same thing people do with pop-ups because they just don't tend to put the thought process behind it. And what you're basically saying is the difference between a great quiz and a terrible quiz is really how much thought that they put behind the actual business use case and the strategy, or if they just put it up there because that's just what they wanted to do.

Yeah, it's amazing to see how many have— you can tell they're the default, right? Like you go into Octane, Give me the default template, you know, enter my questions or use their AI tool now that they have, you know, and just do the bare minimum.

Sure.

And I know people who've run, you know, 10,000 people through their quizzes or more. It's like, and that's not just somebody who lands on your page, right? That's a very active person who wants to get the information. And so it's like, well, you know, it's not a step 1, right? You're on like step 4, right? Step 5. If somebody is getting to the quiz page or a lot of people run paid traffic to quizzes. So it's like, okay, that's your lander, right? Like if that's your lander and you're sending paid traffic, is it worth optimizing that or is it worth just going with the default?

Yeah, that makes a lot of sense.

And it goes on and on, right? Like, you know, what that actually means in how you implement the quiz.

So here I'm gonna pull this up because I wanna show the site and I think that this is fun and 'cause this is what I was looking at too. I'm gonna use, let me use window. So that we can flip around. All right. So this was the site that Jonathan sent me, and basically they're the worst and the best quiz scores that he's kind of put together based on the 3,014 audited commerce quizzes that he put into place. And, you know, when you click on one, and we'll link this down in the show notes so you can go click around on it and look at yourself too. But if you start clicking around, you can kind of start to look through all these different things and look at all these different data points around the quizzes, the questions that were asked, and overall the rankings. And then also for ones that were just as bad too, why Were they bad? What did they have? Without going through them yourself. In fact, you can link through and go back into it and take the quiz yourself if you want to also. But Jonathan, walk us through this. Walk us through what was the idea behind this? What's the goal for you? And then also like, you know, some of the cool things that we can maybe walk through here on the data that you're kind of sharing. So let's start on the top.

Yeah. So first of all, I was doing this for myself, right? That's when I start going down little rabbit holes. I do it for myself initially, like what piques my interest. And, you know, I feel like over time I've found things that have been pretty interesting, even though they're 2 or 3 steps behind, behind what and where most people go, right? So I'm willing to kind of take those 2 or 3 steps. And so I, you know, I would go take a quiz and I've taken quizzes, but what I found is people, you know, I'm a huge page speed person and one of my favorite is the web page optimization, uh, web page test. And one of the things they did is, you know, they showed their score, right? Like you can see your page, but it's like, okay, how does I want information from that, right? Not just what it is. I want to know like how it compares maybe to the rest of the, rest of the, the internet. And so when you go through, if you go back up to the top, right? So this is actually when you— yeah. So there, if you— those little buttons, I tried to— and this is, you know, trying to share information. I've gotten better at it, but I like all the details, right? So you can see exactly how I'm scoring the quiz. I want that to be explicit, and so you can go through here, right, it's like the visitor experience is 50% and the business leverage is 50%, right, and that was a big insight for me, it's like okay, how can— there's always the push-pull, right, you could have a great quiz for the customer and it doesn't serve the business, or you could have a great quiz for the business, but it doesn't serve the customer and you're not going to get a conversion all the way through, right, and so it's like there's a a push-pull. And that's why nobody got an A. And I didn't purposely set it up that way. I set the criteria up beforehand and let the chips fall where they are, where they were. Uh, the other thing down here, if you go down there, is that there's a little, that little tab right below where you just were before. Um, and you can see, so I did it where I categorized all the quizzes, right? So like, are they beauty? Are they, you know, and so I think there's like 12 different categories. And you can see some of them are a bigger deal and they have more data. But you can see how you compare to all the quizzes, or you can see how you compare to your individual category. And I think to me, that was another level of like, hey, can I get this? Yes. Why not make that explicit, right, for people to see? And then when you go down there, there's the visitor UX score, right? To me, that's the customer side. And again, you can see everything exactly how it's scored, right? It's very explicit. I'm not trying to hide things. It's not a black box. You know, I want people to make better quizzes, better user experience, all of those things. So you can see exactly what those are, and again, you can see the business leverage, right? And, you know, what— how the— how I weighed each of those metrics. You know, lead capture is a pretty big one, and people can go back and forth, and I can always make adjustments. I have actually made adjustments talking to Dave a little bit, and he kind of shared his individual numbers, and so I did make a little bit of adjustment from that as well. And then you can go see— go ahead.

Oh, so this is awesome. This is really cool.

Thanks. And then there's always the— again, you can see the little i. So if you hover over the i, you can see the, you know, scroll up right there where it says CRO Flow Score, kind of on the next section. Yep. And then on each of these, you can see like the CRO where I break down like the visitor side Yep, right there, and it shows exactly what that's doing, right? How did they get the score? And you can kind of see, you know, what the average is, and you can go back, and this is where you can go back to the checklist and see, you know, what are these things that he's asking about? And you can see here everything on the, you know, this is one of the better quizzes that's out there. You can see there how much they are above the average in their category and overall, and you can see, you know, one of them, what I found is how many questions do you ask? And that's always a tough one, right? It's like, well, how much are you— and you guys know this, right? Like, if you ask 100 questions and you're going to actually use that data, people have no problem taking it, right? Like, if you have a million, you know, product SKUs, maybe you need to ask a little bit more. But if you have 6, maybe you don't need to ask, do you want, you know, blue, red, or black? It's like, who cares, right? Like, and so it's, again, that's the push-pull. What I found is between 6 and 9 questions, Is the right number for the majority of people.

I mean, I would— 6 to 9 feels like a lot, doesn't it? I mean, Chase, how do you feel about 6 to 9 questions? Do you feel like they're asking too much if you're going to an e-commerce store and they're like, give me 6 to 9 questions to get a coupon or get a, you know, personalization?

Yeah, that definitely feels high. I think like 5 or 6 feels good to me, maybe 7 max. 9 feels like a lot, right?

Yeah, unless it's a super hyper-personalized product. Like I take an allergy medicine that I did an online quiz with and it asked me a lot of questions, but that kind of makes sense because it's like supposed to be like personalized towards you. But let's be honest, and I don't know how much Montana Candle Company personalizes their product display or product options after your quiz. I don't know that answer. However, 6 to 9 questions asking me about candles, man, that's a lot of questions asking me about candles. All right, real quick. It's kind of wild that we're already talking about holiday planning in July right now, right? Just like any other event, there's basically a conversation everywhere. People are already diving into BFCM planning, testing promos, prepping inventory, creative acquisition, the whole thing. And it makes sense because e-com brands, Q4 is when not only when traffic gets expensive, but it's when the traffic is the highest. That's why I find retention.com are super interesting right now. You're already paying to get more people to your site, but the most valuable visitors are still leaving without buying anything or joining your list. And retention.com helps you turn those anonymous traffic into opt-in email subscribers. So you're not just renting attention for the holidays, you're building an audience that you can keep marketing to after the season ends. And they've made it super flexible right now, which is huge. You can start for absolutely free, turn it on when you need it, turn it off when you don't. And you pay as you go, which is crazy because everyone else is charging these big contracts. So if BFCM planning is already on your mind, which it should be, this is probably one of those places that you want to go check it out. Go check it out, retention.com. Go start for free. We'll take you back to your pod.

Yep. And, but they should know that, right? And again, this is, these are not, I don't have their internal numbers, right? I don't know what their actual numbers are. They know those and you can see what that is, right? So we also know the first question is important. And a lot of times this counts email capture as a question, right? So here's all the questions, right, which is nice. You can go see exactly what questions they do ask, and you can see the bottom.

That's cool.

Right, you know, you can— and to me, this is the big one, and you can see why they do well, right? One of the things that they do well that I put in there, and you can hover over, those are their actual images inside their quiz, right? So you don't— I mean, you can go and take it, but this is Another part where you're customizing the quiz, a lot of people use images, a lot of people don't. I would say probably about 30% actually use images, but I was like, okay, I want to pull those down, right? So people, you can go take the quiz and see how it feels. So, you know, going back to, you know, this, right? So does this feel like too many questions? You can see exactly what they are, right?

Like scent strength, that makes sense. Flame or no flame? I didn't know you didn't— I figured candles had all flames, but didn't know that. But I'm also not the type of guy—

Well, electronic candles, right?

Okay.

Favorite scent. And that makes sense. What kind of scents do you like? And then it's like, what kind of scent? Where do you want that scent to be? And then it gives you 15% off. It calculates some results and gives you an explainer. And then what happens? Do we tell? No, we don't want to talk about that.

No.

So that's right. So that was a decision on my end not to kind of go into their— what happens afterwards. But again, on the page, I have a direct link to the quizzes that you can go and see.

Right.

And then it's layered over a second quiz. Okay. Let's try this real quick live here.

But look how fast it is, right? Like, I think that's where before people saw a quiz and you talk about 6 and 9 questions, he just ran through 5 in 20 seconds, right? So with page loads, they used to be pretty bad, right? So like you were like, you know, calculating results.

I always think that people— I don't think this really takes that long, but people like to make people wait. There's probably some—

I will tell you, there is. And I actually worked with a client. I worked with a company called BeenVerified, right? BeenVerified is a huge company, um, you know, hundreds of millions kind of thing. And I worked with them, and I talked to the, the founder, and I don't know if this is public stuff, but he— so I probably shouldn't have said the exact name. But anyway, what they do is they— he's like, we can give that information in 2 seconds. Like, we have it all, right? Like, instant. But people don't at that point value what's actually they're getting, right? It's like, how are you going to pinpoint, you know, my neighbor in the middle of nowhere, what their, you know, phone number is? It's like, well, you know, you have to Give a little bit of that, right? So that's the conversion optimization in play. But again, looking at that, if he had 2 more questions, would you have thought, hey, that's not worth it? If they were good questions, I don't think you would. If they were dumb questions— but each of those questions made sense all along the way.

I mean, I would say that my theory would be if you can keep me engaged, just like a video or any other kind of form of media, and you got me engaged and hooked on the questions and it made sense, I'd probably continue on just because of it. But I'm sure there's some kind of number that like There's probably some kind of drop-off number that looks like that, where it's like the major drop-off probably happens between question one and two, and then after that it probably stabilizes, kind of like a video does.

Right? But then you also think about which who has a better customer, right? Like you have to consider, like if I have a customer that is willing to answer, you know, two questions versus twenty, right? And I don't remember there's a company called Thumbtack, right? They did a study. I love when you guys know all these things. That's so great.

Yeah, we've been on the internet for a long time, dude.

I know, right? Like, I enjoy this conversation because I can say Thumbtack. You're— I didn't— you didn't say anything. You both just went like this. And Thumbtack, they did a study, right? So they help people. It's kind of like an Angie's List is what it does. And Thumbtack, they're like, oh, you know, we can just get them the results, you know, exactly what they want. What they realized, the more questions they asked The better their customer was on the other side. Sure, right? And I think they're at like 12 to 15 questions, right? Because somebody wants to feel that tailored approach. Now they obviously tested it, right? They didn't just come out and say, "Hey, we're making this declaration." They're saying, "No, we're going to test it." And I think they started off with two, right? Where do you live? What do you need done? Okay, here it is. Here's your list. Now they're like, you know, do you need a left-handed plumber or a right-handed plumber? I don't think in that detail, but that's where they found, and that's where I, you know, when you talk about like 5 or 6, I agree, right? The other thing we didn't talk about, and when you go into the scoring, is it's called the Flesch-Kincaid score, which I'm sure you guys know Flesch-Kincaid, right?

Yeah.

Which is how, what level of grade are you using? And so I had the tool go through, grade the quizzes based on their Flesch-Kincaid score. Right. And like, should they be at this level or should they be at this level? And it kind of depends, right? If you're, if you're getting your medication, you don't want it probably to be second grade level, right? And so this is where it's always a custom and a push-pull, but you don't want it to be, you know, a PhD level when you're buying a candle, right? And I think if you go back to the candle company, I'm sure they did well, right? I think they were like at fifth grade, right? Which is right where you want to be, maybe even lower.

Exactly where you want to be.

Yep.

It makes sense.

Question for you guys. Oh, sorry, Jimmy.

Go ahead, Chase.

What do you think about AI's role in all of this? Right? Like, do you think attention spans, number of questions, how you receive and get information is changing at all with now people interacting with AI versus quizzes or more like a static Google? Do you think that has any impact here?

Are you asking me or I thought you're asking Jimmy?

Oh, no, I think he was asking both of us, but I figured you probably have more better thought than I do about this.

Well, I know when I go through, you know, before you would ask Google and it would just give you the results, right? And you're like, where now it is a conversation, right? So that to me is like the Jack principles and we can dive into that, right? But there's a lot of people, again, I've only seen 1 or 2 people use the Jack principles correctly. And so what you do is the question, after you ask the question, you feed that into the next question, right? In a very fun, right? So the Jellyvision was made by people who were improv comics back in the day, right? And now they do healthcare for enterprise, right? Like, that's where they kind of— they moved to. But it's more of a conversation. And I do think a quiz and thinking about it like a conversation as opposed to— and when you have a conversation, you don't just ask the next question. What about this? What about this? What about this? You ask a question, you get the answer, which you already know they're predefined, right? Like, you're giving them the 4 options or 3 options or whatever. And what an interactive conversation, you know, using the JAG principles is you bring that previous answer into the next question, right? In a fun way, right? Oh, you know, oh, you know, you don't want to burn down the house, you know, obviously you're going to choose electric candles, right? Or something like that, right? There's this personalization that can happen. And I think it could have been done before AI. But I do think the person who's using it, AI goes back and forth, especially when you have all your MD files and all those things. kind of dialed in. It's talking to you as you kind of have trained it to talk to you. It's not very static. It's more of somebody or a thing that is used to how you communicate. So I do think it's going to help. I will say it actually makes writing the quizzes much easier as well. You know, I was able to gather the information using, you know, AI, right? It helped to gather that where, you know, this would have been a 6-month project and it was like a week and a half now.

Okay, wait, wait. So here's a great question as I'm thinking about Chase's question. What, you know, AI's kind of taught humans that they can interact in like complete sentences. Do you think the quiz evolves into a more AI-style experience where it's like more open? Like instead of asking what kind of scent do you like, like light, strong, whatever, it's like, hey, how do you want your room to smell? Like, what do you think? And then you would type in, oh, I like it a little bit good, or whatever it might be, however you might want to write it.

Yeah, but I think there's the choice paradox too, right? When you give people too many choices or open-ended, they don't know what they want and they want to be led. By somebody who's helping with the quiz, right? Like, you're the expert who has the quiz. They're not, right? And that's where— so again, that's where I think like a survey is different. I think asking, right? And that's where a chatbot is great, right? A chatbot, you can go back and forth. I think a quiz has a very specific thing where, hey, you know, our site's the expert. We're trying to get you a better customized approach. You know, how do we do that? Well, we do that through this quiz. And the other is, I think with a quiz, It feels as though you're taking it kind of in isolation, which is helpful, right? Like you're not taking it to go and put it in Times Square somewhere. Your results are not going to be Times Square, you know, on the billboards, right? It's going to be personalized. And again, those are part of the questions. And really when you're setting up, the first question can really say, hey, we're— I want you to, you know, take this quiz because of this, right? And the other thing is naming of the quiz, right? Doesn't sound important, but, you know, Jones Road has 2 big quizzes that they run and one is, you know, the skin quiz. You know, what kind of skin type quiz, I think is what they call it. Perfect, right? Like, what's your skin type? It's not—

That makes sense.

You know, we want to— how can we sell you skin products quiz, right? That's not what it is, right? So naming of the quiz is important. It's— again, I'm a huge believer in marginal gains. Small incremental things add up over time. And so that's where I'd say, you know, the questions, how you phrase them, And again, that's where AI can be helpful, right? Like, how can I phrase my first question? And again, that's what I tried to put out here too. You can see how I've already graded and scored and benchmarked, you know, 3,000 quizzes. You don't have to go guess anymore. You know, in other, you know, like Octane, QuizKit, you know, DiGio, they have case studies which are nice, but they're gonna highlight their best ones, right?

That's right.

That's where I think it's helpful to see the worst ones. So if you went back to the site and you looked at some of the worst ones, They're like 50 levels, 50 questions. Like, what are you doing? Right?

Well, this is the whole problem. Like, when you want to ask all the questions you want to ask, I saw like Dr.— and I'm not here to pick on anyone, but Dana Nails, I think, or Dan Nails, uh, has 33 questions in their flow. And that, that's probably why it scored so low. It probably wasn't a terrible— it probably wasn't a terrible one. It was just that they just asked way too many questions.

And I get it.

But again, there's got to be a reason for it and something where it's nails. I don't know how much people need to know that many questions. And the real question I go all the way to the end is like, is it really just to do a product recommendation? And is the product recommendation even any good? And that's the question I wonder is like, what is the lift and what is the drop versus the end result? Like, was it worth it? Because the one thing that we say all the time is that like most marketers collect zero-party data like this stuff. For no good reason except to shove it in their ESP and sit on it because they don't know what to actually do with this stuff. And I get the product recommendation, but I would say there's far more things that you can do with this data than beyond just showing a handful of personalized products or whatever it is. So there's that.

Yes.

All right. So here's the thought I was kind of thinking about, Jonathan. Now quizzes versus popups versus no popups. I mean, this is going to be the age-old question. I think that this evolution is happening right now where people are getting into quizzes, which will qualify a buyer better, 100%, right? But is it better than a quiz? Is it worse than— I mean, better than a popup? Is it worse than a popup? Like, how do you think about this? Hey, you guys want to stay connected with us? Guess what? We send a daily retention newsletter out that you should join right now. If you're not a part of it, it's at ecomemailmarketer.com. It's completely free. We send it to your inbox daily. We do breakdowns. We talk about the latest things. We talk about the latest tools out there. There's so much you can do. Plus, we even talk about the pod, of course. So if you're enjoying the pod, you want to be here, you want to listen to us, you want more email and SMS retention marketing tips, tactics, tools, tricks, whatever it might be, head over to ecomemailmarketer.com. We probably have something free that we're giving away right now too on top of that. So go get that, download it, join our newsletter. We'll see you there.

I think they're trying to solve 2 different problems, right? I think that, you know, the jobs to be done kind of framework, like they're solving 2 different problems, right, how I see it. I think that they're similar because they interrupt in a way where I think a pop-up interrupts, but it's trying to convert a buyer very quickly with a benefit, and a lot of times the only benefit that you can provide on a pop-up is a discount. You don't have to provide that inside a quiz, right, like, and so you can not only qualify your buyer, you can hopefully hang on to more margin, right, along the way, more profit.

Yeah. Well, I mean, a popup, you're right. Like they're usually asking for, trying to give you a discount, which I think I have my own opinion on popups and how people tend to overuse a popup too early before a person has even tried the shop. I think there's always a problem there already, but I almost feel like the quiz is less intrusive and it's telling you like someone would, not everyone's going to feel like someone like myself or Chase probably wouldn't feel helpful, helped by a quiz, but I would almost say that. A lot of people, the general consumer, would want to take a quiz so they can get that more personalized experience, right? Like, that's probably the way that most people are thinking about it, right?

Well, think about your— that again, I'm glad that it rated high, right? I didn't purposely put certain companies high. I didn't choose the example you chose. I mean, obviously I had the tool kind of bubble up which ones were the best ones based on the score. But think about the candles. You did not even know when you went through there, like, oh, I didn't know flames or no flames was even an option. Right? Like, how beneficial now if you were buying candles, right? Like, you can introduce new products to them, right? Like, you didn't even know that that was there. So your quiz can actually be a selling tool by giving them product discovery. I didn't even know that was there, right? And again, it's a great quiz because of that, right? Like, and it was fun seeing you go through it kind of cold, right? And that's also why I had— I debated whether to put a link to their quiz, right? People can change their links. People may not like that I send links directly to them, but again, they had product discovery as a part of the quiz. That's great, right? Like you want people to get 2 or 3 levels deep. If you were going to buy a candle, now you know whether or not I should buy this or this, right? You're making a buying decision. And by you getting that information from that company directly, you're like, oh, they're— you may not say it explicitly, but internally you probably are like, oh, I'm a better educated Candle buyer because I just went through that quiz.

Yeah, that makes a lot of sense actually. Huh, very cool. Now this is super cool information. I think that, you know, a lot of people who were thinking about doing a quiz or are actively doing a quiz, there's a lot to think about with the site. And for the listeners there, we'll make sure we throw the link down below so you can go check it out. In fact, you even give a free audit of your quiz if you want, right? Is that what I'm seeing here?

Yeah, so I have an audit where if you're using, you know, Octane or QuizKit, first of all, they're probably already in it, right? Like, and then when you opt in, it should go directly. I have it set up. If you're, if you're already in the system, it should just redirect you, and it will redirect you right to your benchmark that's already done. If it's a custom quiz or something like that, yeah, I'll go through, and I have the tools pretty well set up now on the backend where, you know, I can, if they point me to the quiz itself, That was one of the hard things is to find the quiz, right? A lot of people don't link directly from their homepage. It's a lot of paid traffic and I understand, so I had to go and find them. And yeah, so when they do that, they'll get this benchmark and their score. It won't be me doing the audit, it will be against kind of the benchmarks and the audit that's already out there compared to everybody else, and so that's all automated, which I think people can gain a lot from. And again, going to the best and the worst, the top 6, you can kind of get a sense of, again, those are the ones I opened up to the world where you can go and click and not only see the quiz itself, but how it's being scored and you can kind of look at it. But I think it just makes a lot of sense to put yours, put their quiz in there and they'll get their own benchmark. Yeah, it's all automated.

Very cool. Well, guys, if you're listening to this pod and you So thought this was interesting, something that's relevant. We highly encourage you to go check out the link, talk to Jonathan if you'd like, and reach out if you guys want to talk a little deeper with him about this stuff. But I thought this was super insightful. I mean, it's just a really cool project just because I don't feel like anyone in the world is really focused on quizzes per se. They are focused on pop-ups, they're focused on CRO, email, all these optimization points. But I don't think I felt like I've ever talked to someone who's really gone and gotten really passionate about quizzes. That's why we wanted to bring you on here. So thanks for being on here, Jonathan.

Yeah.

Thank you. Yeah, I enjoy what you guys do and I follow along. You know how it came up is I was actually watching the videos from the Retention Roadshow. Yeah. I'm like, hey, I need to reach out and say, hey, this is what I got going on.

Yeah, yeah, no, these are cool. This is how we learn about people and it's always cool how we meet people on social too. So with that being said, again, link in the show notes, go check him out, guys. If you want to reach out to Jonathan, if this is something that is really intriguing, I think I learned some stuff around quizzes. I don't really think about quizzes that often, but I'm glad to have kind of stumbled and talked about it here today. We hope you enjoyed this episode, and we'll see you on the next one. See you guys later.

From the open to the click, they're the masters of the flow. Just watch your list grow and your leads turn to go. Did we retain you so far, or do we have to extend it? If you want results, you know you gotta send it. Yeah, you know you. You know you gotta send it. Yeah, you know you gotta send it. Yeah, you know you gotta send it.