Direct Mail & Testing Strategies

Hello everybody.

Welcome to the Magnetic Marketing Podcast. Today we get to listen as Dan Kennedy talks about the importance of split testing, but not just the importance of split testing. How do you make split testing work offline and online, and is there a correlation between the two? Are you able to do one in order to define how to be able to move forward in another? Using a case study published by Wired Magazine as well as Bob Stone's Successful Direct Marketing Method book, Dan Kennedy is going to discuss how to use A/B testing to get the best results possible both online and offline with old strategies and new.

I don't think there's anybody that has had a bigger impact in the field of direct response than Dan Kennedy. The legend of Dan Kennedy should be ignored at your own peril. They're not really lessons, they're kind of laws that you live by. Dan opened my eyes to what small business marketing looks like. Dan teaches strategic direct response that is timeless. His ripple effect touches people who don't even know his name.

The world as we know it was changed because Dan Kennedy became obsessed with marketing.

Welcome to the Magnetic Marketing Podcast with your host, Dan Kennedy. Hi everybody, welcome to this month's Diamond Call. So I'm going to talk with you a little bit about a couple of some direct marketing lessons and strategies, both new and old. I want to talk first of all about testing. And so in the world of direct mail, since the beginning of direct mail, people have figured out that it was a good idea and it was possible to try and test variables. In relatively small quantities that various things mattered: the size of an envelope, the color of an envelope, what was in the return address corner, teaser copy, the first thing people saw when they opened up the envelope, headlines, photo, not photo, this photo versus that photo. Celebrity, no celebrity, etc., etc. And for the most part, you can only do an A/B test. If you test more than one variable at a time, you very quickly get yourself in a position where you don't know anything. So direct mail test matrixes are fairly complicated and can be expensive., and can take some time to wring it out over a period of weeks or months. What I want to tell you today and point you to a very current reference has to do with the role of online testing in improving offline results, and so some of you, make sure you're sitting down because you are about to hear me heartily endorse the use of websites, SEO, paid traffic, all of that online for marketing purposes. And so in this month's Wired magazine, which I suppose it will also surprise some of you that I read it, it has Mark— Larry Anderson on the front cover, the guy that invented the browser. He incidentally was a business partner of my friend Ken McCarthy's way, way, way back when, when people were just figuring out the internet. And this shows, and I'm just going to describe it to you, this shows testing that was done, a series of A/B tests because you can only do one at a time. To improve the response to the website for the Clinton-Bush Haiti Relief Fund, a fundraising site. And there were 1, 2, 3, 4, 5, 6 different A/B tests done in order to keep boosting the revenue production of this site. To the point that they exhausted their ideas and got to what they considered to be maximum productivity. What's significant, before I take you through the test, is that it is easier, cheaper, and faster to A/B test online than it is to A/B test offline. Now, those results are only going to be indicative. They're not going to be perfectly predictive. But they are a reasonable shortcut, at least to a better starting point offline, maybe to settling for these A/B tests in what you do offline, maybe getting to a control, an offline control that then you continue to test against, perhaps forevermore. But online, and I'm now I'm telling you things you know more about than I, but obviously you can rotate sites. You can drive traffic in rotation to 1, 2, 3, 4, however many versions of a site that you like at the same time. And if you drive enough traffic to be statistically significant, you can complete one or more A/B tests literally within hours, days, or weeks. A couple of years ago when we interviewed Tim Ferriss about, the choice of his title for his 4-Hour Workweek book. He explained that he had arrived at that title not based on his choice or preference or anybody else's opinion, but simply by testing a variety of different titles turned into small ads and connected to Google AdWords. That was relatively unusual when he described it. It's now standard operating procedure for the book publishing business, the movie business, the TV business. Pretty much the entire entertainment industry has glommed onto this. So back to this case study. So this is the— this whole case study, which you'll find in the magazine when you go get it, has to do with the— the simple fundraising website, which really consists of an informational page and then a form that people were being driven to by TV, radio, print, online, Bush and Clinton's interviews and appearances, and so forth. So they tested a one-column page with an image, and actually adding the photographic image to the page resulted in about 1% less money donated per page view. However, when they switched it to 2 columns, next A/B test, they reorganized the page into 2 columns with the photographic image, same photographic image that had reduced increased donations by 1% in a single-column format. In a double-column format, the same image, but now the signup form coming up horizontally next to the image and the copy rather than vertically below it, that increased donations by 8% per page view. When they increased the font size on the form, Um, they got a 2% bump in money donated. When they reduced the number of things you had to fill out— this will probably be no surprise to anybody— um, they increased donations by 11%. Uh, when they hid the security icon that tells you that your donations are safe and, um, your information is not shared, that sort of thing. When they hid the security icon to get it out of the way and make the site more, more, um, as they say, elegant, they reduced money donated by 5%. And now here's the biggie. The biggest hit of all was switching the words on the button from "submit" to "support Haiti," and that increased donations per page view by 16%. So overall, so first of all, that's a 2-word change for a 16% bump in donations. And overall, 1, 2, 3, 4 of the 6 A/B tests produced positive results ranging from a 2% bump to a 16% bump, and in total, a 10, 26, 36, 37% bump. And 2 of the tests were negative, resulting in 1% and 5% reduction in donations respectively, or a 6% reduction in donations.. And so these A/B tests yielded a substantial improvement in the productivity of this site. Now, there's a couple things to think about all that. One is many of you on the call now spend a lot of money or spend a lot of time and energy driving traffic to a site, however it is you drive it there. And if you already drive it there with online media or get in the business of driving it there with— excuse me, with offline media, with offline media, as for example clients in infomarketing of mine like Chris Cardell and Michael Kimball do, as clients like Guthrie Renker do for ProActive. If you're spending money offline to drive traffic online, just as if you are spending money on TV to drive callers into a phone room, AB testing that can make that destination site inbound call script, whatever, work more productively is especially important because getting this right may determine whether or not you can spend money offline to drive traffic or how many different ways you can spend money offline, how many different lists you can use, how big a portion of a list you can use. And so sophisticated direct marketers are in fact involved in this all the time with emphasis on the destination that their lead generation advertising drives people to and doing as many A/B tests as they possibly can. Now the migration out the back door of this is if we are going to do direct mail, We can run these A/B tests online faster, cheaper, easier, extract indicative if not perfectly predictive data from it, and use it to create or to modify our direct mail piece, then still run direct mail tests, but now it will be more against a control, so it will be A against a control, B against a control, C against a control, D against a control, with the control already working and working profitably, and be a much more effective and productive offline test. I've just described only the case history in this article. The entire article is important. To get your hands on and read, and I recommend it. Now, if we go way back in time, we go to the perennial sort of textbook of direct marketing. My edition is not young, but they— my edition that I have in front of me dates to 1986, but it keeps getting updated every few years. If you don't have it, It is a book called Successful Direct Marketing Methods by Bob Stone. You can get it from Amazon. I warn you ahead of time, it's not as much fun to read as the Wired— the Wired article is a little heavy lifting. This book, though, is real heavy lifting because it is in fact— it's not for college, it's for business people, but it is very textbooky and in places it is very detailed, but it is the foundational explanation of direct marketing. And so if you've never read it, you probably should. In my edition, pages 102 and 103 have to do with list segmentation, which is an important thing. And so I just want to give you a couple of Stone's points and the main idea of list segmentation internally. So looking at your own list, his key questions list is who are my customers, what do they buy from me, why do they respond to my mailings, how do I retain them and increase their purchases, where do I find others like them, and how do I sell to them more effectively? Now when you go to segment an internal list, so I'll give you— some stone items for that too. Mail order companies might segment by mode of payment—cash, installments, credit card, etc. So if you take mode of payment, something you may never have looked at is which of your customers always buy on installments, which of your customers always buy by prepay, which customers take 2 or 3 pay, which customers take extend-a-pay. Geography is kind of obvious. Type of product purchased by category. So at GKIC, of course, we have people that buy products on sales. We have people that buy products on copywriting. We have info marketing product buyers only. We have people that buy everything across category. Format. Some people may buy hard deliverables. Some people may only buy something they can come into the store or showroom and touch and feel. Others may buy by distance. Length of time they've been a customer. Source. How they sourced is important because it's the best way to sell them. We found years ago people that bought by being in a seminar room pretty much had to be put in a seminar room again or next best thing. Seminar on DVD, seminar on audio CD, or webinar. People that bought by TV infomercial, it's best if they see another infomercial, even if it is delivered off broadcast. People that buy from long-form sales letters, it's best if they see another long-form sales letter. Date of last transaction, so frequency, and frequency of purchase. Do you have buyers that buy every time you put something in front of them? Do you have buyers that only buy every 3 months no matter what you put in front of them? Do you have buyers that only buy at a certain time of year right after they get their tax refund? All of that is internal data that most people have or have the ability to collect, but most don't use. I was just— got a survey a couple days ago. Done by Columbia Business School of the CEOs of Fortune 1000 companies, and 36% of those CEOs say, quote, we have a lot of data about our customers, but we don't know how to use it. And so it's one thing to have data, it's another to use it. Now, if you hook these two things together that I've just talked about, and you think about A/B testing, as a means of driving data, because not only can an A/B test show you one thing works a lot better than the other, the hidden thing that an A/B test can show you is that some people may respond better to the thing that doesn't work as well. And so an A/B test of any variable that produces 60% of the people responding to A but 40% of the people responding to B, even if it's weighted more, 80/20. If in fact the 20% consistently respond to a B-type thing, then they have to be segmented out and they have to be marketed to, communicated to with that B-type thing, and the 80% who respond to the A-type thing have to be talked to with the A. So that might be, say for example, a pain pitch versus a gain pitch. So if 80%, if the pain pitch is more productive with 80% and 20% respond to the gain pitch, but if that 20% more than once, 2 or 3 times demonstrated that they are interested in gain and benefit, So if you take weight loss, if 80% respond to quick weight loss against impending event because there's a wedding coming up, a class reunion coming up, etc., but 20% respond to weight loss because you make a health argument, a longevity argument, a quality of life argument, on the front end you may need to scrap the B and run only with the A.. But within your own customer base, you could have them segmented and you could consistently use both the A and the B. So that's how segmentation and split testing connects and affects both backend marketing to customers and marketing to accumulated unconverted leads. And so this is some fairly sophisticated thinking that is not precluded from fairly small businesses. And in some respects it's easier to do hands-on with a small business and a small list than it is with a big list. But it also integrates online and offline. So there's a little old and new look at media. With that said, we'll start taking calls. Dan, we have our first question, and that's coming from Faj.

Please go ahead. The question I have for you is in regards to something you said earlier when you were talking about the Clinton-Bush Haiti Fund. Site where they changed the word submit to, I think you said support the fund or something.

Support Haiti actually is what it said, yeah.

Thank you. Support Haiti and it was like a 16% return or something. And my question to you is regarding the language in an opt-in. Is it, and I'm going to give you choices because I don't know how else to frame the question, is it better to have the opt-in say claim or receive or instant access gift or item.

Um, well, so here's— so the real answer to this question, yeah, and it hasn't changed, it's just that online's made it easier, less painful to hear the answer than it used to be. So the real answer to this question is none of us know because it's situational and there's no way to tell if we drive certain prospects to a certain place and put a certain set of choices in front of them, uh, what they're going to respond to best unless we actually test. And so my opinion actually counts not much more than yours because ultimately We're both guessing. Now, I may be drawing on a greater range of experience than you are in guessing, but I'm still guessing. And when we don't have to guess, it's better not to guess. So, and that is exactly the point of the A/B test article I referred to in Wired, is so you enumerated 4 or 5 language options. And the best way to figure out which one works best is to drive the same amount of comparable traffic to each one of them, making that the only difference, and see if any one of them significantly outperforms the other. And this is done in— so in an inbound phone room, where let's say we're running an infomercial and the calls are being driven into a phone room to order. By the way, it surprises a lot of people that in an environment like that, more than half of the people don't order. They called to order and they get unsold. But when that's going on and we want to— we have 5 different ways we could we could say a sentence just like you just described 5 different ways. The correct thing for us to do is to take a week's worth of inbound calls, divide them in 5, send them to 5 different operators who use 5 different sentences. And then measure the results at the end of the week. Now, that's not as clean and perfect as doing it online because we have 5 humans involved, and one of them may have a head cold on Tuesday, and one of them may be in a bad mood on Friday, and all that can skew our test. So we have to drive a lot of data to know anything, whereas online you may be able to drive a small amount of data and know something. Often you'll at least know what the dead bang loser is or what the two worst are. So the real answer to a question like you pose is test. And when you systematically, depending upon the numbers and the value, go through that test, and then the headline test, and do you call it a coupon or a certificate or a voucher? So you do those tests. That's how you get something all the way to optimum performance. So I can have an opinion, and my opinion may even be based on having seen 5 things very similar to what you're doing, but the key word is similar, not identical.. And so there's a great charity case. So the norm in fundraising direct mail is to put either 3 or 5 dollar choices in front of you as the suggested donation. So $10, $15, $25, $50, $75. The overwhelming majority of the people are going to pick one of the two middle numbers. And if you get a lot of charity fundraising stuff, 80% of it's going to look like what I just described to you. So there's a classic case, I think it's in Bob Stone's book if I'm not mistaken. There's a classic case of a fundraising campaign for a charity to very affluent individuals. That was averaging about $8,000 per donor with an approach I just described. They tested against that an approach where each donor's solicitation was personalized, and based on their donation history, a specific amount was picked for them. Just a slight stretch above what they might usually give, and the whole form was filled in ahead of time. And to give a different amount, they had to cross it out and write in a different number. And 92% of the people gave the amount of money that was asked of them, and it raised, uh, the average donation from $8,000 to $22,000. Now, the only way on earth you would ever get to that would be split testing. Nobody's opinion would ever tell anybody to do that, except the guy who thought it was a good idea. And so the real, you know, I said years ago you could give every, you could reduce every direct marketing seminar down to a 3x5 card with the word test written on it in big letters. And that's, and that's nearly the truth. So your right to identify— here's 4 ways we could do this, here's 4 ways we could say this. Now you've got to figure out a way to legitimately test the 4. You know, I'll give you another— you know, it's very similar to like last year, I rewrote a series of radio spots for a client, got a significant bump. And I didn't change but hardly a word in any of the spots. I just switched it from being said in third person to being said in first person. And I didn't know I was right, but it was the first logical thing to A/B test. Now beyond that, the A/B test is male voice versus female voice, with music in the background, no music in the background. And when there's enough at stake, you work your way through one of these tests after another.

Okay. Appreciate it. And then I'm just— with also with the, um, the whether it's an educational, uh, report or whether it's a gift or whether it's a move forward type of an instrument, and the same, um, apply there, or is there one that you believe is better than others to move people, keep people in there in the herd, or—

well, initially, essentially, when you're using a giveaway freemium, you have relevant and irrelevant as your choices. So, and irrelevant is, you know, like the Taco Bell stuffed dog. It's like buy a sandwich, get a baseball cap. And so you see that all the time, even in information industries. The ad agent, the Advertising Age magazine control, for new subscribers, which is a trade journal for people in the advertising industry. The bonus that works best is not a series of special reports about how to do better advertising. It's not an expert's book. It's not— it's a coffee mug with a replica of the Advertising Age cover on the front of it. And so various and sundry things have been tested against that. Conversely, for Omaha Steaks, which of course sells food, they've never beat, and they've tried everything, cookbooks, 2-for-1s, you know, a collection of barbecue sauce with your first order. The freemium that works best for them is an oversized desktop calculator. And by the way, that makes no sense. It's completely irrelevant, it has nothing to do, It works for them. So there's enough of these odd instances that tell us that really none of us know what the hell we're doing. And we have to figure it out by testing. Now having said that, the informational bonus when the retention you're looking for is of an information consumer is preferred even if it doesn't work quite as well as some other bonuses because it furthers your case. It is another opportunity to get people interested in your content. It's another opportunity to position yourself as an expert. But there's no empirical evidence to support a universal conclusion that says with this type of offer, this type of premium is best. The biggest mistake people make in tracking response is, I mean, the A/B test is often testing a relatively simple and front-end item. So if we drove them to a website and one had a blue background and one had a yellow background, And the conversion rate was higher with the yellow than with the blue. Okay, we're kind of— maybe we're done with that. But the biggest total data tracking mistake people make is only tracking to the first step rather than to the total customer value over some reasonable period of time. And in big numbers, you can really get yourself in trouble this way. Because the most productive measured against cost per lead or cost per sale is the cheapest. However, if Source A gives us a customer for $50 and Source B gives us a customer for $100, but Source B's customer subsequently spends $1,000 and Source A subsequently spends only $200, that wasn't the productivity bargain it appeared to be. And so most people stop their tracking by source or by variable, typically with first transaction or at best with second transaction. And they never bother to maintain that data cumulatively of those customers over 3 months, 6 months, 12 months, 24 months, you know, etc. And when you go into a mature business that has that data but hasn't used it and you dig, you very often find that they have abandoned what appeared to them to be overly expensive means of acquiring leads and customers in favor of ways that appeared to be more cost-efficient. When in reality what appeared to be overly expensive early turned out to be the bargain over time.

Thank you for listening to the Magnetic Marketing Podcast with Dan Kennedy. If you love hearing in on these lost Dan Kennedy talks and speeches and calls, then please let someone else know about this podcast. That's how you can help it to grow, and the more it grows, the more free Dan Kennedy we can bring to you. Also, Dan would love would love to give you the most incredible free gift ever designed to help you make maximum money in minimum time. This free gift comes with almost $20,000 in pure money-making information for free just for saying maybe. You can get this gift from Dan right now at NoBSLetter.com. Not only do you get the $20,000 gift, you also get a subscription to 2 marketing newsletters that will be hand-delivered by the mailman to your mailbox each and every month, one from Dan Kennedy and one from me, Russell Brunson. To get this gift and your subscription, go to NoBSLetter.com right now.