Web Analytics: An Hour a Day | Avinash Kaushik | Talks at Google

So without further ado, I'd like to introduce Avinash, author of Web Analytics, An Hour a Day. And thank you so much for coming out. I'll turn it over to him. Hi, good afternoon. My ipod. I'm actually now analytics evangelist and also a roving consultant. So I spend too much time in planes and on airports, more than I would like. And my ipod is full of talks that I download from authors at Google Website or the TED Talks. If you haven't tried the TED Talks, they're very, very fantastic. And I like both of these because both of these organizations invite some amazing people to come talk. And so for me, it's an honor to be here and to present to all of you as a part of the author series. So thank you for having me. That's my book. It's one of the early covers and I read it yesterday and I noticed that a bunch of things had actually changed from what you'll see. But I'm going to talk to you a little bit about the book today, and I am going to talk about the web analytics industry and why it finds itself in a bit of a pickle, at least in the last few years and what I think is a new solution or a new way of thinking about analytics around the web. So very quickly, I'll give you a quick back story on the book. The book actually started from the blog. This is my blog, Occam's Riser, and it is about 15 months old. And it has been an amazing experience for me to become a blogger. Somebody had to convince and threaten me. My friend Andy Beale is responsible for me becoming a blogger and getting into this space. And it's amazing how it has sort of transformed a lot of different things. I do. I just checked my stats yesterday. Obviously I am the analytics person. And the blog has about just shy of 5,000 feed subscribers. And about 39,000 people read the blog last month. And it boggles my mind because the target audience for this blog is only 1000 people in the world. So I'm pretty amazed at how many people visit and read it. And, and it started approximately 15 months ago. And before I started the blog, I did a lot of research. I participated in other blogs. I read through many different things and I wanted to come out with a proper voice and make sure I'm doing this right. And perhaps one of the most influential things to me was actually Guy Kawasaki. And a lot of people might know who Guy Kawasaki, he's a venture capitalist. He runs Garage.com, and he used to be the apple evangelist many, many years ago. And in this particular post, he wrote two things that form the core essence of what my blog is all about. And the two things that he talked about is, the first one is eat like a bird and poop like an elephant. It really had a big impact on me because his core thesis was, you learn so much from the world everybody shares, and you are a participant in this wonderful system, and you learn so much, and it's your job as a blogger to go back and share way more than what you learned. And I was working at Intuit at the time, and I said, you know what, it's kind of true. I go to conferences and speeches and I've learned so much from everyone, and this would be my way of sharing with people what little I know. And the second thing that he talked about in this particular post he did in April was think book, not diary. And that also resonated with me quite a bit. And I think if I synthesize what he was trying to say, nobody wants to know your cat got stuck up the tree last night or the girls that turned you down yesterday. Nobody gives a shit. People want to learn something, people want to know something. And his conceptualization of this was, have a start, have an end. At the end, somebody should have learned from everything that was in the middle. And that really resonated with me. So everything that I do on the blog is with these two principles in mind. It has to pass this bar. If it doesn't, it doesn't go on the blog. So it was actually very, very. In spite of that, it was very surprising that six months into blogging, in about October of last year, somebody from Wiley sent me an email and said, we want to make this into a book. I have only been doing it for six months. And my expression could adequately be described by this, because I said, why would people ever want to pay for something that's actually free? Because they had said, simply copy paste all your posts. And that's a book that sounds a bit insane. And the other thing is, I thought, you know, smart people write books. And I never thought that I fell into that category. But after a while I said no. And they kept coming back. And that was very sweet of them. And I decided that I'll actually write the book a few weeks later. And the reason was I thought, I love my blog. It's like my third child. I have two actual children, and I love what I do. And I didn't want to. There was something sort of impure about Monetizing it. So I said, okay, I'm going to write the book. And in turn, what I will do is I will take all of the money that I would make from the book and donate it to charity. So all of the money that I actually make from the book goes to these two charities, the Smile Trains and Doctors Without Borders. Every single cent. And it has been an amazing way for me to raise money, way more money than I would ever have anticipated. The book came out in June and my goal was by December I should have raised $10,000. First time authors make a pathetic amount of money, in case you were not aware of that fact. 10 grand. I'll be thrilled, happy beyond my wildest dreams. Because after all, this is not Harry potter. It does 10 grand in two seconds. Actually, surprisingly, in the first two months that the book has been out, it already raised $10,000. That's just my share of what the book made. Wiley obviously made 15 times that. Happy birthday to them. I'm very, very proud that actually the book raises money for charities more than actually I ever could have done. And the other thing is, a lot of people imagine what does it take to write a book. And it was not as easy as copy pasting it. There's a lot of work that goes into writing a book and I had not imagined, actually I had underestimated by a million percent how much it takes to write a book. Most of my book actually was written in five weekends of Red Bull. I would go to work in the morning just down the street, and I would go to work on Saturday morning at 7 or 8, and I would stay at it until like 9 o' clock in the night, just in my office, close the door, Red Bull computer, and very little eating. So I did feel dizzy by the end of the day. But Red Bull is responsible for the book being finished in five weeks. In the middle of January of 2007. I sent the book out to the publisher and I said, done. And the interesting thing is that is only the start of your pain, because it turns out they call this thing a manuscript. There's a lot more work that needs to get done now. So I went through a copy edit phase, a page edit phase, a line edit phase, and in the last phase, somebody reads every Alphabet that you've written and crosses it with red ink. It's pure torture. So lots more nights, plane flights, staying awake, and in the end, roughly about March, you have a buck. And it is a process that I think people should not underestimate. And I thought, Jesus, I'm never going to write another book. It was way more pain than I thought it would be. And actually I suspect I probably would have given up on it had it not been that I had already decided that I'm going to write this for charity. It's just a great motivation factor for me to write a book. A book that's really good. So they will raise a lot more money. And my contract with Biley was to write 240 pages and the book is actually 480 pages. So I ended up writing a lot more because I wanted to write the whole story. So the nice thing is that the book is doing pretty well. On my blog I have this thing where if you have a book, take a picture of the book with you or anything you like and send it to me and I'll send you a bookmark. And I have pictures from people everywhere in the world who take some amazing shots. That's Sweden, Canada, Holland America, Australia. All over the world people read the book and they take pictures and send it to me. And it is just amazing. I'm just amazed by all these people and their creativity in how they take the pictures. I have a picture from Singapore of a guy with his pants down sitting on a toilet reading the book and I could not include it in the montage. So people are amazing. I'm stunned at how far it has reached. But the other nice thing, I think the one nice thing about going with a publisher for a book is they do some work for you. Not as much as you wish. And so the book is now being translated into Portuguese, Korean, Russian and Chinese. And this sounds very world domination ish. But beyond world domination, one of the nice side effect of doing something like this is that in Chinese the book is being published by Tsinghua University, which is one of the most prestigious universities in China. And so my wife who is from Hong Kong called her mom and said mom, his book is going to be published by Tsinghua University. And she's like wow, he must be something. So after so many years of marriage, I have affection of my mother in law and just that was worth writing the book. So let me give you a quick background on sort of where we are and how we've evolved when it comes to making decisions on the web. And. And the wonderful thing is that web analytics is not as interesting as Sax. That would be a more interesting talk. But it is still a lot of fun. There's a lot of really wonderful things going on with web analytics. So one nice person and his friend invented the web and that was cool. And the first time they plugged in a web server and put a cable behind it, it turns out that it produces data. There were few channels in the world that produce data so automatically as the web. This wire was amazingly powerful and it produces data. Somebody will figure out how to produce a graph in ASCII characters. This is 1995 AnalogX, one of the first log file parsing tools that came out. It's probably one of the most used log file parsing tools today. Over time it got pretty. But it still essentially reported data for it. People essentially parse, logged in, provided data around IP addresses, locations and stuff like that. As marketers began to get hot on the web, they couldn't find a lot of use of it. And so we went on this thing that I call the quest for KPIs. God's answer to every problem web analytics could ever have is find a key performance indicator. And the core essence of the KPI mindset is the problem with us finding insights in our data is we simply have not figured out how to add, subtract, multiply and divide it enough. If only we figure out how to do math on this data better, we will actually get insights. And this continued for quite a few years. It's very much driven between the mix of having webmasters think about data and some sort, and essentially webmaster still putting on a business hat say, hey, we have all this data, how do we make sense of it? And so the result was something like this still really, really hard to understand. And there's a lot of struggle to say, hey, what are the insights here? Data Prettier. Prettier than the ASCII graphs. But is this all there is to it? And so at the end of all this compute power, you use all this compute power and brain power to compute metrics and KPIs and run reports, and you still have this. A very, very unhappy boss. He is unhappy, isn't he? And it's because at the end of all of this, you haven't given him enough data around what decisions they could made around campaigns, how to acquire better people, how to create better experiences, and how to improve that stupid conversion rate from 1 to 2%. You've only been playing with data yourself. I think web analytics is in this soup even now for lots of different companies because we've always been playing with data with an IT mindset. We've been playing with data with simply data data, Numbers, numbers, math, math mindset. True business people have actually not attacked the problem of trying to Find insights. And over the span of time that web analytics has grown up, the use of web by real people and marketers has grown up dramatically. And that clash has led to the problem we have today. Because this KPI driven mindset, applying math to lots of data, ignores fundamental structural business problems that exist on your website. For example, we still analyze our data as if our audience on the website was a monolith. The actual definition for conversion rate is orders divided by unique visitors. All of them. Everybody is not on your website to buy. They're not all the same. Why would you do it? It's just one problem of taking data in aggregate and assuming it's all for the same purpose and trying to find an insight from dilutes your efforts significantly. The other problem with our analysis is we probably live in one of the most frictionless environments in the world to do business in, and yet we behave as if we're alone. If only I have all the data from all my website and all the clicks, I could figure out exactly what is wrong with me. Or hopefully there's something wrong with you, the customer that you're not buying. How stupid of you. The interesting thing is you're not alone. The web Web. You're a part of a food chain. Somebody's out to eat you. It really is amazing. Any dog can put up the website, call it a search engine and compete with Google. Anyone can compete with anyone else. And yet we do all of our analysis as if our world looks like this, as if we're alone. And. And why isn't the fish stuffing itself into my mouth? There is this problem because most web analytics approaches simply look at their own data. The other problem is that if you think about it, at the end of the day, all that you have through your web analytics tools are clicks. Simply clicks. Click, click, click, click, click, click, click, click. If we have time, I will show you this. At the end of my presentation, I have three screenshots. I was looking for a picture of Food Chain to illustrate this core point. And I put Food Chain into Google and I saw a paid ad for ask.com and it said, click here for Food Chain. I'm like, whoa, Google Ask. What's going on? So I clicked on the Ask ad and I went to ask.com and it ran a query for me for Food Chain and showed me pictures. One of them was the picture that I wanted. I clicked on it, it gave me a 404 error from Ask and it's very cute. At the end of the 404 error it said, Try searching on a search engine like Google Circle of Life. I gave you that example because from these clicks, the clicks that I just did, how does Ask know that it had this stupid problem? Probably an anomaly. It's a great search engine, right? It just clicks. We are missing our intent. Especially in a complex ecosystem that the web is today, intent is critical. You really need to understand this. And you only have clicks. Some people hate you, some people are paranoid by you, some people want to tease you. I like that one. None of your clicks actually help you understand this. They don't understand you can't understand intent. You only have the what. But probably one of the biggest problems I find in making headway with web analytics and finding insights and actions is that hippos rule the web. So you take all this massive analysis and thought that you've put into your data and you have complex understanding and you go in front of somebody like Paul, he's a hippo and Paul's the highest paced person in the room. And regardless of what my actual analysis is, Paul's going to decide what's going to happen. He's not going to listen to the data. I'm kidding. Paul is joking. But in a lot of companies, big and small, hippos decide what happens on the web, no matter how much customer insight you take. So how do you fight this hippo? Right? How do you argue with this hippo? If he decides what's going to go on the website, you can bet your last dollar the conversion rate is going to stay stuck at where it was because he is clueless about customers. Because of these four complex problems, these four complex structural business problems, Web analytics has sucked a lot. Notice none of these problems is because you don't have enough data. None of these problems are because you haven't multiplied it enough or divided it enough. Is that we haven't thought about these problems from a business perspective. We haven't had this marketing overlay on the problem of making decisions on the web. And that's the reason that we're in this state that we are, let's say now. And it's because we only slide with words. We only have clicks. We measure them in aggregate. We're solving for hippos. No suit for you. I'm a huge Seinfeld fan. So how do you actually solve this problem? I have this simple solution. And the solution is this is Web Analytics 1.0. This massive circle is the glob of data that you have today. And this little zit is the amount of insight that you can get from it. Today, if you try really hard. You figure out how to solve the four problems they've identified a few minutes ago. And you create this. A new way of thinking about your data. A new way of thinking about new sources of your data. A new way of figuring out how you can get the complete picture. And this is Web Analytics 2.0 clickstream. The largest amount of data is still there. It's golden. It's beautiful. It gets complemented with analyzing multiple outcomes, doing testing and experimentation, bringing the customer's voice to the table, doing competitive intelligence. So you figure out that somebody's going to chomp on you in a few minutes. And at the end of all this work are golden insights. Things that will make a difference to your business, things you can action. I call it the what, the how much, the why, the what else. So far, most web analytics efforts have solved simply for the what. If you want to have massive success with your websites and create experiences that resonate with your customers and you have to figure out how to understand and solve the how much, the what and the what else. And I'll walk you through an example of each of these. So normal clique stream analysis has gotten a lot more prettier than the graphs you've seen before. This is Google Analytics, but you could use other tools. It's actually gotten more pretty. We've gotten to a stage where we can actually understand our data better. But even if you're like, okay, this is nice big chunk of text, I understand what's going on. You can use it much more efficiently. You don't have to look at your metrics in silos and simply drive your decision making based on page views. Sure, page views are nice, but are they solving for my core metric? Engaging customers with my site simply as defined by spending more time on my website? End of the day, notice it's different. Suddenly the most popular page is not the one that's probably doing as well as you should, but it gets better. You press another button and you actually get a view that you can use to make good decisions. In this view, exact same metric. Suck good, suck suck good, good, suck good, good, good. Is it much easier for you to understand the data and start to focus your attention? Nice progress with clickstream and how anybody can use it. But we still had this problem of having everything in averages. Average time on site, average pay views per visitor, average visits per visitor, all averages. So if you look at that metric, it's two visits per visitor, half a million, a million one by the Other averages lie. This is actually the real data for that website. The average page views are not two. This is it. Most people leave you instantly, 40%. Boom. Understanding distribution is a great way for you to make much better decisions with your clickstream data. So if you see averages, ask for the distribution. But I think one of the most interesting evolutions that has happened in pure clickstream tools is the ability to segment. Remember, monoliths don't come to your website. There's no such thing as an aggregated average visitor. Everybody is unique. You're all pretty. You want to figure out how to separate Rachel from Mark to Justin. And now you have lots of tools that allow you to segment your data a lot more efficiently. It is a crime against humanity for you to only have one row of data for any metric on any dashboard report. Crime against humanity because it doesn't give you any context. You want to segment your data so you understand a lot better what the data is trying to tell you. Find the pockets of people, find the micro segments so you can understand behavior better. And in turn figure out what action you should take. The interesting thing, the next one is how much. And it's surprising. I was at a company and they had had a web analytics tool for five years. And I went around, I was the new guy. I knew nothing about web analytics. It's kind of sad. It was just a couple years ago. After the two weeks I turned it off. I simply turned it off. He purchased 200 repositories. Turn it up. No one called me. Not a single human being in a two billion dollar company called me after turning off 200 reports. And the reason is because they were simply producing data and not tying to the outcomes of the company. People only care about outcomes. People are selfish. Everybody wants a pay raise. Everybody wants to improve customer experience. If you cannot produce data that ties to outcomes, you're wasting your time. So people are getting smart to this. So now you can go into a tool and create outcomes. What are my goals? How am I doing in conversion? I like to say if you're using a web analytics tool and you've not defined the two outcomes you want from your website for yourself or your customers, it's like going to play the super bowl naked. You are going to get hurt on top of it. You will be embarrassed. Have goals. If you want to go into the stadium and play the game, suit up. But it's getting better, right? We're getting better at measuring outcomes. Not only do we want to buy keywords that simply send us more traffic we want to figure out through our campaigns how much should we bid for each position so we can optimize for Outcomes. Most amount of visits to your campaign will probably come from T1, but you don't have to get into ego bidding. You can say, hey, my goal is to actually improve conversion, improve time on site, improve page views per visitor, whatever you want. So as you think about outcomes, know that there are tools that allow you to do much more sophisticated analysis well beyond the aggregated numbers that you see on the top. And seek that out because then you will figure out how to save money and increase roi. But the reason that Outcomes is called out in my little diagram is because I encourage people to think about outcomes that are beyond revenue and clickstream. Even if you create a hardcore e commerce website, you can bet that people are coming there for many different purposes. So if Walmart has a website, the outcome for Walmart's website is that you can measure it with a clickstream tool. This is done using a survey, a statistically relevant survey that helps you understand brand impact, that helps you understand likelihood to visit a store. As you think about your website, remember, a website is not a monolith that's used by one type of people. It does not exist only for one reason. Your job is to figure out what are all of the reasons that it exists for and find the best source to measure it. This will get you any amount of dollars to spend you want from your boss. Another example, lots of website exists simply for branding purposes, for marketing purposes, or simply to improve customer satisfaction. JetBlue holds up a plane in New York for 12 hours, won't let people go to the toilet. Then they end up creating a website that is trying to talk to these people and they want to measure if satisfaction is improved. That is an outcome, right? So when you think about measuring how much of your website, think beyond clickstream, think beyond revenue, think of other things you can measure because you can. If you do multiple outcomes analysis, you will be able to find much better actionable insights for your bosses and solve for your customers. Now that's just a beautiful thing. So the next element is the why. Remember our beautiful hippo? This one is a beautiful one. Because hippos can be creative directors. Black turtlenecks, torn jeans. They won't let you do something because it gains their brand perception. So how do you solve for them? How do you get the power to try and experiment on your websites? My recommendation to you is how do you figure out how to be learned, how to learn to be wrong quickly that's your job. And one way you could do it is you could do experimentation and testing. I'm a huge fan of testing and experimentation because experimentation and testing empowers idea democracy. Rather than a hippo crushing all the ideas that you have, you actually take all of those ideas, put up a test and see which one works. Your boss has just as much chance of guessing the right thing for the customer as the janitor. And your job as a person who owns the web experience or the analysis is to figure out how do you create the democracy and testing experience does. This is a multivariate test. This small company in Mountain View. And there's ways to improve this page. All of the things you could try in these boxes. Now it's so simple. You don't actually create different pages. You simply log into the Google Optimizer tool that was used for this particular test, you upload all your brilliant ideas and you let the experiment run. Rather than your boss at the janitor deciding what happens, the customers will tell you what happens. And at the end of the test, it automatically tells you that page better than that. Isn't that cool? You simply throw all the chemicals and wait for them to explode. Now the nice thing about experimentation and testing is not that you get this page at the end. It's the learning you get. So it might seem fairly obvious that find, organize and share your photos is probably not as good as easy way to share and manage your photos. It's kind of eh. But what is brilliant is the fact that the button that says Try now a button works better than free. Download link. I am a huge fan of text links. I hate buttons. Can you imagine people like try more than free? That seems dumbass, but they do. And the thing that got me to download Picasa is this picture. Because very quickly I could see all I do in Photoshop Elements or whatever I was using is I crop, brighten, straighten, auto contrast and that's it. Like, hey, this thing can do this. It's from Google. It must be good. Probably I'm just going to download it. And yet it was proven not to have a positive impact on getting people to download. Surprising. See, my own opinion was wrong. And this page works better. The nice thing about testing is you learn things you never imagined about your customers. There's no way a brainstorming in a conference room in Mountain View is going to figure this out. Let the customers tell you that page up there had a 30% improvement in conversion rate. 3, 0. Trust me, I've done this long enough. That you would die for 2% improvement. 30 is like phenomenal. The nice thing about testing is anybody can do it. So there is a town in Indiana called Popcorn. I was not aware of this, but in Popcorn Indiana there is a company called Dale and Thomas Popcorn marketing genius. They can do multivariate testing. They can pit Oprah against Judaisa, who can sell more popcorn. At the same time they can figure out if text promotions work better than featured picture promotions. At the end of all this test, they learned that the thing that probably had the lowest chance of success, a fat six pound tub of popcorn is the winner. Now you would think if people are buying popcorn on the Internet, you probably want to try a sampler first. What if this sucks? That's the winner. It's really hard. See marketing at its most genius. Take your ideas, create a idea democracy. This is web analytics. And the last thing is remember all these people. The thing is the why is how do you get their voice to the table? How do you run up to Jabjeet and force him to listen to customers? You have to have a scalable model. He doesn't get enough chance to listen to customers. And I'm a big, big, big. Throughout the book you'll see this deep passion for figuring out your customers and solving for them. And one of the things I like to do is a simple three question survey. I'm a big fan of doing surveys in a very strategic way. Surveys have evolved well beyond anybody had imagined. They're very complex, they're very sophisticated. They use cookies, they do tracking, they merge with web analytics. It's amazing what you can do with surveys. And yet my favorite is a three question survey. And the first question asks the customers, why are you here? And that's always a shocker because it turns out that a website that was created essentially for these two reasons has people coming to the site to solve all all of these other reasons. There is nothing on the website that helps people solve this in an efficient way. The second question after why are you here? Is were you able to complete your task? Not conversion rate. That one sucks. Task completion rate. This is beautiful because now you can figure out that 44% of the people who are looking to buy could not. No wonder your conversion rate is 0.1%. Right? You can figure all of these things out too. But the nice thing is always applying a little bit of math magic. And you do distribution. And what is surprising is on this hardcore EE Commerce website, 37% of the people are there for the reason why the site exists. Is it like a shocker? Here's the other one. These were guesses in the survey drop down. Probably you could come for these reasons. For 20% of the site traffic could not even guess why they are there. This is web analytics trying to understand what people are there for, because they're not there for the reasons you think people should come for. And now you have this data, you have a chance to decide are you going to solve for these people or puke on them. It's okay for you to make either one of those decisions, make a conscious choice. And that's my recommendation. But the question that is more interesting is the third one. If you were not able to complete your task today, why not? And when you it's open text, people puke on you, give you tons of data, beautiful data. And in this data are hidden golden nuggets that you want. You're looking for the you idiots and I love that and how stupid or what their mommy likes. And if only you could do that. Customers telling you what you should be doing. Imagine if you could solve for how and what your website does by using customers voices. I call this the segments of discontent strategy. Your job is to find segments of discontent and prioritize those to figure out what should our site be doing. The last one recommendation is to do competitive intelligence. In this article it is also referred as industrial espionage. I'm not sure I'm a fan of that, but I'm a big fan of doing competitive intelligence. The beauty of the web is it throws data everywhere and without you knowing, your ISP is selling it to other people. That's the beauty of it. So you can actually go in now and start with something primitive. So Fireclick publishes an index of the most key metrics. So you can say, hey, not only am I going to think not being a silo like that polar bear, I want to know in the ecosystem, how is my performance compared to other people? Sweet. If you were doing 3% instead of 1.9, you deserve a bonus. And now you can ask for it. Conversion rates, cart abandonment, time on site, keywords, emails, affiliates, all this data waiting for you to do competitive benchmarking. Give context to your data so you can figure out that in the last three months you have improved your performance by 50% and everybody else has improved by 200%. Unhappy outcome. But you can do better. You can go into compete.com, i'm becoming a fan of this company. And you can say I type in Dell, HP and Lenovo. I love my ThinkPad. Say hey, how are traffic trends doing for these websites, me and my competitors? Well, if I am a Lenovo, then, well, I'm not even on the radar. This kind of sucks. The nice thing is I'm improving a lot better right there. But you can do more intelligent things. See, the Alexa days are over. Alexa is like gone over. You have much more intelligent choices because you can go in and then say, what about time on site? Notice HP is the line that's growing much more radically in this graph. And if you look at time on site, it turns out HP is acquiring crappy traffic faster. And notice that one of the biggest growths, nice, beautiful growth is actually Bilnovo. You can get this kind of context about your performance from this free tool. You can get even more complicated things. I'm not going to try to explain Velocity, but this actually helps you understand when your competitors are running campaigns and how they are doing versus you. Sweet. If you have money, you can get hitwise. Very, very nice tool. Also you can use it to figure out who is sending traffic to you and your competitors, Verizon and singular. You can say, hey, where am I doing better than Verizon and where am I doing worse than Verizon? Not only that, you can say, hey, I'm Verizon. I want to know who is sending traffic to my biggest competitor. Maybe I want to poach these people. What a beautiful strategy. Steal from your competitors. You're not in a silo. You can figure out keywords. Google is all about keywords. You can go figure out what keywords are driving traffic to your competitor and what keywords are driving traffic to you and you can see how well they're doing and how badly you're doing. So if I'm Verizon, this initial graph would worry me because a lot of non branded category terms are showing up here for singular and not over there. It's really hard for you to get traffic on category key terms. Something interesting going on. Maybe I should learn more. Or you could do something even simpler. Say, hey, I want to know who is playing in the game called cell phone. Keyword cell phone. Pretty competitive. You can find who you're fighting against. T Mobile could be experiencing 3,000% growth month over month in their web analytics tool. In the competitive landscape, they are insignificant. You want to know that it's good piece of information to have because you could figure out what your strategy should be. So using a free tool, using a paid tool, there's so much golden insights that you can get from competitive intelligence data. Try it There you go. Web Analytics 2.05 Interesting different ways of thinking about data on your web and using qualitative and quantitative data to make holistic decisions that solve for your customers and in the end that will solve for you. So your question is, is this really hard? This is a lot of work and I do have a motivational quote for you from my favorite Mr. Kawasaki. I will let you read it. Life is tough. I think one of the things traps that we as an industry have fallen in is that we have this false sense of security that we have so much data and all we need is a brilliant, genius million dollar tool that will automatically take care of everything. And that's just a bunch of crap. It is hard work to take data and find insights that will impact your bottom line. Make no m. I would even I'm a nice guy. I will prioritize it for you if you want. My recommendation is always start with collecting voice of the customer. Why are you here? What task were you not able to complete? Why not? It's just amazing how much insight you can get. Then figure out what your outcome should be from your website. And by now you probably have already implemented a clickstream tool. But focus on that third because is this too much data for you to think about? Then come back and think about testing, then think about competitive intelligence. It's definitely an evolutionary process. You won't start this with a Bing bag tomorrow morning. And the other nice thing is this expensive. One of the most beautiful thing on the web is everything is free. Just kidding. But if you want you can actually start pretty much for free. So a lot of screenshots that I've shown you in my presentation today are actually from free tools. So if you want to do clickstream analysis you could simply use Google Analytics. If for some reason you're averse to it, there are other solutions in the market you could use. Microsoft is going to come out with a solution in a couple months. But you do have a choice of free tools. You can use Google's analytics or other tools to analyze your outcomes. Simple surveys, type things. You could use the optimizer to do multivariate testing. You could do surveymonkey free. Well it's 19 bucks a month, so. And you could use Compete Fireclick. Those are all free. So your excuse for not having this mindset is probably that you don't want to have that mindset. The hurdle is not the tool. The hurdle is not your ability to implement them. The hurdle is for you to accept and internalize this mindset. And create an execution strategy for your company. That's the biggest hurdle. That's it. I'll be happy to answer any questions you have. Yes, Mr. Crosby, hi now. Hi, Avinash. Thanks for the talk. Always inspirational. Thank you. So for the voice of customer in the survey. Yeah. I'm wondering if you could use internal site search to get a pretty similar feel for what your customers are looking for and that data and compare it to some outcomes to see. Basically it's a survey without popping up. A survey which from my experience Google tends to be fairly averse to. Right. I think figuring out and analyzing internal site search is very important. I have like a whole chapter in my book about internal site search and how much it can tell you about what customer intent is, what things people are looking for, what's not working for them. So it's definitely something you should be doing. It's not quite the crime against humanity category. It's definitely. You're leading a lot of insights by not using data from your internal site search. I'm not sure that it replaces voice of the customer in a sense that you really want to know all the fus about things you're not doing right on your site. That drives change, like anything. I could hammer my executives with all my data. I just give them a few excerpts about the fus and I get the money. I think surveys and voice a customer tends to drive change in such a unique way. And it's also not your opinion of it, which really is amazing psychologically and culturally how it tends to drive change. So definitely use site search. It's a great way to understand customer intent. People are telling you what they're looking for. It's easy for you to look through those keywords and say, hey, does my site actually solve for this? They want to register. Biggest search, key term register. Not a single page on your website about registration. Okay, there's a problem. This is definitely an incremental improvement. Yes. I really like the idea of looking at the open ended responses from surveys, but I know that's a big problem when you reach a certain scale. Yes. So what tools do you recommend to analyze those data automatically? Or is there software that you recommend that can give you some insights or some trends at least within those data? So my personal experience, I've worked with Clear Bridge and a bunch of other open source and proprietary algorithms that tend to promise parsing through open text, voice of the customer and find insights and clusters and the experiences, they're actually not very good. It's really, really hard to get into Brett Crosby's head and what he's saying. It's really actually very hard. So at the moment it's really hard. So in my experience, I flipped the model. My implementation experience is I'm actually not going to parse everything. I don't care. So for example, I would say I want everybody who came to my website looking to buy and then of those, Everybody who rated me 3 or lower, or pick your number, let me just go read them. Or there are tools now that will allow you with statistical significance to find these pockets of discontent. So apply the math first to find the population you want to solve for. So let me find everybody who will help me improve my product category page. Actually, you can apply math to figure out, oh, everybody who rated less than this was here for this reason, gave this score, boom, read their answer. So I found that that actually helps. And this really, for now, isn't a really solid replacement for a human being reading stuff. Also, the other thing that is missing from open text analysis tools is it's missing a soul and it doesn't understand your business and that complicates its life. One day I'm sure Google will invent artificial intelligence. Until that day, you're the best they have. Yes. Thanks for coming. Thank you. Traditional marketing, when you do segmentation analysis, you can look at a bunch of users from gender to age to attitudes towards the brand, for example. Right. Since I move on to the online world, it seems it's a little bit more difficult to do segmentation work because of the fact that we respect your privacy and that it's hard to get that data when you don't have that person present. Right. I know Ebay, for example, actually they do studies on the person actually sitting in front of them and seeing how they compare. Right. I was wondering if you can talk about little bit more about how segmentation studies can be done successfully with the online trial. Yeah. So the question was specifically around how do you segment your customers? And with a more specific emphasis on psychographic and demographic segmentations, I think that was actually the essence of the question. And one of the challenges actually why we have had such a hard time with all this stuff on the web is that most data on the web is anonymous. And that makes it much harder, for example, to do predictive analytics and statistical modeling, pure modeling that you could do in the offline world. Data you have from your box store, it's actually much, much harder to do. I think that in the near term there probably isn't going to be a very Good way in which we would be able to get psychographic demographic segmentation of the data of people on our site. Mostly because they are freaked out about us, all of us. People come to any website and they're freaked out about their privacy already people keep blowing cookies away. They will refuse to activate JavaScript. There's just like a big fear. I think we underestimate how much people worry about privacy. So direct connection to the data is still missing. But if you're a site of any good size, Hawaiis, for example allows you it's obviously paid surveys, but they have taken their data and merged it with the Claritas Prism data. So Prism is a big database that has been around forever online, actually offline and go to claritysk.com, click on free tools, type in your zip code. It will tell you the salary, makeup, the skin color, education level of everybody in your zip code. It's kind of fun. Try it. Now what heywise has done, it is taken the Prism data, merged it in with the competitive IP address data it has for all the visitors and so now you have actually an ability to go into Hawaii's and pick the demographic psychographic category you want and say I want all white women who are not married. I'm not sure why you would want that. But if you do, you can say what websites do they browse on and what zip codes are they in. You could run ads there, you could figure out acquisition. Yes, for the moment it's just you Prism is just a database focused in the US So you could use that. But you have to be a website of a certain scale and size. So my blog probably has bad data. It's not enough signal in all of the noise. But there are options like that. But the other one is I also know companies use primary market research. So one of the best ways to understand, I think it's discounted in the online world actually primary market research is a great way for you to understand your customers. Like a former company I used to be at would simply actually do this annual study once a year of everybody who ended up buying on their website. So it's limiting, right? Just everybody who ended up buying online, everybody who ended up buying in a box store and they would just do primary research, standard market research, and understand all this data and that gave them a much better grasp on it. Capturing it online is still elusive and I think it will be so until people are more comfortable with giving up that data and people are not comfortable at the moment. Yes. So you have a decent amount of People in here who work on the online help centers for Google. Yes, and we use Google Analytics to try to improve it. And a lot of things you said in your presentation are applicable to what we do. But I was wondering if you had any additional advice for the folks who work on online support. I actually I met with somebody who was doing from Dublin who was doing AdWords help center and we exchanged a long conversation a couple hours, figured out the KPIs and most KPIs that I recommended that you track are actually non clickstream. So clickstream is just the data you would use ga to understand content consumption. You would use the data to understand, if possible, do internal site search critical on a health center. It's like a gold mine of data waiting for you if you're able to do that. So use that clickstream data to understand content consumption. Keywords people are searching on what pages do they see? But your core success metrics that you want to track from a help center for any business, Google or otherwise is actually to use some kind of voice of the customer. And one thing you'll find on many enlightened health centers is page level surveys. Did this answer help you? And the interesting thing is there is both a sub follow up for yes and no. If you say yes, you ask them why was it helpful? Did it actually fix your problem? If it says no then you say why did we suck? And they tell you that. But the difference between an e commerce website and a help center is in one case to some extent you don't care about the customer you're trying to sell. So you do care. Not that much. On a help center the customer is actually the problem you're trying to solve. And clickstream tools are very bad at telling you that the problem is solved or was that the right answer? So one mistake a lot of people will say I'm going to run all my analysis on a health center. I will count how many people saw answers, pages, FAQs where problems are solved and if more people see those pages, hooray, happy birthday. No, just the fact that somebody saw the page doesn't mean their problem was solved. Ask them. So page level surveys are something I do recommend. They're easy to do. You can just create it at your own home and you can just dump it into a flat file or an online database. That is something I would recommend. I would recommend internal site search and a couple other things and measure content consumption. That's actually a good use of a clickstream tool. But on a health center you want to figure out what are the ways I can ask them if the problem was solved? Because if it's not solved, they will pick up the phone and call you. Probably cost you 50 bucks. How about Reverse Call? I'm sorry? How about Reverse Call? Yes, you could possibly use Reverse Call, but you still don't know if. Once you read the answer to a problem, you need to ask the customer if it's actually solved because it could be cryptic. They have no idea what button to press, where to log in. That's the challenge with the Help Center. Other Questions Alex saying hi. Yes, you mentioned signal to noise not being good enough on your website. How many page views do you have to have before the web optimizer is useful? So the question was, how many page views do you have to have before the optimizer is useful? And optimizer is one of the multivariate testing tools. And the answer is, let the tool decide. So the way the optimizer tool will work is it will create a lot of recipes of the page, many different variations of the page, and it will measure a success. So you can say, oh, this is the page and conversion is a success, or somebody submitting a lead is a success, or they saw a page with your pretty picture on it and that's success. You define the success. Then it simply puts up all these recipes and it will statistically measure the difference between conversion rate to your success metric. And then that is statistically significant, that 95% statistically significant or higher. It will tell you that we have a clear signal that this is winner. So, for example, if you have a page that is full of these many links and you have another page with these many links, except the first row, you can imagine it will take a long time for you to get a signal because really, it's not that different. You're missing eight more links. We still have 45. Now imagine you had a page with these many links and just one. You can bet the signal will come much faster because the difference is so dramatic. So the speed at which you will get the signal will depend on your experiment and it will depend on the difference at which people convert at your success point. That's beautiful. The nice thing is you don't have to guess. Throw up the experiment and let the tool compute this for you with all its statistical power. It's very, very cool. I like it because also I can't think statistics, so I'm glad somebody else does it for me. I guess that holds. Unless somebody's spamming you. Oh, yeah. All other. You could imagine the competitive intelligence turning into a war where people spam each other. It could. It could. I mean, I think that one of the nice things on the web is so easy to do business and create havoc. And there should be a disclaimer. All normal stupid conditions apply. Thank you.