098 the golden age of data and analytics david boyle masterclass (Strata San Jose March '18)

So when Alistair first asked me to help organize today and then asked me if I'd open the session and set some context, I was very excited. Like he said, I'm very optimistic about the future of analytics and data and driving decisions. I was trying to think of a title for the talk and I came up with this idea that it feels like we're approaching a golden age for data and analytics in the media and entertainment industries. I've been trying to integrate analytics into various businesses for about 20 years. The most recent half of that has been in the entertainment industry. And it's never been easier. The impact's never been greater. We've never had as much support from senior management to do it. It feels like a golden age. But I spent most of my youth reading either comics or math books, so I had to check what the metaphor of a golden age actually really meant. I had to look it up to double check. And sure enough. Yeah, it's an exciting period where things go really, really well. But it alludes to the fact that there's a decline coming. It alludes to the fact that things are great. It's going to get worse. Is that the right metaphor for where we are in data and analytics? Actually, I think it might be. And so I doubled down on that metaphor and that's what I'm going to use today. I think there is an exciting time. There's so much going on. The whole of the agenda today speaks to that. But there is some peril in front of us. There's some challenges, there's some jeopardy. I'm going to outline why I think we face that jeopardy, outline what I think we can all do about that, to try and get us to this golden age and maintain us in this golden age for as long as possible. So, as Alastair said, I work at a company called Masterclass. We make classes with amazing individuals that you can buy online for only $90. And we're a tech company. We're built with data at our heart, like many of you are. That's not the perspective I'm going to talk from today. I'm going to talk from the perspective of what I call the real world, the world outside Silicon Valley, the world where companies are not built on data, where they're old companies. These are 100 year old companies I've made a habit of working for. I've broken that habit. Thank you. But 300 year old companies I used to work for, wrestling with things like creativity and vision and taste and all sorts of things. I'm not going to try and talk about taste in this. I don't want to bias you for the debate later, but you can read between the lines. Sorry, Violeta. And in these companies it's still never been a more exciting time for the use of data. But the challenges that they have are hard to solve using data and they paint a picture for. For the bigger challenges that I think we all have, no matter what business that we're in. So first to the excitement and just a brief look at some of the highlights for me of the agenda today. And I really could have picked any of the topics that people are talking about today. They're all fantastic. Cross channel dynamics. Very exciting. For decades we've wondered about the impact of one channel on another and pricing changes on one channel on another. Distribution changes on one channel and another in music. What's going to happen to the CD when streaming comes along? How will that impact downloads? We didn't know. We spent lots of time on it. Violetta has the answer and we'll be presenting that shortly. Really exciting. These are gold. Personalization at mass scale. Who isn't wrestling with that challenge? We'll hear about that later from Comcast. The world's getting so much more complex. There are so many service providers in so many countries that we all have to work with. How do we do that at scale? How do we get back data at scale? How do we understand the dynamics of users and platforms around the world? We're going to find out the answer later. This is gold. The world is moving so quickly. This change is coming at such an increasing pace. How do we do marketing at a pace of change that rapid? We'll find that out later as well. Social media, how do we get value out of that data? How do we use those signals to better understand the world Again, we'll pick that up later with superfans at 2:00 clock and AD attribution is the famous saying in marketing. Half of my ad spend is wasted. I just don't know which half. It's like a 50, 60 year old phrase. We have the answer today. We're going to Hear it at 3,30. These are gold. They really are. We're approaching this golden age. But how do we stay gold? How do we keep it going? How do we avoid the peril and jeopardy? I argue that there are four big questions that we all need to ask. Four questions that every one of us can ask as we go about our day to day, as we pick up individual projects, as we tackle individual business problems. So I'm Going to walk through each of these one by one, give you a brief example of each in action and hopefully have you keep these in the back of your head as you go through today and as you go about your daily life, let's help get us to this golden age and keep us there. So question one. What is the role of, well, people versus machine? What is the role of technology? What can algorithms do? What can't they do? And how do we make sure that we don't over promise? This is a lesson over promising. What technology can deliver is a lesson I've learned many times in my career. I suspect we all have. The favorite time that I learned this lesson was in the music industry where I was asked to help to predict which single should be the first single from a new band. They'd never released any music before. No one had heard anything by this band. And the person who is owning the relationship, who was a and ring this band had five songs to choose from. He wanted to know which should be the first single. And we'd done amazing analytics in the music industry. We could totally nail this question. So thank God he asked us. We did this predictive work, big data around the world and we came back. There was a clear winner. This was one song was going to be so much more popular than the other four. There were shades of gray amongst those. But one song really stood out. I came back saying, hey, I've got the answer. This song should be the first single. It's the biggest, it's going to be the best. And he said to me, right, well that's not the first single. Then I am gutted because he's not listening. He's asked me to do this work. He's not listening. It turns out that's not what's happening at all. He's solving a different problem. He's solving a problem that's something like how do I help this band to have long term success? And that's not start with the first big song that starts slow and build towards it. Also, he knows information. I don't have constraints that aren't articulated in the problem. He knows that the band aren't ready for a big hit single. They'll fall to bits under the fame they need to practice and gel. So I think I'm solving the problem. I come in with the answer and it's an answer to a small subset of the problem. The problem needs to be solved by a person with skills and expertise and all sorts of knowledge that machines can never have. I was over promising over claiming in my abilities for analytics to answer that question. And the real answer was that my job was to help him make that decision, not to tell him what decision to make. I think it's a mistake. We see a lot in data and analytics and checking ourselves. Looking for the role of people alongside machines is critical. Of course our community doesn't often talk about that. We don't often talk about how limited we are in our ability to answer a question. So I was really excited to find in a blog a minor detail in a Netflix blog post the other day which alluded to this fantastic article. It talks about photography and images and how they find and select and prioritize images from amongst all of the still frames that you've got in videos. At the BBC. I know well the team that do this, they comb through video. They're masters of understanding the show and the context and what makes a hit frame that they can show on video platforms. They comb through them, prioritize them, clean them and end up with a few that are used to promote a few images that are used to promote the videos. Netflix, of course, infinitely smarter and more resourced than the BBC were, have built systems to do this. And the blog post goes into great detail about how they comb through video, identify entities and great looking frames and pull them out. The exciting detail which isn't described in detail in the blog post but appears in the diagram is this small detail at the top. Of course there are some frames that they call product safe and auto published. Great, the machine can do some of that. But look at this, there's a creative and editorial candidates right there. So some level the machine is finding frames that help the people to make decisions. But I'm encouraged that they still have creative expert people who are more efficient and effective, but expert people picking the frames they want to represent the shows not fully automated. So don't overclaim the role of machines versus people. Respect the role of expertise and I think we'll be a lot more successful. So Question one what's the role of man versus machine? My answer is be like Netflix. Don't be like me at the BBC saying I can pick the next song, right? Question 2 Are you favoring tactical acuity at the expense of strategic understanding? Are you relying too much on algorithms and missing out that real understanding of what's going on in the world? The example I see every day is Facebook advertising. We throw out adverts or ad sets to Facebook like Ad Set A here maybe, or Ad Set B and small variants thereof. The amazing Facebook algorithm optimizes, optimizes and optimizes and picks the best variant. And we end up maybe at the top of hill A, success, or maybe at the top of hill B success. What we don't understand in any detail is who those ad sets are working for. What about those ad sets is working? We don't understand the landscape. We can't see that there's hill C right over there. We haven't done an ad that gets us close to it. We haven't got the proposition right. We don't even know that hill exists. Which is okay. We're like semi optimizing. But as soon as the landscape changes, as soon as people get tired of ads of this type, as soon as competitors change, as soon as the national mood changes, so suddenly we're left blind. We know nothing about the landscape and what works. Tactical acuity, but at the expense of strategic vision and understanding Peril. And there was a great blog post about how you might use algorithms to drive that strategic understanding if you're really smart about it. Great post on Medium the other day says that testing can make you thoughtful. If you use testing in the right way without some overarching vision, you'll be left with a gnarly mess of test results and no clear plan. We know about that from the Facebook example. But start with good ideas, do some research, come up with educated hypothesis and concepts you believe in, then build and test them to verify your thinking instead of defining it. Algorithms can be used to make you smarter, but often we don't. Often we use them instead of being smarter. Are you favoring tactical acuity at the expense of strategic understanding? Question 3 Are you investing in tech at the expense of getting good at using that tech? Chess is a wonderful metaphor for this. In chess, computers beat humans at playing chess. We know all about that from Garry Kasparov and playing Deep Blue. But more recently, the lesson is this. Humans playing with chess computers are way more powerful than great chess computers and also way more powerful than great chess players. The combination is what kills it. In fact, it's a clever process, like how you use your computer to play chess against somebody that beats both superior algorithms and superior knowledge. I think this is true in business as well. At emi, as I say, we got pretty good at predicting song success, pretty good at describing audiences and shaping marketing plans based on data and analytics. EMI got bought by another record label who are, let's say, less excited about that capability. I thought, no problems. I'll go work in another industry. I'll transplant that same capability somewhere Else, I went to book publishing. HarperCollins effectively bought that capability from EMI. We rebuilt it in about six months. Three years worth of work. Effectively took six months to get up and running to prove its benefit. Authors loved it, publishers loved it, it worked brilliantly. But they weren't willing to invest in getting good at using it. They wanted to buy the capability and sit it in the corner. They wanted me to do all the hard work. I wanted to train the business. I wasn't allowed to take the business people out of the business for a day of training. I was allowed maybe an hour here of training, maybe an hour there. At emi, we engaged people. It was part of the culture. It was incentivized and encouraged. And it was getting good at using the capability that helped us to win. It wasn't just having it and sitting it in a corner. I'm seeing lots of nods. Something very familiar in analytics. Buy it, stick it in the corner. No, get really, really, really good at how you use it. So are you investing in tech at the expense of getting good at using it? Question three, big risk. Question four, Are you owning the actual decision that's being made? Or. Or are you owning just some inputs to that decision? Think about the music example. I was owning some input into which single the band should release. I wasn't owning which single the band should release. A great example of somebody recognizing that. I read the other day this awesome artificial intelligence platform that's now being used to identify crimes and tag gangs and gang members in crimes. The article was all about how it could be misused, how it could ignite an ethical firestorm by tagging people as gang members when they weren't. The poor data scientist who had worked on this was grilled at the conference when he was presenting and he said, I can't be sure how the tool will be used. I'm just an engineer. The article was not favorable to him for this quote. I love it. He knew what he was doing. He was owning a component of the problem. He was not owning the full problem and being very clear in our work. What are you doing? Are you recommending the answer or are you giving a component to somebody who will recommend it? Be really clear what you do. Don't overclaim. Are you owning a decision or just some inputs to that decision? All well and good. You might ask thoughtful questions. David, thanks very much, but. Feels like a golden age to me. Feels like exciting times. And honestly, data is going to win. Algorithms are going to win. Humans can only resist for so long. There's no jeopardy here. Thanks very much. Good attempt. Here's three reasons why I think there actually might be jeopardy here. Not for all problems all of the time in all businesses, but for many problems many of the time in many businesses. Here's three reasons I think we need to get these questions right. Risk 1 and I'm trying to give you materials wherever possible. Here's a great medium post the other day on the topic Risk one is that actually like common sense is pretty good? It's pretty good whether in this blog post example, whether it's common sense like recommending the most popular topics that day or most popular products that day. Common sense Algorithms. Easy to code, easy to understand, hard to imagine failure, brand risking failure, or algorithmic failure. They're actually pretty good for quite a lot more effort. You can improve on them, sure. But if we over promise, if we under deliver, if we're not clear on our role, I think we risk businesses saying you know what? We'll go with the simple explainable solution instead. Again, in many problems many of the time, and that's either using algorithms like this or I think the equivalent in terms of big bolder decisions is in human curation instead of algorithmic curation. Wasn't exciting when Apple launched Beats as a radio station. Major play for them. Did they have the best AI? Did they go for the most personalized, tailored radio station you could imagine? No. They hired some of the best radio talent there is for from the BBC in England, I might add. This is Zane Lowe, very famous in England. Spent many, many years of his life deeply immersed in music, listening to hundreds of songs a week, testing them in clubs and on the radio, and refining and curating taste. Risk two is that people who could easily have invested in us in algorithms in analytics say, you know what? Screw that, we're going to go with personal brand curation. That's a valid alternative in many situations, just like this one. So simple algorithms is a risk versus what we want to do. Personal curation, expert curation is a risk. And significant reductions in investment in analytics is also a risk. This is another great blog post from the other day. It's titled Hiring data scientists. Step 1 Stop looking for Data Scientists. It opens with Dear World, we're looking for a data scientist to fill an upcoming gap in our business model. We are not exactly sure what you'll be doing, but we are sure that our shareholders will love the idea that we have data scientists. It's funny because it's a blog post, but I swear, a CEO of a company I used to work for. I won't name names. Stood on stage in a company meeting and said almost exactly this. He said, I've just been hanging out Spotify with my friend Daniel and they have the best mathematicians in the world. The best mathematicians in the world. We don't. We need them. And until we have them, we can't compete. Isn't that a lot like this? I said to him afterwards. I grabbed him over coffee and a lunch break at the internal conference. I said, we don't have the problems for the best scientists in the world to look at. We don't have the data. I've talked to them, I bought them lunch. That's it. There's nothing for them to do. We need a business model that supports it. You can't start with data science and then end with the business problems. So it's funny because it's a blog post. It's also true in the real world. I've seen it. And we benefit from that because some of that investment, some of that crazy investment flows down to us. If we over promise and under delivery, if we get our role wrong, then increasingly that investment is going to start to go away and that's going to impact us whether we're behaving in that way or not. So I think it is an exciting time. I'm couldn't be more thrilled about all the things we're going to hear today. We'll be taking copious notes through every session, but at the same time, I think there's real peril unless we get answers to these four questions right again and again, project by project. What is the role of humans versus machine? Don't over promise. Respect expertise. Make sure it's baked in at the appropriate points in each project. Are you favoring tactical acuity at the expense of strategic understanding? I often wonder what marketing directors, when they retire to their smoking room at the Marketing Directors Club or whatever it is they do, often wonder what they talk about these days. We don't know anything about audiences. We don't know anything about what ads work, what ads don't work. We just throw them at Facebook and they optimize for us. What do they talk about? What do they compare notes on? Don't forget to bake into the algorithms strategic understanding so that you're helping people to get smarter. Or don't forget to use them in a way that helps you get smarter, to challenge your thinking, not replace your thinking. Are you investing in tech at the expense of getting good at it? I've seen it in so many businesses. Poor old HarperCollins. Same tech as EMI. It worked just as well. Just the lack of getting good at using it was enough for it to completely fail. And are you owning the decision or just some inputs? It's no problem to say I'm just an engineer. This is just an algorithm. It's actually better to say, I'm just an engineer and this is just an algorithm. There's more to it and I need you as a business leader to work that out. Price optimization is another great example. Algorithmically, here's the optimal price, right? But what does that do in terms of brand? What does that do in terms of competitive response? What does that do in terms of the partner who's charging that brand, whether it's itunes or Amazon? What does it do in terms of their reaction? The algorithmic optimization is awesome, but it's part of the problem and it requires other business inputs. So be excited. I think we're approaching this golden age. It's in our gift to get us there. It's also in our gift to mess it up and cause that decline. If we don't. Keep those questions in mind, I. I hope they're useful during the rest of the day. Thank you very much.

I feel like we should have had the debate. Should engineers own the ethics of their work? We'll do that next time.

There are so many great debate topics. We could spend all day with debates.

It seems like a pretty controversial stand to say I didn't earn that at all. Just an engineer.

Yeah, definitely. I think business leaders increasingly look to. They don't understand some of these questions themselves. They don't understand where the limits are, and they think the limits are infinity. We can do anything sometimes, right? So there's been plenty of cases, I think, where business leaders have said, hey, what should I do? And I've had to say, well, here's what the analytics says, but you need to think about this. I can't actually tell you the answer. What's your vision for the company? What role do you want this company to have in the world? Algorithmically, the best answer for Netflix to optimize on video views might be cat videos. But strategically, do they want to be the platform of cat videos or not? That's a choice, right? No algorithm is going to tell you that. So asking what the optimal answer is comes often and often you've got to push back. It doesn't go down very well. People don't like to acknowledge the limits of their work or, of course, this great capability they've bought. Either they Often outsource thinking to analytics and they shouldn't.

So usually in the industry people have this gut feeling. People that aren't from the analytic side of the industry, they always have this gut feeling. So couldn't you sort of incorporate this gut feeling into your model that will probably give some weight to the data that you have and also their expertise, knowledge.

Yeah, great question. So could we integrate gut instinct into an expertise? Could it be integrated into the model? I don't. We're debating something on this topic later. I don't want to. I've got the mic. So look, I think she's got it. Next. Yeah, this is a dangerous game to play. Look, I think the reality of many business problems, to state it a different way, the reality of many business problems are that they're not single goal optimization, like what's the best price? But multi goal optimization, which is what's the best price given the constraints on brand, given the market, given competitor response, given platform response, given like again on brand, given the role I want my product to play in people's lives, do I want to be a mass product that millions of people have access to? Do I want to be something more special and more refined and more premium? In reality, for many businesses that's a multi goal optimization. And most of those goals are not articulated or articulatable using code or anything like code. Most of them not measurable. There's no data or the data is such a poor proxy for that objective, for that constraint, for that metric. So I think in theory there might be way more gut instinct or expertise that can be baked into algorithms. I think in practice it gets very, very difficult very, very quickly. And I've personally found it better to do a really good job of solving a piece of a puzzle and being very clear what the constraints are around what that recommendation should be used for and what data it contains and what it doesn't. So the decision makers can then take other data into account. I found it better to be very, very clear about that and do a good job on the subset of the problem rather than do a bad job on the aggregate problem and have them not really use or trust the recommendations. So that's my tactics worked pretty well. Love to hear other versions.

I think that's all good to get back to Alison's pointed out a human being. So I don't think we can automated what happens. Yeah, we can't afford to just say let's define this. That's a question we have to address. But at some point the business person Made the decision to automate.

Yeah, that's a good point. Yeah. So the point is that for many decisions, self driving cars is a fantastic example. It's not possible to put a human in the loop for all those decisions. Actually you have to fully automate them. And we'll hear later about the speed of business changes these days, the complexity of business. There are tons of business problems for which that is the case. I think there are also tons of business problems where that would be such a bad representation of the recommendation that it doesn't apply. So I think it's company specific, it's business problem specific and it's particular why I set the context of like big lumbering hundred year old entertainment companies. At their heart, what's happening at their heart, not in terms of distribution optimization. At their heart, what's happening? They're believing in somebody's amazing talent, probably at an early age or probably before it's fully formed. You're believing in that person. You're thinking about their hopes and dreams. You're thinking about their ability to deliver on those hopes and dreams, given their life arc from here onwards and their friends and their family and their ability to support them in that journey. You're betting on their ability to bring collaborators with them and to deliver something and then you're investing in that and curating and hoping that you can help that person on that journey. Like at the heart of the business are decisions that look a lot like that. Where I think my logic applies at the other end, it's absolutely fully automated AI awesome. In the middle is this gray area where we mess it up for one reason or another. And that's the really interesting area for me. Great. One more question maybe and then that should be 30 minutes. Will GDPR have an impact on my golden age? I think for every session for the next three days you could ask that same question. Will GDPR have an impact on this? And you'll terrify every single speaker. So again, I'm going to talk with my hat on in big hundred year old media companies that don't have very much data, so we don't have to worry about GDPR very much. But yeah, I think there are all sorts of complexities that are going to get in the way and that seems like a very big one that's looming. So yeah, I think such a big deal. Luckily not for companies that don't have data. Right, sorry I skipped your question shamelessly. The sessions on GDPR on the wider agenda, I saw them. So that would be a great place to go. All right, Wednesday and Thursday there are GDPR sessions. Okay, great. So time for me to introduce our next speaker, I think. Let me work my magic on slides in a second. I first met Violeta Hennessey last year. She gave an amazing presentation conference I was at and it was just the detail, the level of sophistication, and also the interesting application, which is game changing for again, many of the businesses that I've been in was fascinating. I'm even more fascinated by her background. I remember a time when in media and in companies trying to embrace data. We talked about the credit card industry having fantastic data and data analytics. We wanted to steal experts from the credit card industry. I feel like nowadays it's health care. That's the industry that everyone talks about. That's where there's great, amazing data. That's where exciting work is being done. And that's what Violetta brings to the movie industry. A background in epidemiology, statistical methods, particularly from electronic healthcare databases. And that's the magic for me of her work in movies, which is it brings fresh ways of looking at problems, fresh ways of thinking about problems to crack. Problems that, like I say, we've tackled many times over the years and never been able to crack. So good on you. I'm going to be taking tons of notes. Please welcome Violeta and I'll try and get the slides ready at the same time.