Strategyzer Webinar: The Basics Of Testing Business Ideas

The broadcast is now starting. All attendees are in listen only mode.

Hi everybody, welcome to another Strategizer Strat Chat webinar session. We'll be starting the broadcast momentarily, just waiting for people to come in. In the meantime, we'd love for you to think about today's session and maybe share with us what you hope to learn. You can share that in the questions box in the GoToWebinar panel, or you can tweet it to us at Strategizer with the hashtag strachat. We'll be starting pretty soon. Thanks. Just a reminder, if you've joined us, we'll be starting the broadcast in about a minute. In the meantime, why don't you take a moment to think about what you hope to learn from today's session on the basics of testing business ideas. You can share it in the Questions box in GoToWebinar, or you can tweet it to us at Strategizer with the hashtag Stradchat. We'll be starting soon.

SA.

All right, let's get started. Hi everybody, welcome to another Strap Chat webinar session. Today's topic is going to be the basics of testing business ideas. It's a really interesting one. We've got some great work to expand on from some of our other testing content related webinar sessions that we've done in the past, so we're eager to share where we are today, as well as some fun little challenges and things for you to think about as you're dealing with any of your business idea testing challenges inside your organizations or inside your companies. As always, I'm your moderator Kavi Gupta. I manage content and community as Strategizer and our host Alex Osterwalder, co founder of Strategizer and co author of bestsellers Business Model Generation and Value Proposition Design. If at any point you want to share a comment or feedback, or maybe even just make some suggestions about our content and the webinar broadcast, you can reach us Strategizer on Twitter and use the hashtag Strachat during this broadcast to let us know what you're thinking so we can see your conversations. We also always encourage you to go to blog.stragizer.com for more in depth analysis and content around this topic and many other topics that are related to innovation and strategy. I do want to remind everybody you might have seen the sign up for this on our blog or in the email newsletter, but part of the reason why we're running this webinar is to also gauge interest on a potential testing course that we're working on with our Cloud Academy program. Essentially, we want to help teams get smarter and stronger about testing new business ideas. What I'll be doing throughout the session is sending the link that you see here on the screen to all of you within the GoToWebinar chat box, so you can click and get signed up to be notified when the testing course is live or when it's actually getting closer to completion so you can get firsthand access to it. I'm going to pass the reins over to Alex now. Alex, as you get set up, maybe just give us an update on what's been going on with the testing course content that we've been working on and why we thought it was important in the first place.

Yeah. So thanks, Gabby, for the introduction. See if I can get the screen from you. We've been working on this content in particular because we've seen a lot of basic mistakes. So this particular webinar is going to be really around some of the basics rather than going too deep. But we've seen a whole range of errors that I think can be if the content is available rather than using expensive coaches or trying to search on the Internet for the right information. And then we would go pretty deep as well with a testing library to have a whole series of types of experiments that you could run beyond the interviews. So exciting stuff to get started. I'm just going to quickly draw kind of the context and some of you have already seen me draw this, so maybe a little bit of repetition first. But basically when we innovate or in entrepreneurship, when we create something new, we go from idea and I'll label my drawings so you can understand what it is when we go from idea to business, or first, maybe validated business case or validated business idea. So I'll draw this as a one billion dollar opportunity. If you're in a corporation and if you're a smaller corporation or a startup, that could be obviously a smaller opportunity. But when you go from idea to real business, then, you know, traditionally people used to do this. They used to draw or make a business plan and that used to be taught even at business school. But we now know that business plans are actually the worst way to go from idea to business, simply because the journey of innovation and entrepreneurship. And I'll get a little bit more specific about what I mean with innovation. Innovation and entrepreneurship journey looks like this. See if the GoToWebinar can follow. It's an up and down, it's a four forward and backward, you make progress, you regress. Customers say they don't like it, you're building something, you can't figure out the supply chain, your venture capitalist or your boss who is funding you, takes away the money. And eventually, if you make it, and not everybody makes it, and that's part of innovation and entrepreneurship, eventually you get to the goal. But business plans are not just the worst way to do that. So cross this out. They actually maximize the risk of failure. And I'll tell you why. Because let's say you are selling your business plan if you're in a company to your bosses or if you're in a startup to venture capitalists and the other person says, yeah, I want this. You're actually condemned to execute this business plan that you sold. But the problem is that new ideas never turn out to be like they are in the business plan. And you'll see. I'll quote Steve Blank, who invented the whole lean startup and customer development methodology, likes to say no business plan survives first contact with customers. Because innovation and entrepreneurship is not an execution problem. Business plans are for execution. It's a search problem. And that is what we're going to deal with. And the whole idea of search is we design an idea and we test. It's a very iterative process. In our case, well, it's iterative and we figure out things over time. So I'm going to draw that in a bit. So the whole idea of testing here in the search phase, oops, forgot the C. Is to figure out what could work and what not, what won't work, and iterate our idea and then it becomes an execution problem, but only then. And to frame this also in a established company context, once you have an idea and it's going to work, you change the idea, turn it into value propositions that customers want and business models that are going to work and you create that organization, that company, well then you don't have a search problem anymore and not an execution problem because you just scaled it. Then once you have that business model, and I'm not going to go much into the business model canvas here at the beginning, I'll just assume you know that and otherwise you can Google it. It actually, once you have a business becomes a management problem. Now what we're going to look at is how do we use the testing process for the entire spectrum, but mainly here for what we'll call growth, innovation. Here is where the growth comes from. And every company, you know, ge, Nestle, whatever company you can imagine they were once a startup and Many of the established companies, they now focus on managing a known and proven business model. And they forgot to a certain extent how to create growth innovation. What they're still pretty good at over here is efficiency innovation. And you can use the testing process also for efficiency innovation. Let's say you're going to create a new application for your salesforce. That's still an innovation problem. It's digital transformation, maybe we could call it. Well, you still have some uncertainties, but a lot less. Here uncertainty is lower than over here uncertainty is very high. And then in between, and I'm borrowing here from Clayton Christensen, we have sustaining innovation. So three types of innovation. And in all of them we can use testing our ideas. But where it's really, really indispensable is over here for growth innovation, which is very similar to entrepreneurship, some different constraints. We can't survive without actually using these testing ideas, because otherwise we're going to fail big scale, not the kind of small and quick failure we're talking about. So I'm going to try to go back to my slides now, see if I can make that work. See Cavi. Can we see my slides? Did that transition work.

Now? It's good.

Okay, so I like to use this quote from Scott Cook, the co founder and chairman of Intuit, who likes to say, kind of illustrates the example of the business plan that I showed you before is that for every one of their failures, they had a spreadsheet or business plan that looked awesome. We can always make the numbers look good and shiny and the decks we present around our ideas, that's not too hard. And if we're charismatic and good salespeople, well, we're probably going to get the funding. But entrepreneurship and innovation is not about getting the funding. It's about creating value for customers embedded in a business model that can scale profitably. Right. So what we really have to admit is the quote that I already mentioned before from Steve Blank, The person who launched this whole lean startup movement, who says no business plan survives the first contact with customers. And Steve Blank was an entrepreneur. And I like to ask him, sometimes, Steve, you didn't need these tools and this testing process to become successful. So now you're teaching it. You're using the Strategizer tools. Why do you do so you were able to do it without. Is this just for the stupid people? He says, well, Alex, I would have been a lot faster and I wouldn't have had these huge failures that I had if I knew how to do this. And that's why he's teaching this approach now at Stanford, Columbia and Berkeley, and beyond that, even with the US Government. So pretty interesting to frame it this way. So basically what this is all about is saying however good our ideas are, and I illustrated it here with the value proposition canvas and the business model canvas. We do need to admit that we have no clue. And that is the hard part. It's not very easy to admit that you have no clue, but if you don't, if you don't admit that uncertainty is at its maximum at the beginning, you're actually going to go off and implement something that nobody wants. So let's look at the main components of the testing process so we can dig deeper on one or two of these. And we tried, just recently tried to ask ourselves, what are the main things that people need to focus on when they test? So the first and most important one, and we'll spend quite a little bit of time on this, are the hypothesis not what is your idea per se, what's the business model, but what are the underlying hypothesis, the assumptions made explicit that your idea could actually work, business model, value proposition, or even efficiency, innovation, as I've shown before. And then we're going to quickly dive into how to test this with experiments and what we won't be able to talk too much about, but this is part of our course, our upcoming course is the evidence that's generated from those experiments and then the insights that we, the conclusions and learnings that we can have from this evidence, and then the actions that we take, right, based on the insights and the insights are that either our hypothesis proved to be correct so we can support our hypothesis, or we might learn from the experiments that our hypothesis were wrong. And that is often the case. So more often than not, if you're really doing something new, you're probably wrong. And that's okay because it's designed into the process. And then sometimes we just don't know. There will always be contradicting data, there's always uncertainty. But sometimes you do need to admit that you're going to test a bit more to go deeper. And sometimes you discover completely new avenues, maybe even more interesting avenues that you've set out to test. So insights that lead us into a different direction. But at the end of the day, what's really important is to take action. It's not about testing alone, it's not about evidence, it's not about learning. It's really about making progress from idea to validate the business case and open the business. So it's all about actions okay, let's dive into the hypothesis, which is really core to what we need to do when we test, we need to ask ourselves what actually underlies our business idea. It's the first part of the whole testing process. So quick definition, just to get us on the same page. A hypothesis is basically an assumption made explicit that your business model, value proposition or strategy builds on. It's really this idea of making explicit what you should be learning about. So what you need to learn here is or the hypothesis. And if you understand that, you will start to understand if your idea might actually work. So it's not just about the idea, it's about testing the underlying hypothesis. And then we'll see this a little bit more in the couple of slides that are coming. The hypothesis can be related to desirability, feasibility, viability, or adaptability of a business idea. Understanding what type of hypothesis we're dealing with is actually pretty important. Okay, so that was the definition. Now, of course, we need to ask ourselves, where do these hypotheses come from? Well, they do come from our business idea. Something that we designed and shaped and think could be true. But we don't know, Right? We don't know. So when we start testing, we need to ask, well, you know, if it's related to our value proposition, what are the main assumptions, the hypothesis there. So an example here is that I believe that executives struggle with good meeting plan. That could be an assumption related to my customer jobs pains and gains, as we like to call it. Sometimes people like to call that the problem, problem assumptions. Or it could be related to the other side of the value proposition, how you're actually creating value with products and services. So here we could say an example. We believe that our technology helps increase the quality of meetings. So a hypothesis is not a question. It's something that you believe and you don't know if it's true or not. But you know that you need to go and test that or find the evidence and bring the evidence to the table. Then a hypothesis could also come from our business model. So broader picture, sometimes people are too focused on products and services alone. To a certain extent. The foundation of this testing methodology, the lean startup circle that starts with Build, Measure, learn, lets us forget sometimes that after all, it's not just about the product that customers want. It's really about making more money than you spend to create value. So maybe a business model assumption could be. We believe that executives are willing to pay a subscription fee of $20 for our new service that we have in mind. So this Would be explicitly a hypothesis related to the revenue streams. Could also be that you have a hypothesis related to your environment, coming from things that are happening around you, related to competitors. Maybe you have this hypothesis that competitors can't replicate your intellectual property, mean you might have to get patents on them, or it's too hard to replicate. Or it could be, you know, hypothesis that are related to the regulatory environment. So, you know, if you're Airbnb and Uber, you probably have to assume that regulation might change in your favor or that you could influence that. It's a very strong hypothesis because otherwise you'd be continuing to operate in a legal environment for the rest of your existence, which is not a great situation to be in. Or it could be related to the infrastructure. You have a hypothesis that the infrastructure, in this case broadband mobile, will become available to all the customers that you're targeting, or new type of broadband mobile. So different types of hypothesis. At the end of the day, it's really everything that has to be true for your idea to work. So when you have an idea and you've shaped it with the business model canvas or value proposition canvas, you need to ask what would have to be true for your idea to work. And that will help you generate the hypothesis either related to the value proposition, business model or environment. Okay, now if we ask what different types of hypothesis kind of gave you the answer, but I'll make it even more explicit. We like to use the approach by IDEO to start with. When the Californian design firm that shows us for any kind of industrial design, for any product, we should test desirability, feasibility and viability. We like to overlay this over the business model canvas and say, hey, isn't that kind of the same thing? When we look at the front stage of the business model, we have desirability. Do customers want it? Right. We talked about customer jobs, pains and gains, their problems, and customer solutions for the customer. The value proposition. Or then on the left hand side, behind the curtain, as I like to say, feasibility. Can we actually deliver it? Can we get access to the intellectual property? Can we manage the technology? Can we build this thing that we're setting out to build? Can we manage the algorithms, whatever you are doing? And then of course, viability, what's it worth? Can we make more money than we spend? And we like to add adaptability, everything that's going on around your business model. Okay, so four things. First, one, desirability. Do customers want it? So that's the front stage of the business model and the value proposition canvas Things related to the customer and how we create value for customers. Then feasibility, everything on the left hand side of the business model. Canvas the backstage, as I like to call it. Can we actually deliver? Can we get access to the key resources that we need? Can we mass produce that product that we have in mind? Can we build that platform? So all of the key resources, key activities and partnership relationships that you would need and then the viability, can we earn more money than we spent? So you'd have to have pricing assumptions, revenue assumptions. You need to find evidence for that. And then cost assumptions. Often the teams we see, they don't actually test their cost assumptions and how much it costs. And then it's more of a lucky punch that you make more money than you spent. You need to start testing both sides of the revenue equation. And then adaptability is the market timing, right? Everything that's related to the environment of your business model and value proposition. So you do not design a business model in a vacuum. And though you do want to focus on customers, you still want to ask yourself some questions around competition, but then in particular also around technology changes. You know, if you're building something now, does it hold up in the future? And market changes related to customer trends. Okay, so before we go to just quickly show this slide, but then I want Kavi to launch a poll. Why do we need to look at these different hypotheses? Well, very simple if we're going from idea to business. I drew this before with the search and execution. Now I added a line to it. You know, high uncertainty. Our only task as entrepreneurs and innovators is to reduce uncertainty. And this is basically the drawing that I show executives and senior leaders around the world. It's not about testing per se. It's not about learning and failure. It's about reducing uncertainty and risk. And to reduce uncertainty and risk, we first have to make uncertainty and risk explicit. And we do that with our hypothesis that come from the four elements that we looked at. Kavi, can you launch the poll around the question what types of hypothesis these participants have tested first in their last venture or project in an organization?

Cool. I'm going to launch the poll now. There you go. So to everybody, you can see it. In your last project or venture, which of the following was tested first? And you can select one or more of the following. Desirability, feasibility, viability, or adaptability.

And as a repetition, adaptability, if you forgot, is everything related to the business model environment. Viability is revenues and costs and spreadsheets don't count as Viability testing, that's not evidence, that's just a fantasy. Feasibility is related to R and D and can we build it? And desirability is really evidence that you are building something that people want.

So I'll give about another five seconds for people to submit their answers and then we'll read out the numbers.

Very curious what the result is here. People were on it.

I'm going to close the poll. Here are the results. So desirability 61%, feasibility, 31%, viability, 17% and adaptability at 4%.

Great, thank you Kavi, for sharing this. And my assumption here is that we actually have more experienced people in the call than beginners. Because in most companies and even ventures, I see way too many people starting with feasibility. Can we build it right? And then, you know, engineers and platform people, technology people, very R and D driven, make experiments around the feasibility of the thing that people want to build, products or services. So my assumption here is that people are more experienced than still joining a beginner's call to a certain extent. We'll see that maybe in the questions afterwards. Okay, so we did our little poll. Now the question of course is are all hypothesis equally important? And that's not just type of hypothesis, but in general the hypothesis that we come up with. And we'll do a little exercise around this in a bit real world case. But first, simply to show you what I mean is that, well, when you ask what would have to be true for your idea to work, you're going to have many different hypotheses, many different things that you're assuming around your business model, value proposition and environment. Now most important, and this is a step that I don't always see teams doing, nor in startups, nor in, nor in established company innovation teams. Is most important thing here now is to prioritize what is the most critical assumption, the most important hypothesis that you're going to test first. And the way you want to frame this is that if that hypothesis turns out to be wrong, then everything else you're doing is irrelevant. I'll give you an example from my little world as an entrepreneur. When we started out and said we're going to build software for testing business models, we launched an iPad app. But before launching the iPad app, we asked what should we test first? What we should have tested first, we didn't. We got lucky is do the people we're targeting actually have an iPad? Well, we didn't do that. It turned out we were lucky. All of the executives that we were targeting, they had an iPad and those who didn't, many actually bought a 5, 600 dollar iPad. I don't remember how much the first one cost, but for a $30 app. So that was pretty interesting. But in general you need to ask what's the most important hypothesis to test first. So while I'm seeing a lot of teams now testing, they don't always make explicit what do we need to learn first? So that's a big, big, big no go. You want to ask yourself what do you want to test? What should you test first? So let's do a little exercise. And this is a real world case called Ceres Imaging. And this was a startup team that went through Steve Blank's class and they wanted to build a drone with a hyperspectral camera, fly over fields, collect images and sell those images to farmers. This is what you call precision agriculture. Okay. So they wanted to optimize certain data for farmers and they had a couple of assumptions around that. So let's just rough out this case. I'm not going to go into the details of the business model and value proposition, so I'm going to keep it very high level. They're targeting farmers, they want to sell them data and they say we're going to build these drones that deliver crop data. So zooming into the value proposition canvas, let's look at the customer first. So they had some main assumptions around the customer jobs that these farmers want to manage and treat their fields and forecast potential production. And they had two main things that they thought was correct for farmers, that they want to manage diseases and that they wanted a profitable forecast. So big pain or the diseases profitable forecast, harvest forecast important. And they would come up with this new solution technology driven to help the farmers with targeted interventions, that's the pain reliever and the gain creator is to improve and forecast the harvest. Okay, so that was their rough idea and they asked themselves what are the main hypothesis? So they brainstorm around all of the hypothesis and they came up with a couple that I'm just going to put up here around the business model canvas. And we are going to now look at these in detail and you're going to do a little exercise on your end, see how you perform on this. The question that you need to think about is what are the main the top two to three hypothesis to test first and how could you actually test that at low cost? So focus just on which hypothesis. If you were serous imaging, if you had these hypothesis, which ones would you test first? So I'm going to ask you to Think about this for 30 seconds. I'm going to be silent and write down the first one you would test. Which number is it number three? Is it number seven? Is it number five? The second one and the third one. And you'll just type this into the chat window if you want to participate. Which ones are the top three? Hypothesis? So you put the most important, the most critical hypothesis first, the second most important second, and the third. So which numbers would you test first? Let's do this another 30 seconds so you can read through it. And you just put up your first and second, maybe in the chat window. Cabby, anything showing up yet? Did people. Do people start putting something into the chat window which I can't see while presenting? Nope, not yet.

I think people are still thinking.

Okay, so it might have been a bit ambitious to put a minute. You have to obviously read through some of these. Okay, here's what is the hypothesis?

Here's some test ready?

I got some.

Yep.

So.

I'll read you that. I'll read you the numbers. I actually put the numbers in the top three in their number format. So some people say 8, 3 and 4 from your tiles. 10, 11 and 3, 4, 6. 10, 10, 9. 6. Okay, 6, 9.

Okay, let's look at the solution. And obviously this is not an exact science. Right. Everybody has to kind of figure out themselves as entrepreneurs what they would test first. But there's some general rules. So let me put in red what you shouldn't have put first, and this is the reason I put these up, is this is what the team started to test for first. Let me make fun a little bit of Stanford engineers, very smart people that I love to work with. But of course, you know, they want to build a drone. So if you want, you know, their biggest concern was that they can actually deliver the data from a drone and create value for farmers. So they intended to test these three things first. That they could demonstrate that they could make the drone work, that they could stick together the software and that they could present the data. Now, the challenge with that, obviously, is that it's very costly and very time consuming. So that's not the first thing you want to test because you first want to understand, are you even on the right direction? So those are not the assumptions that matter most. Because if some other assumptions are wrong, you shouldn't even be building this drone. And Steve Blank, who was running that class, likes to say, well, lean is not an engineering process. Keep your eyes on the prize. It was really not about building something. In this particular case, it was really about figuring out what really matters here most. So instead of testing feasibility, which was the case with this team, they should have started with desirability and turns out like these are brilliant kids, right? So they did pivot very quickly in terms of process and they went out to test the right thing. So 4, 7 and 8 would be more or less in the right categories. Definitely. Anything related to customer jobs, pains and gains is what you would want to go figure out first and as quickly as possible. So if we order these things, we could probably have it some list like this. Take a screenshot if you want to keep this in mind. But it's really about starting to test the most critical business hypothesis and in many, many cases, jobs, pains and gains. Not the solution, but the jobs, pains and gains of our customers. The problem, as some people like to say, it's also about objectives. Those are the things that should come at the very top and then you know, you can move down to how you're actually creating value and how they're going to use that. Definitely what's much, much lower down is anything related in this case to the solution. Building the solution. Now this seems obvious and trivial when I show it like this as a learning case, but many companies actually first start in their R and D departments figuring out can they build that thing before they really test jobs, pains and gains. And sometimes it's actually okay to start from a technology point of view, but but not when you're testing an idea. So in this particular case, well, after figuring this out, they did go back to the farmers, these entrepreneurs and the farmers, they talked about the drones and the farmers said, well, I really like your idea there, but isn't there a better way to acquire the data? I don't really care about that, but have you ever seen these crop dusters that fly over my fields? Wouldn't you want to maybe take those airplanes? They're already there. It seems to me like you probably don't even need a drone. This was a big aha moment for the entrepreneurs and they realized that there are a lot of these so called crop dusters out there, aerial applicator businesses that were already flying over the field. So you can see in that little image, we like to call this bricolage in French, cobble together a solution on those airplanes to start gathering the first data. So here the experiment that they conducted based on those much more basic, more critical hypothesis was to use the existing aerial infrastructure. So here now we have an experiment that's very cheap, very quick to perform and addresses the most important hypothesis first. And it turns out that they were so interested, these entrepreneurs, to use the existing infrastructure that they replaced in their business model, in their value proposition, in their solution, they replaced the drones with the existing airplanes. So big shift. But the learning point here was also really that you start with your most important, most critical hypothesis. And the most critical hypothesis is the one that if that turns out to be wrong, if you have to refute that hypothesis, everything else is irrelevant. So when we make this drawing, the biggest uncertainty is what we want to detect, and that's what you test first. That's the most critical assumption. When uncertainty is high, you mostly don't want to build something. That's why build, measure, learn can be a bit of a misleading way of wording things, because people then really want to build something. But you don't have to, right? In many cases, you can do a lot more before you go and build something. And the prototypes, the physical, technological prototypes that are usually costly and more difficult to make, they come a lot later. And that's something now we really see in companies that do this. Well, they don't build anything for a very long time. For a very long time. They test without building something. And that's even possible in B2B, even more so than anywhere else. Okay, now let's move to the experiments 101 and dive quickly into this. So first, quick definition, so we're on the same page. An experiment is simply a procedure to reduce the risk and uncertainty of a business idea. It doesn't have to be about building anything. In many cases, you don't have to build a solution to test your idea. It's really about reducing risk and uncertainty. And your experiments produce either weak or strong evidence that will support or refute a hypothesis. And we'll get to the weak or strong. Very, very important. And the experiments, some of them are fast, some of them are slow, some of them are cheap, and some of them are expensive. Okay, let's look at a couple of examples here. What is an experiment? And this is really the beginner's angle, as we did call this a testing 101. So interviewing customers about their jobs, pains and gains is an experiment. And if we have time, we'll conduct a little exercise there to see how good you are at interviewing. Then using more sophisticated forms of interviews by bringing storyboards of potential value proposition, or making cards of the customer's jobs, pains and gains, and asking your potential customers or stakeholders to force rank the cards to have conversations. Need to be very creative and not Just ask people things. But think about storyboards, cardboards, etc. Or bringing some first. You could call it even prototype. A spec sheet, well designed or brochure makes people believe that a solution already exists. So sometimes we like to call that a minimal viable product. Right. A spec sheet is a minimal viable product. You don't need to build something to get people to comment on a solution. Having a spec sheet or a brochure of a solution, making it feel real is already a pretty good way to go. But all of these three areas here are about what customers will tell us if they're interested or not. So this is really about listening to customers and they will tell us. But we know that what customers say is not necessarily what they do. So we want to look at another couple of experiments, very popular landing pages where we get customers or stakeholders to do something. For example, sign up with their email or click a buy now button or whatever. Then you'll say, well, But I'm in B2B, you know, I sell ship engines. I can't do that. Well, that means you just need to be very creative. There are other ways of doing things to also get customers or stakeholders to perform an action, to really show they have skin in the game. For example, signing a document, even if it's not legally binding. And we'll see an example there in a bit. And then of course, getting customers to interact with prototypes or minimal viable products, things that we have built more expensive and often at a much later stage, as we've seen before. Okay, so say and do are the big distinctions. Experiments that get people just to tell you what they're interested in is not as powerful as getting them to actually perform an action or even pre buy or thinking they're pre buying something. And we'll go into that a little bit more in a bit. Okay, so now let's look at the components of an experiment and look immediately at an example. And it's one. It's very intriguing because it's in a space where we might not be used to testing. So the starting point, very important because we see a lot of mistakes here. It starts with the hypothesis. The first thing you should be doing is prioritizing your hypothesis again, a step that we don't see teams and entrepreneurs do. First, figure out what you need to test first. That is your foundation, the first component of an experiment. What is it that you're going to test? What is it that you want to learn? Which hypothesis are you testing? The second aspect of an experiment is the experiment itself. How are you going to figure out if that hypothesis is true or not, if you can support it or not. And then you make explicit what you're going to measure and you make a threshold explicit. So let's look at an example here. Take our four building blocks of an experiment and map one out. And I'm taking one from Eric Ries's last book. Eric Ries made the lean startup and testing methodology popular around the world with entrepreneurs and increasingly with corporations. He has a wonderful example from the medtech area that he writes about in his last book, the Startup Way. So let's say our assumption is that we believe we're a medical device manufacturer. We believe that surgeons prefer our new medical device over older technology. That's the assumption, that's the critical assumption that we want to test. Now, what we're going to do to verify if that's true or not is we're not just going to ask surgeons, we're going to get them to sign a recommendation letter to their hospital administrators. So you're going to go to the surgeons and say, look, this device doesn't exist yet. But you said you're interested. Could you put a signature here to your hospital administrator recommending it? If we actually start mass producing, that means now the surgeons have skin in the game. It wasn't a big investment to say, yeah, I'm interested, but it starts to become a big investment if they recommend a device that hasn't been produced yet to their hospital administrators. The investment here is their credibility. And they will only sign even if there's nothing legally binding here, if they are really interested. Okay, so what are we going to measure? We're not going to measure what the surgeons say. We're going to measure the percentage of surgeons who are interested and sign the letter, the percentage of all of the surgeons we talk to. And then we define a threshold. Sometimes that's a guess in itself, sometimes it can be very, you know, a hard thing related to production costs, etc. In this case, we'll just say the percentage of surgeons who sign is equal to the percentage of surgeons who already bought a device from us. So if they're not as interested as for the older technology, then probably we can't really support our hypothesis here that surgeons prefer this new medical device over older technology. So this is a very simple way to experiment in a highly regulated environment, medical devices, and without building anything. So that's the key point I want you to take away from this experiment. Also, we did not build anything and still we really get them to go beyond just telling us something, they have skin in the game. Now I'm just going to reformat this in one of the tools that you can get for free on our website, the test card. So we like we created this tool for ourselves because we were not always good at strategizing with testing. Here in the test card we first make the hypothesis explicit. We believe that surgeons prefer our new medical device. Then we make explicit how we're going to test that. What's the experiment? We're going to get surgeons to sign this letter. Then we make the metrics explicit and then we add the threshold to know or to predefine if we were right or wrong. And the reason we put a threshold, even if it's sometimes a guess, is simply because otherwise if you don't put a threshold, everything goes right, everything goes. And we don't want that. Okay, wonderful. So the test card is a very important tool that we created for people to test in a structured way. Another thing that we see is that people just don't perform enough experiments. In particular beginners, also in corporate teams, they don't go far enough. So what you really want to do is create a rough kind of plan or backlog of the experiments that you're going to conduct for each hypothesis were the most important hypothesis that you have. Some of those, let's start with one. We're going to design and build. And again, it doesn't mean we're going to build something physical or a real solution in the case before we actually design a interview series that would end with an ask around signing this recommendation letter. Once we have this clear, we have our list of people that we're going to take test this with. We're going to run the experiment. And while we're running the experiment in particular, if you're a team of two to three people, you already want to start to design the next experiments because you, you want to make sure that you're always moving. And once you finish one experiment, that's when you start to learn. And you can see we followed here the build, measure, learn cycle from Eric Ries using adding some different words because it's not necessarily about building per se in the engineering sense of the work. Okay, so you would have several experiments running at the same time. And very important you need to go beyond the interviews. Okay, just very quickly, what are the characteristics of an experiment? I'll mention three because it will help us think through experimentation a little bit. I think my computer just got stuck here. Have these three, which is cheap versus expensive, fast Versus slow experiments and experiments that produce weak evidence. People saying, yes, I like it, and strong evidence. People, you know, signing up for a recommendation letter where they have skin in the game. So you might be asking, well, why would I do any experiments that don't give me strong evidence? Why would I do any expensive experiments? Because it's not that you use the same type of experiments at any time. What's important is that you keep this drawing in mind. Probably the most important drawing that I'm showing you, which is we go from high uncertainty to high certainty. And that does mean we start with very cheap and quick experiments because we might be completely off on the wrong track. If we're really innovating, we're very likely to be on the wrong track. And it's okay to be wrong because we didn't spend a lot of time on testing. And we can recalibrate and redesign our value proposition business model. The more we reduce uncertainty, the more expensive our experiments can get and the more we can spend time on them. We're still probably going to be wrong. We don't get everything right. But when uncertainty is a bit lower, we're not risking going completely off in the wrong direction and then we can really start as fast as we can. But nevertheless, you shouldn't go right away into call to action experiments if you think you're on completely. The uncertainty is very, very high. So we start to move towards, from saying, towards doing as quick as we can as soon as we feel like we, we're into the right direction. And the technical prototypes should come a lot later. And I remember working with a consumer group that just launched a product at the beginning of this year together with Apple. And in that particular case they were smart enough to not use their technology prototyping budget, which went into the millions before actually testing desirability without building anything. So they were very smart not to go right into tech prototyping. Okay, so the way to frame it here, and David Bland has a pretty interesting way of framing this. A lot of people, different people talk about this, but David Bland makes a nice case about low fidelity testing versus high fidelity testing. Your first experiment should be very fast, should be very cheap, low fidelity, very likely that you don't build anything. And the more, you know, the more you can move into high fidelity, the more you can actually build a solution. Okay, let me stop here with the last exercise and then maybe just quickly go into a new area that we've been working on. But the exercise here is around interviews. I want to do this very Lightweight experiment. Imagine yourself as the chief scientist at a company called Colgate, and you just invented tooth whitening device. So you're the chief scientist at Colgate. You invented this technology. Amazing. Tooth whitening, more efficient, more effective. You can make up the features. Just imagine. Customers. So you have this tooth whitening technology. You're getting out of the building to do your first customer discovery interviews. How are you going to test this? So what I want you to do now is ask yourself to test the new tooth whitening device. Write down two questions for your first customer discovery interview. I want you to type maybe just one into the chat window and then Kavi will read one out. I won't make any comments about it. We'll see afterwards a little bit how to do this. What question would you ask first to test this tooth whitening device? You can make up the features. You're just doing your first customer discovery interview. Let's think about this for 30 seconds. And those of you who are good at this should write a question into the chat window and prove that you're.

Actually good at this.

Scientists, let's go. I'll give you a minute to reflect on the question you would ask. If you were the chief scientist, you went out to do your first customer discovery interviews, what would you ask?

I'm going to start reading some of them out. Alex?

Yep.

Would you like to have wider teeth? Do you feel like your teeth could be wider? Would you like to have wider teeth? How important are whitened teeth to you? Do you have issues of teeth discoloration? Are you satisfied with current tooth whitening tools or technology? Do you want to have whiter teeth? Is tooth whitening important to you? What do you think about white teeth? Would you like to whiten teeth? Tell me about your objectives with your teeth. Do you want your teeth whitened? How do you feel about the color of your teeth? What are the biggest problems that you have regarding your teeth or teeth cleaning? Are you satisfied with the whiteness of your teeth?

Okay, caveat. Let's stop here. This is excellent. Awesome. And the positive thing here is that a lot of people here did focus on the customer first, not the technology. Even though I did say you have the technology. There is some space for improvement here. Red is not so good. Green is good. So many of you ask for opinions, and that is not ideal because when you ask for opinions, you get opinions. And people have opinions on everything. And it doesn't necessarily and probably mostly doesn't reflect what people do in the real world. So would you. And do you Think is not an ideal way to start your customer discovery interview questions. It's better to ask for facts. So when's the last time you googled tooth whitening shows if they ever thought about it before you triggered them? Because if you ask, would you or do you think people will very likely respond positively. But when is the last time refers to a concrete instance. Or tell me about a time when you struggled with your tooth hygiene. Now, maybe not necessarily direct the link to the tooth lining, but if you want to do that, you could ask when's the last time you talked to your dentist about tooth planning? You want to go for the facts. You want to go for the instances when you ask. That's very, very important. Just like we said, getting a surgeon to sign a piece of paper is better than them just saying, yes, they're interested. So here I think I heard a lot of room for improvement. If I look at the second one here, I think participants did pretty good. So you do not want to focus on the solution first, even if you have it in mind. So would you buy this solution? Do you like the features, etc. Is not the first thing to test because you would be framing the conversation around what you think is important. You want to go for the jobs, pains and gains first, a little bit like we've seen when maybe I lifted you in the right direction with the saris imaging exercise. Right? We want to go first. We like to call this the circle. Others like to call this problem validation. We like to call this the circle from the value proposition canvas. Jobs, pains and gains. What are the most important things you are struggling with? Tell me the last time you thought about tooth whitening, in which situations, why did it bother you? Et cetera, et cetera. So here you have a little overview of what you can improve, even for something as simple as a customer discovery interview. So you can imagine when it comes to the more sophisticated experiments, there's a large room for improvement. Now, just to make this very explicit, and I'm going to stop with this part of the slides and then Kavi will send you a link of some of the more advanced work we're doing around innovation metrics. One link to a blog post after this, I'm going to just finish with the importance of not getting stuck in testing. So people who perform lean startup, they often like to say don't get stuck in the meeting room or in analysis paralysis trying to shape your idea forever. But what's also important is to understand a lot of people get stuck in conducting and running experiments we see that much too often. You always want to come out and think about the actions you need to take. So there are basically four different situations in which you take action. The evidence coming from your experiment supports a hypothesis. Then you move on with testing. You go either to perform a more sophisticated test, so after interviews you should get people to do something, a so called call to action experiment. Or you might go on to test the next important hypothesis after you started with the most critical one, then the so next critical hypothesis, next experiment, then the second case, your evidence refutes the hypothesis. Now bad news is you're on the wrong path. Good news is, well, if you did it right, you didn't spend too much time and energy. So it's okay to go back to his square one. And this is what innovation is about. If you're doing everything right, probably you're not innovating enough, right? So in the case of having to refute a hypothesis, fine, you either abandon completely, which is also okay. Some teams should just shut down. And in corporate innovation, there are too many still walking zombies that should have project that should have been killed. Only 1 out of 25 projects actually go somewhere, which means you need to kill 24. That's about the ratio from early stage ventures. Or you need to substantially change your idea, change the value proposition, change the business model. And this is where we see teams struggling most. So they learn about something that works or that doesn't, but then they don't move up, you know, get out of the water again, take their head out and think about the bigger picture moving in. Levels of granularity is very important. Okay, then the next one is, wow, the insights are unclear, the data is pointing in very many different directions, which will always be the case. But sometimes you just can't see any patterns. So you will always have contradicting data. There is no such thing as a clear path in innovation. It's pretty rare. You're very lucky if you see that. But you should be able to read patterns. If you can't see any patterns, then you should conduct more experiments. So in the case of very unclear insights, you should be conducting more experiments, maybe more sophisticated experiments to get more insights. And then sometimes completely new things come up. So you might actually see that. Okay, the customer segment you were talking to is the wrong one. But you need to move from the salespeople towards addressing the CEO a completely different context. And in the case of new insights, everything is open. Maybe it's a new insight that will lead you to abandon your idea, to change it or to continue. So what I want to leave you with and then hand over back to Kavi. He's going to tell you more about the questions and how we answer those. Testing is not about experiments and learning. It's about taking action and moving forward from idea to real business case and ultimately real business. Our task is to reduce uncertainty, create more certainty, reduce the risk by testing and taking action. And it's only actions that get us to move forward. And we've lived that through Strategizer as well. We've had friends to strategize who always pushed us towards taking action and not getting stuck in the learning. And I think that's a very important lesson also for beginners. You need to learn and act every week and ask yourself what are the decisions we need to take every week, even when an experiment is not what you finished. Kavi, over to you.

Awesome. Thank you, Alex. So as we wind down the session, usually we take a few questions from the audience, but given that the hour is coming to an end, what we're actually going to do is address some of the most kind of pattern of questions that we're seeing in the chat box on the blog@blog.stragizer.com just a reminder to everyone that you will get a recording of the session, both a visual and audio video of the recording, and that will come out in about 24 hours. So be sure to check your inbox. And like Alex mentioned, I'm going to include links not only to some of the resources that you saw today, like the testing card, we'll also be sending you a link of the signup form for our testing course that's in the works. And like Alex mentioned, to some key blog posts that display the development of the work that we've been doing around innovation, accounting and some of the other areas of testing. So you'll get that in about 24 hours. And be sure to check your inbox, your spam box, or any of your promotional boxes that you may have based on which email client you use. Alex, thanks a lot. I think that's the end of the webinar for us. Thank you everyone for participating. We had a pretty big house of an audience today, so that's pretty cool. And I look forward to digging into some of the insights from this session as well. So we're going to sign off now. Good afternoon or good evening to everyone, wherever you are. Have a great day.

Thanks everybody. Bye bye.