Start Testing B2B Ideas - Strategyzer Webinar

Welcome everybody. Hello. Welcome to this month's Strap Chat. We're going to get started in a few minutes. We're just going to let people join the call, but very excited to have guests from Zapier and Health Catalyst today, as well as David Bland and Alex Osterbalder. One thing, while we're waiting for people to join, we're trying to make this as kind of interactive as possible and engaging as possible. So we've got a mentimeter in it. We've got some questions that we'd love to ask. The first one is what's one of the things that's holding you back from testing your business ideas? So if you could get your phones, you could scan the QR code or if you go to mentee.com, you can type in 701-82974 and you can start answering those questions. And we'll be using it later on in the session as well. So if you could do that, that would be fantastic. While we're waiting for people to join, what I'll do is I'll actually share the answers so we can start see those coming up on screen as well. Excellent. So while we are waiting, do you want to, should we get a hello from Alex? A warm welcome from Alex and David.

I'm looking forward to these, to the comments that are going to come up. So I already see lack of interest from testers. That's an interesting one.

David.

Yeah, yeah. Hey everyone. Up early here in California for you all. Eager to hang out and share some what we're learning behind the scenes with testing business ideas. So just happy to share everything with you all again. And I love this exercise when you ask this question. Sometimes I have people draw too and it's always so entertaining to see what they draw. It kind of makes the, the fear a little more comical because people draw like stick people not wanting their ideas and stuff too. So this is awesome.

Time commitment, there is a big one, right? Time commitment is probably the biggest preventer, at least in corporate. In corporate settings people don't have time. They have to run a business. They can't invent the future. They have to run a business.

Hello to those of you that have just joined. We've got nearly 160 people on the call so far. We're just going to give people a couple more minutes and then we're going to get started. If you have just joined, you'll notice that I'm sharing my screen. We're doing the Menti Meter, so if you could Answer the. What's your greatest fear about testing your business idea by going to menti.com probably on your phone, or you can do it on your computer as well. And type in 701-82974. I feel like I'm doing the Eurovision or something with the, with the. Calling out the numbers there. So, yeah, if you could go onto there, that will share your answers. And this is always a good fun way to start the session to make it a bit more engaging. Yeah.

And type in, type in where you're joining from.

Right.

Because we can already see people from Poland, Cameroon, California. Oh, that's David. That's David. Any other Californians? Nobody up yet. Norway, Germany. Excellent.

Vancouver.

That's early too. Good. So we hear sometimes people staying up really late to join people in Vancouver, you know, getting up really early to join Argentina. Good. Netherlands. We always have such an international group. So, so fun to watch.

Brilliant. I think we are now on the dot, we're going to start on Swiss time. So welcome everybody. Thank you so much for joining this session. If you haven't already, please can you go to. Go to menti.com and type in the number there that you can see on screen or use the QR code so that you can get engaged with this session as much as possible. Because you want to make it really a nice conversation both ways, not just us like, you know, doing all the talking, but you can get involved as well. Okay, next up we can see, oh, we've got the Strategizer series. Interested to know how many of those books that you've read. Let us know in the chat. That's always good fun. But more importantly, right now I want to talk about who's on today's show. So we're very lucky to have the authors of this book here, which works out very well, testing business ideas. David Bland and Alex Osterwalder. So very pleased to have them. And also on top of that, we've got very special guests today from Zapier, we've got Lars, Lars Vedo, who is a technical product manager there. And also we've got Eric Denner from Health Catalyst, who's a senior vice president now. And they've got some fantastic stories and examples of testing business ideas in the field that I'm very excited about hearing from and I'm sure you are as well. I know many people before the session were telling me how excited they were to hear these examples as well. And don't forget to check them out. You can follow them on Twitter and connect with them as well on LinkedIn. So we've already asked where you're from. Great to see you joining from all over the world. Really, really good fun. Let's talk about what we're going to be doing today. So first we're going to be going through a quick intro from David, setting the scene for testing business ideas. A little bit of important kind of theory that we need to kind of COVID Then what we'll do is we'll go into the Companies of the month. So that's Zapier and Health Catalyst to kind of move on. We've got Hot or not, which has been really good fun. We're getting a lot of really good. Oh, if you could put yourself on mute there. Good. We've got Hot or not afterwards and then we've got the mini assessment to kind of reflect on some of this stuff ourselves. And then lastly, we've got Beyond Customer Interviews, so we'll be going on a little bit further there as well. So that's the session, that's what we're going to be doing. First of all, before we go into it properly, I do want to mention the Testing Business Ideas Masterclass, virtual Masterclass we've got on the 18th to the 20th of May, and if you use this code, you'll be able to get a 10% discount as well. So join us TBIVMC10 and you'll be able to get a discount there as well. It's going to be great. Look forward to seeing you there. On that one. We talked about Mentee. We've done the what's your greatest fear? But continue letting us know because that's always good fun seeing that. But what I want to do is hand over to David to talk about the context for testing business ideas. Can I hand it over to you? I'll stop sharing my screen.

Yes, that's good. So, yeah, basically when Alex and I were writing the book, we kept having this conversation around, well, how do we visualize this? Because so many people are either doing a lot of business design, a little testing, or a lot of testing and a little business design. So I thought I'd just kind of sketch out the process and Alex obviously jump in here as I go. But basically when you're starting off and you have an idea, it's like, what is this idea? How am I going to possibly test it? And your ideas kind of need to get out of your head and meet other people. So your idea inside your head is great, but it's like this perfect little thing that no one else can touch and it doesn't get tested against reality. So it's a really kind of messy journey to how to make a decision on, you know, what to do. And so what we tend to do is jump on the business design side. First we talk about how do we ideate, not just with your idea, but what markets you're going to play in, you know, what are the opportunities. We create business prototypes, which is somewhat different than a prototype. You would hear from kind of like prototyping in a traditional sense. But we're prototyping out canvases, right? To understand visually from a storytelling point of view, what does that look like? We do some assessment, some high level assessment, basically, what are the strengths and weaknesses, what's the environment look like? And then make a decision of where to test. On the testing side, we have this process of going through what are the hypotheses. We use this little two by two called assumptions mapping that I help create, where we basically try to go through the process to see where do we focus for our testing. We design experiments that address our risks, the things that are most important, where we have the least amount of evidence, then we actually look at that and learn. That's an important part of experimentation, is learning from it. Then we go back into the decision making process. I think in reality what happens here is you have your business design side and you have your testing side. You go through this time and time and time and time again until you start to find something that might be a real business that you can scale. And I think Alex maybe can chime in, but what usually kills every new business idea is people try to scale too quickly and they don't necessarily go through a process. They interview five customers, they build it and then they try to scale.

Yeah. The only thing I would add is what I just wrote in the chat window there is that sometimes people, we know everybody, that sometimes you get stuck in analysis paralysis, business design, right? You think about your idea, you make business plans, whatever. But what we also see is actually data paralysis where people test all the time and they learn all the time, but they actually don't make decisions to improve the business model and kind of move to, you know, closer from idea to real business. So you have all kinds of things that get teams stuck. So what David said, going through this, this double loop again and again and again to move to the top, right, that is crucial. And there's no, like one moment and maybe Eric and Lars will talk about that. There's not one moment where all of a sudden you know, lights come up and you see that clear path. There's always contradicting data, but you'll get closer and closer with every single experiment. So the decision, and then moving to the next experiment, getting that data that is going to get you closer is crucial. And, yeah, it's just a lot of hard work.

Yeah. Thank you, Alex. Nobody rewards you for learning faster than everyone else. You have to put that learning into action, and the market's not going to reward you like, oh, well, at least you learned. You have to solve real meaningful jobs, pains and gains and actually put that into action. So thank you, Alex. The decision part is really important, but this is pretty much our guiding light here of how we address the process. And so quite often, you know, what we're doing is we're always extending the tools. Obviously, you know, I draw a lot of inspiration from my co author here, Alex, and we've been kind of layering in these themes of design thinking and testing business ideas. You have a lot of, you know, the desirability so desirable. Do they want this viability, should we do this? And sort of the feasibility, can we do this? Alex is also even extending this beyond into adaptability, but which, who knows, maybe the second edition of book will have adaptability. But for now, Alex, I don't know if you want to maybe explain adaptability to those who aren't familiar with the concept in the book.

Yeah. So you look at desirability, feasibility, and viability. It's about aspects of your business model. But when you design a business model, think of it like designing a house. You design it within a context. When you design a house, build a house, in most countries, you have regulations, you have neighbors. You might want to, you know, your house to look good within the context of neighbors. So adaptability is all about the environment of your business model. Looking at, you know, the regulatory context. If you're in pharma and banking, think Airbnb, they were betting on the fact, you know, that the regulation is going to change in their favor. Or Uber, or you think of the competition. You know, you launch something, is the competition going to adapt too quickly and kill you? So everything going on around your business model, designing your business model, house, you got to think of the context in which you're designing your house.

Thank you. And so really just, we keep extending, we keep testing. We can't necessarily have a perfect plan. We always test even with the tools. What we do is we like using colors. So we use orange for desirability. Often we use green for viability, blue for feasibility and either red or another color for adaptability. And we're always trying to push the conversation forward of, okay, what types of risk do you have? And basically trying to have that conversation as a team so that we understand what types of risks. Because if you have team members that have very, very different perspectives, it's good to come together and have a shared idea of risk. Because if you just treated this in isolation, you might think, oh, desirability is really risky. But you have somebody on the backstage going, no, we actually can't do this because of regulations. So in retrospect, yeah, the customer matters. But if you can't do it because of regulation, that's going to cause some issues. What we do is we have this two by two that I helped create called assumptions mapping. Basically, we have two axes. We have one which is the have evidence and no evidence. This really came out of a whiteboarding session with Alex at Stanford out here in California, when we were whiteboarding out the two by two. I tell this story a lot, but I had this epiphany where I was like this two by two. There's something really I'm struggling with. And he really, really helped out in that conversation. And then important or unimportant, and I know you might say, well, we only write the important things. Well, not everything is the most important thing. So when you're right, when you're pulling these over and you're looking for, okay, where do we focus? You know, and you're looking at the themes, the stuff on the top left, that's important. And you have evidence. It doesn't mean you ignore it. You just have to share and plan and make sure you execute on that part, because you have evidence to support. But the stuff on the top right where you. It's important, but you have no evidence or you have very little evidence, that's where we want you to experiment. Because if you're going to experiment, it should be attached to your risk and not just experimenting to see what happens or experimenting because it's fun. And Alex, I don't know if you've seen this, but when teams get excited about experiments, they're like, let's do a landing page. And it's like, but what hypothesis are you testing with that page? I love this as mechanism for helping teams focus.

And you got to help teams understand, really question them to figure out what's important or unimportant. You were talking about Stanford. Well, guess what? Stanford engineers will love to say, oh, can we build that? That's important. Can we build it or not? That's actually an important question. Well, turns out if you can build it but nobody cares, probably the can we build it? Was not so important. So first you want to understand, often we can't generalize, but you often want to understand, do customers even care? Do they have those jobs pains and gains? It sounds kind of trivial when we joke about it, but in reality, a lot of startups, they focus on can we build it? And corporations that build on R and D research and development, can we actually make that science happen rather than spending a lot of money? Ask first what needs to be true for this to work and the most important hypothesis will bubble up.

Yeah, thank you. This is sort of a high level intro. We're going to deep dive much more on this in the masterclass, but I want to throw it over to Nick because we have some amazing case studies and we have other stuff coming up in the webinar. So back over to you, Nick.

Fantastic. Thank you so much, David. You should be able to see that we've got the testing Business Ideas Company of the month. This is one of my favorite sections to hear the kind of practical examples that we've got here. Now, for the first one, I'm very pleased to be talking about Health Catalyst. So we've got Eric from Health Catalyst, who's a senior vice president and he's really got some great examples of doing this in the business to business context, which is something that we get asked for a lot at Strategizer. So I think it's one without further ado. Let's get into it. So, Eric, I'm going to hand over to you and by the way, we're going to try and take quite a conversational approach. So David and Alex will be jumping in as well. But I'll let you share your screen, Eric, and then let you get into it.

Give me just a second to get Keynote in sharing mode.

That's good because that way we can see the mountain background. I love mountains. It's good to see other things in Swiss mountains. So appreciate that.

Yeah, I told Alex this is the view out my office window at home. So it's a tough view, but I deal with it. Can you see my screen? All right.

Yes. Yes.

Okay, so let me tell you a little bit about Health Catalyst. We are a health data and analytics company where we ingest data from clinical financial operational systems and integrate that in behalf of the client and then help them develop analytics. So our value proposition is data analytics software, a platform and a variety of applications. We focus our attention on the C suite of healthcare providers and we do that through a salesforce, both in terms of new prospects as well as expanding the current relationships. And that happens to be where I focus, is the expansion of current relationships. So our revenue model is that we charge a subscription fee for access to the platform and the applications. In addition, we help clients with the implementation of the platform, the use of the applications. So we have a full time equivalent model where it's not so much we dedicate particular individuals, but they have access to a certain number of hours across the expertise that we provide.

So.

Sorry, I went backwards and didn't want to go backwards. So. So the question that we started having internally was, is there a way that we can expand this relationship? And so I had been familiar with Alex work for some time, so I reached out to him in behalf of one of our founders of the company and he put me in touch with David Bland. And so, so we engaged in a series of workshops. And it began when we, when we first started, David led us in a two day workshop and we had about a dozen ideas that came out of that effort that we whiteboarded. We tried to build a business model around them. We tried to identify the hypotheses that we had and how we would go about testing those. We then engaged in that testing process and it was largely a combination of prototyping some of the ideas both in terms of prototyping the business model, as well as a technical prototype for those that were more product oriented. And then we went out and tested them, demoed them to clients, engaged in a lot of different conversations. And these were with existing clients principally. We didn't really go after new prospects at this point. We wanted a little bit more of a safe environment within which to test. So we moved from ideate to persevere. And then out of that second round of workshops with David and the subsequent testing that we did, we eventually emerged with two ideas that. And I'm just going to talk about one of those ideas that survived to this point and take you back to the business model that emerged out of this. So the idea that we tested through these series of iterations was something that we ended up calling smart sourcing. And that is that in some cases we found that our clients wanted to accelerate the speed to value in terms of adoption of analytics or automation of analytic data for a variety of purposes. And often we found that we already knew how to do these things. And so rather than teach them how to do it, they wanted us to actually take over the function. So this actually has emerged to be, you know, over 5% of our revenue. And this started, let's see, David. We finished our workshops in the fall of 2019. So it's over about a year and a half. We've developed over a $10 million year revenue stream. This again targets the C suite of the healthcare organization, where particularly, and this has been accelerated by Covid, that every healthcare organization in the world is looking for ways to economize and get greater speed to value, particularly in terms of cost reductions, just because of the impact of COVID on their organization. So we certainly had that relationship. But what we began to develop with this smart sourcing idea is we would target data managers over various application areas or data domains. For example, one data domain where there's a lot of work, a lot of manual work to extract analytics from a medical record is in charge extraction, where you're producing registries for governmental entities or whoever. And what we found is that had become a fairly manual process, even in the midst of electronic medical records and electronic financial systems and so forth. So we began to test that idea, and we knew that it was desirable. So ours became much more testing of feasibility and viability. Could we make a business case out of this? So what happened was that data manager, what we ended up doing, as we proved out this business model, is we actually took on the team that was doing that particular function, like chart abstraction. An example would be we rebadged over 70 people and took them into our organization, took on the responsibility for investing in them with training, and turned them into data analysts that were able to do the smart sourcing itself within that client. But because we made it much more efficient, we also began to use these same data analysts to do analytic services for other clients. As we continue to expand. This required an investment in training on our part, a new activity that obviously resulted in increased costs in terms of data analyst salaries and the training itself. But again, it created this whole new resource for us of expanding the pool of analysts that we had to work on the rapidly growing collection of clients that we have. So that's kind of the long and short of what happened.

Quick question, Eric. I'm curious. Before you went to this approach of you had a funnel you tested, Is that how you worked before? Or did that require a bit of change in thinking to actually test to evolve towards this business model?

Yeah, so we had a long history of very creative coming up with all kinds of ideas, but we found that too often we would invest quite a bit of money in something and then in the end find that it really wasn't meeting a market need or it really was not viable. And so this introduced a whole new approach of let's manage the risk down before we invest a lot of resources into it. So it was a substantial change for us.

And I think I just want to emphasize how revolutionary this is because I've never seen, I've seen people outsource before, but I've not seen a company say to a client, we'll actually take your employees and bring them into our company. Okay. I've not heard any other company do that. And so you've taken the data managers that had health industry expertise but not much data skill. You then train them and then turn them into what you're saying. You're leveling them up. They're not just data managers, they're data analysts that bring real insights and then you can add them to your value proposition and they help you with the problem that you had of finding it difficult to find full time equivalents with industry expertise. So it's genius, really.

So one of the things that we noticed in the conversations over the years with our clients is they don't like to get rid of people, they like to treat their employees well. And so that was a wrinkle that came out of these conversations and testing. Various ways that we could do this is, man, we'll take on the function, we'll rebadge your people, and we'll give you an immediate 10% reduction in the cost. And then we're taking the risk of reducing the cost of that function and repurposing people to other tasks and upskilling them. So it was something that we hadn't thought of before until we started getting into the testing.

And what I love is that the business model shift came out of the testing.

Right?

Because sometimes people think, oh, it's always the idea first I need to validate. I'm sure you've seen this, David. Right. People say, oh, I need to validate my idea. But the reality is you adapt your idea based on the things that you see in the market.

Yep.

Yeah, really quickly. One of the things I appreciated from the conversation was just in the workshops together, we had all these different business models. I remember on the wall around your room. And we kept going from one to the other and presenting out and sharing out. And then we said, well, where are the risky assumptions in these? And we ended up doing all these assumption maps around the room, all these two by twos, and we're like, wow, if we're wrong here. It kind of doesn't matter what else is in this model. And I thought that was just a really awesome experience to help guide you all through that and then narrow in on. Okay, what's the riskiest thing? How do we go test that quickly and in your space. Right. Heavily regulated. I remember all these conversations about how to go test, and I'm really, really happy that we found a path forward, because I think some people would easily just throw up their hands and say, oh, I can't test this, so we'll just have to build the whole thing. But you all. I felt like we're really open to the idea of testing.

And, Eric, you've got. I think you've got a little bit to say about what the specific testing that you did was and like the assumption. The specific assumption that you were going to go for.

Yeah.

So.

The biggest challenge we had was the viability piece of this in that we knew we could do it feasibly. I mean, we knew technically how to do it, but we couldn't. We were really struggling with the economics of it, how to make this work such that we could motivate the client to transfer this function to us, but also help them recognize that we were taking a risk. So modeling the financial side of this became the big issue. And frankly, it was just kind of customer value scenarios that we were passing back to. We probably talked to 30 different clients, sharing with them different ways of modeling this. And out of the iterations of those conversations, we resolved on this, on what we have now of an immediate discount to them, with us realizing or taking the risk and realizing a greater return, as we showed that we could actually automate the function. So is that what you were after, Nick?

Yeah. And specifically the percentages, because you were saying that you were thinking that the client, you were able to reduce the cost overall by 10% for them, but then you wanted to be able to make sure. Do you want to.

Oh, yeah. We had an internal target that if we couldn't get to 30% reduction in costs, then it wasn't viable for us. And so that was part of these iterations, is showing that we could actually get to a 30% reduction internally to make it worth our while.

And I just want to connect for everybody who's listening, right back to what David showed at the beginning. That's that double loop. Right. You go to test, you get some data, and you go back to the red part. Right. So, you know, in innovation and entrepreneurship, you go through that double loop all the time. It's not, oh, I have one idea, then I test it and then, oh, it was great. No, you change and adapt. And while, you know, for people who do it might sound kind of obvious, we see too little of the adaptation.

Right.

There's a lot of testing, but the point is to adapt until you get something that really works. And that's why I love this case, Eric.

It's awesome. Well, David was really helpful. I mean, we wanted to kind of jump ahead, and David kept saying, no, let's go back, keep that infinite loop that he was drawing. That was really helpful for us to really get our head around that and not just conceptually how to do it, but the value of it in doing those loops back, back and forth between design and testing.

Brilliant. Thank you so much, Eric. It's been brilliant having you as an example on this. And we'd like you to stay on as well, because we want to have some fun with Hot or Not and that as well. But as we say, it's always great getting more kind of B2B examples. So I really hope that everyone appreciates hearing your example there. But now we've got another wonderful one. I'll just share quickly to set it up. The next company of the month is Zapier, so I'm very excited about this particular one as well. We've got Lars, who's a technical product manager there. Really, really good example. Fascinating seeing this as well. And one point that I do want to make before we go into it is that Strategizer, we now have our own platform that you can do a lot of this stuff on it yourself. But in this case, Lars used Coda to be able to go through this with his team because he was already using. But we obviously really highly recommend that you use the Strategizer platform. But I think it's good that it does show that when a company is creating their own solution, that there's a real need. So I think that's a good thing from Strategizer's perspective, for sure.

Alex, is there any validation? Sounds like validation for us. So that's awesome news.

Lars, please. I'll stop sharing my screen and I'll let you get into it, but I'm really looking forward to seeing this, and I'm sure lots of the people on the call can't wait to hear your story because it's fascinating the way you tell it as well.

Nice. Yeah, thank you so much, Eric. That was great. That's tough to follow. I think mine is a little different. I haven't prepared slides, I'll just be actually showing the work that we did right in Coda. So a little bit about Zapier, first of all is we're founded in 2011. I've been here around two years now, and we have over 400 employees and, you know, little plug. We're also hiring. So if you like what you hear, feel free to follow up. Check out our careers page. Lots of open roles there just to set the stage. Zapier provides automation solutions. We focus on SMBs, and what we really focus on is integrating different systems. We support over 3,000 integrations now and help people set up automations for their workflows. For example, say someone fills out a form on your website, you want to follow up with an email and then schedule a webinar. You might actually use two or three different systems to do that. Maybe your website's on webflow and you're using Typeform to take the form, and that goes to a Google Sheet. And then set up Google Calendar and Zoom. We can actually connect all those systems into one workflow. And so what this project was working on was our Chrome extension. We have a Chrome extension that was built during a hackathon many years ago, actually in 2016. And since then, it had kind of bounced around the company. And lots of people had all these really interesting ideas of what it could do, but we didn't have a dedicated team working on it. Back at the end of last year, my boss and I, we took the masterclass and decided to spin up a new team, which we called Experiments. And the Chrome extension was our first experiment. So with that, I'll share my screen and show some of the work we did. Great. Everyone can see the project plan. Excellent. So what we did was we span up this new team and we basically dedicated Q1, you know, the first three months of the year to trying this experiment out. And we broke that into these three months of work. And each month we met with our stakeholders and we talked about the hypotheses we wanted to test that month, how we were going to test them. And at the end of the month, we reviewed those results and planned for the next month. And what we started with was our business model Canvas. And as they were saying, I kind of ported the stuff into Coda just because that's where my team was working. And this was a new concept. We had two goals. One was to figure out what to do with our Chrome extension, but also to see if this is a new framework that we could use for testing ideas of the company. So I had to educate my team. We had two engineers and a designer that I worked closely with. And so I kind of put this together and linked out to David and Alex's YouTube videos about these different concepts so that it was digestible for my team. And then we walked through these series of exercises as we were deciding what to focus on first. The Business Model Canvas. Here we really focused on desirability. We had a lot of concepts that we wanted to test the feasibility of, so we did build some prototypes and check that out. But on the revenue stream, we knew that this would be a companion to Zapier. So we weren't too focused on that in the beginning. But desirability was our biggest thing because we're builders, we love to build. We were building prototypes every hackathon, and we just wanted to know which one we should really lean into and bring to the market. So the next step was we went and we spoke to a bunch of users. We spoke to potential users after. On the Business Model Canvas. We had narrowed down what users we wanted to focus on, and it was specifically folks in operations, because they were looking for ways to streamline their workflows, save time for themselves and their team, and we wanted to make that easier for them with our product. So we went and did a bunch of user interviews. We did about 17, and then we actually followed up with users as well. And this was a great tip from Alex and David during the masterclass, which was to, you know, on the right here, we have the customer profile, and we would actually show this to people that we were interviewing after we had had a few interviews, and after we had a draft of this and we'd validate that this was, in fact, you know, the gains that they experienced at work and the painful things that they were experiencing and the jobs to be done. And after showing it, I was a little nervous to do that the first time. But showing it to users and getting their feedback and walking them through it really helped us learn more about what was the most important and what language they used to describe it. And then we ended up using that language both in the product and on the landing pages. So after we had that customer profile, we started to look at our value proposition, and we started to get quite specific about what we were trying to solve and how it mapped to the different pains and gains. And there's some other pains that are on the table that we'd love to explore in the future as well. But through the experiments, we're trying to quantify what would be the most Impactful. And we're looking to not just help people automate their workflows, but what we call our mission is to democratize automation, to bring automation to the masses. We're not trying to make automation, you know, 10% better for the few people that have figured it out. We're trying to help more people figure out how to automate their work. So a lot of our experiments were around there, how we could make it more approachable. So after we had the business model canvas and the value proposition map, we wrote down all of our hypotheses around those. And I don't have that graphic, but basically, you know, stickies around both of those. Actually in this one you can kind of see it, some stickies around each one of those sections and what our key assumptions were in those particular sections. And then we dropped those onto the one by one grid. And this was really interesting because, you know, we had lots of opinions and not a lot of evidence. And so that was what we really wanted to lean into because we wanted to work with our stakeholders, our higher ups, and, you know, decide collectively whether we should build a long standing team to work on this product. And we started to see what was really, really important to the decision that we wanted to make at the end of the project, which was do we form a team to continue to work on this? And there was lots of other questions that felt really important going into the project, but ended up not being the showstoppers for whether we go or no go on the project itself. So after we did that exercise of getting all these hypotheses onto this one by one grid, we had this humongous list of hypotheses and these are what we reviewed every month to decide which ones to focus on. And this is where our leadership was actually really, really helpful because we were just really trying to focus on the core value props that we were considering bringing to market and trying to not dilute that message with other ideas. So each month we chose what ended up being three. We were more ambitious to start picking four to five, but it ended up that we could really execute on three. And we went quite deep into each one. Each month we had our three hypotheses which we would test out. This one, for example. We believe users want to see data from Google Sheets while they're doing their work. Someone might be filing expenses, I'll give a quick example of that in a sec. And want to see data from a sheet, but they don't want to go and open up the Google Sheet, log into that Account search for that sheet, find the specific row that they're looking for. So we wanted to test that concept. And how we did that was by building a landing page. Did a couple other tests as well, but a landing page was one of those, and then circulated that landing page across social media, across our community of users and different experts that use our platform as well. And we ended up with this learning card, which is, you know, it's in the desirability category. Do people care about seeing data from Google Sheets? This is the test. We're going to share a landing page with our super user community. And this was our observation. So we shared it with that community and online, various different platforms. And we ended up getting a nice conversion rate on that 15% conversion rate. And then we followed up with each of those signups to do interviews. I'll show you what that looked like. This is just the post in the community improving our Chrome extension. Hello, super users. This is what we're working on. We did this single mock up, just a quick example. And you know, it's surfacing data from other systems while the user's doing their work. And that helped us validate that, that was getting some traction. You see, those conversion rates and those types of really small facts were what we were looking for. And so at the end, this is, you know, the framework that we learned in the Masterclass. This evidence scorecard was really the end goal. This desirability category. You can see what evidence we gathered. Each one of these pieces of evidence was backed by hypotheses that has a test card. And so when we, or when leadership looks at this, they're like, oh, okay, our value proposition is around this core concept. What kind of proof do we have? And they can go and drill into each one of these things and see how well that actually worked. Oh, there's like 15% conversion rates on that concept. And so to sum that up, what the end result was is we validated this concept and we're still working on this. We're in month three right now. Now what we're doing actually next week is launching an MVP of this concept. You saw this screenshot is a full, fleshed out, fancy example of where we could get to. But we still wanted further proof. And some of the best evidence is when you ship it to customers and see how they use it in production. So here's a really quick demo. Imagine that we kept our expenses in this Google sheet. And imagine I wanted to get a better microphone, which, you know, if I'm doing more webinars I'm hearing all these clear voices, maybe I should, but I don't know if I have enough room in my budget left. I can go and open up our chrome extension and I can say, you know, how much budget do I have left? And I'm looking for 180 bucks and let's see what it says from the sheet. Demo Gods, be kind. Yes, that worked. So it went, it looked up that Google sheet. I see I've got 250 bucks left. So I could go and submit this as an expense and hopefully my team would approve it. I didn't have to flip tabs. Keeps me right in my workflow. I didn't get interrupted here. Don't have to go find what sheet it was in. So we're about to ship that and see how users enjoy it and see if we want to invest more and go further down that path.

I don't think David's sponsored, but I think that's the mic you're using. David isn't.

Is.

That's why.

And he's sounding Good.

Question. So, you know, you really applied the things you learned at the master class really nicely, put them together very systematically. What would you say was the biggest, the hardest thing that you underestimate? Right. Because there's one thing, you know, learning it at a masterclass and then applying it in a real world context, what is the thing that you didn't expect to be as difficult, you know, when you, when you started to apply the methodology?

Yeah, I think one of the hardest things was, you know, moving really quickly. We had, we spun up this new team, we had one new hire as well. And like I said, we were really ambitious and we thought we should test four or five or six hypotheses each month. But it was in around month two when we got into a groove and we had some user interviews and some people we were following up with that went really smoothly. And once we started getting landing pages live, that was really helpful. But it took us a while to get all those gears going smoothly. We're only a 400 person company, but still we're working with other departments and putting things out into the wild and really wanted to make sure we're doing it well.

Awesome example. Two great examples is amazing. Love seeing these real cases.

And how much did you spend as well? Was it, you know, to do this? Was it expensive? Because I see someone asked that in the chat.

Okay, yeah, so.

Oh, I see.

What size is your team as well? And that's where the expense would have been. We didn't really spend money outside of that. We almost ran some ads, but we ended up just using our own channels and reaching out to our own customers via email and things like that, and leveraging our own social networks to get some traction that way, going and looking at Reddit and joining some conversations there. So in one way, we didn't actually spend any money except for the salaries of our employees. And it was actually part of the masterclass as well, how much you invest at what stage of the experiment. And so, you know, I was also doing other projects and my team was doing other projects. So not everyone was full time on this project.

One thing that you said, Lars, that I found really interesting was that you went, you could have used lots of different methodologies with your team because your team knew lots of different ways to do this. What you were clear on was you, because both you and your manager went to the masterclass, you went, we're going to stick to this methodology. We're going to do exactly this. We're not going to worry about trying to do other things and mix it up. We're just going to stick to this. And that helped you, right?

Big time. Doing it together really helped. You know, we had that shared language before we kind of started with the team. I did like a very quick summary of the class and kind of walked them through what we were doing and showed those deliverables and our meeting cadence and things like that. And yeah, there was this piece about focus that really helped. The fact that normally builders want to build and we just wanted to ship products to our users, but we actually really resisted that urge. And we made it very clear to ourselves and our stakeholders that we were going to experiment and learn and build prototypes and test them internally and test them with select users. And we didn't ship big updates to production for quite a while there. And that was another part that was uncomfortable because we were used to just continuously pumping out updates.

I think just one thing to emphasize for everybody watching and listening, in both cases, Eric and Lars sent. Eric said, we want to go and jump to do it. We see that in every company. Lars said, builders build. And that's the hard thing. Holding off of doing that, even like, oh, we need to build an mvp. No, you don't. So in both cases, there was a ton of learning that went on before anything was built. So I just want to emphasize this because, you know, people want to act too quickly. And the point is to go fast, not to get stuck in analysis paralysis or data paralysis. But, you know, there's the right speed for this kind of process.

And you've set me up perfectly there as well, Alex, because I think we're going to move well, first of all, thank you very much, Lars, for that as well. And we're going to move on to the next segment now, which is the hot or not. So we're going to have Lars and Eric stick with us on this because this is good fun. If you haven't already, please get your phones out or just go to mentee.com on another table tab and you can either use your phones to use the QR code if you open up your camera app, or you can just type in 701-82974 for this next section, which is a bit of fun. But as I was saying, Alex did the perfect segue there because the first one, if I move over now to Menti, the first one that we have is actually starting testing minimum viable products. So let's move on to that. So for this segment, what we do is we ask if you haven't heard of this one before, this is the third time now that we've played this game. This is it's just a fun discussion where we come up with a topic and then we ask you whether it's hot or not, just to have a little bit of fun, create a little bit of controversy. And we'll go through the panelists and then we'll ask you or then we'll see what we'll reflect back on what's being said. But I can see already it's pretty hot. People are liking testing minimum viable products straight away. Alex, I can see you smiling the most. So do you want to go hot or not on that? And do you want to explain it briefly?

So I'm laughing because I think and maybe people didn't read the sentence right. Start testing with minimal viable product. So because people love building and Eric Ries, our friend Eric, framed it as build, measure, learn. People take build to measure, take build to literally and start building stuff. You should not start with building. You need to test the circle you saw. Lars was showing the hypothesis around jobs, pains and gains. So for me, start with the building a minimal viable product means you don't understand how this works. So it's not.

Yeah, similar. The not part here, basically the idea of there's so much you can learn without building and even MVPs if you're doing something like a concoda or low code or no code, it still takes time, right, to put it together and you can learn so much before that through other methods. So I feel like we want to stop here. Well, we want to start with this, but you could learn so much more before building anything. So I'm going to go with not on this one as well.

Lars, how about you? Do you want to chip in?

Yeah, like I said, I definitely love to build, but I'd also go with knots because we did some feasibility testing of some of what we considered our best ideas and actually disproved that those ideas were going to be widely applicable. Had we reflexively gone into that and just tried to build those out, we would have spent a lot of time and not had as good results.

Great. Thank you. And, Eric, are you going to be a rebel on this one or which way are you going?

No. So in prior lives, I've been a Chief Information officer five times, and I wish I could kill minimum viable product.

Okay, let's move on to the next one. And we'll maybe start. We'll go around the reverse. So, customer interviews. Eric, hot or not?

I would say hot. I'm finding the sooner I can get in front of a customer and start getting feedback, the better.

Great. Lars, how about you?

Yeah, red hot. That was where we started, and it was so helpful. And then for the rest of the experiments, we were always referring back, we're like, oh, you remember Samantha, that company? She said this. Do you think this feature would help her or him? It was really helpful.

Thank you, David.

Yeah, I'm going to go hot as well on this one. Not just at the beginning, but continuing to do it throughout your process. I have teams that they do interviews every week still. And so it's not like you stop talking to your customers. So, yeah, definitely hot on this one.

We've had a lot of agreement on this so far. Alex, where are we going to go on this one?

I'll be the rebel.

I'll go with not.

I'll go with not. And of course, I'd say not for the way most customer interviews are done today. So absolutely agree with what Eric just said. Get in front of customers as fast as you can. But I think the practice people underestimate how hard it is to do great customer interviews. They ask for opinions rather than facts. Or Lars just showed you can put the customer profile in front of your customers to really understand priorities. So the reason I'll go with not is because the practice of customer interviews is really bad today. But obviously I agree with Eric, David and Lars.

Well, 157 people were with Hot and two people with Not. So you're not alone, Alex, you're not alone. So it's okay.

Story of my life.

And then last one, this one for a bit of fun. Lean startup, hot or not. David, do you want to go first on this one?

Yeah. I'm going to be kind of controversial in this and say not in the sense of. It's a quirky brand. I love it. I'm certainly an early adopter of it. And I really appreciate all of Steve Blank and Eric Ries's work. However, it became the underpinnings of everything else. So the language is used everywhere. But as a thing, I feel like I don't think it's hot. I feel like the practices have become kind of assimilated maybe to all these other different sort of methodologies. But I'm going to go with not on it just because as a thing, I don't see a lot around it anymore.

Eric, do you want to go ahead?

Sure. It's an over abused term that's become something to everyone and tons of definitions. So I'd say not interesting.

Lars, go ahead.

For the sake of a little controversy.

There we go.

I'll go hot for the exact same reasons that they go, not the fact that you kind of inspired this movement of being lean and scrappy. And I saw one of the questions was saying, we're not an engineering company. How do we build landing pages? There's tools. And we had that conversation internally where our leadership was saying, just go and do it. Use what tools you can to be scrappy and get it done. For that reason, I'll go with hot.

No, fair enough. Fair enough, Alex.

So it's a tricky one. Right. So I'll go. So we create a balance here. I'll go with hot in the sense that I think the term is abused. Right. And it's probably not the right one to talk to senior leaders because as soon as you use the word startup, some CEOs, I know, they shut off. But I think what's hot is that we're now getting a lot more professional. So what Steve Blank and Eric Ries launched and then, you know, we brought in some of the tools. Now it's maturing. So people who still use, you know, the foundations, just the foundations, they're not, you know, professional today because, you know, Steve's thinking evolved, Eric's thinking evolved, our thinking evolved a lot with new tools. So I'll say hot because we're getting a lot better at it.

Brilliant. Okay, well, that's been, I think, the least controversial. Hot or not that we've had, there's been a lot of agreement there compared to the last few weeks, but that was a lot of fun. We're going to move back over now to the next section which is the mini assessment that we've got. Just I think it's really important for us to share a little, have a little bit of time to reflect on how we can apply some of this stuff ourselves and how we're doing ourselves. So what we do is still on the mentimeter. Oh we're going to go through this. Oh I've always, I've had, always had the fire.

Most important animation of any presentation.

Right.

I've learned that from Alex. So for the testing business ideas mini assessment, if we can go back to the Mentimeter, what we're going to do is ask a few questions that are going to help us reflect. So. You can see here again. Excellent. So we've got the first one. I think a lot of these were from David as well but feel free to jump in Alex and everyone else. So the first one is when was the last time you talked to customers? So A, all the time within the last week, B sometimes within the last three, sometimes within the last three months, C rarely within the last year and D, never.

It's anonymous so you can tell the truth because we do know from practice that probably some of you didn't talk to customers. It depends on the framing, right. How it's the last time.

Maybe some of the people that voted customer interviews as not hot might have been the ones that are saying never. I don't know.

And the thing just to add maybe is that when you are an established company, the bigger you get, the less some of the leaders actually talk to customers. And it's not by bad will, it's just like you focus on the processes more. I see Eric nodding.

Right.

And it's the problem of the design of a scaling company. Even startups that scale, they move away from the customer is very dangerous.

Okay, next up, has your manager ever said I don't know in front of you and why is this an important one as well?

Yeah, I mean this is. It gets at the mindset, right? We try to show you tools and techniques and without the right mindset it's simply a check the box exercise to get to whatever you wanted to build. And that's really what we're focused on is helping change mindset as well. And I feel like this is a really telling one. I was at a large corporation in Canada and I stood up for their exec Team. And I was like, when's the last time you said I don't know? And there was like some people going, yeah, I say that all the time. Others going, I never say I don't know because people will think I don't know. So it's a really interesting question. And not that you should analyze all your leadership with this, but I love it because it gives me an idea of the insights of hey, are you going to be open to testing? Because if you never say I don't know, then maybe you're just going to go build it.

Anyway, here's a quick, interesting story. You know this one, Nick, right? I listened to Harry Stebbing's podcast, one of the best 20 minute VC and he had the founder of Segment on the show. It's a company valued at something like $4 billion. And the story goes the founder thought the team is working on a stupid idea, including him. And he didn't want to tell them, I know that this is a stupid idea. So he said we're going to launch a landing page to prove it's a stupid idea. And it turned out that became the founding story of Segment because it took off overnight. So the CEO wanted to disprove the idea and by actually doing that they found a multi billion dollar business. So funny story. Go listen to Harry Stebbing's podcast, it's awesome.

And keep an eye out because we're going to be making a video hopefully with Peter as well at some point. Okay, next up. So we're not doing badly. Let's move on. So do you need permission to run experiments? Yes, no or I don't know.

This goes back to the pirate story, right? Do you need to call yourself a pirate or rebel?

You know, I think of Twitter in this one because Twitter. Well, not when Jack Dorsey before him, when Dick was there, you know, there was no permission, but also there was, I don't know if there was a strategy. So it's kind of a trick question because you could obviously run experiments if you're giving full permission, but it should be connected to a hypothesis in a strategy. But this is really interesting. There's kind of a break on this one. Not lopsided.

Quick question to Eric and Lars. Maybe did you maybe not just need permission, but did you have to convince others to run experiments? I'd be curious about that.

I wouldn't say convince. So the sponsor of our work was one of the co founders of the company and I wouldn't say that we had to convince him, but it was a new practice we just hadn't. We hadn't formalized hypotheses and tested. We just didn't. We weren't formal about it.

So.

I don't know if that makes.

Sense, but that's great support.

That's awesome.

When leaders understand that, that's great. Lars, did you need to convince anybody? Your boss?

Thankfully, my boss took the course and was really excited about it, and so he did most of the convincing. I don't know how hard he had to convince folks when he was making that choice, but it was a change of years for us to say we're not going to ship for three months. That was kind of a big deal. And say we're going to focus on learning. So there was, you know, some, some convincing there for sure. But, you know, we run experiments and.

Such all the time and we've just moved on.

Lars, making the case to bring your boss to the master class. So thank you for making that case for us.

So when was the last time you killed a business idea? A, all the time. B, sometimes, C, rarely. And D, never. Already We've got sometimes seems to be fairly popular, but also rarely, few that are saying all the time.

Someone we.

Don'T kill, we wound them. That's very interesting framing. Go ahead.

I like to ask the question, how many of you are working on an idea that nobody cares about? And you always have a big room, Everybody holding up there, a lot of people holding up their hand because the processes are not designed to test and kill.

Right.

So unfortunately, a lot of teams know they're working on something nobody wants. So this is a. We use this as a metric, you know, how good is your kill rate? Or if you want to be more gentle, how good are you at retiring ideas? Killing sounds a bit brutal.

Eric, when was the last time that you killed an idea?

Last week.

Oh, that was good. There you go. And Lars, how about you?

Yeah, we've killed quite a bit of ideas. And you know, I take those as a moment of pride. It's like part of my resume, all the ideas I've killed. But we also have a code word in our product management weekly meetings, we say pomegranate when we think a project is getting over invested in and it just makes us take a step back and think about whether we should still be investing in this thing.

I love that. Where can you get that kind of insight? That's fun only here on Strategizer's webinars. And it sets us. We're not quite finished, but it sets us up well for the Next webinar, which is going to be about why it's key to kill your ideas in innovation. So, beautiful segment there. Or sorry, segue to next month's. But before we do that, we're going to wrap up the mini assessment. So that was super fun and it's great having Lars and Eric here to get their perspectives. But just one last thing we're going to move on to before we wrap up the webinar is kind of. We have this last segment which is going beyond. So we're going to go beyond customer interviews. I'm going to actually, I'm going to stop sharing my screen and I'm going to let. David, do you want to go ahead and share your screen for the last part of the session today?

Yeah, for sure. I love customer interviews. Again, I voted hot. I guess it was weird saying that. But they don't test feasibility, right? So don't try to test feasibility with customer interviews. Usually you're testing desirability front stage, maybe a little bit of viability. They don't take long, they don't cost a lot of. They generate kind of light evidence. But that qualitative evidence is great for insights into using the notes you have into your marketing copy of your landing page and your ads brochures. It's very much voice of the customer. So I still love them. But we can go beyond. And there's a lot. We cover a lot of these in our masterclass and in the book. We put a lot of time and effort, energy into applying a taxonomy to 44 different experiments because people would just do customer interviews and then build the whole thing. And when we asked, well, what else are you doing? They would just kind of shrug. So here are all the different things that you have, discovery wise and also validation wise. So when we think going beyond, there are all kinds of things we do to pair with customer interviews and that will deep dive in the masterclass. But I just wanted to stress that we do do more than just customer interviews and I'm sure, Alex, you'd probably reinforce that message as well.

Yeah, it's a great start.

Right.

But even in customer interviews, you can get more sophisticated to understand the priorities of the customers.

Wow. Thank you so much as well. That was a great kind of wrap up there, David. And before we finish, I want to say a massive thank you to Eric and Lars for taking the time to put these presentations really together. Even though you were going through your story, that is still fantastic. So if everyone can share their love for that in the chat, I think they'd really appreciate that. So, yeah, thank you so much from them. Thank you so much for David and Alex for joining as well. Before we go, there's always, we always want to get your feedback. So I'm going to launch the poll. Could you please share with us, on a scale of 10 to 1 in this case, how valuable did you find this webinar? 10 being extremely valuable. Absolutely. You'd share it with your friends, you say definitely join next time. And one being not so valuable. That was a waste of time, complete waste of time. So if you could share that in the chat, that'd be fantastic. In the poll that we've got going, I'm going to give you a little bit more time to vote for that and then we'll wrap up. But I'll share my screen again because what I want to do is we had a question about the code for the next masterclass, so I'll put it up on screen again. As you can see here, if you want to book yourself on the virtual masterclass, which is going to be from the 18th to the 21st of May, you can use the JoinustBivMC10 and you'll be able to get a 10% discount. It's gonna be fantastic. David's gonna be there, Alex is gonna be there, I'll be there. It's gonna be a lot of fun. So we look forward to seeing you there as well. I think we, I think everyone's voted, so yeah, if you could just give us your votes, that'd be good for how you found the session. Apart from that, I think we're pretty much good. We look forward to seeing you on the next one which will be about why killing ideas is key in innovation. We'll have Tendai Vicky joining us next time as well on 9 June. At the moment you can register or you can get notifies when that will be open. It's going to be open, I hear tomorrow. So if you sign up for the, for the notification strategizer.com webinars, we'll be able to notify you there. What's your final thoughts, Alex, on this? What's your wrap up?

I think this was awesome, right? To see how two teams, two groups, two companies have worked on this and I'm looking forward to Tendai the idea killer because he can't do innovation if you're not good at killing ideas.

Lars, any last points that you want to share? I don't know. Any shameless plugs you want to make as well? Lastly, before we go well, shameless plug, don't tempt me.

Please go install the Chrome extension. Try it out. New feature, that thing I demoed launches next week, so I'm really excited to see how people use that. And there's feedback surveys in there, so reach out anytime. I'm easy to find on LinkedIn and Twitter.

And lastly, we've got David because I think Eric's just popped off because we've gone three minutes over, excuse my non Swiss time this month, but I think it was worth it, right? For three minutes more. David, what's your final thought?

Yeah, I just appreciate everyone coming on Eric and Lars and sharing, being transparent and sharing what they're working on, how they apply these concepts. We do this all the time behind the scenes and it's just so rare that people are willing to come out and talk about it because we're always working on new stuff and so I think it's amazing and I just want to thank you all again because we all do this behind the scenes every day. But it's always great to take a step back and share with the larger community. So thank you so much, you.

It's been an absolute pleasure having running this session for you, this webinar for you all. I look forward to seeing you in the next webinar on the 9th of June. Take care everybody. Have a fantastic rest of your day.

Bye everybody. Take care.