DEMOlishDAY thanks to the Kauffman Foundation #LEArN #SXSWi 2015

So like people. Yeah. I mean I don't like to talk like that especially since I'm on the air being broadcast. But. How was, You know, me? Let's get this over with. I'll hear it. Process.

I'm great.

Yes.

Travis.

Okay. Yeah. Awesome. Sam. It. It's. That.

Because I.

Am very excited. So push the. So guide the teams. So guy. The team is telling a story specifically of like there was conflict with the team and one person wanted to push it and one person didn't. Okay.

So am I sitting up here?

Oh yeah. Okay. Gary. Yes, Sam. It. They.

Nice to meet you in person.

A little more pop.

I mean, thanks for coming out for it too. How did your kids yesterday?

I would say pretty good actually. A lot of strong teams.

Was it a like showcase or competition or what was. Yeah, I checked out the site so it's looking pretty good now. So it's kind of clean and I mean I didn't even know. I've only been in miss him for.

Here.

And like obviously I'm a Uber. I'm not connected and that's connected with different companies. But it's cool to dive in and explore where you guys are.

Yeah.

Well, I mean I think it's. It's a somewhat valuable spin on things. A lot of times people pitch their business like how it exists now or like and like the Q and A helps sometimes collapse like how you get there.

But I've been on for kind of.

Tell a little bit more about like where they actually like change their strategy. How long you sell.

The way we did we got together.

Oh, the house.

I haven't been there yet either.

Going tomorrow.

So I just. I know I just got in Yesterday at like 1pm there's just so much stuff to do. But I think I'm gonna go by tomorrow morning before I take off.

Yeah, I leave tomorrow.

Pretty good.

Are you.

Is there like. Do you have any specific goals? Like you're looking for first customers, potentially partners.

So. Only one person is watching. Melissa. It.

Radio ram.

Platform. All right, we'll turn the mic on rapid. All right. We have a hard stop at 6 so I can be a dilly dier. I'll have another water, please.

Good brother.

Sit here. Yeah. Jillian, not to put you on the spot. Nope. All right. Bring it in. Pretty close if you're here. It's not a huge crowd, but it's quality. Two minute warning for demolish day. One minute warning for demolish day. All right, so I'm going to talk for two seconds. Just say thank you to Kaufman up Global. And to Gary. Gary's going to tell you something about an incredible prize that we have for one of these demolish people. It's a little bit of a knock on demo days. There is a clear need for demo days and I love seeing them in the right place and time. But sometimes entrepreneurs are a little too stubborn and I was one of those entrepreneurs. So since it's Austin, since it's weird, since I'm weird, we're gonna have a demolish day. We're gonna crush things that people are holding on to, maybe we're not learning from. So yeah, I'm going to stop talking. Thank you to Kaufman, who lets me run these weird experiments not just in Austin, but in Kansas City. Take it away. All right, so Gary, we're going to, we're going to have you come up. We're going to demolish something. We're going to have a pinata. Brad and I actually went looking for a pinata and our demolished story was marginal returns looking for the pinata store. That was not planned, but that really happened. So at some point, you know, entrepreneurs need to be passionate and need to listen to their vision and themselves. But on the other hand, they need to listen to the market and the facts. And the facts were that we could.

Not find pinata store.

We did our homework. We were told where it was. Police officer at the gas station told us it was out of business. So instead we got a piggy bank. So instead of each team, kind of like being distracting and gimmicky and taking a swing out at each pitch, you demolish something once for a photo and for fun, if you're on meerkat, I hear that's like a new interesting thing. Take a video, take a picture. I think there's some a hashtag up there. Demolish day.

Pretty self explanatory.

And then we're going to get to the pitches. Not a VC bitch. It's not for money. It's not. Check out my great idea. We're trying to get entrepreneurs a little bit of a lean startup event disguised as demolish day. So we want people to talk maybe about an experiment they ran, something they learned and something that they improved as.

A result of their learning.

The more entrepreneurs that waste, less resources in my mind, the better. It's okay to have a vision up in. Sometimes waste is just waste. Introduce the prize and smash something and we'll get straight to the pitches. We got a hard stop at six o', clock, about five minutes per pitch.

Thanks, Adam.

Thanks Health Global. Thanks Kaufman.

It's Gary Shapiro here.

I head the Consumer Electronics Association. It's a nonprofit that produces the world's largest innovation event international CES. Although there is 3,600 exhibitors, we reserve part of the event for startups. It's called Eureka Park. It's the buzz area. The show has over 350 counties. The winner of U5.1 will get a free exhibit in Eureka park if you reject it. I really.

If you don't want it, leave.

No, that's fine. Thanks.

For the other giant up under your names. Remember them? I introduce but we have a celebrity. Rachel's going to introduce. Rachel Hart, you introduce a celebrity Catch.

Okay.

Well, I'm probably dating myself but we.

Have a real world rule.

It's probably now called the challenge. Celebrity guest judge here.

Wes. Wes.

I don't even know your last names on tv. So you're just Wes in all caps. And Wes runs a silo Peak dinner.

City and he's going to be one.

Of the guest judges at the so this is pretty tight.

And the other four people. Big job in Detroit. My favorite place in the world. So supposed to do.

Yeah. So get. So I won't push the. I won't. I won't push a hashtag. I promise time. We.

All right, everybody.

Everyone's alive. No. No liability. All right. Good.

Do you have an order?

I have. I don't know. Oh, last thing before we get. We knew. I knew there might be an issue with the pinata. Who's here from up Global about state cheese. There was a photo booth at the time. This seemed like a good prop to like to match demolish stuff with. So say cheese. At a photo booth last night I promised them if they gave me the prop it would get a plug. Carla, are you still back there? Say cheese dot com. Should we get this with the judges or with the people that are actually demolishing? All right, here you go. All right. You have about five minutes. As everyone can tell already I'm not super strict on time as the as in general sense. We are done by six. Each team will have about three or four minutes. My goal is to tell an interesting story and tell the audience how they can avoid the mistake that you made. And also to show that you actually learned a lesson and you're able to sort of negotiate the difference between pure passion and reality. Which is something I wasn't even that great at. Okay. Getting.

First of all, it's the worst. My name is Bruce and I'm co founder of course.

Pathetic.

Where we organize Free book courses in a course catalog format. And we're offering you prepass soon. That's a small pitch. Now I'll tell you problems when we started, let me actually start. A lot of you might know there's a lot of free online learning these days. Khan Academy probably heard of edX, Coursera Udemy, there's thousands, thousands of thousands of courses. So we thought, hey, it's a good idea to aggregate these courses. Not the only one to think that, but that's what we're thinking. So because Khan Academy right Now has about 15 million users you meet, this is something like 5 million registered users. So if you can get organized and send it to other sites that are very successful. So what we didn't figure out is that not all learners are the same because you have like casual learners and not so casual learning people who are going to college courses. So we attempted to build something for both and so we spent about a month figuring this out. And there's a lot of problems. And if you're a casual learner, you might go to Common Academy and watch a video for like five minutes on one specific part of algebra. And if you're a pro learner, you may take a moot course From Harvard that's 14 weeks long in computer science. So we built a site and we thought, hey, it's going to be great. You can have everybody in one spot. And what happened? It sucked, quite frankly, because when you go there, you couldn't figure out, hey, is this a small, is this a short course, is this a long course? And people who are looking for short learning for casual learning park the same for, look for more in depth. Kind of like casual gamers on your iPhone may not be playing Halo, but we spent a month of our time, more than a month of our time working on this and coming out with a demo. And then it didn't work. And so we decided to forego ignore something to demolish casual learning. And we came back and decided we're only going to focus on one thing, which is people who are going to take the higher group courses, full college courses, and that's all we're going to do. Now this sounds like, hey, that's no big deal, but there's a lot more casual learners. So. And there's a lot more courses on that are casual learning. So you'll see your competitors have like 18,000 courses listed. And you look at your site and you're like, oh, now we look inadequate because we only have a few thousand courses. They can say 18,000 courses. But I hope that that painful lesson is spending all that time and instead of just focusing in the beginning paying out because we have some services will come to us just because we're focusing on a higher ed learning and we look at our competitors and even though they have more courses, we think in the long run are our tokens will provide more value because we just weren't providing value. I'm not trying to turn this into the pitch. The bottom line is before you start something and waste your time, really think about your audience. And that's what we did not do. We thought everybody was the same group and we waste a lot of time doing something else.

So if this doesn't work and you.

Start something new, what would you do differently?

You would, you would actually go talk to people. Very basic set of lessons here.

Right.

So it should be not really great for everybody who probably knows this stuff, but you find out why they're learning, what their interests are. You know, if you're doing something independent, something else, a game and why are they playing the game. And then you go talk to them and you see what they need and you find somebody else who's in the macro market but not but split up to be a little more about it, find out their needs first. And that's the most basic. But I think a lot of people make that same mistake. I was just at another presentation who for and they did the same mistake we did. And it's sad to see people doing the same mistake that we did and wasting all their time trying to be everybody do everything. There's kind of a joke in startups are in politics.

So you can fool all the people.

Some of the time or fool some of the people all the time. And you should definitely pick for fooling some of the people all the time, but not fully obviously providing but it's the same basic point. And so that is the not most insightful thing. But a lot of them don't figure that out. And I would say when you go south by southwest, ucs new startups, they can really learn probably to not to figure out their customer first like you should. And that's it.

So that's exactly the format we want. I'm going to keep my own comments to a minimum but judges, let's have a conversation. Push them right away if you start in five minutes each. So it's like people want to know, like I remember fights with my own co. We want to hear the juicy stuff. So drag it out of them. They've been warned that you're going to really pull stuff out of them. Like one co founder wanted to change it, one co founder didn't. How did you learn it? How you know, how did you run an experiment? Like let's really pull some stuff out of them, let's pull the juicy details out of them. And when you're first coming out, you can say who you're, who you're from. So when you say something, since we didn't introduce all the judges in the beginning, nice job. And if you have one more minute of. While we do some logo stuff.

Yeah. So my name is Brendan Rice. I'm with Biz Dumb. We're a startup accelerator in Detroit, in Cleveland. My question is it's, it's great that you identified this as a mistake and you kind of found your niche and you're going forward. What are you doing? How are you operating your business differently now to make sure because it's still a process you're still changing on. How are you operating your business differently now to make sure you don't stumble.

Into a similar mistake moving forward? I can see Eleanor, I would say two things. One, we are, I said, asking customers, we're also thinking about money. Sometimes this is a funny thing too. We didn't think about where you're going to make money. I mean we didn't think of it at all. Obviously we did, but we didn't put enough thought about it into it. And if we were to have it again thought about where the money's going to come from, we would probably ignored some of those casual.

Some of those.

Casual learners because you can make money but it's not very easily. And we talked to a few people who have made money in like kind of a consumer oriented sites and they told us not doing fine.

So all right, next I need to be judged too.

I'm Paul Hyrun. I'm one of the co organizers of St. Louis One Million Cups. I started out in the corporate well as an independent business person. Got sucked into the corporate world. Was an M and a for Fortune 125 company then ultimately moved to mentoring startups. Had a few successful exits. So one of those startups that approached me after one day cups, he said, you want a cup of coffee? And I said I just had about five. But we went and talked and he explained his idea and I said this sounds great but you don't have a core value proposition. So I worked with him for two months to develop a core value proposition. I said, okay, now you're ready to start talking to people about it. So they went out and they had 3,000 interviews, actually more than 3,000 interviews of business people of different kinds to find out, okay, do you really have this problem and would this really solve your problem? If not, what would you like to see? So out of that three weeks of learning, a whole bunch of people talked to a whole bunch of people. We found that enterprise was not the market. Even small to medium business, which a lot of people said, oh, this would be perfect for, that was not the right market. We found that individuals were the ones who really resonated with this because they have a problem. And they said, if I use this and I you bring it into my business, eventually the business will use it. But they're not, I'm not using the tools they're providing me. Now it's all around intelligently scheduling a group of people. So we said, okay, great, we learned that, now let's test it. So we built an alpha and went out and got 300 people to look at the alpha and give us feedback. Now we're about to release the beta and one of the potential investors has had a successful exit, but his exit is in around a website, this information. But his statement was, you're doing this all wrong. You need to go out and talk to businesses, small medium businesses, and get them to sign up and pay you for this. And it's like you're not listening. We did that. We had 3,000 interviews. And the moral of the story is even if you have a hypothesis, you go out and test that hypothesis, you're probably still going to have to do another test. Because the whole idea of Lean Startup is it's scientific method, is it repeatable?

Right?

So for that Investor, we took three interns last week and made 300 more calls on small to medium businesses without tainting it to ask them, if you had this, do you have this problem? Would this solve your problem and would you pay for it? And the resounding answer was no.

So, so what's the mistake?

Did you have a bad investor?

So the mistake is not moving on from that investor. One of the three co founders said, well, we need to run the experiment. We need to run the experiment. My point was startups can't afford the distraction to answer a question they already have the answer to. If we didn't learn it for 3,000 interviews, we're not going to learn it for the next 300. So the mistake was going ahead and trying to do that, because I personally don't believe that investor will anyway, even though we went out and took three interns time over three days to do 300 calls. So the less it is, you're gonna fail, fail quickly, but don't go back and repeat that failure.

I was also excited for his story because also listening to something other than the market. So in this case it was an investor. In my case it was press. You know, so a lot of investing, a lot of founders, you see them going after press, going, you know, listening to investors, we all want to hear mentors and smart people's points of view and also. And take that in to be respectful. But to me, doing these customer interviews and learning from the market is the ultimate thing that should be driving you. You're chasing press, listening to investors, someone else, your mom, you know, something like that, it feels good. But ultimately I believe that that ends up with more waste. That's what we're trying to avoid. So in this case it was an investor. In my case it was press. I mentor a lot of startups. Now. I see people driven by press when they should be driven by solving a problem for a customer. That's what I've been very emphatic about over the past couple years. So I'm glad. Thank you for sharing that story about an investor. Push him a little bit for two minutes bonus and then we'll move on. Couple judges on the other side of the room just yell, real. Yeah.

Why?

Why would you, you.

Why not?

I come out of publishing, at least sell books a year before they're published, and if we didn't get enough sales, we didn't publish the book. And so when I, I left out a part of the story that was trying to be free.

Matthew, the founder, I said, you need.

To get out and test this with people. He said, we don't have a product, we have to build it first. I said, what are you going to build? And he was stumped. So I said, look, we can build a paper prototype. We can put on paper exactly how it's going to work, what it's going to look like, how it's going to feel. And I'm a UX UI guy anymore. I don't code at the code level anymore. It's all architecture, us. So we really developed a paper prototype that people could see. And frankly, people thought we'd already built it because we made it interactive. So they thought they were actually using the product. And that's when they got, they thought it was magical. It's like, can I have it tomorrow? It's like, no, there is no database. It's a chemo but having that functional prototype, as you said, that is a real experiment. And there's a movement in the lean startup world where people are saying when you go out, tell people what you're asking, tell people what you're testing. I think that's the dumbest thing in the world.

Science has proven before.

There's something called the Hawthorne effect. The Hawthorne effect, they went into a hospital and they said, well, this improve productivity. We paint the walls paint. They told everybody we want to improve productivity. Sort of painting walls paint. Productivity went up. Okay, we're going to test if it's going to improve productivity if we paint the walls green. They painted the walls green. Productivity went up. They said, okay, what's the ugliest color we can find? They found the ugliest color. Asked a bunch of people. They painted back. All the productivity went up. The Hawthorne effect says if people know what you're testing, they're going to perform to your what they think your expectation is. So with another startup I have that's around voice Biometrics, we wanted to avoid that so we went to Rose Hulman Institute and we had, we made a web page. It was a trivia contest and students could go on and answer the questions. The answer to Paul Wright ever Edwards in the and it would be drawing for 100 bucks. We used the voice signature service, voice authentication to make sure that they didn't do more than once a day. So at the end of the three month test we went back and asked them what were we testing. They said the questions data picture of the website, how the questions were randomized, how we did the how we chose the winner. Nobody guessed we were testing when people use the west angel to get to something they wanted to do. So we scored on that because they didn't know we were testing and we got real valid, real world results.

Excellent. Okay, I think we're good on time. We only three teams left so we can do a one minute bonus question. All right. I like it, I like it. I can I can iterate and be flexible like I teach.

Hi, I'm Gillian Morris. I'm the founder and CEO of Hitlist. We make a mobile app where you can list places you want to travel and we alert you when there are cheap flights to get there. We have about a quarter million users now. Started a year and a half ago, but kind of two and a half years ago I actually went through a lean startup workshop with Adam in Istanbul, Turkey. So I actually have sort of two things that I can talk about and I'm Curious. Which would be more useful for the audience. One was our journey to finding a backend co founder and the other is why we put it in our ideas.

Yes. Second one. Okay, I like it. That part I. That part I admire. All right.

So the first year, I went through a lean startup workshop. At the time, I knew almost nothing about the tech world, didn't know how to code, although I was teaching myself and recruited a co founder and we worked for a year diligently on a product. Now you hear how I said hit list? I think, do most people understand what Hitlist does? You list places you want to go, you get alerts when your chief likes to get there. So the first year that I was in this world, I spent trying to explain to people what I did by saying this was a different idea. So if you live in San Francisco and your best friend lives in New York and you really want to see each other, it's a flight search engine that will find you the cheapest place that both of you can fly to and meet up. So ideally, you know someone that you want to see that's more important than. Than the location that you're going. I thought this was a great idea, and probably about 10% of the people that I talked to also thought it was a great idea. I always wanted that. And it's so easy, just like Adam was saying, to listen to an investor, to listen to the press, and to completely lose track of the site that only 10% of the people that I was talking to actually had any interest in my product. But those 10% of people, sometimes they were really smart, sometimes they were influential investor types or something like this. Strangely enough, none of them seemed to want to invest in the idea, but some of them liked the idea. So, yeah, we spent a lot of time, a lot of resources, a lot of false starts in building this. We tried to be lean. We tried to experiment, we tried to talk to our customers, but we only listened to the ones that we wanted to hear. So it's nothing like getting something into the market at scale. We managed by, I can't even tell you how many. How much effort to get around 9,000 users, which was painful. And none of them really came back the second time. But that was sort of a registered user number, things like this. It wasn't until I realized, really started just talking to people over and over and over again and listening and trying out different phrases and trying out different ideas about a year. And we realized perhaps we weren't actually on the right thing. When I hit on the phrase, this place is wrong to go and we alert you and there are cheap flights to get there. It's a different idea, but it's just something that people get. I can say it. It's one sentence and all of a sudden everyone nods their head. It's not sort of, oh, my cousin might use that, or I know some kids in college who might be interested in flying to meet their friend in San Francisco or something like this. But it's something that pretty much everyone can get behind and everyone can understand. And even if you are the 1% who is not at all price sensitive about airlines, people are still value sensitive. So again, talking to customers.

Pretty important.

And listening to the market as a whole, not just ones that you want to do.

So when you pivoted, was it like.

You walked in the middle night and.

Start to pivot, or was it over time? How long did it take and what pushed you over the edge?

We came to.

Can you repeat the question, please?

Oh, he asked, was it a sort of overnight revelation of we needed to change or not? And I think it was a slow, mounting realization. And then it was sort of one day where we really just did say, let's put everything else on hold and start building this idea. And I remember the next time that I went to like a startup meetup event and I told someone what I was doing and that feeling, when I looked, I said, we alert you on their cheap flights to go to the places you want to go. And he was like, oh, where do I get that? How do I download that? And I was just like, why hasn't it. Why haven't been. Why haven't I been getting this reaction in the entire last year that I was doing this? It was the most exhilarating and overwhelming feeling of relief to know that it was actually something that more broadly, people wanted. Does that answer your question? We did actually end up sort of saying Trip Common was the old startup. We're like, Hit List is a product made by the Trip Common team. It's one of many things we're going to do to make it easier for you to travel with your friends. All this stuff, we got that out for about two months before we just. Wholesale manager said, we are Hit List. This is what we build. This is where we are, and we're better off for doing that. Didn't deluse the brand.

So it sounds like if you want.

To, I told you, you can also introduce yourself.

Hi, I'm Melissa from the Enterprise Network. So it sounds like you, like many entrepreneurs, went through A period of denial before you decided to pivot. What were some other examples of the false vanity metrics that you clung to during that time?

Like you said, the registered user's number.

Great question.

Registered users was one. But retention is really the metric that you need to look at. And this is something that our head of product now was at Pachay, Urbano and Glassdoor, two companies that had their own products, which went viral in their day. And in the case of Patient, Urbano didn't really retain users. Realizing that until you have a product that retains users, anything, any resources, whether it's time or money that you spend on marketing, you're basically renting your users.

You'Re not acquiring them.

That was a phrase a friend of mine used once, and I just think it's spot on. You're really just getting people's attention for a little bit. And as soon as you stop giving them the free swag, as soon as you stop giving them the, you know, $25 off your flight, sorry, off your Uber ride or whatever it is, it's going to be a lot harder to keep it. So, yeah, paying attention to attention.

Excellent point.

I wouldn't maybe end up. That's great stuff. That's what we're here for. Nice work.

And right on schedule.

A timer works for you guys.

All right. Hello, everyone. My name is Lakshika and I'm the CEO and co founder of Taste for Free. So we're a little bit newer than a lot of these other panelists over here and, you know, but we have learned a lot of different things around and, you know, we're really excited to present exactly what we've learned today. So the story I'm going to tell today is very interesting. It's about how our co founders decided to change everything under our rug on a day and it caused a really big fight and how we got over that. So this kind of started around three months ago when we were all ready to create our ui. All right, so we're three engineers and we think that engineers can do absolutely everything.

All right.

I mean, it's one of those things that we mainly have as a tool is that we think, oh, everybody else just has random degrees and we can do everything. So we decided, all right, we can tackle design. So we take a look at a whole bunch of different things and a whole bunch of different websites that are very similar to us. So Taste for Free is a disc centric rating system and we have lots of very beautiful pictures. So what we did was look at a lot of Galleries of beautiful, beautiful pictures. Looked at Pinterest, looked at MySpace as they changed, and we were like, you know, we can really work on this. We can revolutionize design. So what did we do? We're like, oh, we have to have horizontal scrolling. This is a new thing. I mean, everyone has vertical scrolling. Everyone does, but the cool new thing is horizontal scrolling. Yeah, I know. Look at those snarky horizons. Yeah, I get it. But we were left sema conditions. It'll be really awesome. Okay. Yeah. So, you know, we started on a design, and we're like, all right, this is great. This went great.

You know? Yeah.

We had stumbles. We were. There were lots of issues. There were lots of things that we were not getting because horizontal. So, you know, then we go through everything, and we're like, all right, I think we got it. Well, so it's about time that we're. We're doing one major component of our website, and we can't get it. We go through 10, 30, 40 different designs. We're just bashing our heads, you know? And we already coded a good month, right? We've already coded a good month with this horizontal, beautiful design that we're calling.

Right.

Yeah. So we're bashing our heads. We don't know how to get it.

And then.

And then we realized, hey, okay, so my space hangers on the sky. Okay, let's look at them. We just saw them, and they have rhizome.

Score.

Let's look at them. All right, so we go back and we're like, all right, let's look at it. Well, they went vertical. Okay, all right, so we. Well, it's MySpace.

They don't have anything.

They screw up one thing, screw up again. All right, so, yeah, micro lesson number two was when somebody else does something, you know, please, please look at it, because that's how it should be done. All right, so then, you know, we go on, and finally the demon comes, right? And we come back to one of our co founders. He's, you know, in a pile of. A whole bunch of scribbled designs and looking like, literally like Beautiful Mind, you know, the guy is, like, going crazy over here. What happened? What happened? Benny, we have to go.

Virgil.

Okay, well, we just spent an entire month and a half working on this design and how beautiful it is. You were the biggest proponent for it, and you wanted it so badly. What happened? We can't do it.

We can't.

We have to take out everything that we've done. We have to just grab it. And of course, you Know, it escalates and becomes a little bit more emotional than it should have. So that's the other thing. Don't get emotional with a particular part of your product because it's not worth it. So he slams the door, goes outside. My other co founder comes to me, hey, maybe we should look at his design. Ah, yeah, okay. All right, let's look at it. You know, we slowly pop a couple of beers and start working on it. Throughout the night. Throughout the night, we were done. We were done with the design because it worked so much better with a vertical design. And that's the one thing that, you know, we honestly, we were so blindsided by this idea that we could revolutionize design. We could change the way that people are looking at things. And you know, it is a gallery gallery. People like sweeping through different pictures. We could do this. And the reality is a, you know, when you're an engineer, you need to really understand that it's not about revolutionizing every, you know, you need to look at the other designs and what people are doing. And you really need to understand that. The second thing was, you know, if somebody else changes things, they did it for a reason. And the third thing was that make sure that you're open minded and really understanding of what your co founders have to say.

I'm going to ask one question. This is great. How did you define work better? And then I have a canned response here. First of all, this is for horizontal scrolling. So erase that. Show them what it is.

Make mistakes. Austin, Texas.

All right, so how did you define, define work better? And then I'm gonna also, Timothy is going to jump into this design conversation. We'll try not to go on too long. But how did you define work better?

Of course. Yeah. So what, what worked really well for us with the horizontal swirling was that our pictures were wider than they were tall. And so that would really looked really beautiful if you did have a really nice flow of horizontal scrolling. But like, what didn't work for us was this food card that we pop up. So essentially when you click on one of the cards for our food, you get a taste graph. So essentially telling you exactly how salty is it, how sweet is it, how mommy is it. So we take the dishes and define them with our taste wrap, but we also have a whole bunch of comments that people can leave about the restaurant dishes themselves. The commons tend to go a lot longer and there, you know, you can't. There's no horizontal scrolling for going longer this way. And we realized that, you know, there's A reason why things naturally.

I promise I wouldn't do this, but there were three questions and this. Was there a metric that went up because of parts that was sort of. I planted that question for a reason. So I used to get in fights all the time. My co founder, it happens all the time at every startup. People have opinions, that's fine. Test it against reality. And that's sort of where the hypothesis. That's sort of where I've become a nerd about lean startup. I believe that doing vertical scrolling will increase comments by 20% and then that either happens or it doesn't. So this is all the stuff I've been doing at labs. This is all the mentoring I do. So that. Was there a metric? And it's okay if there wasn't. Yeah.

So we just started three months ago. So for us it was, you know, when we showed it to more people, more of our friends, because we did do all right, do you like this better or do you like this better? And of course we had that with our friends and family and they always thought that it was a lot more smoother and easier to use when it was all right.

So since you're young, you get a break going forward. No opinions. We're going to hold you accountable to a metric. We're all going to hold her accountable to a metric. In a month, we're all going to come back down to Austin. Gary volunteered another booth. No, he didn't. All right, there are three questions.

Nathan with the Kaufman Foundation. So was this your first blow up with your co founders and what are you guys doing for the next time to better handle these types of situations?

So that was actually the biggest and the only real blow up that we had with the co founders. You know, what we really discovered from that conversation was but we need to have smaller discussions leading up to if somebody feels like they are not liking a particular part of the website or they feel not comfortable in a particular part of it, or they hear something that somebody else has said that we say it right away and we address it right away. So we're very at this point now we open up our conversations a lot more and make sure that everybody is heard. Every morning we say, say we have discussion, but we're now have discussion. We're actually living together now. So, you know, it's a little bit crazy. Let's start with life. I guess so. I mean, it's a lot easier, you know, because we are daily talking about what can we do to fix things. And no, don't let it Set on things. That's.

Chris Kovak, Kansas City.

You.

You had a hypothesis and you invested a lot of time developing it. How much time did you put into testing the concept outside the tracks? I mean, somebody who had maybe was.

In your network, but you weren't friends with them.

So is that a couple days?

Is that a week more?

So actually the first thing that we did even before, you know, we decided to quit our jobs or anything like that, we wanted to make sure that it was a worth investment. We were getting paid really well, and so we wanted to make sure that we didn't quit for anything. So we got really involved with the food and tech community here in Austin and we actually reached out to the organizers there and set up a game with them and saw is this idea actually something that people would want to see. And these are people that are interested in food and they're interested in tech, but you know, they're outside of our network. Right. We wouldn't have access to these people otherwise. So we really contacted them and got in touch with people like bloggers. You know, we talked to, we interviewed a whole bunch of people in the community and just to see if this is a thing that Austin solved for the walk. So we spent a lot of time, at least a good three, four months before we even quit. We went in January to actually do it. So we reality. But, you know, before that, we had been doing a lot of getting involved in community, making sure that this was something that people would really appreciate.

Go for it.

So you learned not to reinvent the wheel. Right. Do you think you have enough diversity in your management team and your co founders so that that doesn't happen again?

Yeah, so diversity is a very interesting thing because, you know, as one CEO and then we have, you know, two guys and three engineers. Right. But the interesting thing about us is that. So I was actually going into the path for product management, and so that was what I had studied up on and actually worked when I worked. The thing is that we understand that we have a blind spot almost. So we have surrounded ourselves with a lot of mentors and a lot of people that, you know, will think outside of the box with us, you know, because we, we can contact somebody who is in the food community and say, hey, how are the restaurant people actually thinking about this thing? Or how are these people actually? So, you know, we get a wide variety of interesting aspects to our product. And as we understand it, it's just three of us and we're a little bit blinded with this idea that you know, we're all about the metrics and we're, you know, a lot about the numbers and how we can quantify things. Right. But design and, you know, user experience is not necessarily. It is quantifiable.

I have someone. I just brought someone in that might disagree with you on that.

I don't mean to crash the party, but.

Hi.

So, hi, I'm Eric Reese. Yeah, was that. Yeah, I'm the author of the new startup, and we're just setting up upstairs for a lean startup launch party. So I happened to be coming by and I was asked to do this.

It was forced, so I might get yelled at later, but I don't know.

I don't even know whose party I'm crashing or what, but I was sat in the seat. But I'm like a startup advice robot. Someone says you can't. Can't measure the impact of design, then, like, that's automatically the thing flows and.

But I'm sympathetic.

I mean, first of all, this is very courageous. I have to take the thoughts and comments from absolutely anybody. Kudos to you for that. And here's the thing. The purpose of design, what is it? Why do we care about the design? I think, you know, in a business context, we care because we want to change people's behavior. The good design is one that's easy to use, that, like, people will respond to emotionally or like, ultimately, if you have a design, it's like, well, it's so good, but no aspect of it is measurable in the world. It's a purely, like, artistic thing that people experience internally, but doesn't change how they think behave, doesn't change what they say or do. It's like, wait a minute, What. What is this design, really? So I think a good design is one that changes people's behavior for the better. And therefore, it is metrical. But it's really, I think there's a temptation for people to be like, well, therefore, whatever makes the metrics go up is a good design. And like, I haven't seen startups go down that dark road and they're selling corn in no time. Like, it's bad news. We don't want to do that. You have to say, look, the purpose of our design, the experiments that we do, is to see, like, hey, is our vision really right? Let's find a better design that helps.

Us get there sooner.

So that's my two cents. So I'm the party crasher.

It's your party. After it's the one south by is one big party.

We're all party crash.

We have. So we built in a little extra time thanks to Gary for that. So we'll talk about that at the end. We have, we actually had some office hours before, so we actually pushed some of these teams to run experiments in between office hours and tonight.

I love it.

So we have you and who else? We had one guy that ran an experiment. Is he still here? All right, so we don't have to balance, we don't have to factor that into the event anymore. All right, so you get, we don't want you to get penalized. You get your full five minutes, then we'll talk about your experiment. We'll open it up for questions and then I think it's open bar at six. But don't worry. Eric downstairs. Okay.

Hey guys, sorry my voice is a little hoarse after three minutes to stop by, but my name is Ben Cornell. I'm the founder and CEO of a company in Chicago called Sherpa. Sherpa is evening package delivery. So users ship all their packages and other online purchases to our receiving facility. We upgrades to the receiving facility in Lincoln Park. We use our address and a unique code that we give them. We identify those packages that receiving, notify them and they can schedule a one hour window that same night. When we first started, we thought we would operate our deliveries on like UberX or lift model, using independent contractors with their own vehicles. They're out there doing rigs for postmates, taking Lyft rides and maybe they can drop off a few packages here and there as they go. We pivoted I guess to using cycle messengers actually. So we put a big bet on the cycle messenger community. They are full time employees of ours. We don't use an independent contractor model. We get a higher level of dedication feel out of, out of these messengers. They basically. It's a strong culture. I don't know if you guys have ever worked for Pisco messages or anything like that, but it's a society really, it's a way of life and it's an identity for you. So we sponsor messengers every year at SEGA Messenger World Championships. We have as our COO the 33rd fastest messenger in the world and we, we find that, you know, it's not, it's not a lot super in vogue. Obviously a lot of startups are participating in the on demand economy and we feel that we get a much higher quality delivery out of messenger. Also they're not paid very well so they work for us for 12 bucks an hour. Our average delivery is 550 for getting 2, 3, 4, 5 delivery is how the messenger, they're, they're sort of paying their own salary plus making some money for us. We operate full repair shops at a lot of our offices on street. And I guess, you know, for us, we feel like, you know, we had talked to a lot of customers before we started this. We feel like we knew what they wanted and what they had responded to was that they didn't feel that nutritional parcel carriers cared about last mile delivery. I mean, we've all seen videos of a FedEx guy throwing a TV over a fence or, you know, some angry UPS guy kicking your package. So what we wanted to provide was.

An employee in a uniform.

You see his face every single time he delivers a package. And we wanted to build really a culture around last mile package delivery beyond just offering as a service. So we tapped into, tapped into that key and I guess, like, see if there's a lesson there. It's, you know, think outside, sort of take the trends.

Did you go over the mistakes?

Yeah. What'd you demolish?

We demolished the independent contractor model with drivers using their own cars. We don't do that anymore. We use bicycle messengers. They're employees of our company dressed in uniform. Instead of going sort of independent contractor, we will in house spend 30 seconds.

How'd you make the decision? Was, was it painful? Was it easy? What led you? Was there an explicit experiment? Was it by accident?

You know, I met a bicycle messenger one day when I was having something deliberate and we had like a 30 minute conversation just about sort of the culture. And I was, I was so moved that these guys were like chefs. I mean, chefs. If, you know chefs, they don't get paid a whole lot. They have a very particular skill set. They pride themselves on working hard under extreme conditions. And it's all about identity and pride and frankly, bragging rights on these guys. So we pay them. I think we get by for a lot less than what we have to pay contractors. I mean, they are very satisfied. 12 to $15 an hour. And so we modeled it out and it looked great. So we put a big bet down on it.

Just bicycles.

I'm just getting a skeptical look here.

Right.

Yeah.

So we do have vehicles as well. We'll throw a bicycle messenger in. Bang.

As well.

Yeah.

We had a bad winter this year. We never miss the delivery.

Yeah.

Our plan to scale is that we don't want to open up a receiving facility in every city that we go. We see a model where we give an app on our back end. We love you Know put that out to small business owners that have late hours. 10:11pm they're open. That's the latest Windows 10 11. So they're operating an app. They operate as our receiving facilities. We have messengers are already out working doing dinner deliveries. So we will probably go to an independent contractor model at some point with couriers to have this excess capacity already out running other stuff at night. They packages for us as well. So there's existing courier services already. Why not use them as your backbone for delivery? He said why not use an existing courier service for our backbone? We feel that that's really our core competency is actually delivering packages. We don't want to outsource that. So we sort of built a courier company from the ground up by hiring again one of the best couriers who operated career companies in the past and just bring it all over sort of in house so we can, we can really put right about it. I would say so. She said once the opportunity caused Bicycle messengers aren't the fastest. I would submit to you that they are the most efficient and thereby the fastest way to get anything private city in the windows that we operate in. So we don't deliver it until 6pm So 6 to 7 is our first window. It's very popular. The city is absolutely clogged with traffic at that time. Our messengers work on these eight foot long damaged cargo bikes. They have a three foot forward cargo area and these bikes can penetrate city traffic in a way that cars just cannot. So if you've got long distances and you're trying to operate, you know, from these rush hour carriers. We did a race. We did a race between a car. 10 drops Friday rush hour traffic. We kind of made a big zone. We have a very large zone. We cut it down into the zone that one messenger would work. We're currently putting it up into a video. So put that online soon. Carvey courier from Sherpa.

Yeah.

Long story short.

Did you get your question in Nathan? Okay. Awesome. All right. So we were gonna end a little bit early for like the judges but we can. That stuff doesn't affect my hard stop. So there's nothing more than I love that someone that actually sometimes I feel bad like pushing people. Run an experiment. Run an experiment. But I blame this man. So anyone who actually does it, you know it Goldilocks, right? Like you push people or do you let them do what they want? So anyone who actually.

I can't make you run an experiment.

But if you do, I think you should be rewarded for it. So this guy Came in for office hours. I think he did run an experiment. You want to come up and did we go through that or am I wasn't you? Me, yeah. White shirt, white shirt, sunglasses. Sure.

I could talk about what I did.

All right, so we tried. We weren't going to get too nerdy on this.

Serious, right. I didn't expect this.

I told you it might happen.

All right. I'm Ron Benson and our company is in Views and we provide. We connect brands with audiences by having interactive audience engagement technology. So the base of our technology allows mobile to screen interactivity. One of the solutions that we've come up with is to have places like south by Southwest, large screens outside of where people are gathered, lines that stretch around box for people to actually be engaged by brands in interactive content on screens. Now, the idea I've been testing since I've been here with the demo that I brought along, and it's been resonating with the folks that I talk with. However, speaking with Adam, he's like, no, that's just not going to work. What you do is kind of like pushing people to give the answer that they're looking for a hobbit. So I went. So I went around today at his behest and spoke with people within this space here primarily and told them simply that that screen that you're looking at up here could be controlled by their phones. And if they wanted to do it, didn't prompt them as to why they would do it, but just to see if they'd actually take their phones out of their pockets. And to a person, everyone did. So I don't know if that proves that what we have might work. I didn't give them any reasons why, but it confirms what I've been thinking about to this point, and what I've been seeing from the folks that I've spoken with is that people are willing to actually engage with the screens that they come across if that opportunity is presented to them.

All right, so I'll say how we designed it. How would you have designed that experiment better? And then I'll tell them how I thought we. We could have designed it better.

I like it a lot.

I mean, it's a lot better than.

Just asking people what they want.

My question is, what did they think they were getting out of?

Yeah, you just told them to do.

It and they complied with you.

They complied.

So, yeah. My concern is, are they testing. Are we testing, like, your charismatic ability to get somebody to comply, or are we testing that the value that you. Your software might Offer is real, but.

I think that makes.

So that's like, okay, that's nice to have.

Sure.

But I like, I assume that when they texted the thing, nothing happened, Right?

Nothing happened. Yeah.

So, okay, I like that part of it.

So.

Because, like, if nobody had texted anything.

Nobody would have to apologize.

So, like, that's awesome.

So there's two parts of an experiment. What you're testing and how you're testing. He gets an A plus for how he tested it. We tried to tie it specifically to a unique value proposition, which was either cutting a line or being less bored by playing a game. We missed that part with the experiment. We weren't planning on ending with this note. We take chances, run experiment. It doesn't always work out the way it's supposed to. Nice job. We're going to run it again. This time where they understand the value proposition and why.

But I've already done that.

You don't get happy hour until you.

Come back at 6:30.

You get happy hour.

I've done that. And to a person, again, everyone actually takes.

Excellent.

Because they know that they can skip the line if they actually play an interactive quiz online using their mobile browsers.

And the takeaway, again, is to see that they'll actually take an action, not just say, hey, that's a great idea, and then not use it after you spend a year building it.

Oh, I've got a demo if anyone.

Wants to see it.

Nice.

All right, we can do the judging at the end. Let's open it up. You have questions for them if they have questions for you. Free for all for three minutes. I'll hard stop in three minutes.

Now.

Not.

Not to get a second bite of the apple, by the way. So I was just saying one of the most sometimes again about, you know, I'm not trying to. One very quick story. A lot of stories are very similar, like, hey, talk to your customers or something like that. One of them is actually talk to your competitors. We have about five competitors and we view them as enemies. Horrible people. They are, you know, bad people. So we would, you know, we check out what they're doing and, you know, work on what we're doing and hopefully we're better. But my two competitors did talk to each other. They didn't do each other's enemies. They did knew each other's competitors. I had thought about doing that because one of my competitors actually lived. I'm from Seattle. I wasn't living in Seattle, but I was back home. And my competitor works at Amazon during the day and he literally lived like 15 minutes from my mom. But I didn't go say hi to him, even though we're in the same building.

Two.

So what happens? Apparently he didn't want it to be a competitor anymore. He just wanted to get out of business. And he told one of our other competitors this and he offered his site for sale come for like some high price. What happens? Our competitor's like, I can't pay that. A month later he comes back and.

Offers his site up for sale for.

$1,000, which is nothing relatively to what he had and my competitor obviously what because he's not stupid. And I found out and I started talking to my competitors how I found out because I was like, hey you now this is slightly dark to you. Like what happened there? And so the lesson is view your competitors as competitors for sure, but not as enemies. And because maybe one day they will become your friends or sell yourself or cheat. And I, we lost out really good deals because I asked my story.

15 seconds each. What's the next goal and how. What experiment would you run to test that goal? Literally 15 seconds.

Next goal for schedule that I talked about is to get the data into a small cohort, get their feedback, massively feedback from them on their use. We are tracking and we built a system so that it tracks every click. So we're going to use metrics and numbers, not what the co founders think should happen.

Maybe end of that. Just getting from feedback scare you sometimes just getting just we're gonna get feedback scare you the way.

All right, so but well, but like.

You can't have it both ways.

People to experiment and talk to customers and say they can't be like don't do not too much. It's like, well how much exactly? Like I think the key is as long as you know, as a founder you feel like you are really focusing in on the 1:1 and like critical thing. You absolutely have to know that's the key. Too much feedback, too much data. Like too much noise can be distracting and it can make you feel like.

It makes, it makes it easy for.

You to kid yourself because you're like.

Oh, somewhere in this.

Like my own data is probably something that's helping me with the right. I'm doing a good job. Like that's what Google Analytics is for. It's always at least one graph. It's up to the right no matter how bad things are going.

Right.

So I could just like, you know, rather frame it as like get feedback and talk to customers. I try not to use that language anymore. It's just like, what do you really need to learn? What's the one critical assumption that needs to be tested? And like, can you force yourself today? That is the only thing I'm going to focus on. I'm not going to kid myself because like all these awesome banning metrics.

All right, on that note, how we run the experiment. I'll see you after class. What's the one thing you want you need to learn? And we'll really end on that one thing each 10 seconds or less. Microphone goes dead after 10 seconds. Go in order. Now you go serve it. Start with the one thing you really need to learn. Within two weeks, we're rolling out a.

Program to apartment buildings and wedding planners. So we're doing the same thing that we did with consumers is we're just, we're finding a lot of wedding planners, a lot of people who manage apartment buildings finding out what they're looking for.

One thing you want to know is what they're looking for.

We want to know what dad does.

Nice job.

Okay, so we just ran an experiment. He told us to go out and we asked a whole bunch of questions to 20 people and found out a lot of things, you know, more than. Because now we have the baseline and now we just need to expand past that baseline. So that's the next thing.

So I'm, I make hit list with some app where you list places you want to travel. We alert you in there cheap flights to get there. A core part of our value proposition is that we're actually going to get people to book trips that they would not have taken otherwise. And there's basically no way we can prove that other than, you know, people sort of, you know, anecdotally unless we're testing against the control group of our own user base. So we're looking at segmenting and essentially a B testing within our quarter million users. Essentially people with M and later in the Alphabet are going to be one group and you know, A up to that thank you in another.

And.

And then take a survey probably in a month and see if we can see significant differences.

The schedule main metric is people are invited to a meeting.

Do they ever turn around and schedule meetings?

And what does that look like?

At what point in the process do.

They see it of value? Do they probably three things before they start scheduling be where do they start after the first invite? Understand value problems. If not, what can we do to change that?

How much they would pay for the test that we, that we offer to get credit for our courses pretty simple.

And like, they doing the Academy Awards where they start playing music, they're pouring ice, it's time to go. Let's go. Thank you very much. Whatever you have here.

Thanks, man. That was awesome.

See you. Three viewers, maybe.