Pilots, POCs, and Trials (mistaken causality and never-ending pilots)

Hey Rob, how's it going, man? Good to see you. Yeah, you too.

One month when this comes out until the book comes out.

How's that? How's that going? I've seen a bunch of activity on LinkedIn and yeah, yeah. Just like people about pre-ordering.

Yeah, I guess I don't, I have no idea. I have no idea. Oh, one of our friends sent us a picture of him having bought 1,000 copies, which I'm pretty sure was AI generated because it had the, it had the delivery date in the past. It was like 2025. Delivery date, but when I first saw it, I was like, yes, I'm gonna, you know, we're gonna make it.

Yeah, we're going somewhere. Yeah. You know, I, I was totally fooled by that. Yeah. I looked at it, I was like, what are you doing? But he's also a partner at BCG. And so I was like, I don't know, maybe that's the thing that partners at BCG do.

Yeah. For a second I was like, oh man, I need to send him a, a like really big thing. We're gonna send him a really nice $30,000 on, yeah.

Yeah. Well, any, you know, just in case anybody didn't, didn't see any of that, if you preorder the book and then send us a screengrab of your purchase, you get links to Rob's Claude skill, which is a super helpful, like, sales call review skill and stuff. I'll put a link just so that people can see it. But anything you want to, anything you want to refresh on, on that one, Rob?

If you preorder the book and tell people about the book, it will make me feel less like I have wasted the last 3 years of my life writing it. And so for no other reason, perhaps, like, you know, maybe tell people about it if you care about Rob's mental health and self-esteem and things like that.

Speaking of self-esteem, this one, this was a personally painful— like, I felt like an idiot while I was reading this week's blog. And it's, it's so good. It's called Pilots, POCs, and Trials: Mistaken Causality. And never-ending pilots. This was a conversation we had in the, uh, in the Founder Community, uh, maybe a couple weeks back. There was some back and forth around like, when, when does it make sense to do free pilots? Um, I am the king of free pilots that do not convert into sales. And, uh, and so that's been, uh, this reading this, it was like, oh man, everything's so much more clear now. So, um, anyways, if, if the listener has ever done a free pilot that didn't convert or is wondering about when do pilots make sense, or this is a super practical, uh, one that I'm excited to dive into. Um, Rob, there's probably other context that you want to hit before we actually talk about that subject.

Yeah, this is, uh, so this is a chance to have two conversations at the same time. We're gonna have it at two levels. We're gonna have it at the, the let's talk about pilots, and then more deeply, like, let's talk about causality and the pilot conversation is the practical, it's, it's, it's the brownie that you, that you get. It's the, it's the, um, you know, it's the dessert, but the real meal is the causality conversation, which is what I'm most excited to talk about. And, um, yeah, and it's just like, this is one of the super common things that I got backwards. Like I remember personally going to like board updates or monthly investor updates and saying, I, it's definitely going to convert in, in June and like definitely going to convert in July and like August. I have no idea why it's not converting. And, you know, investors continue like, is this real? Like what's going on here? And so I've been in that place where you have pilots that are just not converting and it's usually because we're thinking about the entire thing backwards.

Yeah, let's, let's hit two parts of this real quick. One is you start this off with a question from a startup that you received. And just for context, so this is the type of thing that other founders are, um, you know, wrestling with. Uh, you're right, we are, uh, this is from the question that you received, so it's from another founder. Uh, we are getting into pilots and they are going well, users are getting value out of the product and usage is going up, but we're struggling to align on success criteria with our champions and deals are kind of dragging along in the pilot stage. What do we do? And we're, I can't wait to dig into that because there's like a, there's a fundamental thing at the base of that that's like buried in it, but that's like what it feels like to be in that situation. And then you do this, as you've alluded to, this super annoying thing, which is not give the answer directly, but you say we need to understand the fundamentals of what's going on here. And you write, when you understand this, you realize that fixing things that prevent purchases does not equal doing the things that cause purchases. So let's, let's start on that and like, tease the difference out between things that prevent purchases or fixing things that prevent purchases and doing things that cause purchases. Rob, can you just like go off on whatever direction you want to take that?

Yeah. So, so this is the kind of, so this is true when I talk about demos, when I talk about group demos, when I talk about steps in a sales process, which is that we conflate things that cause a purchase With things that prevent a purchase. They're just like two categories of things. There's a bright line between them. There are basically two things that cause a purchase. Do they have pull, AKA are they trying to do something and blocked by all their available options? That's the first thing. Second thing is, do we offer something that they believe fits their pull? It unblocks them. So they can do the thing they're trying to do. That's what causes purchases. There is nothing else in that category. In, but then there's a lot of things that can prevent the purchase, even if the causes are present. Now, if the causes are absent, none of the things that prevent purchases matter. Because nothing's going to cause them to purchase. So on the list of things that prevent purchases, you know, pricing. If you price really, really like ridiculously high, more than the, you know, more than the annual budget of Estonia, right? Like you might, they might not buy. They, they won't buy even if they have pull, cuz they're like, yeah, we can't afford that. Or if your sales process convinces them not to buy. Or if there's a variety of different things, if you just give weird vibes, they might not, they, they, they could not buy. So there's all these different things that can prevent somebody from buying. They can prevent a purchase. Causes versus preventions. The POC. So given that a proof of concept or a pilot or a trial is not on the causes. Then it's on the things that can prevent, because another thing that's on the prevent list is time, right? Time can prevent a purchase. And so adding additional steps to a sales process, adding all these things, like it, it is something that prevents. And so we can go the next step here, but you had something you wanted to add.

Yeah. And just to like emphasize the difference of this, you brought up one that was interesting, which was like, Um, SOC 2 compliance. Yes. Um, like, yeah, uh, that, like, compliance is a really good one where nobody buys because you are compliant, but they won't buy if you're, if you're not compliant or if, you know, you don't, you, you don't meet something that is on their, like, IT security or on their legal requirements. Um, and I, I remember in my outbound, early outbound, um, I, I used to, uh, so SOC 2 is less relevant for us. There's a regulation called COPPA, uh, Child Online Privacy Act, which it regulates, uh, kids and family media. And I used to include COPPA actually in my outbound because it was like, that's relevant. Like they probably think about that. And obviously no, like now in retrospect, like it's, it's literally like leading with something that would prevent rather than identifying the pull that would cause them to buy. And then as a one sentence response to a question, maybe if it comes up, then it's like, oh yeah, we know, we know about COPPA and we're compliant with it. And you know, like that's, but that's so far away from it. I don't know, Rob, like other things. Commentary on that or things you would add to—

Yeah. So it's like, like IT review, security review, like these things are things that will not cause them to buy. Right. So I had a founder reach out and they're like, hey, yeah, we don't really have a champion, but somebody passed us to procurement. It was like, okay, cool. They sent you there to die because procurement is not never going to say yes. They can't say they can only say no or not no, right? They can't say yes. And so there's all of these other things that will not cause them to buy and in fact will probably prevent you, prevent them from buying. And so there, there's those two lists of things. So if you, if the causes aren't present, they're not going to buy. If the causes are present, then, right, like the preventers or whatever we'll call those, the anti-causes, may prevent them from buying still. If you just focus on the preventers, the anti-causes, you're not going to get anywhere. It's not, it's not, it's, it's like going in again, pitching COPPA compliance, pitching SOC 2 compliance, right? It's usually like, okay, cool. I don't, or another example of this is, uh, making your product easier to use or free to try. This is a common thing. Yeah. Does it matter if nobody wants it? Nobody has pull.

Yeah.

And so, and so there's all of these things on, on there where they are preventers. Now we need to approach causes and preventers extremely, or anti-causes totally differently. The causes are things that we should focus on and emphasize the anti-causes are I don't know if that's the right word. Anyway, we haven't figured out the right word for this, but the things that—

leave a comment with the branding that Rob should use for whatever the, whatever that second thing is, the thing that does not cause a purchase, but can prevent a purchase.

Yeah. Yeah. Yeah. Um, um, so the things that prevent a purchase, we want to avoid those when possible. And if we can't avoid them, we want to minimize them. Make them as small as humanly possible. Security review is the perfect thing. You don't want to say, hey, by the way, hey, you forgot the security review. Do you want to— shouldn't we add 14 steps of a security review? If they don't bring up a security review, you don't want to bring it up necessarily. You would like to avoid that.

Yeah. With the— with these anti-causes or TBD on the brand, it feels like get out of your own way is kind of like the rule of thumb and Uh, the, the trap I was, I was falling into was you have to spend time on like SOC 2 compliance. Like you have to spend time on that for COPA compliance. Like we had to spend time on that. Um, and so if I came at it from a supply perspective, it's like, this is something that I have done and I should, I should talk about the thing that I've done. But that's not, that's like totally the wrong frame for the conversation, which should be entirely pool-focused and then addressing any of these to them. I forget exactly your words there, but it's like, what is the least I can do to get these anti-causes like out of the way to not prevent the pull from like causing the purchase to happen?

It's like treating them like checkboxes rather than like open-ended questions. And we want to, we just want to minimize their size and impact on the process. And so, right, like now we can jump to POC. So when you realize that what a pilot does is prevent a purchase, right? They, they, you want to avoid it. Uh, what a lot of people do is they propose a pilot because it seems like the right thing to do because it seems like a smaller thing to say yes to. But this is like actually shooting yourself in the foot. If the customer does not request a pilot, do not propose one.

Yeah. God, I feel like this is one of those Interstellar moments where I'm looking at myself from like 18 months ago and being like, do not say that, dear God, please do not say that. It's gonna push this back 4 months and then it's either not gonna happen or it's gonna be frustrating. Um, yeah, that's, um, so rule number 1, the, if the potential customer doesn't propose a pilot, don't propose one. You know, for, it's also, For me, it was avoiding the like quote unquote scarier thing, which is actually expecting them to pay like real money for, rather than like doing it.

Or actually seeing if they had pull, like actually testing whether they had pull and whether they thought we fit versus like just giving them something easy to say yes to, or that they'd feel bad saying no to in hopes that maybe eventually they'll finally get it and love us and, and, you know, give our life meaning.

Yeah. All right. You've that. So rule number 1, uh, just straight up, do not go out of your way to propose a pilot. Um, you're just adding an obstacle. Um, rule number 2, if the potential customer asks for a pilot, don't just say yes. And here's the, here's, it's a super interesting way that you break this down. It's like, all right, What if, what if they bring it up? How do I navigate that? What's, what's the right way to handle that conversation? And what is going on, Rob? Like when, when a pilot is asking, or excuse me, when a customer is asking for a pilot, um, you know, what's, what's going on behind that?

We don't know is the thing. We need to know why they're asking, right? It could, they could be asking for a pilot. Because pilots are just a required checkbox. They have to say, yeah, we, some big enterprises do the way that we buy software. The way we buy AI tools is we have to demonstrate that we have used them in a pilot scenario and da da da da. And that's just part of our procurement process. Okay, cool. Right. That's gonna lead to a pilot that takes a certain shape. On the other hand, it might be that they're, they're asking for a pilot because they have no idea what we've been talking about. And this is a good way to get us just to like, you know, throw this, kick the ball down the, down the court. Right. And, and, and, or they could be asking for a pilot because they don't believe that what we're saying is physically possible. And so they haven't made the decision to buy. Once they see the results, then they'll consider whether it's worth buying or not. Then they'll start thinking about buying it. And so there's all of these different scenarios. So we have to pull that out of them to figure out What's actually, what's the demand behind the request for the pilot, the request for the supply? Why are they asking for it? And based, and, and usually we have to, you know, test whether it's like, okay, cool. We do a pilot usually after we've like, you've described your pull and said, yes, we fit pull. And we've gone through all the other kind of like checklist items for you to assess fit. And then the pilot is like the last step before purchase. So what else do you need to know to assess fit first?

So it feels like there's kind of a spectrum here. There's the, it's almost like the customer doesn't even want to do the pilot. They just have to do it. Like their pull is so strong that it's like, we have to jump over this and then I'm going to buy. It's like, how quick, you know, in that case, it's how quickly can we get through this thing? And then there's these two others that you've, you just talked through. One is, Maybe if you can prove it to me, like, like skepticism, there's, but it's pull. There's pull on the other side of that skepticism. And if there's a, I'm, I'm too nice to say no. And if you want to do this free thing for me, I'll ghost you later on. But I like, I don't want to do that. I'm not, I'm not going to say no to you right now, which is like the wrong place to invest any of your time or effort or energy. But before we get into how to design the pilot, How do you specifically interrogate to understand? There's probably, there's probably a couple other variations that are in there. Let's talk about how to interrogate that. Not, not like waterboard, but yeah. How do you have a conversation that teases out what is going on behind the request for a pilot?

Yeah, I, I, you could, you could literally just say, yeah, cool. What's going on behind the request for the pilot? Like you could just ask that question. The, the broader question is, hey, yeah, we, We sometimes do pilots, but that is after you've confirmed that this fits your needs and like you want to bring this into the organization, just so long as the pilot doesn't go poorly. So what else do you need to understand to make sure this fits your needs so you confirm that like, as long as the pilot doesn't go poorly, Like, we're good here.

Yeah.

That's a very specific wording that we will get to in a sec. Oh, there's like specific wording there. As long as it does, like, it's not what, what do you need to see as success from the pilot? It's, it's like, you're already convinced before the pilot. So, which is gonna be like, like as an example. If you frame that with the poss— like, what do you need to see from a pilot? Or here's how we typically approach pilots. It's like a 3-month thing or whatever. They might say, listen, this is just a bo— like in their mind, it's like, okay, I'm just gonna do whatever you say there. Like if, if this is how you've done it before, but I just need to check a box. It could be a 1-week thing that we call a pilot or a trial or whatever. And we can just like, like we just get past that, but they're not saying it. So you have to get behind. The, you have to get behind to figure out what are they like, have they already assessed that this fits their needs? And then what function is the pilot performing in their heads? And it shouldn't be a, if the pilot goes well, then we should, it should be as long as the pilot doesn't go poorly.

Yeah.

This is done.

Yeah. Let's, let's go back to the original quote. I want you, I want your reactions to this. What are they doing? Like, where did the thing, the train go off the rails basically in the original framing of this, the pilot, what, what the founder struggling with? I'll read it again real quick. We are getting into pilots. They are going well. Users are getting value out of the product and usage is going up, but we're struggling to align on success criteria with our champions. And deals are kind of dragging along in the pilot stage. What do we do? What's going on there, Rob?

So, so what they are doing is before the pilot, they're, they're just offering up a pilot is what they're doing. Turns out they didn't need to offer a pilot by default. People now just buy. Okay. That's, that's problem one. But problem two was when they were offering up a pilot, they were asking the customer, the potential, the champion. To come up with success criteria. In the, in the champion's mind, they were already sold on it. And so they had to rack their brains for success criteria. It took them a while to get there. The startup would propose like a 3-month pilot and somewhere in that realm, the champion, the, they just still hadn't figured out what success criteria was. Other priorities were popping up and the deal was getting kind of like derailed. The champion would stop showing up to the weekly check-ins because they had other things to do, right? Like it's just all of these, we've just, what the, what the founder was doing here is they were introducing a thing that can only prevent a purchase and expanding it and treating it more like an audition where this champion is the judge and should know what they're judging based off of and need to think about what they're judging. So you can see how what we're doing is we're just increasing the surface area for things to go wrong.

Yeah. And it, it's coming from a good place. Like, it's like, I want to create value for the customer. Like, the way to create value is to define success criteria, to stay close, to define, you know, and, and it ends up, like you mentioned earlier, just being totally counterproductive. Uh, you, you mentioned something else earlier in the post that I thought was, was really provocative, which is sometimes the pilot quote unquote works like that. Sometimes the, the pilot, it might even start as something else in it and it's something happens and it leads to a purchase. You know, in my case, I read that and my biggest sale came after like a pilot. And so what I did as a result of that was assume, okay, a pilot is like the path to more deals that are similar to this, which is, you know, but what you just talked through the example was and they had, they had, they've, there's a reason that they had a 3-month pilot. It's probably like their earliest customers like needed to see it or something like that and it led to sales. And so it became, you know, the pilot became part of a sales process and you were like, yeah, just like, you know, we're going in and doing surgery and just like taking that part of, of the process out. But there's like this dangerous thing that's going on, which is pilots don't always fail. Like, pilots can also succeed, and that can lead to misinterpreting, like, what's, what's going on.

Yeah. And it's, it's, when a person buys after a pilot, you might think that the pilot caused them to buy when in fact it might not have. And the, they might have bought despite the pilot. Not because of the pilot. Even if the pilot went well, it could have been a total negative for them to have done, right? Like, these are the things where you actually need this underlying model of causality. Otherwise, what you're gonna do is you're gonna just look at the things that happen without a model in your mind, what causes and what prevents. And say, well, everyone we asked to lick sandpaper wound up buying. Therefore, right? Like, and this is like super common, like demos are the perfect example of this. We think people buy because of our demos. They often buy despite our demos. Same with sales process. A sales process does not convince somebody to buy. The only function of a sales process is to convince somebody not to buy. And so we have to design everything knowing whether it is a cause or an opposite of a cause, a prevent thing. And when we don't do that, we wind up with these clunky and usually metastasizing, you know, oh, you know, our first pilot went super well at 3 months, but then our next 3 didn't. Let's try a 6-month pilot. Maybe that will convince them, right? And it tends to go in the opposite direction when you don't know what causes versus what prevents. Yeah, boy, I'm—

you said like 6 things in there that just made me cringe thinking about some of the things I've done. Um, let's get into, uh, what you should frame this as, and then we'll, and then we'll wrap. Um, uh, you know, you've, you've established at this point, the question is not what do they need to see in order to decide the pilot is a success, which is closer to what that founder was framing it as. This is what you write: we need the pilot to be closer to— what are the conditions under which you won't buy as a result of the pilot? And then, assuming this doesn't happen, is there anything else that needs to happen before you purchase? What— bring us like behind the curtain. What's going on with those two questions? Or like, are those like literally the questions you would say like ask the person when they, you know, when you're before committing to it?

Roughly like we would have a conversation about, about what the exact question needs to be. But what we're, what we're basically doing here is we are no longer treating the pilot as a cause of success versus as a potential cause of failure. And we're like, we and the customer, have to treat it that way. If we frame it to the customer as like, what success criteria would make you want to buy it? That implies that they haven't yet made that decision leading up to the pilot, which implies, well, then maybe we should hold off on the procurement process. Maybe we should hold off on all the other people who we need to get bought in, right? Like that, that has all of those neg— So we need to make sure that we are both treating the pilot as something that is a checkbox that is by default checked unless, you know, like something bad could happen during the— if the users hate it, they don't use it at all. Okay, cool. Those are the negatives that can take this train off the track. Yeah. It can derail the train, but the train is already headed there.

Yeah. It's like you've listened to a call of mine 3 days ago, which you did not, because my framing was totally flipped on that. But so the— it's almost like it feels to me like the client should sort of need to kind of drag you into a client, into a pilot, and you should only allow yourself to be dragged into it if you know that on the other side is a purchase. And also it should be as painless as possible while you're going through the pilot. Like what's How, okay, let's say that you understand the pool, you understand what they, what would cause, what would cause them to not purchase on as a result of the pilot, how should you design the pilot or like what principles should go into the way that you, or what you give or something like that?

Yeah, it's basically like what would cause you to not buy and how quickly can we get Can we like show that that's not the case? And that's it.

You're designing around that thing.

Yes, exactly. Right. And so if it's like, if the users hate it, well, what if we do a training with the users, give them 2 weeks with it where we're going to check in with them and then get a final thumbs up, thumbs down from them. Right. And we're designing around the thing that can prevent a purchase, not designing around the thing that would cause a purchase. Yeah.

Uh, Rob, let's, uh, let's wrap this one up pretty, pretty, um, soon. Are there other things we haven't hit either related to pilots or this kind of causality, anti-causality that we haven't hit yet?

Yeah. So I would say this, this causality, anti-causality, whatever is, is, um, the broader thing that we are, we are exploring here. And most of what we've been focused on is the causality side of things. Pole, fitting pole, not not fitting pole, like, you know, all, all those kinds of pieces. This is still like, there are some implications of this that I haven't yet, that like, I, I haven't found a way to articulate. And so things like when you do discovery, you are looking for the presence of causes, which means you need to assume the absence of causes and test for the app, like there's like that kind of thing, which makes sense in my mind. But as I just said it, I saw like the light left your eyes. And so we're, we're, we're going to keep working on those, but there's something fundamental here. And, and this is like a V1 of that.

You think that you're going to write a textbook about, about Pol and about what, you know, about sales and startups. You're actually going to write like a, Intro to Logic textbook that's going to help dingoes like me actually understand some of the things you say. Awesome. Any actions that you would encourage founders to take after having listened to this?

Buy the book.

Awesome. Buy the book. One thing this has me doing personally is thinking through, okay, where am I conflating a cause with something that could prevent or that could failed to prevent or whatever the right way of saying that. What is actually causing and have I sneakily brought in things that I am— it's kind of like, you know, a wolf in sheep's clothing a little bit. That's like kind of in my sales process that's actually potentially, you know, sabotaging like my sales. And so that can be like a quick diagnosis for folks to do. Anything else, Rob, we want to, we want to add?

Oh yeah, I just had a design. You want to design in the causes and to the extent possible you want to design out or minimize the preventions. And so that's a, yeah. All right. I'm going to think about that for the next 4 days.

Awesome. Hey Rob, what's your, what's your deadlift at?

Still 405. I figured out I figured out what I was doing wrong. So this week, hitting 5 at 4:05, moving up to 4:10 next week. Mark my words.

Awesome.

All right.

We'll wait for that, for that update. Take it easy, everybody. We'll see you next week.