5 Lessons Learned Evolving from a Sales-Led to Product Led with Planday: Workshop Wednesday LIVE

Hello everyone. Welcome in. Thank you for joining us. Happy Wednesday. Just gonna let everyone get in, get you connected to audio and get settled. Welcome to this week's Workshop Wednesday. Super excited to have our guest speaker for this week. So thanks for coming back. Workshop Wednesday As a reminder, we're here every Wednesday at 10:00am Pacific Time, rain or shine. Share detailed, interactive and hands on workshops. We always bring along a SaaS expert to get the opportunity to interact and directly connect with the speaker themselves, get your questions answered and also connect with other fellow community members. This week we're going to hear from Fred Limfeard. He's the director of Growth Marketing at Plan Day. So make sure you get your questions ready. He's going to be discussing five lessons learned evolving from sales led to product led go to market strategy. We aim to keep these as interactive as possible so we do encourage you to turn your video on to come off me when it comes time for the discussion and interaction so you can get your questions answered but also so we can make this a little bit more of a community feel from behind our screens. All right Fred, welcome. Thanks so much for joining us.

Thank you so much and happy to be here. So great to see so many people talking about or listening to this topic which is quite in my opinion and quite important topic especially since as you all know it's very sort of hot right now with oh we have to go product led and you know we need to you know, have less human touch but there's actually some, some, some real implications to that also what can, what can happen if you kind of try to do it wrong or fast that I'm going to try to go over here. But yeah, so to introduce myself I go by Fred. I'm an advisor on growth related topics for a company called fastpring and some others and I work at my day job as the director of growth at Planet which is a B2B SAS company and I'm going to kind of take some of the learnings I've seen from there and other companies especially when you come from a sales and marketing led business. That's how you've grown and now you now you're starting to see things and you want to try to get into go more product led. So let me amazing continue Fred before.

You kick it off I'm going to interrupt you and we're going to ask you some questions just to get to know you a little bit better and to kick off the conversation. So Danny already started us off but we'd love to hear where Everyone is tuning in from. The great thing about these workshops is we really do get people that are joining from all over the world. So Fred did a similar session for us in Barcelona this last June at Saster Europa, and it was really, really popular. So we wanted to bring it to our global community. So we've got Phoenix, Boston, San Diego. I'm over here in San Francisco, snowy Denver. Oh, my gosh, this is amazing. Another San Diego. Very cool. India, South Africa, Bangladesh, Montreal, Dublin, Turkey, Vancouver, London. And it keeps going. So, yeah, amazing. Thanks, everyone. All right, Fred, are you ready? Really quickly, how would you define what you do in your current role?

Yeah, that's. I've been trying to explain that to my kids.

Exactly.

Basically. Basically, my role is usually looking at. For everything from acquisition to retention to monetization. So the big picture and how we can scale and grow based on the, you know, the opportunities and goals that lies ahead and creating sort of a, like a culture of more experimentation. Rather than looking at the historical data and trying to come up with forecasts on how to get to the point, it's actually more about the experimentation and those kind of things. So I work cross functionally across a lot of departments and I have focus areas depending on what objectives the companies have. And then you go in and you go deep and you really try to move a metric that has a big impact on the. On the bottom line.

Amazing. What project are you working on today that you're most excited about?

Well, today what I'm working on is a lot about taking a product that has been born from sales and marketing that and trying to make it much easier and self serve, which is a little bit about the topic at hand here. So.

Yeah, we'll be hearing about that shortly. Okay, one more question and then I'll let you get to it. Well, I guess I have two questions. Someone wants to know where you're located. Copenhagen.

So I'm in Sweden now. I'm a Swede, but I work for Plan Day, which is based in Copenhagen and south of Sweden is used across the bridge and there's Denmark and Copenhagen. So Sweden and Denmark, one can say.

Yeah, you're split. Okay, last question. What's your favorite SaaS product?

Jesus. I would say now it's actually a mix between Notion and actually attribution platform. And yeah, this was hard. You kind of took me there. I'll have to get back to you on that.

Okay, well, we'll go with Notion for now. All right, enough questions for you. Let's let you pass it over to Take it away on your content. Thanks so much for joining us.

All right, cool. So let's go ahead with the five lessons learned and see if we can get some discussions going afterwards. So basically you think you hear an organization from the C levels, et cetera, you want to go PLG and you know, you have a company that's been scaled, you know, with people and where you call people and you explain the product right. And you might have outbound sales, you have some inbound motions, you book demos, et cetera. You're noticing that cost CAC or cost per acquisition is becoming higher and it's also increasing. So it's not really looking scalable. Do you have a drive to sort of decrease the human touches and maybe even exploring if we can reach smaller accounts? So basically here is where some companies go really hard and they use let's do it and blah blah. But there's actually, what I say is you got to take it a little bit easier and take a step back because there is an underestimate what it actually means by implementing product LED growth and what it takes. And if you rush it too fast, you can risk some destructive consequences, for instance in the organization. So what you got to do is kind of look in the mirror and kind of see, okay, how much can we do with this product? How, how much can a person that are landing in this product understand it without any person talking to them and how fast can that go and is it even possible? And then you need to kind of form this, some sort of cross functional plan or you can end up as the new bomb picture that I'm showing here. And a lot of bad things can happen, you know, because you have people that are being paid usually saves people with commission. And you might tell them that don't call these people anymore because they're going to have to learn themselves. And you know, there's all sorts of things that can happen that is counterproductive. So basically when you, the lesson one here is kind of implementing product LED growth tactics, it's very healthy. Even if you're an enterprise company, you implementing the tactics and trying it out, but you shouldn't stop doing what, what works today. And, and what I mean by that is you know, we still have to pay the bills, right. You know what you have and you know how much money that brings in. Yeah, you might know that it's not scaling but it's paying the bills. So becoming less reliant on human touch, you know, it takes time, planning and, and you don't really want to mess with that. So you want to, you want to continue optimizing what you have, try to plug as much holes in the leaking bucket because you're going to have to pay for this new initiative anyways. You don't want to introduce a ton of changes super fast, especially when it comes to the product and your operations and even marketing and channels, et cetera. What you really want to do is implement these processes, mapping up what kind of human touches do we have, kind of learn from them, set up these feedback loops where you're actually trying to figure out okay, well what can I replace, how can I reduce number of calls made or length on the telephone calls to do a demo, etc. So you need to evaluate what can be moved to self serve. Lesson number two is that you have to realize that you're not going to be doing any product LED growth anytime soon and going full plg. Probably never going to happen with, with the product. Depending on what product you have. It's when you look at PLG companies, those that are true plg, they are born that way, right? They are, they are born with the product first and then they are scaling it with, with maybe adding an AE motion or something like that. So you kind of just have to realize that. Because if you realize that yeah, we're not going to go fully plz, then you kind of can open yourself up to how do we kind of take what we have now and, and make it more efficient, effective and adding these tactics to it. So as I said before, it basically wasn't built to sell itself your product. So you want to kind of go towards this product led sales, which is the word used days before, before I call it more sales, product assisted. And it's basically if you're a sales lead, you know, it's acquisition and it's primarily led by the sales teams. But if you're fully product led, you have the features and the usage in the product, that's what kind of sells the product. So you want to look for the middle there where it's kind of driven by customer expectations. You want to figure out how they can try your product earlier and then you kind of want to look at, okay, well so how can we get to this point without negatively affecting our current operations and goals? When do we get there? As an example, when can we expect X percentage of trial signups to convert with one to two human touches or less, or maybe you have four touches, how can we get to two touches, etc. Then you need to look at what needs to be True. For this to happen, do I have the right skill set? Do I have the right people? Do I have the right organization? Maybe I have at many times you have the right people and competences, but you might not just have the right organization so they don't get used in that way. And then also look at is there changes I need to do in the product, systems integrations, et cetera. Number three is, and I think this is important and this is going back to really change management because this is kind of really what it is and it is to have a dedicated team to drive changes in parallel of the existing sales LED motion that you already have. Right. So basically, so basically you don't want to take the people you have that are working in your motion and tell them to also do product LED growth immediately. Right. Because that's going to inadvertently take focus away on the optimizations and the stuff they're doing on your inbound and etc. And then that's not going to work as good as. So you kind of want to look at the new team to drive this change in parallel. Now I am a big advocate that the growth team is super suited to drive this change. Right. And a growth team never wants to be stuck inside a marketing department or maybe a product department. Given the fact that right now I am stuck in the marketing department without we can leave that like that. You want to have some sort of understanding top to bottom that they are here to drive this change. They're going to experiment across marketing, across the product, across sales. Right. And so they are suited to report to the CEO or maybe similar so they can have that autonomy to drive that change. And they want to work in parallel with the existing org just to see how they can exchange these human touches with maybe cells or stuff. And they also, if you don't have to really restart sort of this ideation and experimentation culture where they kind of pull in insights and ideas from all these different departments and they start to experiment around that and they might just also figure out, you know, if we, if we need new resources, they can help hire those and they can even see if we can retrain existing people. Or maybe they spot people inside organization that they can kind of move to this new team maybe. And basically, if you look at it here, the classic sales marketing led organization, you have sort of sales marketing success engineering product on one line there, but on the product led, it's kind of very much driven on, on product. And I believe it's going to be some sort of in the middle where you're working a lot with the product to sort of understand when you need to reach out by a salesperson. So instead of calling them before they've seen the product or talked with the product, you want to drive your inbound as normal and you want to push that into a trial. And then you want to figure out when they reach the trial, when should I reach out to them. So you're looking on the account level. How are people in an account using our product and how can I figure out to reach out in the right time, etc. Number four is understand and plan sort of beyond your product market fit, which is impacted of changes that you do within your product model, channel and markets. So what I mean by that is you have a product market fit now, but that is not only based on your product, it's based on your whole motion, right? So if you would for instance, take your product and make, add a trial to it at a freemium, you're going to change these things around. And it's basically kind of these moving points which this guy, Brian Belfort explain really good that you have these product market, product fit, you have a product channel fit, you have a channel model fit and all these things. So when I change something in the product, things might happen in the channel, right. Or if I change something with my model, I might need to change the channel. So just understanding that one of these fits affects the others. And number five, the last is sort of don't try to go super, super broad. You have to start small and then expand the scope and experiment, learn and implement. And what I mean by that and going into growth and experimentation here is that you don't want to experiment on two wide segments or verticals because characteristics can often differ a lot on these micro segment levels. Right. So and, and if you're experimenting too wide, you know that it can give you false positives and negatives. You might not get your stat sign or statistical significance and it's not really what you want to make decisions on. And a good, and a good example of that is. All right, so I'm a company and I'm going to divide up segments and we put the 0 to 50 as small because we sell to, you know, 50 to 250, 250 to 505 or two, you know, thousand we might have all these. So we call 0 to 50 small. But the fact is actually that, you know, it's a big difference between someone that's 0 to 5 and 40 to 50 employee company. Right. And it's also different if it's the hospitality industry or maybe it's a retail industry. So what I mean by this is don't go that wide. You kind of want to look at. Okay, well let's say we spot that, okay, we're going to start with one to nine employees in a sector and we're going to start experimenting and see if we can reduce that. Maybe it's an opportunity just by making that self serve. Because we have seen in the data, et cetera, that we're spending so much time on just that segment in that, you know, industry and that size. Right. So by dialing down in there, you can, you can. And experimenting on that level, you can actually see that. And it's also when you look at the data, if you look at the data in this way, you will spot these patterns because once you have 0 to 50, it's averaging it out, right. So you won't spot these things where it might make a difference. So the top takeaways basically to sum up is that you want to assess your sort of maturity level across your organization and the product. So how far can I go? And is there tons of changes that needs to happen in product for it to even be possible and then form a plan? Right. And then you, I think you have to understand that, you know, we're still paying the bills with this. We still have people that are being paid by, with getting commissions, etc. You know, they're not gonna, you don't want them to stop calling customers and don't aim to be fully product led, you know, directly or ever try to try to try it out and see where it lands because it can really be destructive in my opinion. And you need a high level speed and experimentation to do this. So you need a dedicated team that has mandate and autonomy to be able to do this, that are not being stuck in a silo somewhere. So I think actually this is it and I'm sorry if I rushed through. It's a really interesting topic and yeah, I would love to your questions or discussions on what you guys are experiencing and thinking.

That's perfect. Thanks Brad. Great time for discussion. I think this is a great glimpse into it. Of course you can really, really go deep and spend a lot of time here. So appreciate the breaking it down for us and I think it's great to kick off the discussions if you want to go ahead and unshare your screen so that way we can all see each other a little bit better and easier. I'm going to start by going through some of the Chat questions. We also have, we're live streaming this on our YouTube and our Twitter and LinkedIn. So we've got some folks watching from there that I've got some questions coming in through as well. But like I said, we want to make this interactive. So if you're comfortable coming off mute to ask the questions live, we'd love that. First question up is from Mike Latch. I don't know if you're comfortable coming off mute so you can share your question.

I'm perfectly comfortable coming off mute.

Perfect. Go for it, Mike.

Oh, my question above, you actually answered it, Fred. When you were talking, which was you were talking about, you had the picture of the atomic explosion and you're talking about destructive consequences. And I was, I was just curious. You had some examples. I'm like, you just mean like morale killers, you know, where you know the wholesale team is like sort of a gun. We're all going to have a job, you know, or anyway just like examples of yeah things was the question.

Yeah, I, I, you know just that thing that you, you're gonna stop thinking that you have to experiment in a way that you just tell people to stop calling people without kind of considering that these are humans that they are actually their salaries are based on making calls and certain aspects and it's super destructive at overall. If you say from one week to another, hey, by the way, don't call anyone up until this point on this site anymore without actually thinking about the consequences then what you should do instead is figuring out a way that you know, you can pay them differently maybe or have some sort of different commission scheme before you tell them to, you know, stop call as an example. Gotcha.

I had another question. Do you want me to get back in the queue, man, or go for it? Go ahead and ask it now. I didn't get a chance to write it. I was writing it. Do you have any. So in some of my experiences at tech companies, some small bit large, I've come to the conclusion that I really like serving and selling to larger enterprise. Just because the kind of, you know, if you're like a team player, you like getting in there rolling your sleeves and really helping people. It's very expensive to do that and doing that for a small company like you.

You go out of business, do you.

Have any like ideas, thoughts, anything like after saying that, does anything pop in your head about product led growth stuff in terms of enterprise clients? Like anything you remember happening, lessons learned, you know that anything popping your head?

Yeah, absolutely. So this is a really interesting so I've been buying SaaS for like 13 years. So I've been in so many different sales processes, both being being an enterprise, you know, customer and being part of it. And I think that we shoot ourselves in the foot a little bit when we say, oh, you have to do this when you do enterprise, or you have to do this when it's small. In fact, that, you know, if I am in an enterprise and I'm in a buying process, I want to have a free trial, you know. And so being able to offer that I think is offering a free trial even if it's an enterprise, instead of saying, you know, the classic, hey, you know, no, you can't have a free trial. Oh, I have to. We actually don't do free trials. You know, you kind of, why not give out a free trial even for enterprise? Or you might be a part of your product that you can showcase, maybe even on the website. Or me, maybe in some sort of environment sandbox. You see, people are using to provide to allow these people to try it out. Because, you know, in enterprise there's, there's a huge buying committee, right? You're going to have, you know, maybe a techy, geeky person like me that like, I cannot trust anyone's word. I have to go in and poke around and touch things to even consider this. So that's why I think that these product led growth goes great with enterprise. If you look at the, the buying committee, right, and can apply it to them, you can actually speed up that process. Instead of having, you know, the classic qualified qualification call and then you're going to set up a product demo and it's going to go through this very long process. So I don't know if that answers.

Basically I was just kind of curious what, just what would pop into your head. You might have said something like, yeah, cutting edge PLG stuff is 17 times in automatically shows the screen to their lawyer and then, and then the CIO can pop in on this, you know, like, like true plg. No humans landing a million dollars. I've never heard of such a thing. Someday it might be like. No, I was just curious if you. Anyway, you answered my question.

Thank you. Yeah, yeah, absolutely. I'd love to continue to talk outside this on that topic. Cool, thank you.

Thanks, Fred.

Yeah, thank you. Thank you so much.

All right, thanks, Mike. Okay, David Kopeck, are you comfortable coming off me? You dropped a couple questions into the chat, which would be great ones to ask. Okay, I'm going to go ahead and ask his question. Then, Fred, in a recent podcast interview, you discussed how many companies are reactionary to Churn, but you stated that you see it as a lever for growth. In this podcast, you define voluntary, deliberate cancellation and involuntary churn, like a credit card expiring. Do you have any recent examples of strategies that are automated through qualitative feedback surveys? Pre voluntary churn.

Okay, well, cool. Thank you for catching that one. Churn is a hot topic too, so repeat the last part there. So do I have any examples of.

Any examples of strategies that are automated through qualitative feedback surveys? Pre voluntary churn?

Yeah, yeah, definitely. Setting up the feedback loop is kind of the most important part. And I'm happy to share outside this, how to do this, especially when you cancel your subscription, to have that continuously feed into sort of a, maybe an automation system or something. Something where you can actually trigger actions based on that data. So let's say I cancel my subscription. You get the classic transactional email saying, thank you, and I'll put the survey in there where I ask why and why did you cancel? And then I asked, what can I. What do I have to do to get you back? And I'll take that together with all the information about you, send that into the automation platform and I trigger, you know, hyper personalized workflows based on. Okay, you said you, it was too expensive. Okay, I'll give, maybe I'll send you an offer or something. Or maybe you said we were missing a feature. Okay, I'll take that and send to the product team and then when that feature comes, that email will go out to you. So setting up these automated feedback loops is kind of. Yeah, most of the important things I do to kind of start working with these things, kind of building an engine around this qualitative feedback. That makes sense.

Okay, awesome. Any ideas for workflows to give technical support or initiate an immediate response to the data obtained?

Could you, could you repeat that one or clarify it maybe? So workflows for helping technical support to understand sort of the information about the issue that the person are having. Is that what we're talking about?

Yeah, this is off that last question. So I think it's more about the qualitative feedback surveys.

Yeah, I mean, it's. I, you know, I would base. Especially if we're going to Churn, or is that if you see that people are struggling in the product and you can spot that behavior, which, which, which you can, then you can, you know, trigger a customer support person to. Oh, sorry, A customer success person to reach out, for instance, or Maybe if you want to minimize customer support time by using PLD tactics, one of the good things would be to provide the support people with the information about the struggles in the product. Let's say an example so you can see in the product, for instance, let's say you're using Intercom and they have these little tool tips in there. Like you can spot if people maybe try to look at certain help articles too much and click on the tool tips and are missing things. So you can kind of send that to the customer support agent or even, you know, being more proactive to actually start sending out content first to reduce things. You can do a lot of these different things. I mean, just by giving them information about why they want to talk with you, that then you don't have to ask these questions. I mean, if you put that out on a year, then you're going to save a ton of time. So it's all these incremental things to make it more efficient. And it also makes it easier for the user, you know, they don't have to explain everything. You already know it kind of.

Yeah. All right. Just looking through the chat, Nikhil, when you were talking about setting up the growth team, sort of like pricing. Pricing used to be in marketing product nowadays has their own team thinking about it. Pricing can be a growth lever. So pricing can be part of the growth team. John, I know, I see you're nodding your head and your next step to have a question you comfortable coming off mute so you can ask?

Yeah, absolutely. So this is fantastic.

Thank you.

One of the things, Fred, you mentioned was to make sure that we give room for the PLG team to act autonomously. And just curious what you've seen work in terms of kind of providing that air cover. Do they report up through the same CRO or do they report up through a different line, you know, to provide them the room that they need to be successful?

That's a, that's a good question. Thank you so much. Yeah, so what I've seen is, you know, when, when you, when you try to drive that change from within one department, it's usually not very efficient because, you know, it's the same as I, I advise, don't put your data and analytics team inside, you know, the, under the CRO or something. It should kind of be in, within. So I, I think you should have it under the CEO or something in the middle. And then of course you're trying to just talk about, you know, or that growth person would actually come in and automatically start talking with People and trying to explain like this is how we're going to drive change and it's going to be more about the process. So we don't know yet how this is going to work. So that means that you as a product department can't start build yet. Right? Because you don't know. So what us, the growth team would do is we would create an environment where we can work maybe on top of the product and try onboarding stuff and stuff like that much faster. Then I would try to explain the value of the product teams by being able to get the insights from these experiments to prioritize the roadmap and become more efficient. And I think it works better if that entity is like in the middle. They're just like on top working with every team based on you have your sort of high level okrs that is always going to be top on that. But you might have a theme that okay, we're going to dive into this here and, and I go into the sales department and trying to work with them or we go into the product department. Maybe, maybe you have someone in charge of monetization and try to run experiments on that because. Because everyone has their day to day job no matter what. It takes a lot of effort to. Oh, now I'm going to have to run experimentation on top of all these other things that are paying the bills today, if that makes sense. So yeah, under the CEO or some sort of change management function depending on how your organization looks. It's ideal. I would say maybe you have a chief growth officer and that would be the. But, but I don't think that. I think the growth role is what I've seen the tre kind of the growth role is not very, very understandable if, if an organization not as mature on that they're not gonna, they might not gonna have maybe a growth person that works across product and sales and marketing. The way you can see, you know like Facebook and stuff like that. Well, they have like these you know, acquisition growth teams that are only running that and they're. If that makes sense. Yeah, that does.

I just had it and thank you. That's. That's helpful. Just a follow up question was based on your experience, did you learn that certain cohorts of the customers as we broke them out, you had a great example of don't try to boil the ocean, right. Work on a specific segment or vertical. Did you find that there were, you know, certain cohorts that had a better response rate than others? Thank you so much for letting me ask my questions.

Yeah, thank you. Yeah, absolutely. I mean I, when I dived into the data and looking at it, you know, on the small segment, I could see that, you know, 0 to 9 gave, you know, a, you know, let's say €200 ARR. But if I just went from 10 to 20 or 30, that tripled. And the sales cycle length, the time to value from first touch was even shorter. So that tells me that, hey, this zero to nine is not giving us anything, you know. And so what pops into my head then? Like what? Maybe we should give away our, maybe we should can give away our product for free to these people. You know, build up a lead engine, maybe that could work. And maybe we can just steer in and find out how does these 10 to 20, 30 people trigger? How many people are they there? How many touch points is needed? And I look at, you know, first touch to opportunity one, right. And where are we losing deals, where are we winning deals? You know, and when it's just a big segment, it's when you're generalizing just zero to maybe 50, you simply don't see it. You become blind unless you kind of look deeper. That makes sense. Awesome, man, thank you.

So, similar question in the chat here, just off that. So I don't know if you have anything else to add, but when you choose a small cohort of the target market to experience. Sorry, when you choose a small cohort of the target market to experiment, what are the key metrics to be kept in terms of pricing as the new team wants to decrease the price of the cac. Oh my gosh, this is a wordy one. You might have to read yourself, right?

No, I think I, I see it, I think I get it actually. So I, I see, I see it here on the side. Yeah, let me, let me. So this one is really good, right? Because one thing that, that, that is the hard is like, let's say you have a, if you're a sales and marketing company, you're going to know that salespeople has discounting, right. As a lever to close the deal. Okay. So if you, and it's kind of baked into the model. So like if you try to, for instance, put in, you know, the classes classic, you know, this is how much you pay per month and this is how much you make if you pay yearly. You know, you're going to have users that have seen that, that come in and then there's the salesperson that's like, oh, they think that we can, we're going to give them 20 on a 400 user, you know, license or something like that. So, so pricing becomes, you want to differentiate. If you try product, the product led where you want to be like on much smaller customers, you want to really differentiate that. And it comes down to the model, right? So what's your model? You know, how are you doing the. Do you have usage based billing? So okay, after five users then they cost this much, after 10 users is this much. But then after maybe 50 then you have to contact sales. Right. So you kind of have to figure out that you have a clear different, different differentiation line between where the sales and marketing motion starts and the sort of the self serve new stuff that you really have no idea how it's going to work is going to work. So that has to be differentiated in your, in your, like on your pricing page. Right? You know, you know the, I think that's super important. So pricing and packaging is super, super important. And also features, you can't just go and throw both of them in the mix and then I don't think it's going to work in a efficient way. This is a good question. The pricing is super important there actually. Thank you for that one.

Amazing. Anyone else have any other questions they want to ask? Live. Take away have something different in the PLG pricing packaging to not interfere with a slg. Love that. Thanks John.

For sure.

All right, last call for questions. I think that was a great one to end on actually. One more. Do you really get to statistical significance?

Do you want me to, do we have time for me to ask that question?

Yeah, absolutely.

So Fred, like I have a very technical background. You know, if you're like scientists you.

Get to really, you get to run.

Real experiments and really understand what like statistical significance means. And, and that in my experience that doesn't happen very often in like real. But you know, if you're Google, you really do have a lot of data.

Right.

You really can't get a million people that just see a red button. Everybody else sees like, like, like do you really, do you really get that? That, do you really get the opportunity to say look, I've proved that this doesn't correlate, this causes this. Now let's do this. Like does that like do you really get that or you still really getting information you have make judgment calls and then make your business case, you know, and make changes. Does that question make sense?

Yeah, yeah, absolutely. I come from sort of, you know, conversion rate optimization and I kind of use that term because I thought that it would make sense. But it's, it's Absolutely. It's absolutely what you're saying. It's kind of, you know, first of all, you know, when you do testing, let's say on website and whatnot, you know, you have to kind of like, you don't test maybe web visitors and mobile visitors together in the same task because you can get all these different things. So that's kind of what I meant with that. But in the end, you know, you know, if you're, you know, in an A B test, right, kind of sense, then yeah, test statistical significance kind of me is the way to call it. But I would say in this case, when you're running growth experiments, it's a different type of testing and it's more about probability and confidence level and level of risk that you can trust this. So I would say it's more towards more type of qualitative testing. I mean, it depends on what you can, what you feel comfortable. I mean, to me, I want to collect all these different parameters and it will increase my, my, my sort of confidence level that this is the right thing to do and they can, you know, I think when you're trying these different things, it's. Yeah, you might not run it as an A B test, but it could be that you're running it as an A B test on the pricing page, you know, where, where I'm looking at people that are on the pricing page to kind of, you know, click on the free trial button because of something, then I want to just only show my variation to those that are small. And the way I would do this, I would probably figure out how to output an enrichment tool, right. To, to, to see every, every, every company that I pick up that are in this vertical or control group, I'm only testing that towards, you know, in the same type of setting, one size, maybe higher, and see if it changes if that, if that makes sense. But, but it's absolutely, absolutely right. You know, in B2B you don't have the amount of volume as I did when I used to work E commerce, you know, so you don't, you're not as, you know, as you said, you don't, you're not, you don't have the opportunity as much to look really, really directly on statistical significance in that means, if that makes sense. Awesome. Yeah, thanks, Fred. Yeah.

Yes, I understand. You answer my question.

I feel like this could be a whole nother session. You drop that in the chat. So, um, maybe for a future workshop Wednesday, we'll have to add it to the agenda. So. Thank you, Fred. For sharing your insights and just for a great interactive discussion. And thanks everyone for joining us for this workshop Wednesday. Next week we were are actually headed to Singapore for Saster's first APAC event. So it's our inaugural APAC event. We're super excited to be there. We will be live streaming so be sure to check it out and watch out for the content to come in the coming weeks. But our workshop Wednesday will be back in two weeks so in two weeks time we are excited. We're going to have venture capital firm Iconic Growth, Doug and his colleague Christine are going to be talking through some tactics for building resilience and efficiencies in scaling in 2023. So don't miss out. We will see everyone in two weeks and thanks again for joining and thank you so much.

Please add me on LinkedIn. I would love to continue to discuss these topics outside and thank you so much Disaster and for hosting me and everyone for asking questions. Thank you so much.

Thanks. I appreciate it. Bye everyone.