Growth Tactics For Technical Buyers with Weights & Biases

edition of Workshop Wednesday. Every Wednesday at 10 AM Pacific time, we are here week after week to share detailed and interactive workshops with a guest speaker. So we are so thrilled today to have Lavanya Shukla, VP of Growth at Weights Biases. She's going to walk us through her playbook to building a growth team. We're thrilled to have her present these frameworks, and she's going to have plenty of time for questions. So please be ready to have those in the chat so we can do a deep dive with her later on in the session. As always, we aim to keep these sessions as interactive as possible. So please have your cameras on, drop those questions into the chat. I'll be helping to queue those up throughout the session. I would love for you to even come on to ask that question directly to Lavanya. She would love to speak to you as well. And we'd also love for you to participate just in the chat. So please let us know where you're tuning in from. And Lavanya, welcome. Where are you tuning in from this morning?

Lavanya. Thank you so much. I just got back from Mexico. I was in London for a bit. And right now, I'm in India. So I've been all over the place.

Wow, all over the place. So we always have international folks too. So thanks for joining on the other side of the world. But let's just get started with a few warm-up questions before we dive in so the audience can get to know you a little bit better. First up, how would you define what you do in your day-to-day role as VP of Growth?

So growth teams, not everyone has a growth team, and it's slightly different for every company. In terms of metrics, I own the pipeline and the user growth number. And in terms of what we do, my team is a mix of machine learning engineers and also the marketing team, so it's kind of like a mixture of both.

Very interesting. Um, yeah, excited for you to dive more into this during your presentation. Uh, what project are you and the team working on today that you're most excited about?

Uh, so as everyone here probably knows, ChatGPT is a thing, and like every week new LLMs are coming out, and we build tools for these LLM folks. So we're super excited because our work is finally going into all of the world. So we're just trying to capture as much of that attention as possible, and it's been pretty exciting.

Great. And in the last 12 months, what's the hardest thing that you've worked on?

I think it's the other side of this coin. So we knew how to get machine learning engineers to use us. We had a playbook, we had a flywheel that was going, and now all of a sudden all of these software engineers are building ML models. So figuring out how to rebuild our business, both our product, our go-to-market strategy, and our growth machinery for these new users has been exciting, but also challenging.

Yeah, I'm sure a lot of other folks in the room are dealing with similar things too with the rapidly changing times. What's the current— what's your current favorite SaaS platform or tool?

I think so. I'll give a shout out to something that's being built by one of our friends. Her name is Mercy. She used to run the growth team at Slack. She's built this tool called Panobi, P-A-N-O-B-I, and it's for managing all of your growth experiments, and it's pretty cool. Awesome.

Well, thanks so much. We always love hearing about up-and-coming apps. So without further ado, Lavanya, I'll pass the mic over to you. Everyone, again, we'll have plenty of time for Q&A, so please make sure to drop those questions throughout and we'll get them queued up. But Lavanya, take it away.

Thank you for having me, and let's get started. So thank you for having me, everyone. I'm super excited to be here. As you come up with questions, please drop them in the chat, and then we'll take it at the end. First, I'm going to tell you just a little bit about what Weights Biases does, just so you have a little bit of context on where I'm coming from. So we are an AI developer platform, and essentially, we help you train and deploy all of your machine learning models. So we're used by all of the big companies putting models in production. But for me, at least, more excitingly, all of the people building these LLM models, OpenAI uses us for ChatGPT and GPT-4. Stability uses us. Midjourney uses us. And like, that's kind of what we're doing. We were founded by 3 very technical founders, Lukas, Chris, and Sean. All of them are machine learning engineers, and this has really helped us put the machine learning engineer first no matter what we do. That's why we don't have a marketing team. We have a growth team that is equal parts machine learning engineers, and equal parts marketing. And also this has made us very data and experimentation driven, and you'll see it throughout this talk. So just wanted to set the tone for where I'm coming from. So today what I want to talk to you about though is how to create exponential growth systematically. And I created this framework for it, FUEL. So essentially what you want to do is You want to create flywheels, and we're going to learn about the difference between flywheels and funnels soon. You want to make sure you understand the user decisions that are behind every metric that you're trying to move and how you can inspire them to make these decisions. You want to run data-driven experiments to influence these decisions, and you want them to have very clear start and end dates, very clear hypotheses, very clear attribution. And then finally, you want to capture the learnings from running these playbooks from running these experiments and convert them into playbooks. And after you're done with these, with capturing these learnings, you want to feed them back into your flywheel and do it all over again and again and again. So the personas that you're going after, the experiments you're running, are all going to be different, but hopefully this framework is general enough that all of you can use it to think through any growth problem. So let's get started with the flywheels. This is probably my favorite of the four. So essentially flywheels are network effects in action. We want to create flywheels, not funnels. And funnels will lead to linear growth. As you can see here, you generate some awareness, then you get people to get more and more engaged in your product. But with flywheels, you add this extra step here where you take your most engaged users and you turn them into champions for you. You turn them into your sales team so you don't have to build a big sales team inside. And you do that by getting them to build publicly on top of your product. So, you know, good examples of these are, if you can, like GitHub. People are creating repos that are public and then sharing them with other people. And those public repos that then drawing in other users to sign up for GitHub, right? So you want a component of your product that is available, that people can build on publicly and then use that to join other users. So that's our north star, especially in this market. We don't want to pay for users. We don't want to pay to get linear growth. We want to create exponential growth. So at Weights Biases, we have two flywheels that work. Here is our content flywheel. And essentially what's happening here is machine learning engineers write about the latest models, the latest papers, and they share the performance as Weights and Biases plots. So here you can see examples. So they write a blog post and then these are Weights and Biases plots that are interactive, that are embedded in. So you'll notice these are helpful pieces of content, but they're not about the company. They're just showcasing the company, but they're actually about things that people really care about. So if you can get people to write this user-generated content and distribute it through both social media and then also through organic search, we can start to draw in a lot more users who are coming in. And organic search is actually one of our most significant ways that people find out about us. And also organic search is important because these are users who are actually searching for solutions to problems. So they're very high intent. So if you can show up as a first result with something that's useful to them, then they'll— and you can showcase your product, then they'll sign up for the product and they'll become returning engaged users for a long time. So second thing you want to do is you want to set a North Star metric. So you want to create Live Views, but then you also want to optimize them around this North Star metric. And you want to be really obsessive about measuring it. So in our team, we created a culture of reporting on these metrics every single week. Our selfish team was tied to moving these metrics, you know, and we would find the bottlenecks each week and fix them. And we were very honest about reporting these numbers. The people who work out well in our team, at least, are people who are very honest and very curious. So they're not afraid to be like, look, the metric hasn't moved, the metric hasn't moved for weeks, you know, but like I'm trying these things to move it and like this is how it's doing. So you wanna ask yourself these tough questions about why there are bottlenecks in the funnels and you wanna be curious about finding those answers. But then also this, oops, the second piece of advice is what makes a good North Star metric. And in my opinion, there are 3 components to it. First, you wanna figure out who your user is. So who are your most valuable users? What are their titles? Are you going after the technical persona, the buyer persona, or the IT person who's helping them integrate your product? And then also, or like what deployment are they using? Are cloud users more valuable to you or people who are using their own machines? So first, who are the right users? Second, what is the minimum valuable activity that they're doing in the product? To be counted as users. So it should be high value enough. You don't want people who are just liking your posts to be considered active users. They should be people creating posts. It has to be an activity that takes a certain amount of effort. And then the third thing is what cadence are they doing this activity at. So for some products like Slack, you want it to be daily because if you're not using your messaging app daily, you don't have a habit of using that app. But for like a tax software, it could be once every 6 months because that's not something you use every day. So you got to figure out what's frequent enough in your product to form a habit. So then you have this user activity cadence combo, and you'll iterate it over— iterate on it over time as well. So the last thing on— so I feel that I wish someone had told me was Firstly, start measuring everything as soon as possible. Make sure you instrument every step of your entire funnel. So every channel that people come from, social, events, white papers, make sure everything is instrumented. And then every single step that they take in your product should be instrumented. Because if you're not measuring, you're not learning. And these are things that could save your business. So you want to make sure you're capturing this data. Also, Irish Seminair told me to fix attribution so you can actually trust your data. So things like someone watching a YouTube video and then signing up for your product are way harder to attribute than someone coming to your website and clicking sign up. But you have to be creative, creative about measuring these sources. And you want to make sure every— you have a full picture. Because even if you're small now and you don't really care, Future investors will really care about the ROI of your experiments. So that's flywheels. That's probably the biggest section. But now let's get into users. So for users, the main thing is you want to understand your persona, your intent, and your channel, and you want to have a fit across these. So persona is who's the user. Intent is what problem are they trying to solve. Distribution channel is Where are they hanging out? So for the same persona, if I'm trying to get them through social, I put out a very different type of content than when I try to get them through an event or when I try to get them through organic search. So all of your experiments should have a clearly defined persona, intent, and distribution channel before you start developing that content. And then ideally, you want to prioritize and design the ideal experience for your most important PIC combos. Here is a look at what our personas look like, just to crystallize it. But the— when it comes to users, the most important thing is behind every user metric, behind every metric that you're trying to move, is a decision that you want your users to make. And your job is to understand that decision at each step of the funnel and make it insanely easy to get the user to make that decision. So you wanna go through your entire funnel from the first time they see your post on Twitter to the time when they become an engaged habitual user. And you wanna ask yourself, would I take this next action if I didn't care about this company at all? So would you click on that social media post? Would you sign up at the landing page that you were at? Would you come back another day? Did you see enough of the product that you would come back another day? And if not, you either want to reduce the cognitive load to take the next step, or if you can't reduce the cognitive load because they have to fill out that form, then you want to give users more motivation to get through that step so that they can experience the value from your product. So that's users. Next up, let's get into experiments. So there's two types of experiments that you want to run. One, you want to optimize existing channels, and two, you want to find new channels. And typically you're running experiments in both of those buckets. I want to give you some advice for how to come up with good experiments. So first step, go where your users are. It's very hard to get users to build new habits, but if you integrate into places that they already are, then it's easier to get them to tack on the habit of using you. For example, for us, we went to places where machine learning engineers were hanging out at scale. So either schools or other complementary tools or other communities, and then we made it so easy as you were using these other tools for you to also use Weights Biases. The second thing to do when coming up with experiments is take advantage of what your audience is interested in and excited about. So we started running these paper reading groups where new machine learning papers would come out and we would, I would have my machine learning engineers explaining those papers to people. And while they were explaining the paper, they would see the results in Weights and Biases. So it was an easy integration of the product. But like, we were not building these programs to talk about Weights and Biases. We were talk, building these programs to talk about the latest machine learning papers. And the amount of people who care about the machine learning papers was far higher than people who care about Weights and Biases. So take advantage of what your audience is excited about and put in your product in there. Next, you want to build a community, and like community-led growth is one of the most underappreciated ways to grow, I think, especially for technical audiences. Ideally, what you want to do is you want to either teach people something new Or you want to make it easy for them to keep up with what's going on. Or you want to connect them with their mentors and heroes. Or you want to teach them best practices. And while doing all of that, you want to just like integrate your product into your programs. So those are just some of the ways to come up with innovative experiments. But when you actually come up with these experiments, here's some advice on how to run them. And I also wish someone had told me about this before I got started. So you have limited amount of time and resources as startups, but there's an infinite amount of stuff to try. So what— to run good experiments, what you want to do is, one, you want to have clear start and end dates for your experiments, and you also want to have clear attribution. It should be very clear if someone came in through a particular experiment, and if you can't measure it, you shouldn't run the experiment because you're not actually learning anything. And there should be a very clear metric of success. And ideally, it should be pretty, it should be something that takes the user some amount of effort. So someone just clicking on like an ad is not a successful experiment. Someone actually signing up for the product and becoming an engaged user is when that ad was successful. So you want to have clear metrics of success also. You want to obviously like run a portfolio of experiments. So Some are experiments that are high impact, some are low impact, some are high effort, some are low effort, some you have really high confidence in, some you have medium to low confidence in, and you want to kind of have a portfolio. Also, you want to have very short iteration cycles of no more than 2 weeks. If you're not changing something every 2 weeks for an experiment, then you're taking far too long. Startups with really long iteration cycles will just die, because there's so many unknowns. If you're not learning something new every 2 weeks, then you're moving way too slow. And you can have experiments that run for more than 6, more than 2 weeks, but you want to keep changing something. So you put out a blog post, change something about it every 2 weeks. You put out, you run a new event, for the next one change something. Make sure you're always learning. Also, kill your darlings fast. Like, you know, this is a thing in literature, but like this is also relevant to experiments. You should not be scared to try lots of ideas, but you should also always kill the ones that don't work super fast. And you should build a culture where people are not afraid to try things. It's totally okay for your experiments to fail, but it's not okay for experiments to fail when you didn't capture a learning from them. And so how we do this is we reward interesting experiments. So an experiment where what happened was different from your intuition, that is a good thing, you know, so you want to reward experiments like that. As well. So that's experiments. The last step is learnings. So the real output of a growth team is not just user growth or not just pipeline growth. It's actually building a machine that leads to scalable, repeatable growth. And you want to be— over time, you want to get so good at running experiments that you can predict very closely, if I do this, I'm going to get 100 users. If I do this, I'm going to get 1,000 users. If you can have that type of predictability, that's going to take you a long, long way. So you want to capture these learnings on why something worked or why something didn't. And then you want to build playbooks out of this. So over time, hopefully you have like, if I want to get machine learning engineers, this is the most effective way to run an event. If I want to get the buyer persona, This is the most effective type of content and nurture campaign to run. You know, you want to have playbooks for all of this and you want to keep updating those playbooks. And that's all that I had for you guys. So again, the strategies, the personas, the experiments that you run will be very different for you. But hopefully this FUEL framework can be used to think through any growth problem that you have. And with that, I'll open to questions.

Amazing. Thanks so much, Lavanya. Everyone, please start to drop in your questions. We do have one queued up right now. And I think our attendee is going to come off chat or come off mute to ask. Chen, do you have your question if you're still on?

Yeah, I'm here. Thank you. Thank you so much. I have many questions, but just to respect everybody's time, this last one. What we found, our growth team, the biggest challenge was Um, our growth team is exactly as you said, it's more funnel-oriented instead of flywheel-oriented because we don't have the content. And what the challenge we found is it's very hard for us to find very good product marketers or like, or like technical people that can create those content because those people usually they want to build products, they want to focus on other things. So I was going to ask how you manage your team and how you make sure that you can find those people and get them motivated to produce this amount of content to, to get the flywheel running? Thank you.

That's a great question. So I actually do have a slide on how to build a growth team. But before I get into that, there has to be something about your product that allows your users to build something publicly, right? So you got to have that. The second thing is not every technical software engineer, machine learning engineer would want to write content, but there's more than enough who are motivated by teaching people. And that is the type of person you want to hire, people who want to build publicly, people who want to give back. And developer evangelists is kind of software— the industry figured it out by calling them dev evangelists. But I found like if you go on Twitter, and that's how I hire people, I literally just go on Twitter and see like who is sharing, who is giving these talks, you know, because those types of people are looking for these roles. And they aren't that many roles for that kind of person, you know. The only challenge there is you want to find someone who cares about your brand, not just their brand. See, a lot of influencers will care about their brand, right? But I would go to Twitter and find the people who are doing the best writing around your field.

Thank you. Thank you.

Lavanya, I'd love for you to walk the group through this slide.

Okay, sorry. Let me do that in a sec. So my background was I've been programming since I was 10. I'm not a marketing person. So like, I didn't know how to do marketing or even growth because growth wasn't like a big thing, right? So essentially what we did was we tried to figure out, um, who are the people who are like super embedded in the community and are also very data-driven and are very creative. So like, first of all, like, if you are running a growth team, you need people who are very data-driven. It made it easier for us because anyone who's a machine learning engineer is data-driven, is very experimentation-driven, right? But like most technical people are data-driven. The second thing is we wanted to only hire machine learning engineers because they knew our users inside out. And again, I went to Twitter to find the people who were like talking the most loudly about machine learning, right? And then these people, we wanted to find people who cared about building in public to answer the question that was just asked, you know? 'Cause if you care about people who wanna build in public, they will wanna give talks, they will wanna build repos that everyone can use. They're driven by that. And then also, this is not necessary, but ideally you do want to find someone who's either a great writer or a great speaker. It's hard to find both, but if not, then you have to tack on an additional person who's presenting their work, right? And then we also care about these 4 values. For me, the most important thing beyond smarts is someone who can break through walls. Because in growth, it's not like you have a playbook that you're following. Every 6 months our job changes. Like right now, our entire industry is changing, right? So you want You want someone who sees a problem, is not afraid, and finds a creative way to get around the wall and solve the problem. So that gumption is like the most important thing. And then once you hire this kind of person, so essentially to TL;DR this, you don't wanna hire marketing people. You wanna hire people who are exactly like your users, but just more public-facing versions of your users. And then you wanna talk to them about your strategy and your prioritization constantly, 'cause these people are not necessarily good at building funnels or they're not good at business metrics. So you wanna tie their work work to the business metrics. So the conversation I'm having right now, it's great we're writing these machine learning blog posts, but which of those are turning into pipeline or turning into user growth? It's not enough to just write a blog post. You also want to think about how to convert people into users from this blog post, right? So those are the conversations that then you need to have with them.

Right. Lavanya, we have another question from Kashal. Kashal, would you like to come on off mute and ask your question?

Absolutely. Thank you, Taylor. And yeah, thank you, Lavinia, for the great presentation. My question was around the user generation content part. So from the perspective of a company with a product that doesn't have ability to create UGC right now, what would you say, you know, what types of user generated contents would be most ideal for these type of flywheels?

Yeah, so ultimately you have to figure out a way to build that into your product, right? But that's not today. So the other things you could do are create communities. So Discord or wherever, Twitter, wherever your users hang out and then build a space where your ideal users can come hang out and it's not about your product, it's about the industry. So it's not about Weights Biases, it's about machine learning. And then just start talking about like what these people are excited about. Start bringing the people that these people want to talk to, you know, in your community. So it then becomes like they're still creating content and then like, you know, people will find connections, people will start running events. It's not really about your product, but it is your community. So that halo effect still carries through. So if you can't do it in your product, do it in your community.

That's great advice. Thanks so much.

But do it in your product though.

Absolutely. Yeah, we're trying to figure out ways to kind of incorporate user-generated content. But yeah, it's just been interesting to kind of navigate that side.

Totally. Thank you.

Lavanya, could you share some tips about your best recommendations for growing your community and growing your followers?

Yeah. So growing your community and followers, two different questions. So let's start with the community first. So what you want to do is like anything, like you want to first get 5 people who are insanely excited about like starting a community. There's a lot of people that really want to be a part of something and really want to be the ones like leading the charge, but they can't do it in big communities, right? So you got to find those people and then make them feel like rock stars. So when we first started, we had this guy Eric, and like he was just posting everything he was reading in our community. And like he was so excited. There were like 50 people, and like he would get a couple likes, and he became well known within the community as the guy who knows what's going on in ML. And slowly, all these amazing people would come just to see what Eric was saying, you know. So that made Eric feel amazing. And then he became a part of our community for so long. Then other people were like, oh, I want to be like Eric. So they started like sharing stuff, and that drew in other people, right. Find a couple people who will be contributors and then make them feel like rock stars. And then we made sure that we were talking on Twitter about how cool Eric was and like other people who were creating content around our brand. So that feeling of belonging, that feeling of being made to feel like a hero is the key to getting people into your community. Now the second question was getting followers and that's mostly for your own Brand are you thinking, Taylor, or like for the community, for the company?

I guess probably company is what I was thinking, but you could touch on both.

Gotcha. So yeah, to get followers, I think like again, if you build a community, then you'll have more people talking about your product on Twitter or what have you. And then you can like make sure anytime someone tweets at you, you like make them feel like the smartest person ever, like the most special person ever. You know, you highlight every post, I think. Hugging Face in our space does this really well. Anytime someone builds something on Hugging Face, they just make them feel like amazing, and then more people want to do it. That's just the key.

Great. And Christine has a follow-up question about community as well.

Oh, hi.

Yeah, um, how do you keep the community on task or guide it back to your brand or content if it starts to move off in a completely different direction? You're acknowledging and recognizing them and making them feel like the heroes, but it goes in a totally different direction than what your brand is. Do you like kind of let that go runaway, or do you put in something to like gently guide them back to what you wanted to build it for in the first place?

Yeah, so that's a hard one because you don't want to stop this magic that's happening. It's so hard to create that magic, right? Ideally, you want to figure out a way, if people are doing that, maybe there is something interesting there. So maybe you need to like figure out how to meld your product and your community into wherever they're going. I would start by asking users, hey, why are you gravitating towards this and not that? See if there are even any users who want to do the thing that you want to do, because we had that too. Like, you know, when we were trying to get users to create these blog posts, like most of them were like, We're coders, we're not writers. I'm not writing a blog post, you know, so then we kind of were like, okay, we'll build a community around coding the thing that you want to do. Some people will do the thing we wanted them to do, not everyone, and that's just fine. So I don't have a great answer, but just talk to people and ask them why this and not that, and if you can create enough users to do this.

We have another question up from Bernard. Bernard, would you like to come off mute and ask? I will ask for Bernard. So Bernard's question is, can you elaborate on the growth tech stack and why you chose it? How does it track community ROI and attribution?

Yeah, so we have Marketo, Salesforce Outreach for like the pipeline piece. The, the name of the tool is totally skipping my mind. We use this for our community Yeah, if I think of it, I'll send the name later, but essentially it's like a CRM for all of your community users, and you can bring in all of the people who are on Twitter, LinkedIn, people who are in your product. You can build— they have an API, so you can track any user activity. So we suck all of that data in, and then essentially what they do is they sort the people in your community by reach, love, and they have another metric. So like if someone with 50,000 followers on Twitter comes in, they go right up to the top. If someone with 10,000, with 10 followers does a bunch of stuff in your product, that's love. The love metric increases, they bubble up to the top and they have a mixture of those three, like parameters that they use to sort your users. And then you can tag them in certain ways. You can message them in certain ways. And I know I should know the name of this, but I'll think of it. Posted in here. And then we also built a lot of stuff custom. Like if I could just show you guys one of our pride and joys. So it's called Fully Connected. And we spent an insane amount of engineering resources. So this is our blogging platform. I'm sorry, it's slow. That's probably me, not our actual website. And this is where people are posting content. So here you can see all the articles, all the news, all the events. We have a podcast too. All of that is here. But essentially, if you click into any of these, and these are written by my team a little bit, but mostly it's other people. So like this guy Vincent, he wrote this beautiful blog post about how to fine-tune a new model that came out, and then he put Weights and Biases plots and he put code in there, you know, and people really, really like it. So this infrastructure was all built custom because we realized we really want to invest in this UGC. This is the weights and biases version.

We have a couple follow-up questions. Chen, I know you had a couple other questions, so I'll pass it over to you.

Thank you, thank you. It's me again. I have another kind of organizational related question, is two parts. One is, what kind of persona do you have on your team? Like, like, how do you manage them? And then secondly, how do you enable your sales team and also influence your product team externally?

Totally. Also, sorry, I just thought of the name of the tool. It's called Orbit. I think the domain is orbit.love. Yep, that's the tool for community. Okay, so coming back to this question, that has changed over time, like the structure of the team, and I'll just quickly walk you through what it looked like. So initially we were 10 people. I joined, I was the machine learning engineer creating content, and then the first few hires were all machine learning engineers and one writer, just to make sure someone was editing all the work that the machine learning engineers were doing. And then we were also creating code. So the first 10 hires, machine learning engineers and writers. Then we hired an events person because we realized we were doing a bunch of events. Machine learning engineers are not the best at following up and doing the logistics. And then we started to own this pipeline number, not just the user growth number. So then we brought in the marketing function. So our SEO guy, our dimension person, our video guy, all of that, you know, nurture person, ABM person. So, but starting with the technical persona made sure that no matter what the marketing side did, it was so rooted in our users because the machine learning engineers were talking to all of them. They were helping write the nurture emails, they were helping write the ads. So it was all focused on the machine learning engineers, and that actually comes from our founders. So Our CEO is a machine learning engineer. Our other two founders are machine learning engineers. They look for that messaging. If I just started talking to the exact persona, they'd be like, "Lavanya, what are you doing?" You know?

For sure. We'll come back to you with another question.

It's me again. So I wanted to see if you had any insights or tips around tracking attribution for hard-to-track channels like YouTube, TikTok, and you know, all these new form media out there? It is really hard.

And there's no silver bullet, but like the thing, and it'll depend on your product. So sorry, I don't have an amazing answer, but what we do is essentially what you're trying to do is when someone looks at the YouTube video, you just want them to click something that has attribution. You can put UTM parameters in there. So like, it's all about like How can I get someone to click it? You know, and like all the creative ways to do that. We don't gate any content because our whole thing is like, you know, anyone can access anything and then if they choose. So I guess, yeah, just figure out how to get them to click something that has your teams.

That makes sense. One thing that I had in mind was like, you know, during the onboarding process, adding a field that says, hey, where did you find us? I wanted to see if you had any data around like effectiveness of those.

Yeah, that was super effective in the early days because it helped us almost confirm what we knew. And then there was a couple of things like we were sponsoring this big guy on YouTube, Janek, and we could never see it in our attribution. But so many people were like, oh, I heard about you from Janek. So that was helpful. But then you want to take it out or like put it for one week every year because the answers don't change that much and it's still there, you know. Thank you.

Lavanya, a lot of folks probably don't have a team set up quite like yours with the mix of technical engineering folks and marketing. What are your best recommendations for other groups of how those two personas can work better together?

Yeah, that's a good question. So from the top, it— you have to from the CEO onwards, you have to tell the team like, hey, our focus is this technical persona. That's who we're talking to first. Because even the execs, like if you talk to the technical persona, you're talking at an elevated level and those people want to be around those people. They want to know what's going on, you know. So first from the top, say that's our first priority. And then everything that you do, even the marketing team, whatever they do has to be from the standpoint of the machine learning persona. So they are always keeping these technical people in the loop no matter what they're doing. The other thing, we do have a little bit of this tussle now where it's like, OK, if you're optimizing for pipeline, we want to do things, some things that are not good for the machine learning persona. So where's the trade-off? But even then, we found if you still focus on the machine learning persona, we do end up getting the execs like, this is what's right for the company, at least for us. This leads to more pipeline. For example, we run, we go to these conferences, the biggest conferences that have machine learning engineers. That leads to more pipeline than like a re:Invent, Amazon re:Invent, or NVIDIA GTC. You would think that GTC would lead to more pipeline, but not really, you know? Because the execs that are coming there don't care as much to like follow up with you and like actually go and like sign up for your product. But if you get the execs that are going to a technical conference, really care about getting to know, like trying new products or what have you. I don't know if that answered your question.

No, thank you so much, Lavanya. Any other questions while we have Lavanya here? Lavanya, thank you so much as always, everyone. This full session will be live on YouTube immediately after, so make sure that you're subscribed to the SAS page. And the slides will also be available on SlideShare. So make sure that you're following us there as well. And we will be back next week again, as always, for Workshop Wednesday. And we will be having Sterling Snow, partner at Peloton Venture Partners and the former CRO of Divi. He is a fan favorite here at Saastr. So really excited to have him back to talk about some revenue topics. And we look forward to seeing you then. So thanks everyone. Lavanya, thank you so much. And we'll see you next week, hopefully.

Okay.