SaaStr AI Day: Why Every Role Is a Prompt Engineering Role: No Code AI Tips with Runway's CEO

drop in the chat where you're joining us from. Uh, we love to see all the different regions people are joining and attending from. Okay. Perfect. We can go ahead and get started. So like I said, welcome everyone to AI Day, our very first, uh, AI Day ever. So we appreciate everyone being here. Um, we are so thrilled to have Siki Chen with us today, who is the founder and CEO of Runway. And this afternoon, he's going to be discussing why every role is a prompt engineering role. He's going to give us some no-code AI tips. So just a quick note for everyone watching, uh, we do like to make these sessions as interactive as possible. So if you're comfortable, you can turn your camera on. Uh, like I said, drop in the chat where you're tuning in from, and then if you have any questions pop up while Siki is doing his presentation, please feel free to drop them in the chat below, and he will have a few minutes at the end to answer any questions you may have. So please, Siki, take it away.

All right. Thank you, Taylor. Thanks for coming today. Really excited to talk about AI with this fine group for Saastr. But I'm Siki. I run a company called Runway.com. We're a next-generation finance platform. And outside of my day job, I actually am really, really into AI, fairly deep into it. In fact, I built what was at one time, I think, the number 1 ranked GPT in the ChatGPT store. I think today is number 3. And the things I would like to talk to you today are 3. The first is why every role in a company has become a prompt engineering role. And with that, we'll love to show you 4 of the AI tools we built without any code to increase our operational leverage on running one way, and how we develop an AI-first culture to have people who are— have technical backgrounds or even have no engineering background whatsoever to develop these AI tools and use AI to increase your leverage. So let's start with what this weird belief is. So the way I think about AI inside companies is that it is a force multiplier for human capital. What AI can do, I could personally consider the greatest invention since fire. But what it allows you to do is turn one person's ability to get output to 2 or 3 or 10 or 100. And it is— there's never been a time to be an operator inside a technology company. And if you ask Sam Altman, he has this amazing little clip that I want to quickly play for you all and how he thinks about what AI can enable humans to do now. You said it, Sam. Um, but seriously, yeah, this is happening and it's happening now. And so let's not play that. So what I would love to start doing is just show you some of the tools we built at Runway. And aside from one of them, most of them were built with no engineering support whatsoever. And I'd love to show you how it works and even the prompts that we use to create some of these. So the first one that is interesting is sales qualification. So when you sign up and you enter an email on our website to enter our waiting list at runway.com, I'm going to just show you the full Zapier flow of what happens. So when an email comes into our website, it goes to a database and then it gets, goes through a Zapier automation. And we have a ChatGPT prompt to filter out all of the free email accounts. And we just described, hey, Gmail and Yahoo are free mail accounts and filter out anything like that. And it'll just do that. And then we enrich this email with Clearbit information. So we know where this company, where this person works, how big the company is, and what their role is, and see if it's a qualified lead. And then we will actually draft a customized email for that person and put that draft inside the draft folder in Gmail of a sales representative. So all they have to do when they come into work in the morning is go to their draft folder and press send. So I will show you some example outputs of, uh, of what this looks like. So we also link it to Slack. And so when Jeff Bezos, the Amazon CEO, signs up for Runway, obviously this is a sample test email. This is what we get in Slack. And everything that's below where it says Runway sells Copilot is actually written by GPT-4. So it'll say Jeff Bezos is the CEO of Amazon and showed an interest in our services. Amazon is a global powerhouse of 100,000 employees and so on and so on. And given Amazon's impressive revenue team size and strong alignment with our ICP, They are a highly qualified lead with a significant potential for collaboration. All of this is written by AI. And then the next step is we'll actually draft an email. So this isn't a real lead. It's not Jeff Bezos in this particular case. But this email would be in the drafts folder already. And it says, hey Greg, Cyrus item come through. We'd love to show you Runway and see if we can be helpful to your company. In this case, it's a pharma company. While you're busy metering out doses of innovation in pharma, let's inject some fiscal vitality into your veins. With Runway's financial planning. So we try to have some kind of clever connection to what the company does and what we do in the copy. And so how does that work? So this is actually the raw prompt that we're using here in the raw prompt they're using to qualify the leads. And it says, you are a highly intelligent sales qualification bot and will examine raw data from Clearbit And we describe what our ICP is. We just tell them leads are qualified, will always have at least 30 people, $1 million in AR, be based in the United States, be executive business decision maker. And because we have AI, it's able to reason about that from the raw data in Clearbit. And the thing that I would point out here is you can, the way you would engineer this, this is the same way you would describe it to a person who works in your team, right? This is what ICP is. Here's how you think about it. And that is exactly the same way that you would prompt an AI to do it for you. And you can automate that without no-code through something like Zapier or some other tool. And we have these other things where we say the quality of writing should be 11/10. And when you do these, like, when you add these words of encouragement the same way you would encourage a person, it does a slightly better job. Here is the same prompt for how we write the copy. You are a phenomenal copywriter. They'll examine the raw customer data and personalize below template. And return a full body template, not just a snippet. And so we ask it to be authentic, irreverent, casual, edgy, witty, and just make sure the customization is one short sentence. Be genuinely witty, authentic, clever. And we give them some context of what Runway does, and this is how the copy is created. So that's one example of how we do it. And happy to answer questions at the end. Another thing that we built here at Runway, and this is for my own personal use, is we triage emails in Runway. Uh, uh, uh, we triage emails with AI. And so here's a pretty basic, um, Zapier automation that you can build very quickly, but basically a new email comes in and I will have GPT-4 read the email and depending on the contents of the email, it would add a correct label. And if it's not an important email, it'll actually skip the inbox. So here's what that prompt looks like. You are a super intelligent email organizer. Give us some information about an email. You will reason out loud about the contents of the email. And also return the best label to categorize it. The thing that I want to highlight here is asking ChatGPT or, or GPT-4 to reason out loud helps become— make it become smarter. In the same way that, you know, when we're thinking about a hard problem as human beings, speaking out loud or, like, writing things down helps us reason a little bit better, GPT and AI works the same way. And so I give it a lot of context about What kinds of emails go to what label, right? So family emails are like, I tell them who my family is and my relationships with, with those people and knows to put things related to kids in kids and family to family. Uh, very important emails. I describe what's really extremely important to me. These are things that are really appropriate to me for as the CEO of Runway or things that are from our internal tools like Hyde or Slack or important conversations with investors. And the other thing I also do is I try to give it a bit more encouragement. Begin your response by taking a deep breath, then reason out loud. Now, you, you might think that I might be making this up, but there, this is actually from academic papers. When you tell it to take a deep breath or think out loud, the capabilities of the prompt actually increases, and you can read about these papers in arXiv. And so we really try to put every trick in the book to make it as capable and smart as possible. I see there's a question, reason out loud, why do this for AI? That's a really great question. So the thing about the way LLMs work is they generate tokens. And by having a reason out loud, you are generating more tokens and thereby giving it more context to make a decision. And so if I just had to say, okay, like, say a yes or no, the capabilities of it is gonna be lower because it didn't have time to think. And so what it's actually doing is you're giving it more time to think by asking to generate more tokens, which is what taking a deep breath and reasoning out loud does for an LM. It is like really, it's, it's actually intuitive. It's just surprising to people because this is not the way you would interact with any other kind of technology. But AI is AI precisely because it is like uncannily weirdly human. And if you ask it to do human tricks, it will like try to do better. One of the interesting papers I read recently, and this is an aside, is there was a paper that talked about how if you ask AI, if you tell AI my job depends on it, it will try a bit harder, which is kind of funny. But anyway, another thing, you know, this is like something we built inside a Runway product is if you think about scenario planning, for example, in finance, if you, you as a CEO have a question about some scenario, you might ask the CFO a question about how you can increase revenue next year and not run out of money. And, you know, the CFO or whoever owns the model will go to your spreadsheet and make some changes. And that's something you can automate with AI. And that's like, it's part of the things that we built here. So I'll just play this quick demo video of how we can do that with Runway.

So imagine you're doing some annual planning and you have a question about how you grow revenue. So So he asked Runway, hey Runway, can you figure out a plan for us to triple revenue next year and not run out of money? And because we have the concept of plans at Runway, it will look at the semantic structure of your model and any of your existing plans and said we'll create a scenario and named it 3x revenue, named a page after it, put some visualizations in place and then explains what it did. So here's another triples of revenue 2023. It focused on 3 key areas, increasing sales on account, decreasing churn, and boosting expansion MMR. And what's cool is once it's in Runway, you can also fine-tune it. So you can say, by the end of 2025, we expect to have, and you can reference anything in your model text. So ARR value as of December. AR, right? And the cool thing is because plans exist, as you change your plan, let's say you move your marketing plan up, what's going to happen to this $12 million in AR in '25? Well, it'll increase by $3 million in real time. So some of the AI capabilities we're building at Runway, that's pretty handy.

So that is obviously a tool that requires a bit of engineering for that to work. Um, the first way we use it, um, and I know we're at time, so I'm going to go pretty quick. We built this bot called Jamf Session in Runway, and what it will do is it'll let you— it'll give feedback and critique any document that you have in Runway in the voice of whoever you want. So here we have a document for Runway values, and I just tag @JamfSession with Steve Jobs, and this bot will go through this document. It will know that it got notified and read through it and quote passages from it and give specific pieces of feedback in the voice of this person. So whether that's Steve Jobs in this case or Elon Musk, Elon Musk or Daffy Duck, it will have a conversation. You'll be able to have a conversation with them. And this is already working. This is a real instance of usage in our own Notion document. But, you know, this is Steve Jobs critiquing our values. I always believe in the nation needs urgency, but, you know, it needs to be balanced with impeccable focus on quality, etc. And this is available to everyone in our company. And this is the prompt. We talk, we ask it for the insights to be really thought through and deep and non-obvious, even to an expert on the topic. And you'll notice at the end, I also add this trick where the feedback wants to be deeply original opinion, thought-provoking, and clear. My job and life itself depends on it. And so what we find is that these LLMs actually do respond to guilt-tripping. And it will increase its own capabilities if you add it. So how do we do this? How do you actually, like, build such a culture in your own company? So there's 3 things that work for us is one, we made it really clear that this is a strategic priority for the company. Every department lead knows that it is our goal to never have a company more than 100 people. And what we want to do is scale with increasing capabilities of AI such that one person can do the job of 10 and 100. And in response, another thing that we do is before we open up a headcount and add more, more people to a company, we first think about what can we do to automate this? AI tools are increasing day by day very rapidly, and there's things you can do immediately, probably before you need to hire another person. And that's the first thing we think about. And the third thing is we run hackathons often at Runway, and sometimes we just run AI-specific hackathon. And the key thing here is typically in hackathon, you will have like, you know, the engineers do some work and they'll be able to do a demo. And everyone else kind of just watches. But with an AI and no-code tools, you can involve the whole team. Like 3 out of 4 of these tools was developed with no engineering involvement whatsoever. And so that's how we've done it at Runway. And that's what I want to leave you with. That is the truth that every role is becoming a prompt engineering role in every company. And if your company does not adjust to it, you will be outcompeted in terms of operational efficiency by 10-person companies, 100-person companies doing the job of 1,000 and 10,000 people. And you can do a lot today without having to write any code. And it's imperative that every organization develops an AI-first culture today. So that's me. And that's it. Thanks for listening, and I'm happy to answer any more questions.

Amazing. Thank you so much. Those were tons of helpful tips. So we did get a few questions on our livestream channels. So First one being, what are some ways as a CEO that you make AI a strategic priority?

I— so actually, I see our head of sales, Richard, Richard, in the, in the, in the chat. But yeah, I mean, he'll tell you one of the things before we hired, I asked him, he asked me what the vision is for sales. And it's like, I want to have the most AI-leveraged sales team we have. And so As a CEO, you know, you set the strategic priorities. And if you make clear that this is a strategic priority in every function and you hold people accountable to them, that is how you do it.

Amazing. Okay. And then another one from our livestream. What are you seeing your customers are demanding now as a result of AI?

I mean, the biggest thing for any customer is like, what is your biggest pain? And they're always like a biggest pain. And when you, when you're Product does more, you uncover more pains as it does more. And so one of the biggest pains that you might have in finance is saving time, right? So think about like how much time it takes you to operationalize your model every month and get— pull data from like 20 different data sources, how much time it takes to collaborate with the rest of the team, report out to different investors and get context from different departments. All of that can be made better through AI-enabled tools. AI can look at your data, and be able to, like, transform it in the way you want to. It can help you collaborate with your team and explain how the model works so that people actually understand how the business works. And it can help you create scenarios faster, as you've seen in that video. And so most of this is pretty specific to our customers and is around saving time and collaboration. Awesome.

And then Patrick just dropped another question in the chat. Is there a way for you to identify which enterprise workflows are best placed to be optimized with AI?

Yeah, that's a great question. And really, it's like, anything that you haven't automated before that takes a lot of time that you do repeatedly, right? So, you know, what is a workflow that just happens all the time? In our case, the reason why we automated Suave's sales qualification is we had his website go viral a few months ago, and we were getting slammed with like hundreds of leads a minute. And so a human being would have to— actually did have to go through all the leads, and it's almost more than a full-time job. So the thing about AI is that it is artificial intelligence, and it's actually quite intelligent. So a lot of things that used to be done by humans where you have to, like, use human reasoning to make a decision, AI can do it for you. So basically any repetitive workflow that's currently done by a human that you think might require some kind of human reasoning, can be probably automated with AI very quickly.

All right. Yeah. Amazing tips. So we are at time. So thank you so much, Sikui, for joining us and giving us all of those amazing tips. And thank you to all of our viewers who attended this session. This was recorded so you can reference it in the future on all of our social channels and reference his slides as well. So thank you again, Sikui. We really appreciated this presentation and hope everyone has a great rest of your day.