Adding AI to SaaS: Inside the AI Product Strategies of Figma, Cloudflare, GitHub and Ramp

This is unbelievable. We have here an all-star panel. In my mind, this is like if we had the Marc Andreessens in the world in 1995 talking about the internet. These four have built some of the most widely used AI features in production right now, and we're gonna hear the real stories behind them. What we're gonna do is the first half, each person's just gonna tell the story of the biggest AI feature of their company. And then we're gonna discuss as a group the critical lessons learned. Y'all ready?

Yeah.

We're starting with Mario, who is the CPO at GitHub and personally helped build Copilot to where it is today. Copilot has 1.8 million paying users from 77,000 companies. It is the most used AI dev tool in production today, and you're gonna hear the story.

Thank you. How are you doing today? There you go. So today I want to talk to you about Copilot. It's interesting. And it's called— which is one of the co-founders of GitHub recently posted GitHub won back in 2008 because of two things. One is it was the right product at the right time. And it was Git is where everything was coming together. Where all open source was coming together and working in a decentralized way, and none of the centralized version control systems could actually handle that. And the second thing that he listed off why GitHub won was because the founders had taste. And that taste actually talked about them being dev-centric, them being developers and creating a tool for them. And that has been at the heart of GitHub since day one. It's this developer love, this obsession about what do developers need to be successful. I have been doing dev tools for 20+ years now. Actually, I'll probably retire doing dev tools. I love doing that for a living and making tools for all of you. If you think about it, humanity already invented the fire and the wheel, so everything else we're going to be doing from here on, It's gonna be powered by software. So I get a lot of energy about giving you all the best tools, uh, possibly. And that's what happened with Copilot at the very beginning. If you think about it, like the first time that I saw it was in 2021. This was before ChatGPT was a thing, and we were just playing with this model called Codex, and that Codex model was a technology. You could do really interesting things. When it comes to natural language, but we didn't know what to do with it. And that, that's actually a challenge we face constantly in this technology world, which is you got something cool, you have a tech demo, how do you turn that into a product? And what we ended up doing was obsessing about that developer experience. We had introduced Ghost Text. We work out really hard in making the latency be 100 milliseconds, 150 milliseconds, no matter where you were in the world. And then we ended up doing a lot of prompt engineering. So if you are in the audience and you're great up on prompt engineering, I wanna talk to you and hire you. But like, it's because it's a real thing. Like, it takes a lot to actually do the— pass the right context to these models and get the right answer. And that's how we are where we are today is by generating value, not through a technology, but a product. And the story of Copilot is that is one of being in the right place at the right time, and also having that taste and really being a steward of those developers and the people using it. So what is the future then? And for us, the future looks a little bit past the developer in this case. And I was recently in a podcast where I was saying, we need to be able to reach a billion people and make them software developers. And you're not going to be able to do that overall by being professional developers. So what I'm really excited about the next generation of Copilot is transforming into this product that your grandmother, your child, or you yourself, even if you're not a programmer, can actually interact and create something overall. And if we're able to do that, I think we're going to really accelerate humanity forward. So thanks for that.

Amazing. Next is Diego Zackx. He's the VP of Design at Ramped. Which is a financial ops platform, but probably all of you have heard of Ramp because they're one of the fastest growing SaaS platforms in history. They reached $300 million of ARR in about 3 years. 25,000 businesses use Ramp. They, and they use AI to save those 25,000 businesses about $1 billion and 10 million minutes. And, and it's just incredible what they've built.

Yeah, thank, thanks for having me and, and thank y'all for coming. Who, who here is expensing this trip?

Yeah.

Amazing. That's what we do. We make that super painless for everyone. We use a lot of AI for this. We build software that helps companies spend. All non-payroll spend goes through us, so we serve the finance function, but we also serve the employees, right? The people who are spending to do the job. So we see ourselves as a We're really a company that sits in the middle of that relationship between finance and the rest of the humans in the company. And it's easy to, when you're submitting an expense report, it's easy to forget that finance people are people too, and they're just, they just want to help you do your job. So we make their lives a lot easier and hopefully make all of your lives easier as well. We've been using AI from the beginning and it's really been at the core, but I, I do want, you know, in all of this AI stuff, we, we talk a lot about the tech, the AI itself, and, and I think for us at least, AI is not the product, is the product, and AI is one of the ways in which we make people's lives easier. So the, the main thing that, that I try to focus our practice around AI around is, does it feel like magic? And magic could be achieved through AI, like some really complex thing, but it's also sending an email at the right time to someone. There's putting the button in the right place and having a really tight piece of copy that just makes something super easy to understand, painless for someone. So if it feels like magic, it's AI in my book. In order to get to that, first we root in real user problems. We make that technology, the experience disappear. We ask not how can we make the experience better or slightly faster, is like, how do we make it go away? That's really where AI shines, where, hey, and I'll show a couple of examples of that. Then finally, how do you operate at the right altitude between, hey, the AI is just being helpful, providing a suggestion, saying, hey, you might be missing some information, to fully doing something for me. There's a huge range between you might want to book a flight to go to Saster to I just show up at the airport and it's taken care for me. That last bit that I was talking about, that's where we're exploring at the edges of what is possible with technology. So in order to do that, we hire just the best engineering talent, people who are builders, people who are curious, and we make it low risk for them to just play with AI and try stuff. So we have a range of things. If you're expensing a dinner, you just spend on a ramp card. We look at all the data of the merchant, the network, the company, your spending. We generate some memos, possible reasons for that expense. You just reply with number 2 and everything is done. That's it. So AI is not here, like it's just doing it in the background. And then you go from the more obvious things to the much more magical things. So that expense was categorized by the accounting team. You upload an invoice, they're super hard to read. We transcribe the whole thing to how much did we spend on marketing? and instead of just very confidently like spitting out a number, we produce and manipulate the ramp UI so that you can actually see the data and get to that answer yourself. And then like, it's like a first draft of a report or a spreadsheet. And then we also have those amazing engineers who, when paired with that designer, get to explore a lot of experimental stuff. So if you've ever had to do a thing 50 times on your computer, by the time you've done the third one, third round, you're like, come on, why can't you just figure out that I'm gonna do this another 40 times? So that's what we built. Once you do something on RAMP a couple times, we essentially predict the next action. And instead of clicking around, we just make it a keyboard command and just hit enter and just get through the list. And here's where like AI is not always the solution. The problem here is not that it takes a lot of clicks to do 50 things, is that we're making you do 50 things. So what if we don't even— that's the actual solution. Don't make me do 50 things at once instead of trying to AI suggest it. But so big range, let people play and just pair them with designers to figure out how to bring that tech into a real user problem. And yeah, explore.

Next is Dane Connect. He is the SVP of emerging technologies and incubation at Cloudflare. Cloudflare is one of the largest computing networks. They, it's a $30 billion company. A third of the Fortune 500 uses them. It's quickly become one of the leading AI clouds. It's where millions of developers are deploying AI apps. And Dane's only mistake in his whole career was 10 years ago hiring me.

Thanks Danny, and thanks, uh, for having me. So yeah, a lot of you guys know what Cloudflare for— sitting in front of a large portion of the internet, uh, and protecting it from DDoS attacks and, uh, helping make it faster. Um, about, uh, 10 years ago, we started actually opening it up so that we wanted to be able to make it so that everybody could build applications the way Cloudflare builds them, where we don't have to worry about scale, we don't have to worry about which— picking a region for where it's gonna run, and, uh, really making it a cloud where not just we can build our internal applications, but everyone here can build those applications. And in the past year, you know, we continue to add primitives for how you can build these full-stack applications. We've deployed things, I think things like databases, storage, and then most recently we added GPU storage, so you can start to build AI-enabled applications. So whatever you write in, where GitHub makes, GitHub Copilot makes it easy for you to write that software and focus on building your business logic and get your applications out faster. We want to be the place for you to deploy those applications and we hide the infrastructure, the scaling challenges that you might have. So if your app goes viral, you won't have to worry about it falling over when all of those users are finally there to see what great things you've built. But when we started building, deploying, adding the AI primitive, we were trying to figure out where's the best place that people really wanna run AI? And the answer was a lot of actual places based on the circumstances. If you have a really powerful iPhone, the best place is the device. But a lot of people don't have those powerful iPhones. A lot of places where you wanna run this stuff is not gonna be there. So the next best place we figured was gonna be the edge. And so that's why we made the decision to deploy the GPUs to the edge and make it built into the Workers platform or serverless platform where you can quickly call any of the open source models, deploy your models, and run that Workers inference right there at the edge or fall back and use our AI gateway. We need to route back to one of the larger LLMs like OpenAI to, ask some of the larger questions so your app can handle all different types of, uh, uh, the AI workloads, uh, that you possibly need. So yeah, in the past year, uh, we've, uh, retrofitted, uh, our 300 cities, 500, over 500 data centers, uh, with, uh, GPUs. Over 170 of them, uh, have them today and, uh, pushing out more bigger, faster inference, uh, every day. And, uh, really excited to see what you guys build. And really what, what you said at Ramp really resonated was AI's become really powerful when you're building into your application, not as a chat interface, but really as something that disappears and, uh, becomes part of it. And, uh, uh, wanna help you guys do that.

Amazing.

And lastly, we have Vincent van der Muelen, who is leading AI efforts at Figma.

Hey everyone. It's going to be hard to close out this amazing lineup, but I'm going to try. So my time at Figma actually started outside of Figma as a fan. So I was at a design tool startup called Diagram and we were making these AI plugins for Figma. And one day Dylan called and he was like, do you want to join? And for me, it was like being invited to Santa's workshop. Of course, like I want to be where the magic happens. So Diagram decided to join Figma as of last year, and now I get to build these AI features at Figma itself, which is truly a dream come true. Last year, we set about creating the first AI features for Figma. And where do you even start? One of our first projects that we did as a company in the AI space was JamBots, and this actually came out of a hackathon. And the idea was that you can go to ChatGPT and you can brainstorm and you can riff. But how do we make it more fun? Can we make a widget for our online whiteboarding tool, FigJam, that lets you interact with ChatGPT and other APIs in a much more expressive, visual, collaborative way? So we built this widget that makes it a ton of fun to interact with ChatGPT, and that was our first foray into AI, and it really made us want more and made us very excited about what we could do in this space. So we started there and then for the past year, we've been wondering, we've been very focused on building our first AI features and creating this bundle for more professional designers. So when we decided to set out, when we made this bundle, we really wanted to focus on, on, we really wanted to focus on complementing designers and not replacing them. So we asked ourselves, what are features that designers might want? So one of the features that we focused on was AI search. As designers, we lose designs all the time. Can we help you find them faster using AI? Another one was make prototype. Creating prototypes takes a lot of work. What if we can get you 80% of the way there using AI? And then of course, we've all had the pain of naming our own layers. What if AI would just do that for us? So this is our AI bundle, and there's many more features, and we're now excited to iterate on these. And when we think about our AI features, we really think about a few key principles that we always try to adhere to. Um, Jago and like RAMP, we really try to root this in existing user behaviors. So we try to ask ourselves, what problems are users experiencing? And is this even a problem that AI needs to solve? And then from there, we try to build these prototypes and the key there is really frequent iteration. So we have an internal Figma build that we ship experiments to constantly. We also do internal research. We do external research because once you have a prototype, it still takes a ton of time and a ton of living with the prototype to get it right. And then of course, AI is notorious for not having the right level of quality. So how do we actually evaluate and make sure that these AI features have enough, have the right quality to begin with? So a big focus is evals. So we set for every feature that we build, we set up these little tests where we have the expected perfect outputs and we compare, we then compare that expected perfect output to the AI outputs. And we do that for every change. And only when the expected output is close enough only when the AI output is close enough to the expected output, that's when we actually put these features into the wild. And that's how we kind of combat these AI features sometimes being unreliable. And then lastly, we really think that AI is a space that we should all be involved in. So it's a very multidisciplinary effort. So I myself am a product engineer, but we also have designers, we have ML engineers, we have researchers, a ton of infra people, and together we one, think that leads to better ideas, but we also think it truly helps making these features great and keeping them up as they scale. So, I'm excited to chat about these topics today.

So, you've now met the incredible panelists. By way of introduction, I'm Dani. I'm the CEO at Jamf. We help 135,000 people fix bugs faster. Some of the largest companies in the world now use Jamf. Probably if you use the internet today, you've used a website built with Jamf. We're in the business of helping people build and ship great products, and here we have people who have built the most widely used AI features in the world. I'm, I'm so honored to get to moderate this panel. And the way that we came up with the questions is we asked every single one of them, what questions are you dying to know from the other panelists? What are the tactical things that GitHub wants to learn from Figma and vice versa? And so those are the questions we have for you now. Are you ready? So, AI is changing really fast. Also, all of you work at companies that do planning. And so how do you do product roadmap to be able to adapt, do things on the fly as things are possible, but also can have a roadmap meeting and, and be able to forecast what you're gonna work on this year, this quarter?

Mm-hmm.

That's it.

So I work in product, so this is something that I live and breathe every day. And what we try to do at GitHub is this concept of strategic roadmaps. And in a strategic roadmap, you don't have, what are we doing next quarter? What are we doing next month? Or anything like that. You have more, what are these bets that you're gonna be taking? And what is the question you're trying to answer on those bets that informs the next stepping stone of it? So it has this cone of what I would say confidence of where you're going. That is always rooted then on the strategy. So what we try to do is say, okay, this quarter it's gonna look like this, and we're trying to learn X, Y, and Z. And based on what we learned from this, we're gonna have this hypothesis and the next quarter might look like X and Y and Z. And we try to plan out in a strategic way, more or less a year, knowing that anything that we actually say 4 quarters from now is completely untrue. and if we actually are doing really good and sticking to that, then we played the lotto probably very significant amount of time. So it, that's more or less the way that we give ourselves, and even with our customers, we have a lot of enterprise customers as an example, and they want predictability, but in AI you cannot have predictability. Uh, LLMs are not predictable even on that. So on that end, what we try to do is making sure that we are achieving the right value in the product through a strategy. And then learning constantly about it. And then we revisit that strategic roadmap in what we call MPRs, of monthly product reviews, and sometimes it's every 6 weeks. It doesn't have to be every 4 weeks overall. And we try to ask the same question, okay, what does the next quarter look like and how can we, what did we learn and how can we affect it? And that's how we try to stay more or less as nimble as a startup, let's say. but still have the scale that we have today.

So, uh, Cloudflare does a sort of similar where, uh, the farther you go out, the less fidelity you have. We divide it into 3 different groups, uh, where we have a kind of a core product organization that really focusing on what is, what do we need to ship to customers next quarter or the quarter after that? What does sales need to drive revenue, to reduce churn, to make the customers happier? and, uh, we call that, uh, incremental innovation. Uh, then we have a, uh, another group my, my, my team runs is, uh, the, the second horizon where we kind of really focus on growing TAM and really trying to, uh, do things that are, uh, 12, 18 months out. Uh, we're, uh, uh, focusing on what are those kind of those next big strategic pillars that we want to start experimenting with, investigating, and starting to gain a better understanding of. And then we even have a third horizon where the research, a research team, which really focuses on not even with the idea of how you productize something, but really just kind of more the fundamental technology of, for us, how you build a better internet. And a lot of the work over the past few years has been in the AI space, in which that will help inform the roadmap of the other two horizons as we go forward. So, you know, I think it's, you know, you want to know where you're going long-term, but you also want to remain agile. And, you know, I think every 4 or 6 weeks, really, if you're not reviewing the roadmap and really asking yourself, is the next thing that we're building the right thing? You're probably waiting too long.

I think one of the things where, and I think you guys are coming from the product world, Vincent and I coming from the design world, I think designers are very comfortable doing 99 things, finding 99 ways that something does not work before something makes sense. So at least for us, design being such a core part of Ramp, we're very comfortable in not really knowing where we're gonna end up 18 months from now. We do know what we wanna do in the next 3 months. That's just execution mode. We don't actually know exactly how that's gonna work out, but we got that quarter locked down. But understanding that We're making two assumptions. One, it's okay not to know exactly what shape that's going to take. And that's part of the, like, core underlying assumption that we're going to figure it out. And that same kind of optimism that's backed by people who work really hard and are actually super smart, we apply the same thing to AI. It's, this thing doesn't really work today, but in two weeks there's another model and it's Mike Worthen. So even if the latency is like 800 milliseconds now, we're probably going to be able to get it to 100 within the next 4 months. So it's really about focusing not so much on the things that we cannot do, but on the things that we want to do and being highly optimistic about what we can achieve within that year, rather than trying to figure it out, decide today with the limitations that we have today, which in the world of AI that it's like a complete, that just leads to complete failure within weeks.

Yeah, I think it's the same for us. One thing we like to do is at least somewhat like skate towards where the puck is headed. So we try to, and this is hard, but we try to somewhat predict, okay, what do these AI models look like a few years from now? Like it doesn't take a genius right now to figure out that in a few years these LLMs are going to understand video.

So.

How might that affect us, our roadmap and what we want to do? But we also try to stay very flexible. So like at Figma, we have a lot of hackathons and one of our big hackathons is called Maker Week. And when Maker Week happens, we go into that week of hacking with the full expectation that this might completely change our roadmap. And in fact, right now, like the big project I'm working on and that I expect to work now work on for the next year actually came out of a hackathon, like 4 weeks ago or something. So just to show you, like, you don't have to have these plans set in stone. They can change and give your employees, like, the opportunity to actually come forward with these great AI ideas.

Yeah.

I think one thing I heard across is also giving the people in teams room to experiment. Uh, I think it's even more important, uh, in the speed of the AI technologies moving today, where you give people, uh, all your teams a chance to actually Um, not write long docs and, uh, sit in an ivory tower and say, this is what's gonna happen, but actually go and say, how, how can I use these and touch them, use them, feel them? And, uh, so making sure that your teams have those tools so that they actually can experiment and, and the time.

Dane, what does that look like in your team? How do you give people time to experiment, like tactically, day to day?

I think early on we, we really start with very small teams where we give 2 or 3 people that are passionate about an idea, just really the chance to just go ahead and build something and prove something out. You know, I love demos, so I love watching a team of a couple engineers and product people, designers go off and come back in a few weeks with the greatest possible end-to-end demo that they can do. So like actually proving out things. from the start to the finish, uh, and seeing how fast they can do that and seeing what they learn. Um, and then really investing behind that. Uh, so, you know, which kind of goes back to the experimentation, like you need to invest in and create a place where it's okay to fail. And because the failure might be that it might not be a failure of the idea, it might be a failure of the models just aren't there yet and you need to wait for the technology to catch up. And, but the speed things are going, it will.

I think Vincent was talking about evaluation and at GitHub, we invest in this thing we call Coffee, which is Compiler Offline Evaluation. And that's one area that I have found to be a place that you have to overinvest in AI if you really, if you need to experiment, because you could experiment all day in AI and not accomplish anything without being able to understand how evals are going. So for us, the way that we translate and we give the ability and the autonomy to the teams is saying, anytime you experiment, you're doing offline evals and move quickly to that offline eval place. Because even if you get an offline 95, by the way, usually that means the product, when it gets to market, it's a really bad product, at least in AI world, in my opinion. So you're going to have to then do online eval to try to figure it out. Is it going to fail 50% of the time? Or 30% of the time or 20% of the time in the user. So that, that's the way that we encourage a little bit of that. And we also then have a GitHub Next team that looks at Verizon 3 items or Horizon 3 items like Dane was saying as well. So that's a little bit how we do it.

What does that look like in practice?

The GitHub or coffee?

The eval, like you have a team that's doing a test suite or like how do you implement this in an organization?

Yeah, it's not that— I would say it's very analytical is what I'm trying to say is you have this thing, you end up figuring out what are the test cases that you need to go and run and that you need to pick the right data. So you need people that are in applied science that really understand that. You need deep product and engineering to understand, okay, how are we going to actually then create the evaluation? To be able to score on that. So it's very similar to benchmarking if you want to think about it. I don't like benchmarks out there, by the way, because you could game every single one of them, in my opinion. But what I do like about benchmarking is that it gives you a view into a set of scenarios that then you could then figure out, are you getting better on those or not? So I think about offline evals mainly as benchmarks on are you going in the right direction with your hypothesis or not? And you just run a lot of them. Just recently I was reading a post by one of the teams and what they were doing is, you know, a lot of these models right now can actually do function calling and, you know, 4.0 is really good at being able to say, okay, I'm gonna call this function instead of this function. But when you have 30,000 functions that you could call, and that's a lot, we don't have that many, but imagine that world, then how do you make then the model pick the right one becomes a really interesting challenge. And we were trying with, can we create agents in between with domain expertise? And then 4.0 can just go and pick the right agent, and then this agent figures out the functions on it. So you end up running that through evals and try to figure it out, hey, is the hypothesis or not? And you get back an answer usually within 2 to 3 weeks on some of those things. And then you just move forward from there.

What's so interesting about what Mario's talking about is testing in software is solved problem. We sort of know how to do it. We can come up with a list of edge cases. We can run through tests. We can even automate our tests. But with AI, it's non-deterministic software. It's really hard to know, is this ready to go out to users? Does this meet our product quality bar? I'm curious if any of you have any other good systems that you recommend or use internally for when AI is good enough to ship.

Well, so we, yeah, I, I wouldn't recommend this per se because it's a lot of work, but we, uh, build a lot of our own eval systems and I think it's important when you do evals that you do your evals in a way that matches your product. So Figma is a highly visual tool, so it wouldn't come to anyone's surprise that we do visual evals and that we also do our evals inside of Figma. So for instance, for our search feature, we would have the AI search model just spit out all the search results on the Figma infinite canvas, and then we could go in with a Figma plugin to actually rate those search results as good or bad. And we would do that every time we'd update our search model. So it's a really fun visual Figma-like way to inspect the results. And I, yeah, and that's the benefit of building your own evals. You could eval tooling, you can really make it fit to your use case and optimize what's important, which in our case is being able to zoom in on the design and being able to see, hey, is this actually a good result or does this just look like a good result at the surface level?

One thing that, one thing that all of us in this room are trying to figure out for our own products is how important is it to be first to market in AI versus sit back, watch the market, see what works, let everyone else do R&D, and then build it as well. And curious for the four of you, how do you think about that in your own organization?

I strongly believe that you have to put yourself out of a job every 18 months or someone else will do it for you. So that is the philosophy at Rabbit. It's all about velocity. It's about getting there first and second and third and fourth and fifth. That's really the key. It, we, in order to do that, we've, we set up basically two key beliefs, which is all we do is betting. As long as we're right 52% of the time, we're going to win. So make a lot of bets. Very quickly. And the second is make it really cheap and easy to correct, or when you're, whenever you're wrong, it's really easy to recover from and just do the next thing. So it's really everything around Ramp is geared around velocity of our ability to just make changes to our product instantly, every day, multiple times a day. And when we see something in the market that works, that someone got right, we have the muscle to just like literally go do it. And 2 weeks later we have something that kind of works or doesn't work at all. But then 2 weeks later it starts to work.

Cloudflare is very similar, actually. I spoke with your CTO a little while ago about this. You know, we focus on velocity and shots on goal. You want to get as many things out there as possible, get early user feedback, know that you can continue to, uh, uh, improve it and over time. Uh, but until you hear what the customers have to say, you, you, in a lot of cases, you're just guessing, uh, what the reaction's gonna be. And especially now that, uh, and I think that velocity is only gonna increase with tools like Copilot and, uh, all the developer tools that are being AI enabled. Today.

The alternative to what you're describing is what I see happen by default at a lot of companies or most companies, which is as people progress in their careers, they want to start making only good guesses, only the right guess. I want to be right most of the time. I want to be right all of the time. And then what that creates is it just takes you 6 months to have some confidence or more confidence in a guess, but you might still be wrong at the end of it. So it's this like self-fulfilling prophecy of if you are not just fearless and wrong all the time, you're going to still be wrong. It's just going to be super slow and probably someone's going to take your job. So it's— maybe. Yeah.

Yeah. You also said something about specifically about AI features. And so I think that's actually from a product perspective, I think that's a mistake. I think yes, marketing can slap AI on any of our stuff and hopefully get more attention and ride waves. But from a product perspective, the product team's focus, I like to encourage them to, what customer problem are we solving? And AI happens to be a great tool in our tool chest these days that it wasn't there a few years ago. We should use that as much as possible to help solve those customer problems faster, more efficiently, better, but not necessarily think about it as how can I ship AI features faster? It's how can I solve customers' problems faster?

I think we similarly want to move fast, but at the same time, I do think quality is very important, right? Like Figma, for instance, is a tool you use on a daily basis. If we start shipping AI features that don't meet your quality bar, very frustrating experience. So what I do like about what we do is, is that we have this plugin ecosystem, right? So even if sometimes we take our time to really get a feature right, we can encourage our plugin creators. Creators to fill in the gaps, right? So we have something like 200 AI Figma plugins right now. And I, of course, myself used to make them. So I'm sure GitHub might have something similar with their like extension ecosystem. That could be a powerful way to strike the balance of delivering quality features while also keeping up speed and giving your users like AI value on day one.

So A lot of us here are building out applied AI teams in our companies. You all have done that very successfully. What's the recipe for a great internal applied AI team? Who should be on this team? What should be the makeup of this team? Who is fit to lead this team?

I'll start this one then, maybe. And what I would say is there's two components, in my opinion, creating an AI product. One of them is the applied science component. And look, there's not that many PhD people out there in the world. Universities do a very good job of constraining that inflow. So you definitely need PhDs, in my opinion, in many of these fields. They're really involved overall. But one of the cool things is like having these LLMs, like OpenAI is innovating. You get Anthropic innovating, you get Mistral innovating. So you don't need access. They actually democratize a little bit. Getting, having to invest in very deep applied science and very deep in MLOps, in my opinion. So that's one of the benefits. I think you do need an applied science team. You need them to actually have a significant amount of understanding. So how LLMs and neural networks sometimes work, you need people that are very good in data and can create this evaluation suite and all of those items. But you also need, and you started to see this more, AI engineers, which is you take a developer today and now they have to inject intelligence into an application. In all of our applications now have intelligence, which means that we need to then move where the traditional developer is and then allow them to innovate on top of this technology overall and interact with a, a bigger ecosystem that they were not used to before. So for me, the right thing is not to say you need a bunch of applied science. I think you also need to grow the company to be AI engineers plus applied science. And that's what at the end might create the right product overall instead of just investing in the deep tech.

So for example, what does the Copilot team makeup look like?

We probably don't talk too much out there about that.

Um, it's not like a super secret sauce, but it's interesting.

What I would say is We try to not centralize it. There shouldn't be just one Copilot team. There's a Copilot Intelligence Platform team, and that's true, but I would say the repos team is a Copilot team. The PR team is a Copilot team. The issues team is a Copilot team. So the people that we have in VS Code and IDE, they're in the Copilot teams. So I think that's the way that I want— that I would phrase it both internally and externally. And so, whole company is centered now on that Copilot.

And when you talk about building AI engineering as a skill across the entire organization, is there like a reskilling program happening internally? Are you hiring for that? How does that happen?

So, for us, it's like, uh, we look at like enthusiasm a lot, right? So like we have some dedicated ML engineers who like have been doing ML for a long time, but a lot of us are just like engineers from all over the company who are so excited about AI, um, and that then came into like our org to build these features. So we, we have some, I guess, reskilling programs or whatever. We have some, sometimes we have certain days where whatever research will share out research and there will be some education. But what we'll see is a lot of these people reskill automatically because they're so passionate about AI and they're just constantly building side projects in their own time that we don't even need to do the reskilling. These people just are bursting out of the seams of ideas and they're like, toying around with Copilot and all these other tools in their free time, and we just need to give them a place to shine. And yeah, that's a lot of our AI engineers, to be honest.

So what's the makeup of AI at Figma as a team? Is it—

yeah, we have a mix where I think we're also starting to realize that, okay, we do need like a fundamental team of ML engineers and like people who really can build our, build custom models. But then, yeah, we do have a lot of AI product engineers like myself. We, yeah, we have designers who are like specifically focused on AI. And then, yeah, what I haven't heard a lot about is research, which we think is crucially important. So we have a lot of researchers who will basically focus on, yeah, doing a lot of research with like our experimental AI prototypes, both inside and outside the company. So we can get a pulse of how our customers responding and what is the quality actually like. So it's truly a very like multidisciplinary effort at Figma.

I think one of the key things to remember, at least for us, is AI can be customer-facing, but in our, at least in my experience, it's been mostly internal-facing. That's where the biggest transformation has come. So we know what we need. We have this many support specialists. We have this many customers. How do we make each person more effective? How do we build AI tooling for them? Everything. So our underwriting models, our support system, our operation system. So we've actually just have spent most of the things that actually hit production that are more experimental are internal to transform. The way that we operate. So I think what we do is we have an applied AI team who are just constantly all day, like plugged into whatever is happening on AI. They know about it like the day before it goes out. And they are basically the inspiration for everyone else in the company to say, this part of my job really sucks. I would like it to go away. And then They can say, oh, try this model. And then we give that person the tools to actually just go do it, just figure it out. And then they try. So we really look for people who are builders, who are curious and are going to figure it out on their own. And whenever they get stuck, they're going to raise their hand and say, hey, I need, I need help. We obviously have parts of the product that can't break. We move billions of dollars. Cannot mess with that. You don't experiment there. But we also have product that it's fine if it breaks, we fix it right there. So it's really a balance of what can you break, what can't you break. And if it's really risky to make a decision, you have to think about it more. You have to get the experts, the people who actually know really what they're doing, but everyone else, they're gonna get pretty far on their own if you just give them the space and just tell them to go do the thing. A lot of times you just have to tell people like, hey, You can do something about this.

I've talked mainly about workers AI and how we give everyone else the ability to deploy workers AI-enabled applications. At Cloudflare, we actually think about it in 3 different ways internally. One is operational AI. What tools are we providing to our employees so they can do their jobs more efficiently, they can build better processes, deliver more value to themselves and to the customers. from an internal perspective. And, and recently we actually created an operational AI team that other teams can kind of like a McKinsey consulting group inside for, people wanting to go solve those problems. Like this, this just takes me too long to do this every day. How can you make it quicker and faster? And using a team that has some best practices to help, teach that team how to build those applications. Um, and then our product teams for the the products that we actually ship to our customers, we've been using AI and ML for years. Every single request that goes through us goes through a security model that decides is the request good or bad. And there's, that AI proficiency is built into every team from our security products to the products that show our analytics. And then, and obviously the third one is what we've talked about, which is offering it as a developer tool for everyone else.

I have one last question, which is imagine your product 5 years from now. What do you think it will look like? Is it the same product with AI features or do you think it's radically transformed with AI?

What I would say is it looks more AI native. One of the beauties of what we're living through right now is that these LLMs can actually understand natural language in a way that none of the other tools before could. So I think if we really lean in into that and go and extract the maximum value of natural language, it's something that is innate to us as human, then the product looks completely different. So, and you can hold me to that. 5 years from now, if GitHub continues to look the same way as it is today, I shelled you all.

For us, we actually measure engagement time and we want that to go down. So hopefully 5 years from now. Nobody's using Ramf. It just works and it's just doing everything in the background. So that's where I think separating that, that experience from the people who actually use it to do their job and the people who have to do their jobs, like instead of it. So we wanna make it go away for all of us and then have finance teams that have 1,000 accountants, 1,000 AP clerks, 1,000 like per person, that would be truly the magic there.

Yeah, I kind of hope somewhere, I hope AI kind of disappears as a thing that we talk about in the forefront and go back to the business value that we're creating. And I think in 5 years, I think the models are going to be exponentially better. Hopefully there's something even beyond LLMs, uh, that are more efficient and, uh, can better reason and, uh, can really just, I can't even imagine what they'll be able to do, uh, as far as enabling us to just do our jobs and live our lives better. Yeah.

5 years from now, nobody of course knows, but I do think some level of role collapse is going to happen, which is super exciting. So designers will be able to create Software engineers will be able to create designs and all of these roles are going to start blending together more. But I think what we believe at Figma is that regardless of what happens there, some fundamental, some truths are fundamental and that's this AI design tool of the future will probably be visual. You probably still want to collaborate with your coworkers and we, yeah, we believe that regardless of what happens, yeah, we can invest in those principles and be all right and Yeah, create something nice.

We're out of time. You all, this has been amazing. Thank you so much. And thank all of you.

Thank you everyone.