Using AI to Empower Creators fr Roblox Studio Head Stef Corazza

We have this unique synergy and collaboration with the community where basically we told the community, hey, give us access to your data to train AI. We are going to make the best AI companion, the best AI assistant that we can. And that assistant goes back into studio and is free. Right? So we're not making money off your data, we're actually helping you create more. And so we found the overwhelming majority of the creators in our community gave us permission to use their data for training. And that's why I was mentioning earlier, we have not only one of the largest data set in the world, but also the most multimodal.

Welcome to training Data. Today we have an amazing guest in Steph Coraza. He leads generative AI at Roblox, one of the largest gaming platforms on the planet. Roblox has 79 million daily active users and they have a creator economy that pays out hundreds of millions to creators. Because of this, Roblox is uniquely positioned to transform how games are made and played with AI. Steph is a founder at heart. He started mixamo, a pioneering AI company for character animation and was acquired by Adobe. Roblox brought him in to revolutionize how games are made with AI at their own platform. Under Steph's leadership, roblox is pushing the boundaries of AI and gaming from their groundbreaking AI assistant, which lets you generate games with simple natural language all the way through their 3D foundation model of technologies. Welcome staff to training data. Today we get to talk about games. In particular, we're talking about AI at roblox. Roblox is one of the largest gaming universes on the planet. And I say universe instead of platform. Platform is a pretty overused term. And really Roblox is created much more than a platform. It's a place where you can create, it's a place where you can play, where you can meet new friends. And it's all done online virtually. Now we are technologists, but we're also investors. And I want to spend a moment on how remarkable of a business Roblox is. It's obviously an amazing technology and we'll get to there, but the business is exceptional. You've got a $29 billion market cap company with over three and a half billion of run rate revenue. An amazing stat here is they're really building an economy. It's not just a selfish company, it's a company that actually has produced over 800 million for their creators, for the people actually building on Roblox over the course of a year. 70 billion hours of gameplay on Roblox per year 70 billion. And they're able to deliver cash operating profit. So $600 million operating profit. That's because they've catered to a huge audience. 75, 79 million daily active users over time they've actually shifted up in age and you've got 46 million of those daily active users are actually now over the age of 13. 3.8 million daily active voice users. And the numbers just continue to grow on this amazing business. And so Steph, we're so excited to have you here today. We could not have asked for a better person in the category of AI gaming. From machine learning to computer vision to biomedical engineering. You have a pretty impressive technical background, an amazing journey from what was initially biomedical to generative AI efforts. And we were hoping that you could kick it off by just telling us how you got here. How did you get to becoming the head of generative AI when you started off as an engineer in a very different field many years ago?

And thank you for the great introduction about Roblox. It's really like an amazing example of a compounding effect and we are every day, we are mesmerizing ourselves about the success. My journey started at Stanford about 20 years ago when I came from Italy as part of my exchange program and I was focusing on computer vision, machine learning for the measurement of human motion. And so you have basically two markets there. One is the biomedical that you mentioned and the other one is animation. And so we were basically at the boundary between the two. And at some point I realized there was a much bigger opportunity in the animation space. And so I basically work on like video based motion capture and animation creation solution that then led to the spinoff of Mixamo, the company that I started in 2008. That letter got acquired by Adobe in 2015 and is still today one of the most used machine learning services in the industry to rig and animate characters. And so after a few great years at Adobe, actually seven of them, I helped build the 3D offering there, including products that my team built like Adobe Stager and Adobe Aero for ar. And then we acquired Legarithmic and so we built the full 3D portfolio. And then after that I was really passionate about Genai. I was working with the CTO of Adobe, trying to figure out completely new ways to generate things. And that's when basically Roblox reached out and I had, you know, breakfast a few times with Dave and we talk about it and I really realized that potential of the gen AI head was really finding in the gaming space and specifically In Roblox being a platform where so much data exists, you know, 15 million experiences every day, every year are played, I found basically Roblox to be the place where this could really blossom with a massive impact worldwide.

So before we get into the AI components, can you tell us a little bit more about the platform? You're an engineer by background and so scale is incredibly exciting to you. I would imagine the sheer scale at Roblox is pretty mind boggling, I think to anyone, especially users. There are obviously platforms that have more daily actives, the Facebooks of the world, there are platforms that have more monthly actives, et cetera. But it's very rare to have the amount of bandwidth and compute and graphics and everything in one place. Maybe you can share a little bit about what that means to have 15 million daily sessions and 79 million monthly actives and a little bit about the stats and what that means technically.

Yeah, we always like to talk about the daily active, which is the 79 million that you mentioned. But the monthly active active that I think we don't communicate to the outside world. It's even more staggering in being in the several hundreds of millions. Right. So it's a massive community that is growing very healthy and pretty fast. And sometimes people ask us where are those games coming from that people play? Some of those games are now worldwide franchises with tens of millions of concurrent players. And then sometimes I give this number which is also like a reminder of the scale. So every day we have roughly 90,000 experiences in games published on Roblox. And so that gives you the scale of the human creativity if you want. And how much really this is becoming more and more of a creation platform and game development platform that has like incredible numbers in terms of scale of creation and also like it's an economy on its own as you mentioned, you know, we, we paid out, you know, $800 million to our creators. And so people have jobs, people buying houses, people have like companies, they are now, you know, we have game studios, they have like more than 100 people, some of them are VC funded. So it's basically creating its own like creation economy and tied to that. And so that I think is very humbling and at the same time has incredible potential. Because the uniqueness I think of Roblox is that we are one of the most vertically integrated companies on the planet. We own our own data center. We have many data centers around the world where we own the hardware, but then we also own the app that distributes all these games and the players and all the services like video chat and live chat and chat translation and so on. But then we also have the creation tool, which is Roblox Studio, that I have the privilege of leading with my team. And then also we basically have all the services for creations that go with it. So it's all the way from like the bare metal CPU GPU cluster all the way to the creation tools. With the big difference that we only charge our. We only basically take some revenue ourselves. We, when our creators make money, right, we make money when they make money. There's no upfront fee to get into the game, there's no upfront fee for the tool. The tool is free. A lot of services are free. And so there's really very little friction to start using Roblox. And if your game has one user or a million users, you don't have to worry about anything. We scale it for you. We pay for all those cpu, GPU instances, storage in the cloud and everything. And it's completely opaque to you as the creator. So I think that's the uniqueness. And as part of the fully vertical integration, we are also able to subsidize AI, right? So we are one of the few companies out there with a full, a full fledged AI assistant offering for game development, for code creation, material, texture, assets, everything. And it's all free to the creators.

I'd love to get into that. Maybe. Can you just walk us through today? What is the experience of creating a game? Like, who are the typical creators on your platform? Is it a high school student? Is it a professional game developer? And what types of games are they creating?

That's a great question. So we have several million creators on a monthly basis on the platform, so big numbers there as well. And usually I think the average age is in like the mid-20s. It's a little bit older than our player demographics, of course. Um, and typically the majority of them are doing like world building. They're like building stuff, they're artists, they are making games. And then we have about 30, 40% that actually are coding. Right. And then of course there's an overlap between the two audiences. But basically, roughly this is what we are seeing. And it's used for the most different creations you can imagine, from like natural disaster simulations to learning to the more classic, like a gaming experience, events, concerts, fashion, design. We are seeing like new type of experience popping up on a, on a daily basis, which is, which is very fascinating.

What's your favorite game?

I've been playing lately, Racing Empire quite a bit. I like car games. So that's a good one that I really like.

Steph, I was actually surprised to hear that you said 30 to 40% of the developers are actually coding as opposed to world building. Why is that? And maybe that kind of parlays into what you've created with Assistant.

Yeah, so there's just. It's a skill that is harder to master and to get into. Right. A lot of people just like go from players and they want to create something and so they start like building the world. And that is probably more intuitive than writing code where you have to understand this like high level constructs apply them to gain interactivity. And so that is one of the things that we wanted to tackle with Assistant. We wanted to basically remove that friction of having to learn to code, having to learn like programming language in order to create interactivity. And so that was like one of the initial inspirations. And that's why, you know, Code Assist was the first feature that we released. And then now this was like in March 2023, which feels like a two decades ago. And now, you know, a year and a half later, we basically have a.

Full.

A full, basically Assistant that has game development capabilities. They go from writing code, auto completing code, explaining code, debugging code, applying scripts to parts and objects in your scene. So that's all the coding stuff. Then we have documentation, right. People ask, how do I do xyz? And usually they have to browse through dev forums and documents on the Internet instead. Now a system can basically summarize those information for them. And then the third aspect is creation of assets. So we rolled out a material generator, we rolled out a texture generator which was a lot more complex, where you can Basically texture any 3D object with quite good fidelity and resolution just from a text prompt. And so all these together is what we call Assistant with an umbrella name. And it basically is allowing now to create entire like simple games from scratch just by typing natural language. In the future there will be more like multimodal input through images. But basically right now we have like simple games that people are making also as a test where like they're only using Assistant. And so you can imagine you can make that game on your phone, you know, with a microphone. You just speak to it. And then a system will just generate the world, will create the forest, will create the enemy, the balls that you have to fight, and then also add all the game mechanics to it, everything automated.

Are the games created with Assistants, like how good are they? If you had to give a score out of 10, the game's created from coding versus what people are doing right now with AI versus where it's going to.

I mean, if you can code, of course, you get to another level of sophistication in terms of the complication of the gameplay and all the nuances that make a game fun to play. So of course we're not there. The examples that I see are mostly like, we do game jams and we spend like two hours to make a game with Assistant and what can come out is pretty incredible, but we haven't had in the community. I'm sure they're going to take it to the next level. The goal is not to exclusively use Assistant. The goal is to basically combine Assistant and learn skills through Assistant. So maybe at the beginning, the first script, Assistant will write it for you and will attach it to a part. Then you know where the script should go and then you know, you know how to, I don't know, make some hobby platform, move up and down and then you learn on the way. So we see Assistant as a companion, that it shows you by doing how you make a game. And then over time, people really develop skills that otherwise it's hard to acquire.

Stefano, have you seen that the type of development has changed? As in the Assistant not only has language like the documentation you described, it does also have code. It even has images. I saw that it's. And we'll get into the technical specifications in a little bit. But as a user, you have language, you have code, you have images. Have you seen the behavior change for how people develop in Roblox, as in tactically, have you seen the number of people actually coding going up? I would have guessed that outside looking in because the barriers to entry of coding have gone down. But does that just mean that more people are developing in general and the ratios actually stay the same?

That's a great question. So what we have seen, we have measured the productivity of people that use Assistant versus not. And so we have found that people that use assistant create 180% more code.

Wow.

And so the individuals are a lot more productive.

That's benchmarks to people who already wrote code. Or does that also say just more? Oh, wow, okay.

Two cohorts of coders, one using Assistant and Code Assist. Code Assist is the, you know, suggests code and a system that creates from scratch, but they all kind of integrate into a similar user workflow. And then if you're looking the same cohort comparison for Material Generator, creators that use material generator create 60% more materials. So also on the art front, there's more productivity. And then if you look at the final goal, which is how much they publish. Right. Because publishing the game is the ultimate goal. The lift is about 30%. So people that use assistant publish 30% more than people that are not using it. And remember that this is now still in beta. Mostly it's going to get out of beta soon. I can't give you the exact day, but it's going to be relatively soon. And so we're going to see even a broader adoption, of course there and impact.

One more question on this, on Sonia's quality question, like what about usage? So they're publishing 30% more. Do you have any KPIs that actually give you a sense of if the Assistant games, the hours spent, the Robux spent on them, whatever the KPIs might be, if they are also that 30% lift?

Yeah, that's a great question. So our number one KPI right now is around retention. And so we are seeing like a week over week retention of people using Assistant that is much higher than any other features that we roll out. And also we are seeing that over time that retention in the long run increases quite a bit. So we have people that are using Assistant with Studio, we are seeing significant lift in the overall retention of Studio. And then of course we see a high retention on Assistant. So the number of daily users has been growing organically and very steady. We don't do any marketing, of course on this and it's free, which is, I think the best marketing is not cheap, let me tell you that. I think Roblox is very generous on that. But we have this unique synergy and collaboration with the community where basically we told the community, hey, give us access to your data to train AI. We are going to make the best AI companion, the best AI Assistant that we can. And that Assistant goes back into Studio and is free. Right. So we're not making money off your data, we actually helping you create more. And so we found the overwhelming majority of the creators in our community gave us permission to use their data for training. And that's why I was mentioning earlier, we have not only one of the largest data set in the world, but also the most multimodal if you want. Because we have, we have code, we have Images, we have 3D assets, we have audio, video. All that is part of a gaming experience. And also with the interactivity is the glue for all of that. And with the analytics on the usage. Right. So it's a very, very powerful data set that we are very treating with A lot of respect and a lot of the best practices to keep that data, of course, very secure, but at the same time allows us to really harvest the value. And ultimately what we are doing is we are teaching AI game development, right? Basically that's what we are doing, right? We're not teaching how to make an image, we are not teaching how to write code. We are teaching game development. That's the ultimate goal. And so it's gonna. All these tools, they are at the beginning will feel a little bit, hey, this is a tool to make a material, this is to make the textures to write code. We are already seeing that they are converging. We already started that process of converging some of those like lower level tools into larger ones where basically the AI is actually learning how to develop a game as opposed to just how to do like a small task.

I love the vision of the AI learning how to develop a game. And the question I have is, if you break up game development into its component parts, which parts do you think are most likely to be taken over by AI? Well, in the near term versus what do you think humans will be uniquely good at for a while?

That's a great question. So we don't see AI as taking over, by the way.

I know.

I think the best parallel that we made at RDC like a couple of weeks ago was AI is your dishwasher, right? Nobody wants to wash the dishes. And so we are really focusing on task. Especially with the last release of Assistant Actions, we are focusing on the task that you don't want to do, right? Washing the dishes, doing the laundry. So we have introduced amongst the assistant capabilities a new capability where Assistant can basically make large scale modifications to the data model. I'll give an example. I have made a beautiful open world game with a huge forest. This forest has 100,000 trees. Now all of a sudden, I want these trees to actually follow the seasons and, you know, get more leaves, more yellow because fall is coming. The amount of work that will take me to implement that will be huge. Assistant can do that with three lines of text. I can say, select all the trees or I can say select all the pine trees. Actually those don't become yellow, so that'll be the wrong one. Sorry. Select all the trees other than the pine trees and make the leaves yellow. Right. I can just give these three liners. Our system can go, can select, you know, the 57,000 trees. There are no pine trees and can change the color of the leaves for me in just a few seconds. So that's the kind of task that we are seeing a lot of value. And honestly, this was the feature that the community loved the most. Of all the AI features we released in studio in the last two years, this was by a landslide the one that got the highest score and appreciation for the community because again, it was the dishwasher. We are not replacing your talent, which we believe is irreplaceable, but we actually helping you on the task that you don't want to do.

I am always so impressed by Roblox's emphasis, genuine emphasis on community and teaching. You've said this in the past few minutes. You've talked about community quite a bit, about teaching how to code and how to create games and really uplifting the entire community, frankly. And I just want to say that this is very much true all the way to the core of the business. I got to follow my really good friend Craig Sherman to board meetings in 2017 and 2018 at Roblox. And even behind closed doors years ago, this was always the focus. It wasn't the. The banality that a lot of board meetings on monetization and financials. It was community. It was uplifting community. Teaching them how to code, teaching everyone how to use Roblox in a way that actually benefits themselves in their own learning. And this dishwasher analogy sounds very consistent with the ethos of your business. How impressive. How did you implement it? It sounds like a segment anything type of algorithm. Was it a segmentation approach or this particular feature? How did you do it?

That's a good question. So we basically have found a way to give Assistant. So Assistant is really good at generating code, right? As you know, is based on LLMs and some of them actually we are supporting as open source project like Starcoder and we train with our own data. So over time Assistant is getting better and better at generating code. Some of the code instead of running runtime, you can actually run it edit time in Studio. Studio has a command bar where you can just execute some of that code. And so Assistant actions basically is creating code that is executed directly in Studio at edit time. And because we integrated in, Studio has full awareness of the data model of your scene. And so it knows, oh, this is a tree, this is a car, it knows what you did and has full awareness of that. And so we have combined the ability to generate code with the ability to create using the same code LUA commands in Studio and the awareness of the data model. Those three things coming together have basically unleashed the power of the LLM onto Like data model manipulation.

Do you have these things labeled or is it dynamically determining that this is a car?

There is a good amount of inference that just happens. Cool. Yeah.

How do you imagine the creation experience evolves, let's say 10 years from now? How do you see assistant evolving?

So when we met about roughly two years ago and we said, okay, how is AI going to impact creation and where should we start? The one paradigm shift that we envisioned, that we thought was gonna happen in the industry, not just the Roblox, was a shift between fine control on the creation to capturing intent. Right? And so I spent quite a bit of time in the Photoshop land where basically there you give control to the color of the individual pixel of an image, even though most people don't need it, but some might, right. And so there it's all about 100% control, non destructive workflows, but 100% control on the actual artifact that you're generating. And then we are now moving towards a complete different generation of tools where the tool is successful and can produce good outcome only as long as it can capture the intent from the user. And so all the digital tools that we have seen in the last 30 years, it's all about surfacing more control to a user and then the user will figure out how to, how to use that control. Right. Instead we are migrating from control to capturing intent. And so we're going to see probably quite a big change and there's going to be a thousand startups trying to do in different ways and there will be new UX paradigm popping up. But we can see audio as an input, we can see high level gesture, not just with your hands, but also with mouse and keyboards. So things that are just providing an input into what you want to do. We are seeing multimodal input. I can describe a word better if I can type and I can provide maybe a concept art and maybe some high level sketching on top of it, right? So this is very different from having the phenomenally granular control, but at the same time the velocity is like two orders of magnitude faster. The challenge in all this is that for, you know, for casual creators or people that are, you know, have a limited amount of time to spend on probably what AI is providing, it's already good enough to share on TikTok or to make an experience on Roblox and invite your friends over, but for people that want to spend a lot more time, you still have to provide that fine control. Right? And so the challenge is, how do I make, how do you Make a tool that is really good at capturing all the initial intent, but at the same time allows the real pros to iterate with the same level of control of the traditional tools. So that's a little bit the challenge that I think a lot of companies like, like we are facing in studio, a lot of other companies are also facing.

We have a really fun episode of training data with, with this team that created a company called Dust. And one of the founders, Gabriel Hubert, talks a lot about rasterization versus vectorization. And this is definitely your language as a graphics person. They also were Stanford computer vision type folks. And it seems like that transformation has happened and the vectorization kind of what you're describing, like you can, you can expand it to a great deal based on intent, you can shrink it to very small and almost like fractals go down into more and more detail. When do you think we will get there? What do you think? We'll get to the point where you can say, hey, this is the attent. And then you can go in and in a fractal level of detail, change each pixel also based on intent, but on a much smaller scale.

Yeah, that's a million dollar question. So what we're trying to do is allow Assistant first to be able to perform those operations. Right. So the same AI that gives me the rough slot machine type of input, I throw in some text, an image, I pull the lever. Let's see what we get. We want basically a system to also be able to go in and do the fine grain change. So hey, only the trees that are above five feet, can you just change the color of that? So and then can you take the texture of the tree and then open up and I'm going to paint over it. Right. And so we want to allow AI to already be able to go beyond one shot and allow for iteration. We believe iteration is a fundamental way people create and so we want to make sure that AI can support that from the get go. And then we will always have a fallback with some tools and maybe they're. It's going to be more like a progressive disclosure where we don't throw it in front of every user. Right. But only the users that want to go deeper. Then you know, we can basically pull the curtain and then allow them to go a little bit deeper in some cases. Honestly, we would just like interop with our other tools. There are things that only in Blender you can do or only Photoshop or Substance Painter in our studio won't become, you know, this crazy Place where you can do everything, but it's hard to do anything right. So we are very, I think, committed to studio being good at what it does and then it's okay to have, you know, for really the pros that want to go deep to have great interoperability with external tools. We talked a lot about Studio, but actually the thing that we are the most excited is taking all this AI goodness and bringing to inexperience creation. And so we think that's going to be the next frontier. So two aspects in the industry, we are at the very beginning and we are curious, we don't know how it's going to play out and we are super curious. Using AI for substantially different gameplay. Like, you know, companies like Ego Live are experimenting on that and then taking all the ability to create the AI has unleashed and bring it to inexperience creation. So I'm there, I'm playing my game on Roblox now. I want to modify this level. I want with my friends, create something new that we can play together. Those are like completely different type of creation experiences that we're going to see be unleashed by AI. So right now we are incubating in studio. We are making it solid, robust, we're making sure the output is high quality. But we are very excited to bring those APIs to inexperience and we think that's going to be the real impact.

I'd love to dig more into that, maybe starting with just the in game, the gameplay experience, like how do you think from the player's perspective these games will be different?

I think they can be with basically no effort or very little effort from the developer. These games can be more personalized and also they can be always different. Right? Like if you play the same game, but there's an LLM and is aware of your past, all the things that you have done in past sessions and then you come back, then you can make the game different, more interesting, can change the challenge, can spin a little bit the story, right? So all these things, when you have like a very smart AI backend who keeps track of what's happening, the history, and knows also who you are, it can really morph the game onto something that is really more enjoyable for you specifically. So we think that will be a big opportunity. There's a lot of companies that are now doing experiments and we are very excited to basically provide AI as a platform and then let them experiment with different gameplays.

That sounds Stefano, technically very hard. And I remember even for the simplest type of Roblox gaming, which there isn't really a simple type. You're generating digital worlds continuously. It's actually a very, very heavy lift. You mentioned having your own data centers, right? Like low latency, being able to play with people internationally, across the world. How do you think about adding this new level of complexity, this AI inference complexity for gameplay, as in generating the world, varying it on the fly. What will have to change from an infrastructure perspective?

That's a great question. So I think the first step towards that will be npc so non playable characters that you hook up to an LLM that doesn't need. That's pretty interesting, right? That's pretty straightforward and I think that's going to be the first rep and we're going to see what kind of impact that creates. Then we have seen other experiences where they actually want to create the whole world and of course there are some challenges there. And let me tell you, I think what I think is the biggest challenge is actually moderation because Roblox Safety is our number one product and we want to keep the platform safe and we want people to connect with civility. And when you allow people to create anything, of course the bar goes a little bit higher and so we open this Pandora box, but we also have to build the guardrail so things stay positive and everybody can have a positive experience on the platform. So I think that is more of a challenge than like a latency infrastructure because we can cdn, we can cache things, we can pre generate some of the content, we can use a level of detail. So there's a lot of things that the gaming industry has developed in order to basically stream. We are using streaming as well, right in the platform so we can basically in real time generate and stream more assets. I think the challenge will be more like if you have now the control of the game that you're playing, how do you make sure that everybody else has a great experience? I think that's going to be the challenge both on the moderation side and also like in making it fun. We have infinite respect for game developers because they know how to make things fun. And not every player has mastered the same skills of making games fun for the last 20 years. So it's like how do you give this freedom to create while you somehow control the gameplay and the story so it stays compelling? So these are all things that we are very eager to learn from the community, honestly. And I think that's the beauty of being a platform. We don't need to have any opinion about it and we don't need to figure it out ourselves. We just provide the APIs to the community. And the community with the infinite creativity has. We'll figure it out.

So I've heard NPCs as that first stepping stone in game AI frequently. What do you think is going to be the next stepping stone for how, you know, game mechanics or gameplay might change? Because I imagine you're not going to go from NPCs to entire worlds, right? Like, is there another kind of. What's the second stepping stone?

I've seen like, games where they limit the creation to one specific item. So for example, you. There's a very popular game on Roblox called Build a Boat. You go into this game, you're building a boat, and then you're sailing that boat, and then you're racing other players. And so you don't create the world. The world is predefined by the developer of the experience. But you're basically allowed that constrained creativity. As in, these are the materials that you can use to build the boat, and this is the size that has to be. And then you can go crazy and build whatever thing you want. And because Roblox is such a physics sandbox, right, we have aerodynamics, fluid dynamics, and so we can actually see what you create and like simulate the wind and you're going to see the outcome of what you created. And so I think experiences like that right now are a little bit more difficult without AI. And AI, I think can. Can really power those up quite a bit, where you can create your race car, you can create your boat, your airplanes, like some elements of the gameplay based on what you create. You can have, you know, an advantage in the game, but you're not complet changing the game itself.

Constrained creativity. That's a great answer.

Exactly.

I have two follow ups from that. The first is you just mentioned physics engines, and there's been a lot of enthusiasm for years now around pins and these neural physics engines, et cetera. Is that now in the physics Engine at Roblox, if you can disclose, are you using neural networks as part of the physics engines as part of the. You mentioned aerodynamics, right? Like, are you guys at the point where you're estimating Navier Stokes because it's cheap enough, because you can use neural networks? Or is that maybe in the future? And then my second question is about NPCs. Would you guys ever allow a world or a game that is purely NPCs? Is it something you watch as opposed to something that you participate in that's a great question.

So on the physics side, I think the use of neural networks for physics has not been proven as effective. I think there is more like there's an infinite amount of real world approximations that you can use that are like very computationally efficient. And I think that's more important than how good you're approximating the world. And so on that front, I'm not sure neural networks are going to provide that much value, honestly, especially when you have already implemented all the basic functionalities and people are already using them successfully.

Especially for a purely digital world. I mean if you're trying to. If there's a noisy real world, I imagine it's different from like Roblox world where everything can work in the physics estimations. Is that fair?

Yeah, that's, that's fair. So maybe I would say that's all I can say. It's basically maybe. I think at the moment it's still not fully proven as a path. And then on the, on the NPC front, you know, if you think about a game where all the players are NPCs and you're just watching it, to me it sounds like tv, right?

Yeah.

So people watch this all the time. And you know, these NPCs can be super smart and do super fun things, can race, can do whatever they like and maybe it's just fun to watch. So definitely we are seeing such a huge community of people that just watch other people play games. And so those players could be LLMs in the future and still generate great entertainment and fun. So I will not exclude that. I think at the beginning we're going to find a hybrid model which is like NPCs existed, where people will inject NPCs in the game to just make it more interesting and have people that also can populate games that just launched. They don't have a thousand concurrent players yet. And so you can basically populate huge world with characters that are interesting to talk to without having to have the real people there right at the beginning. So I think there's going to be a lot of potential on that front for sure.

What about for user generated content? Like how do you imagine UGC changes now that generative AI is really coming into the experience?

That's a great question. We have seen some of this with the genai features that we released for avatar creation. So avatar creation is another huge area of creativity. There's a massive community in Roblox. They work on like making assets and clothing and accessories for avatars. And a lot of People make a living doing that. And so we have found that there were a lot more people, like always, that had amazing creative ideas, but may not have the tools to do 3D modeling, for example. 3D modeling is a very narrow, specific skill. They would have to learn blender or Maya. And so not all of them were up for the challenge, but they had a great idea about what outfit to create. And so there we experimented with using images as an input or text as an input and generate avatars. And we found that it substantially democratized how many avatars could be created. And this is an area we launched, like an early beta of our avatar auto setup, and we are doubling down, that we're going to do a lot more in the coming months that you're going to see. But basically, we want to allow people to create avatars, clothing, accessories, with just multimodal, easy input. Again, there's more people that have great ideas than people that can actually execute on those. And so we want to basically remove those barriers and really leverage AI for that type of creative expression as well.

Do you think that generative AI blurs the line between what it means to be a user versus a creator? Do you imagine those two become kind of the same thing? And I imagine you think of them distinctly today.

Oh, for sure, yeah. And I mean, think about, you know, what happened in the music scene, right? The boundary between the composer and the people that could play an instrument. And people in the audience were just so set at the beginning. And then, you know, I think karaoke completely blurred that. Right. And then now everybody can create things on GarageBand. And now with AI, people can just type lyrics and the genre and then a full song is created. So they're like the boundary got completely burned, I think, in the audio and music space. And for game development, it's going to take a little bit longer because it's just more complex as a type of content. But I think it's just entropy only increases. And so we're going to see that for sure.

Stefano, you mentioned that you guys have an absurd amount of data. You guys have an insane amount of video data. I think Robox is still the number one VR application in the world by a lot, including on VR headsets, and then also obviously personal computers, all sorts of devices, et cetera, Ton of audio data, ton of textual data, kind of everything data. And so you mentioned this. I've been kind of walked by it, but obviously Sonia and I are very curious with all that training data. Yes. The name of the show. Are you guys going to have a world model. I think I know the answer to this. I think it might even be announced already, depending on when this airs. Are you going to have a world model? What's it going to look like? What will be its boundaries and where do you go from there?

Yeah, that's a great question. So, yeah, we do have an enormous amount of data and I think the challenge for us is less about gathering data and more like being able to use it, because there's a lot that goes into, from the raw data to be able to train an LLM or AI in general. So that is where the work is focusing. We have announced just a couple of weeks ago that Roblox is working on a 3D foundational model. Our intent is to open source that. And basically what it does is it will allow the digital synthesis of scenes and words from multimodal input. So that's the goal. And I think also we can go a little bit beyond that. As I said before, our goal is basically to teach game development to AI. And so it's not just about world creation. I think that will be the first maybe area that we can really attack in a meaningful way. But then there's going to be all the interactivity of that world, the ability for things to move, band doors to open, characters to run around, right. And then the next level will be all the interactive gameplay. And so we are seeing the data and we also can see from the data what works, what is fun for the, for the user. And so we will be able to guide AI to not just make games, but actually make games that are fun because you can see which games get more traction or like which specific levels get played more. And so there could be in the inference aspects of quality of the experience, not just like, hey, I'm just going to generate whatever you ask me, but I can also, like, if I can pick, I can pick things that are like more fun to engage with. And so again, that one is going to be, you know, it's going to take a while to figure out, but you know, we have the data, they can support it. I think it's more about on us to be able to pick up, you know, there's going to be a lot of like unsupervised learning that we have to do. And so we have to pick up, we have to find ways to pick up signals of intent from our creators that then we can use in an unsupervised way to classify things and then figure out what AI should be learning from.

I'd imagine because of the physics engine, you guys have such a competitive advantage with object placement, object interaction. The struggle with a lot of training data for these video and role models is like, they're two dimensional and there's no concept of. I mean, maybe you can infer from it the concept of what is three dimensional and how objects interact. But is that something that you guys are leaning on particularly heavily? I'd imagine it's a big advantage to the 3D model you create.

Yeah, it's a huge advantage for gaining also consistency over time and spatial consistency. If you actually have the full 3D model, and if you have a full 3D scene, then you can anchor the spatial and the temporal coherence in a much stronger way than if you're estimating those things from a 2D. I mean, the classic example is if you're trying to stylize video and you use just video to video, you know, you're gonna see drifts and artifacts and everything. Then, you know, with ControlNet and the depth map. Right, which is like a, you know, two and a half D approach, this video to video type of stylization became a little bit more reliable and more consistent temporarily. But you still, like, don't have. You have basically the situation where, like, if a character is looking at you and then you stylize that face and then they look away and then they look back, they're like a different person. Right. We have all seen those examples. And so the only way I think to overcome that is to actually train with 3D data. And the industry went so fast, so far with like just 2D. Right. And is now making magic with 2D. And I think there's still like the work to incorporate 2D information into those, into those algorithms still has to come. And I think it's going to bring to fruition this like, temporal and spatial coherence that right now we don't have, or that it's so hard to, you know, to generate out of 2D data. Right. Because it just is not rich enough.

Just to make sure I understand what you just said, do you think that 2D is going to lead to a dead end and you have to start over from 3D? Or do you think that 2D can kind of get there with scale as well?

I think if the goal is to operate on a single image, 2D has done fantastic. And so if you need to texture a character or like stylize an image, all these things is fantastic if you want to actually have a video. So if you add the basic time on the axis. And now you are also maybe moving the camera. So the camera is moving and then, you know, there's like 2,000 frames ahead of you. They have to be consistent at that point. If you're not using 3D data, you're at the disadvantage and you're going to be dealing with drift. And yes, you can enforce it in many possible way, but the problem of 2D data is that it has occlusion. So. So if you're looking at me and my arm is behind my back and then at some point my arm pops out, you have nothing to enforce consistency of the look of my hand because you just have not seen it before. Right. So all these problems don't exist if you're operating on a 3D representation of your scene, because even the data they are not seeing from the camera is there and it's available. So I think the industry has gone like, you know, in a amazing. I made amazing progress in trying to cope with that, which is a fundamental lack of data. And then if that data is actually used and can be incorporated in, let's say, a stable diffusion, then we're going to see a much better outcome.

Steph, in our homework for this episode, we heard that you're passionate about neural rendering. Can you tell us what that is?

Yeah, I don't know where you're at that, but yes, it's true. I'm very passionate because I have a strong belief that neural networks will change the way we're going to stylize games and make games visually a lot more compelling than they are today. And so if you look at the history of game development, the creation of assets and the style of the game were always tied together. If you're making a Super Mario and that's the style of the game, you can just on the flip of a coin, say, okay, make it look like a Call of Duty. It would just not work. You will have to redo all the assets from scratch. But if you're using neural rendering or generative rendering techniques, you're able with some text description and maybe some reference images to restyle your game in real time. And then you leave the geometry where it is so it's physically consistent. Right. Your physics capsule is in the same place, but the visual look can be very different. And this can be used also to make games completely photorealistic. Even though you're using some low resolution meshes and textures, you can use this as a final pass. Like in games, you add typically bloom at the end to make things look really cool. And so it could be a final generative rendering pass, make it photorealistic or stylize it in this specific way that I like. And you can restyle your game in a way that actually doesn't add any extra assets that you have to download, doesn't change the assets. It's really like a compute layer that happens at the end right now is very compute intensive. But you know, looking at the speed at which things have moved, you can imagine that in the future people developers can just add this filter to their game, describe the style, give a reference image, and then have the game look beautiful without changing, you know, a single geometry in the asset. So I'm a believer that in five years this will be the way games will be built and it is going to run also, at least on the high end phones. And yeah, we're very excited about it.

Steph, does it ever become the end gamer's choice, the way it's rendered? As in I put my skin or style into whatever game I'm playing?

That's a good question. I think if the developer of the game allows that freedom to the player, why not, right? I think it's going to be an artistic choice. Hey, you can play my game and you can make in any style you want. And again, you can still play with people that will visually see a different game, but the game is still consistent. All the physics, all the gameplay still hold.

Stefano, thank you so much for joining us today. We learned an incredible amount, certainly about the scale and complexity of Roblox, not just from technology, but also the scale of impact. The 79 million daily actives, the hundreds of millions of monthly actives, the amount of technical depth that's necessary to get there. We learned about how you're using AI as dishwashers to empower your customers and developers and gamers, not just in the assistant, but also in the code assistant and the material generator. And we learned a little bit about what you think the future is going to look like. We might be using infrastructure for guardrails to make sure that the civility of Roblox is preserved. We might be watching some NPC television and we certainly will have a 3D world created by Roblox. Can't wait to live in that.

Oasis, thank you so much for having me. It was a pleasure.

Thank you.

Sa.