This is a YouTube video titled "How Delphi Scaled Agent-Generated Data with Materialize | 5-Min Engineering Deep Dive" featuring a technical discussion primarily between Sam Spelsburg and another speaker about the challenges and solutions in scaling data infrastructure for the Delphi platform. The conversation covers topics such as digital knowledge transfer, user interface complexity, AI expectations, and the integration of Materialize to improve performance and scalability. The content is a focused engineering deep dive rather than a casual conversation and does not include a speaker named Delphi Demo.
Delphi is a platform that lets people create a digital version of their mind. Our customers right now are people who fundamentally have some reason for someone to learn from them. They have some sort of expertise, special way of doing things, and we want to give them the ability to increase their impact. And so we have the evolution of knowledge transfer. But Delphi is the first way that people can actually create a digital representation that scales a one to one to many. This is the creator view. And so you have your insights dashboard where you can see conversations that have happened, you can see insights, and you can see the quality of your Delphi things like this. This view, though, quickly became complex. You can see that all of these green dots are people that are actually using it right now. These are all live users. This audience table has an overview of people, the messages, the last time they were active tags. All these different things get pulled together from a bunch of different places inside different SQL tables. S3 bucket. Once this table started getting really big for people to get the overview of their audience and, and the complexity started increasing. It would take two minutes, sometimes it would just timeout or fail.
This is something that we're seeing with some of our customers where you could say, hey, I have this message tab, but the status update is too complex to calculate. Let's just remove it from the ui. Or hey, saying when the last activity happened, that's causing some instability. So let's just remove it from the ui. You can do that. But the rest of the market, when your customers are looking at what's coming out of the faang companies, there's starting to be a gap between customer expectations and what everyone else is able to deliver. And so one of the things that's really cool about what Sam's been able to do is say, hey, we are going to provide those modern experiences with the team that we have access to.
Not to interrupt, but I think it's such a good point because especially with AI, everyone's bar for what's good is just getting higher and higher and higher and higher. And so it's not sufficient anymore to have a product that's just okay or just good, but it needs to be really good and it needs to be magical. And you have to be leveraging new tools like materialize if you want to make sure that it's going to be, you know, meet that bar of expectation.
There's a lot of things that are just breaking right now where, okay, there's some CTO CIO initiative. We need to be AI native. Let's put all of our data together. We have our vector database. Let's try to bring in our fresh context and then serve it to these LLMs. And as people are wrestling with what it means to deliver fresh context to LLMs or to make it available for agents, they're seeing that their infrastructure never had a chance of supporting that. And so what do you do from there? We think about Materialize as you can think of it as a developer productivity tool. This is the fastest way to create what we're calling live data products. And not just putting data together and hoping that it eventually ties out. But the underlying promise is that these are views and queries that you can trust.
I'm very bullish on this kind of system to serve data because if we think about where things are going, we have model context. Protocol is huge right now and only going to become more important as agents and LMS are actually just making queries directly and they're just accessing the data at incredible scale. What are we going to do? We see maybe one trend is serverless like Supabase. Maybe we'll just continue to scale to infinity. But that doesn't really help if you are still having to do all the work every single time that the agents and the LLMs are just calling you over and over again. And. And especially if you have complex transformations like the one you can see like a little bit on the right in this slide in our conversation view. How are we going to do that at scale with agents? We need something that's actually doing the work upfront. Since last summer when we integrated Materialize into some of the core areas in Studio, we had a 120x reduction in page load times. We went from sometimes two minutes and then breaking down to under a second half a second for it to. To load the audience and the conversation views.
