Keynote: Building Intelligent Systems on Real-time Data

Jay Kreps, CEO of Confluent, announced IBM's acquisition and explained why AI systems need real-time, high-quality data instead of static warehouse extracts. Confluent introduced Confluent Intelligence, a suite of tools including Tableflow, a Real-Time Context Engine, and streaming agents that let companies build AI applications on continuously updated data. ElevenLabs and Getir shared how real-time architectures improved voice AI latency and cut costs.
Chapters
Please welcome Jay Kreps, CEO and co-founder of Confluent. Hello everyone, welcome to Current London. This is always one of my favorite times of year. I think it's a remarkable thing to to be able to get everybody together and talk about all the interesting stuff that's happening in the world of data streaming. That's ultimately what community is about. And that's how I wanna start the talk is actually just by giving some thanks to the larger community in this area, the people who help build open source Kafka, the people who help build open source Flink, all the people who build the surrounding projects in that ecosystem, the partner technologies, all the people building applications in that space, that ecosystem is really thriving. And that's what makes this area special. It's what helps us all do what we do. It's why Confluent does what it does. I think it's really a special thing. And over the last 12 months, it's been a really rich time in that community. We've had over 80, KIPs and FLIPs, these are the new feature proposals that have gone into the open source. Over 400 people have contributed to Kafka and Flink. So really rich and thriving development and of very meaningful features. So in Kafka, the queues for Kafka, this is the other side of messaging, proposed probably in tickets for the last, 7 years people have asked for this and finally it's been done. And I think done in a really elegant way. Similar work on Flink agents, which we'll talk about in a bit, which brings kind of native agent capabilities into Flink. Very exciting and meaningful features, big and small that have been added. And there's been changes for Confluent as well. So Confluent is now part of IBM. This is something that I'm very excited about. It's an awesome opportunity for us to join forces with an organization that's been contributing to open source for many years, you know, going back to Red Hat and Linux, uh, development. And I think it's going to be an excellent steward in this area and allow us to continue to invest and to innovate and keep contributing, uh, to this larger world. And with that, I'm going to turn it into the area that I'm going to talk about, which is all about building intelligent systems with real-time data. You know, I think right now probably every conversation in tech in some way touches on AI. It's, it's almost unavoidable, uh, sometimes annoying, but always actually very interesting. Um, and I think for all of us, we're kind of figuring out how to build systems in this domain. And I think this is part of a bigger shift in our industry in how we think about data, the infrastructure for data, how it's used. We're kind of going from a world where, you know, if a company considered itself data-driven, that would mean that you had good business intelligence, you had a data lake or a data warehouse, you had someplace you could put all the data, you could run some reports, you could generate insights, executives could understand what was going on and make good decisions. And I think, you know, the rise of AI really broadens the reach of what's doable. You know, it's not just about showing reports to the people in the company to make decisions. It's actually about building systems that can autonomously take decisions, that can take action, that can close the loop, that can do things within the organization. And I think we're still in the early days of realizing this in a lot of companies, but I think it's a very exciting thing. And I think it makes us have to rethink a lot of what we do. And whenever you're rethinking things from first principles, I think it's actually very exciting. I feel like it's one of the most exciting times in the industry, you know, since, since I've gotten into it. And so I'm going to talk a little bit about what this means. You know, I think if you think about the adoption of AI, it's almost like we've gone through a couple of mini waves in this broader rollout. You know, the first thing was chatbots where, you know, you can ask a simple question And the use of data in a chatbot is kind of easy because it's either right there in the model or maybe it's doing some kind of web search or kind of broadly available data. You know, a lot of the next use cases were things like Claude Code or Codex or Cowork, you know, the, the kind of knowledge work companion that's helping you do things. Here the data is again pretty easy because very often the data is your code base. You check it out and everything is right there. It has all the context it needs. It can look anything up that it can read through. And I think we've also started to experiment and build more complete business systems that take advantage of AI in the rest of the organization to take over some part of the business, carry out some function more autonomously. And this is where it starts to get hard. And so if you talk to companies, this is where they're struggling to put it together. And I think— And ultimately, the reason for this is, you know, this is where all the unique messy bits of enterprise data come together with AI. These are where the two things have to actually play together. And that's a lot harder than it sounds, at least anybody who hasn't seen the ugly interior of the data world inside of any company, which is, you know, ultimately always a bracing thing. And it requires us to think, a little bit different. These kind of applications really do work differently from traditional software. You know, ultimately, if you think about classical software, it's a system of rules, right? You're building a set of if-else for loops that are, you know, discrete, they're predictable, they're deterministic, they encode all of your business logic. Once you start to bring in data, and probabilistic models, you're making decisions that are not fully specified in business logic, that aren't testable in the same way. And so when we think about how do we build and validate this kind of system, it is actually a very different kind of art and science. You know, if you build classical software, you know it works 'cause you wrote a bunch of unit tests that precisely go through all the aspects of your business logic test that you do what you're supposed to. And when you run those tests, the data actually doesn't matter. You, you can run on fake test data that you've brought in. It doesn't have to be the real thing from the real customers. It doesn't have to be what you would have in production. You can validate business logic in a vacuum without any of that. But when we think about these AI systems, that's actually not the case. You know, if you're building something which is gonna go support your customers and solve their problems. You're gonna need real customer data, you're gonna need real customer problems. It would be impossible to validate that your system worked without that. There's no way you can just kind of plug in some lorem ipsum fake text and validate that you're actually getting good results. And the nature of that validation is suddenly more statistical. You know, you're running evaluations, you're saying, okay, We're able to get to the right answer 90% of the time. We're able to get to the right answer 98% of the time. But it's not like unit tests, it's not 100%. It's a quality metric about the business that you're improving. And this is a really different way of thinking. Ultimately, your software development and your data are suddenly really intertwined. You have to work with these two things together. And this is, I think, a problem in a lot of these AI applications that people are struggling to figure out how to do it. What's the architecture for working with AI and data? How can we make this easy? And the, the problem ultimately comes back to the fact that there's a lot of messy stuff across the company. There's a lot of databases and old apps and legacy systems. You know, there's a lot of weird data that is all specific to a thing that was built at a current time. And how can I plug that into something that needs that context data to make the right decisions? How can I open that up in a way that's reliable, that is safe. And, you know, there's usually a couple paths people go down when they think about this. You know, the first one is maybe we'll just open up to our agent all the data we have. We'll just give it the keys to the databases, we'll give it all the REST APIs, we'll kind of just MCPify everything we've got, and we'll let it fish around and find the information that it needs. And it almost seems like this would work. You know, it's actually good enough for a demo, but it doesn't really translate into production. When you really look at the kind of systems you can build, this gives you real-time access to the data that's needed for the problems. But a lot of this data is so messy and specific to the given application that it doesn't really give you the results you would want. It tends to consume a lot of tokens. It typically isn't safe. You actually don't want an agent mucking around in a bunch of production systems. It's going to access data it shouldn't have. Some of that's gonna come back in outputs that shouldn't be there. And this isn't a controlled environment that you can work with, that you can evaluate, that you can iterate on. So ultimately, you're gonna have to produce some kind of, you know, context data that you can feed to this agent that can power it that's a more controlled environment. And the next approach to this that people often go through is, well, if I have to collect a lot of context data, where is that data? And one place you can find it is in the lakehouse, the data warehouse. And they think, maybe I'll, you know, take this, join it together, get exactly the dataset I want. I'll ship that off to some database specific to my AI application. I'll serve it out of that. So this is kind of the next thought process people often go through. You know, maybe they build some set of SQL queries, Spark jobs, something that kind of agglomerates what they need, strips out the stuff that they don't, and load this up into some kind of operational database. Maybe it's a Postgres or MongoDB, something to serve it to the AI application. And this, this almost has the opposite problem, right? This does give you well-structured, curated data that's been kind of processed and is meant for use. You get to sample it down to just what's needed. It's safe. You're not handing anything over to the agent that you don't want. That's not part of this use case domain, but it's ultimately not real time. You know, you're gonna be using data that's probably hours old, could be days old, and it kind of doesn't really work if you're trying to have this agent interact live with customers, interact with people in the organization, take decisions about the operation of the business. It doesn't work to operate off of a stale version of reality if you're actually trying to work in the real world. If you're actually trying to close the loop. And so this doesn't quite get us there either. And, you know, so inevitably when you think about this, you come to something that kind of has the best of both worlds. You need something where you can create these context datasets, but you have to be able to do it in real time. And this kind of leads you to a streaming architecture, I think very naturally, the ability to gather data from the right sources, be able to process it and curate it in the right way, be able to continuously populate some data source that will be the context data for this agent or this application. And to do this well, it's not as easy as just having a purely real-time flow because at the end of the day, we still have to be able to iterate on this dataset. We have to be able to work with it. We have to be able to run evaluations. We have to be able to try out, adding new data, and see how that changes our evals. And to do that effectively, to build that kind of pipeline, is often very complicated in a purely real-time thing. And so a lot of the work that's gone into streaming systems has been about making streaming systems more than just pure real-time, make it a kind of generalization of batch processing, something where you can build out a complex data pipeline and taste— test it offline, and then be able to have it run continuously as data arrives, as data flows, translate it into production. And this kind of offline to online motion, I think is really important to be able to bring complex data processing into any operational environment. You need both that ability to work in a safe offline environment and then an ability to run it live. And to do this is harder than it sounds. You know, for Confluent, we've been focused on this problem for some time, and you have to bring together all the layers of the stack to make this effective. And this, you know, this starts at the storage layer. So we've put work into a set of functionality we call Tableflow, and this is about taking the real-time streams of data that are in Confluent, being able to replicate those into tables of data that are in object storage, and open up those tables in common table formats like Iceberg or Delta. And this is more than just a connector between these two things. This is actually very deep integration where you have the real-time stream of what's happening in the world, and then you have the long-term representation of that as a table of data. And these two things feed off of each other. They're in some sense one continuous thing that goes from one to the other. And you know, this is a very powerful way of populating your lakehouse or warehouse. These open table formats actually expose this data to any modern analytics tool, but it also gives you the ability to work seamlessly from the historical data all the way to what's happening right now. It gives you this ability to go from batch to streaming. And all because ultimately you have a representation of data that spans both of these. And to make that easy to do, you need to have not just the data, but also the processing of the data. And you know, this is where Flink comes in. I think Flink has done a very good job of unifying the semantics between batch processing and streaming processing, making it possible to build a SQL query or application that can run on data at a point in time, you know, kind of a stored table, a snapshot of what has happened, and then translate that into what continuously is happening. And that's very much this, you know, kind of batch-to-streaming journey that these applications need to take. And this allows you to build offline, very sophisticated pipelines, test them, and then translate them into continuous operation. And when you put these two things together, it makes at least the first part of these overall flows much easier. Now I can capture data from all the complicated systems I have in the organization. I can work on enriching that data, joining together the things I need, filtering it, getting it down to just what's gonna be the right context data for my application. And I can iterate on that until I have the right thing. But I still have this last step where I'm kind of loading this up into some serving database. Taking the streaming data and I'm inserting it into Postgres or putting it into MongoDB. And this last step here, as I go into the operational database, it does have some challenges. And so, you know, there's nothing inherently wrong with that, but there's a couple of things I need to get right. So first of all, I'm mapping between the streaming world and all the semantics there and into this serving database and how it thinks about data. I have two different security models between these two systems. And at the end of the day, anytime I'm doing these big batch loads of data from one system to the other, I'm gonna cause a lot of load on this production system. So anytime I make any kind of serious change to the data, that's gonna be a big operation. Then the incremental updates may be small, but if I go back and restructure things again, that's another batch load that, that may overwhelm the production survey. And so ultimately, like, the challenge I have is I'm trying to knit together the streaming world and the data represented there, and the serving world, which is a totally separate system, and I'm trying to build something which glues these together at the application layer, and find some way of orchestrating this. So, there is an alternative approach to this, and this is something that we've been working on at Confluent to really try and unify these at a more foundational layer. And the idea here is very simple. You know, I have already stored in object storage a table that represents this data. I have already the ability to query that data. The problem I have is that, you know, if I do a point-in-time query on the data in object storage, it's quite slow. These Flink queries are slow. The data in object storage is going to take time to retrieve. You know, we're talking about something that would probably be on the order of you know, a second, several seconds, you know, at the least, and could be more. And so, but what if I could speed that up? What if I could have a fast path query execution layer and, you know, caching that would keep that data in memory on SSDs, some way it could be accessed very quickly? You know, if I could do that, then I could actually have a very continuous flow from the stream processing to the live point-in-time queries that my agent needs to be able to issue. This is exactly what we've done with a piece of functionality we call our real-time context engine. And this allows us to take the real-time streams of data, process it, be able to structure it as tables, and serve it up through MCP to agents or any technology that would use this. And the, the access for the agent is very simple. It's issuing SQL queries, you know, which work against this data just like normal SQL. And the agent can pick the SQL query, can tune it, change keys, sort it in different ways, pick how it wants to access it. This is ultimately a live fast SQL engine on top of this cache data, and it has all the capabilities of this. And this real-time context engine is now generally available, I'm proud to announce. So there's a QR code here, check it out.
It's—
yeah, that's right. It's very easy to use this stuff. And I think what's really cool about this is you get this whole path. So from the connectors that are gonna capture real-time data, the transformations that you're gonna build, the queries against that context that your agent needs to write, it's all in one system. So there's not some complicated set of translations where you're handing off from system A to system B and system B to system C. You know, it's one thing to be able to build and iterate on these datasets. And that's really important because that iteration on data, this is ultimately how we make our application better. Making context data better, that's actually the iteration cycle. It's unlikely that you're going to be building new LLMs, but it is extremely likely. In fact, it's certain that you're going to be building better context data and you're going to be doing that continuously if you work in this area. And this use case of building materialized views, it's not unique to AI. Our first application is this context engine, but you can imagine a lot more you can do with this. If you can create new streaming datasets and issue fast queries against them, that has use cases for analytics. It has use cases for building derived caches of data. It has use cases for personalization, for observability. There's a lot that you can do with this. So we're just getting started in this area. We felt like these AI use cases were so common in customers that really helping complete that for context data was the first step. But we intend to open this up more broadly. So that's a little bit about context data. I'm going to also talk about how to build a certain type of agent, how to build something that actually uses the context data. There's, there's lots of frameworks out there that help you do this. It's kind of a zoo of different APIs and technologies right now. You know, but we did feel like there was an area which was under-addressed, which was about how you could actually build an agent that reacts to business events, something that you plug in. And when the customer does this, it takes some action, you know, something that keys off what's happening in the business and acts in the background. so not something that's kind of a coworker, but something that's actually like a, a full background worker. And this actually has a lot of advantages when you translate it into a streaming model, something where you have the stream of business events, you can take action on them, you can produce output events of what you want to occur. The nice thing about this is you can run it, you can run it offline, you can translate it into an operational environment, you can do it in a way that's safe, You can monitor all the outputs and actions it takes. You can replay them and try them again with different context data, see if it's better. So this is ultimately a very kind of safe and powerful architecture for building this kind of AI system. And the architecture is really simple. Ultimately, you're taking a stream of events, and then we've added agent API to Flink that makes it very easy to build a simple agent in Flink that will take action on the data. And Flink can help you with all the data prep that is needed. It can help you subscribe to the different systems, and it can help you actually take the action, call out to the LLM. And this, this works with virtually any LLM provider, you know, things that are available from the Frontier Labs, from Anthropic or OpenAI, things that are available from the cloud providers, as well as some of the third-party serving layers. And one of the nice things you can see in this picture is the inputs and the outputs here are just event streams. And so I can run this not just in production on the live events that are happening, I can run it on offline snapshots of data and compare what would've happened, you know, what the simulation would've been if I ran it again with a different model, with different context data in a different environment. And that's one of the powerful things about this. So that's a little bit about how to apply LLMs. To streaming data, but what about doing something similar with good old-fashioned machine learning algorithms? So we don't talk about these as much anymore. They still actually have their applications. It turns out that there's a couple of advantages from classical machine learning techniques that sometimes get overlooked. One of 'em is it's very cheap, it's very scalable, and it's very fast, right? So those three make it something that you can kind of use without thinking. You're not gonna have your boss checking your token budget. You can apply it to as much data as you want, and it will not be the slowest thing in that pipeline. And this same functionality that allows you to apply LLMs, we, we have the ability to bring other models to bear, classical machine learning models for anomaly detection, for forecasting, for fraud detection, things that work naturally with streaming data. We've tried to produce capabilities that are easy to use. So this is not something where you have to be a data scientist and go get a PhD to use it. These are relatively self-tuning models that are easy to adapt and you can apply in the flow of data directly in Flink. So I've talked through kind of 3 different problem domains, you know, serving context to AI ops and agents, building these agents to work with streaming data and some of the traditional machine learning capabilities. When you put all these together, this is what we call Confluent Intelligence. And we had an early access version of this that has been available now for some time. We've been working with customers to bring this into production, to harden it, to make use of it. And I'm very excited that we're now announcing the GA of this functionality across Real-Time Context Engine, streaming agents, the machine learning functions. And you can go check this out, read all about it, try it yourself, put it in practice in your environment. Um, and I think this has been a very fun one for our team to work on. This is very exciting functionality that we've found a ton of uses for inside of Confluent itself, as well as with our customers and partners that we've been developing looking for. And you can use each of these features on their own, or you can put it together. Like, one pattern we've seen is it's actually not uncommon that there's some type of stream of business events that are coming, uh, that you'll apply some of these anomaly detection algorithms to look for the unusual cases on a very high-volume stream. But when that happens, you wanna know, well, okay, this was something that was anomalous, investigate it. Why did it occur, right? And that's where you can hook up an LLM and actually have a kind of streaming agent do a deeper dive, reach out to other systems, check what happened, get, you know, some of the related data. And, you know, to power this agent, you can build a set of context datasets that it may want to make use of that help it get to the right decisions. So you could use each of these pieces on their own, or you can hook it all together into one architecture that combines it. And all of this I think is ultimately part of this bigger transition that I talked about, kind of moving out of this world of just business intelligence, just kind of having insights for people and being able to harness data and apply it as part of these AI systems to actually drive the business forward, to actually carry out the business. And I, I think, you know, closely related to this is this transition in the infrastructure layer. As we go from batch systems, the kind of lake where the data just sits, to something that's very active where we're kind of taking action off the data all the time. And this use of real-time data, the rise of streaming, I think is kind of an inexorable part of this trend. And to talk a little bit more about AI and real-time, I'm really excited to welcome to the stage somebody who knows quite a lot about this. So, uh, we're going to have a guest from ElevenLabs, our own AI leader Sean Falconer, and Alex Holt, who's the head of engineering at ElevenLabs, are going to come tell us a little bit about what they're up to. So guys, come on up. Thank you all. Please welcome Alex Holt, Head of Forward Deployed Engineering at ElevenLabs, and Sean Faulkner, Head of AI at Confluent.
All right, well, thank you. Thank you, London. Alex, welcome. Thank you. Yeah, thanks so much for being here. So, you know, for those that maybe don't know a lot about ElevenLabs and the things that you do there, you know, can you maybe start off with that? You know, who are you guys? What do you do? Can you tell us a little bit about that? Yeah.
So ElevenLabs are an AI research and product company. And actually, you've all already heard what we do. So we started out building voice models and every speaker that's being introduced today The voice on the tannoy is an ElevenLabs artificially generated voice. And so you've all heard kind of the output of what we do.
We're doing a live demo right now.
We're doing a live demo. And so the mission of ElevenLabs is pretty simple. So our goal is to help companies to communicate with their employees, with their customers, and with the world. And we do that by producing world-leading voice and audio AI models and products to deploy those models for content creation and our creative platforms, but also for building intelligent voice and multimodal agents for use cases like customer support or sales or learning and development or even video game characters.
Yeah, amazing. So we just heard Jay talk a lot about you know, the essentially why real-time data matters for AI. And obviously that's something that we, we think a lot about at, at Confluent, and I spend a lot of my time thinking about that problem. But for the types of experiences that you're building at ElevenLabs with these voice agents, like, why, why does that matter? Why do you need essentially real-time data for the types of applications that you're building? Yep.
Um, so I mean, we do a lot with, with— we create voice agents, um, and latency is super obvious in voice. If you're talking to somebody and they take ages to respond, it's going to sound really weird. If you're talking to customer support and they're not answering you with it in real time, it's going to sound— it's going to sound weird. It's not going to sound like a real person. It's not going to sound caring or emotive. And you're going to very quickly hit the zero button. Can I talk to a human? Can I speak to a human? And so latency is super important and it's important in both the voice lab It's creating expressive, realistic-sounding agents. It's also important on the data layer so that those agents are saying the right thing, and they have to immediately to hand have all of the information they need to actually do their function.
Yeah, I mean, that makes a lot of sense. There's kind of two constraints around sort of real-time when it comes to something like voice, where you have the latency problem, like you said, is someone— essentially, the voice agent takes a long time to respond. It feels very unnatural and awkward, and people are going to look for talking to a real human. And then also you have the challenge where once the voice agent's actually responding, you want it to be able to give essentially a meaningful, accurate response. And if it doesn't understand what's happening right now, it's not going to be able to do that effectively.
100%. I think when we're thinking about building voice agents, really what we're trying to do is we're changing the way that our customers communicate with their customers or with their employees. And there's two parts to that. One is it's got to sound emotive and caring, and it's got to fit the tone of whatever message it's trying to— that agent is trying to achieve. And that can be done through our models and through the way that we stitch those together with kind of an emotional understanding. It also needs to say the right thing. You know, our goal is that when you are communicating with customers, if it's a customer support use case and you call in, you are with one of our AI agents, It sounds like the most emotive, caring, understanding individual, and the most intelligent person in the customer support org that has all of the information of every time I've ever called in before, all of the understanding of the product I'm calling about, and the services offered by my business. And, you know, it's analogous in sales. It's gotta be the most emotive, caring, and interesting salesperson I've ever spoken to, that also knows absolutely everything about the product in the same way that my engineering team does. Yeah, and so you need both of those pieces. You need the emotive voice engine, and you need the ability to connect to the real data sources in real time and take actions in my source system so that those conversations have substance and action.
Yeah, I would think the bar is also really high from a customer experience standpoint, where if the voice agent responds in a way where it actually leads to sort of a misleading result or misrepresentation or doesn't understand the customer, that results in churn, essentially. Which is going to really impact the business as well.
And again, it's important on both sides. If you get through to an AI agent and it doesn't sound human, then before it says anything, you're kind of— we've all been on those IVRs where it says, press 0 to go through to this, press 1 to go through to this, and you're immediately kind of going, okay, what's the fastest route to get through to an actual human? Human, human, human. And so that needs to be really solid and it needs to sound emotive and caring and real and like somebody you actually want to spend time engaging with and listening to. And then as you're saying, the second part is really important, which is once I have that conversation with it, I need to know that's actually resolved my issue and not just giving me lip service or hallucinating, making things up, or worse from a business perspective, like giving me refunds when I perhaps don't deserve them. Yeah.
So, you know, I work with a lot of, you know, enterprise customers and talk to a lot of them. And I think predominantly in the industry, we see this challenge around what I would term the agentic data divide, where we have these really powerful models. They continue to get better. I don't think that that's going to slow down anytime soon. But the businesses essentially can't deliver the right data at the right time to actually feed these models to make them smart about their business. The models themselves are very smart about general information, maybe a little dumb when it comes to your domain-specific information. They have to bridge that gap between the data that they have is very domain-specific and makes them who they are as a company, and essentially the models. So how are you sort of doing that in your world of these, you know, production voice agents? And why is closing that gap also important?
Yeah, I mean, I think you kind of touched on it, which is I think the models right now are really great. And for a lot of the use cases that are actually dealing with enterprises, the models that you can buy off the shelf are smart enough. They can, you know, a lot of the use cases that we're doing, you know, don't require models that are kind of PhD levels of intelligence in specific fields. They require something that's empathetic and something that can understand a process and follow a process and feel human doing that. And I think a lot of the foundation models are already there, but the thing that's missing is the context of, as an enterprise, my business, the way that my business operates, the procedures that my customer support agents follow, or my sales, uh, team follows, um, and the context about the customer that's calling as well. And so it really is connecting to those foundational models, those procedures, and the data, and having the confidence that it's actually going to use that data and follow those procedures. And that's one of the, the big challenges that we've solved for our— with our, uh, agents platform is the ability to configure those connections and test and be certain of the fact that the agent's gonna follow them.
Yeah, I mean, if you can have the most powerful model in the world and you're building this sort of beautiful castle, but if the foundation of that castle is sort of a junk heap of data, it's not gonna be very effective and the castle's gonna kind of fall over.
I mean, if you were to take Einstein and put him in a customer support center, I suspect he'd be really, really bad at it, right? Like, you don't, you know, the models are, you know, It needs to be good enough. And I think we're there for a lot of these cases that we're seeing.
Yeah. So I was at Google I/O, I think, probably 6 years ago or so, when they did this amazing demo. It's a very famous demo where they played essentially a voice agent at the time booking a haircut for somebody. And it was an incredible demo. It sounded very, very real. People were blown away by it. And I don't think that that actually ever materialized as a product, as far as I know. And I think that that is like a very common experience for anybody who's, you know, building with AI today is essentially it's very— you can build these incredible demos, but companies really struggle to essentially move beyond the demo to production.
And you're—
ElevenLabs is essentially a company that has done this effectively. So we're in a place where we'd love to be able to learn from you. So what are the things that people tend to underestimate with these types of investments? And then what are some of the lessons or advice that you can give essentially everybody in the audience?
Yep. I think the truth is it is really, really hard to get enterprise-scale and great AI into production. There's no one tool that's kind of press a button and it's done and deployed and ready. And moving from demo to reality is hard. The way that we tackle it at ElevenLabs is that we, we have products that kind of go from the full end of the spectrum, from developer tools for you to build the experience that really matches the way that your business works APIs for our models, APIs for our voice engine platform, a full platform for configuring agents quickly. We also have a user-forward deployed engineering team and with it where we will partner with our customers in a very, very deep way. We embed, we work out what the KPIs and metrics are actually trying to achieve, are understand their business and work as a single team to actually deliver agents to that specification together using those tools. And so, you know, the, the, the, the, Advice here is it is— it does require configuration, it does require time and, and, and development resources, um, and it requires tooling that makes treating agents almost as code. So developing, versioning, testing, iterating, monitoring once it's been released, um, you need that, that kind of full DevOps suites for, for agents. Um, and if you have those things, then you can build truly enterprise-grade production agents today.
Yeah, I mean, I think it's well said. Like, I think that, that is the piece where, uh, I, I also see, you know, people underestimating. Everybody wants this kind of AI magic box to just work like a magic box, and in order to get the box to actually do the magic, it takes a lot of actual sort of engineering, data engineering work, and you can't get away from that as a business. So that's something that, you know, everybody essentially has to invest in.
100%.
All right, well, thank you so much, Alex, for, for being here, and we We are going to welcome one of Confluent's customers up next, and thank you everyone.
Thanks everyone.
Please welcome Can Suzeri, Engineering Director at Getir, and Mike Agnic, DSP General Manager at Confluent.
It's great to be back in London with all of you, and it's great to be here, John, with you. So maybe we could just start with a quick intro. Tell us about yourself. Tell us a little bit about Gitir.
Yes, thank you, Mike. Hello everyone. I am John Sioziri, Engineering Director at Gitir. So at Gitir, delivery in minutes is not just a slogan, it's a customer promise. Let's think about it. But when your whole business is built around speed, the real-time data becomes absolutely essential. We have a motto at Getir. Speed is our currency. And if you are serving more than 14 million users, then even a small delay is not just a technical issue. It can affect customer experience, operational decisions, and the overall quality of the service. That is why real-time data and event-driven architecture matter so much for Getir. It helps us make better decisions, react faster, and keep the business running with the level of accuracy and consistency that this kind of customer promise requires. So Mike, when I think about real-time data at Getir, I don't see it just as a technical issue. I see it as one of the most important parts of the business.
Well, I, I'd like to come back to that, but I think maybe, uh, tell us a little bit about this journey to 40 million users. There must have been a moment when your current approaches kind of stopped working and you needed to take a fresh look at your architecture. Can you tell us that story?
Yes. As we scaled, we started to understand that most of our services were too tightly connected. Even though we had many services, they were still created a lot of hard dependencies between each other. So the system behaved more like a distributed monolith than the microservice architecture we wanted to create. And I can clearly say that distributed monolith is the worst thing that you can ever face as a tech team because it slows you down, creates hidden dependencies, complexity is everywhere, but ownership and flexibility are not. So at that point, we realized that we needed a different model, a different approach. We needed clearer boundaries, domain context, and data ownership to be able to build a loosely coupled system with using event-driven architecture.
It's a very kind of normal story to run into both data scaling and infrastructure scaling challenges at the same time. And it sounds like you, you know, you undertook this and you leaned pretty heavily on streaming. But what were the— we don't just wanna re-architect things for the sake of it. What were the impacts that you saw in the business the way your engineering team operated, maybe even the culture of Getir?
Yeah, Mike, the impact of that transformation was very clear, both for the business and for engineering. For example, we saw 60% improvement in homepage and checkout performance, a 60% reduction in technical incidents, a 70% reduction in database cost and overall TCO, and 50% improvement in developer velocity. These numbers are very important for me because they show that that was not just an architectural cleanup. It created measurable business, operational, and engineering volume. It also changed how our teams spend their time. Instead of dealing with, you know, infrastructure complexity or dependencies between services, our engineers could focus much more on business problems, product improvements, and faster execution within their own domain. So that improves focus, speed, and engineering efficiency in a very practical way.
When we were talking yesterday, you brought up this idea of shifting left governance and data processing closer to the source of data. And I know you started to use Flink to do some of that processing. What does that mean to you? What ability does that give you to kind of create a new kind of product? And what does that mean?
From my experience, real-time systems need high-quality data first because if your data is not reliable or available at the right moment, then real-time system doesn't really help you. You are just making faster decisions, but probably the wrong ones because your data is weak. That's why we adapted a shift-left mindset around data quality. We try to improve the data closer to the source before it moves across the organization. And of course, Flink is important here because with Flink we can work with the data while things are happening, not after everything's finished. So this type of transformation is also about having the right partner with you. For us, that partner was Confluent because I believe that you can find technology anywhere, but you can't find a real partnership.
Well, I appreciate it. You're making me blush. But the— Thank you.
It's the truth.
I think before— I'd say one of the things I've gotten to do a lot of in the last couple of years is spend a lot of time with customers big and small all around the globe. And I'd say that as AI has emerged and become much more dominant there, it's been really clear to me the organizations that are getting authentic value out of AI are the ones who have invested in getting their data understandable by kind of all of the humans and now the systems that need to get at it. So I think governing, cleaning, providing high-quality data early, I think, is, is really one of these inflection points for us as we all start to use AI. So as we, as we close things out, what's next for Getir? I'd love to hear kind of— you've got this foundation, what comes next? And maybe you could leave the audience with a piece of advice. Having been through this transformation, maybe there's some folks here who are a few years behind where you are. If you could leave them with one piece of advice on, on what they should do, what would that be?
Yeah, of course. For us, data streaming is not just a side topic. It's a core business capability. And AI is also part of that feature. But again, I believe that the main point is data. Because if you want AI to create a real business value, then you need clean and reliable data at the right moment. That's why the real opportunity is not just only experiment with GenAI or agents. It is to connect those capabilities with a strong real-time data foundation. And if I had one message for leaders, it will be this. Real-time culture starts when the leaders treat data streaming as a business priority, not just a technical topic. Thank you.
Great.
Well, John, thank you for being here. Everybody please join me in thanking John. Thank you.
Thank you.
It's a pleasure.
Thank you. Please welcome Sean Klaus, Chief Product Officer of Confluent.
Ready? Good morning, and a big thank you to Getir and ElevenLabs. What an incredible set of use cases, and honestly, those are just a couple of examples of why real-time data is the lifeblood of AI. Without it, your AI agents are going to be reasoning over yesterday's truth. And I'd like to dive a little bit deeper into the opportunity ahead of us here today. Uh, and as part of that, in the next 20 minutes, we're going to show you how the data streaming platform produces AI-ready data, continuously contextualized and governed data ready for use by any downstream system. But instead of just talking about it, let's build something fun. We're going to be talking about River Racing. They're an F1 team about to race in the Silverstone Grand Prix. Their goal? Well, they want to use AI to improve their pit strategy. Should they pit now, wait a couple of laps, or even stay out? Now, it might sound simple, but actually effectively operating a strategy like this requires real-time AI-ready data. And to make that a reality, they're going to have to use every capability of the data streaming platform. First, they're going to have to connect and stream a variety of different important information, including information from the car's sensors. They're going to need race standing data. They're going to need driver profiles and more. And then they need to convert that data into contextualized, well-governed, continuous AI-ready data that they can then deliver to the AI to make recommendations and to take action. Now, our mission for the data streaming platform is really simple. First, we want to make streaming ubiquitous. We want to make it available for any use case, anywhere, any team, all the way from the smallest microservice use case, all the way up to the AI agents that are transforming your business. Because when streaming is everywhere, it unlocks a better way to build. You can shift governance and processing left, closer to the stream, do that work once, and then reuse the data everywhere you need it, in analytics, in AI, and for your real-time experiences. You— the stream once, use everywhere. Now, let's start with that first step, ubiquity. Many of you in the audience probably know that streaming is a more, more powerful foundation. It's better for innovation today and in the future, but honestly, it can feel too complicated, risky, or costly to scale up to all of the potential use cases. But none of those things are laws of physics. Let's take the first part, complexity. The era of agentic engineering is upon us, and the data streaming platform is coming along for the ride. We're excited to announce our new Confluent MCP servers and skills that make AI-driven development with Confluent super easy. With the MCP tools, the entire data streaming platform becomes a surf— callable surface area for all of your AI agents. And with skills, we've packaged up best practices into reusable agent capabilities. And that means that you can stand up any streaming infrastructure you need for any project in minutes using natural language from the tools you're already using, like Cursor, Cloud Code, or any MCP-enabled tooling. So let's see some of this in action. River needs data from the car sensors, and they need the race standing data, which is coming in from the FIA's MQ server. We're gonna see just how easy it is to get all of this moving with Confluent, And for that, please welcome back Sean Falconer. All right.
Thanks, Sean. So I'm doing something a little bit different today. I'm going to put aside my race helmet and put on my data engineering hat. And do some AI programming to help my team at River build a real-time pitch strategy system. Let's get into it. Please cue the demo. All right, so starting off with Claude code, which has access to Confluence fully managed MCP server that you can use for Confluence development. I'm going to ask a pretty simple question, which is what data do I have access to? Claude thinks for a moment and uses the MCP tools available to identify the car telemetry topic, which is streaming in live telemetry metrics from our car that's on the track right now. I can also take a look at the schema and see all the various fields and telemetry metrics that are coming from this car. I can ask follow-up questions as well. For example, I can ask to see a sample message, and then Claude will stand up a Kafka consumer using the tools available and actually pull in live traffic. Now, this is only half the story. We have other data besides the car metrics that we also wanna bring in. For example, race standings available from our organizer. So I tell Claude about this, and then Claude is able to not only tell me what I need, which is IBM's MQ source connector, but the exact steps I actually need to follow to deploy this. So I need to provide some information about the credentials and a little bit of information about the queue configuration. and then I can go ahead and ask Quad to deploy that. So we're gonna do that. And then just like that, we are live. The connector's been deployed and now we're bringing in live traffic. So we, if we look at the topics here, we see the race standings topic, the various messages, and this is bringing in that driver information and where they are on the track right now. So the amazing thing about all this is that we were able to stand up a fully managed connector in just a couple minutes. And bring in live traffic from MQ and do that all by expressing essentially our need using natural language and not having to think too deeply about the underlying data infrastructure or the management of this system. So we have everything that we need now to build our real-time pit recommendation system, and we're gonna get back to building that in just a couple minutes. But for now, I'm gonna hand this back to Sean.
All right, so thank you, Sean. As you just saw, Agentic Engineering cuts through all of the complexity of creating and managing the streaming infrastructure we need for this project. But we did say that our goal was to show and get streaming to be ubiquitous, and folks still do worry about the cost and risk of scaling up streaming to all the different data use cases. And that's where we created Cora, which is our Apache Kafka engine built ground up for the cloud. Cora is, is what lets us offer elastic, auto-scaling clusters that instantly auto-scale, that are 10 times faster, 40 to 70% less expensive, and offer a rock-solid 99.99% SLA. Now, this isn't just theoretical. We operate more than 30,000 clusters in over 100 cloud regions, sustaining more than 8.2 trillion messages a day. Now, some of you might already know about Cora, and you might already know that it's a cost-effective, scalable, easy-to-use way to scale out streaming. But there's another reason that some of you might not yet have made the move. Migrations, they can be brutal. It can require months of planning, mapping out different producers and consumers, and coordinating cutovers across different application teams. And honestly, sometimes the effort and the risk can seem more than the reward. But have you ever wondered, why can't there just be a button? A button I just press and the migration just happens and I don't have to think about anything else? Well, that's what we've created with KCP, Kafka Copy Paste. It's our new tool that means that even complicated Kafka migrations can be achieved in just a few days with zero downtime. How does it work? Well, first, it maps out all of your existing streaming environment, then it provisions the new infrastructure, it migrates the data, and then it seamlessly moves your clients across. The best part? No disruption. Your real-time experiences keep working, your migration happens while your business keeps going. Now, look, I know that many vendors have made bold claims about simple migration over the years towards their tools. I totally get it. And honestly, at Confluent, we've always acknowledged that migrating mission-critical real-time infrastructure is complicated. But KCP is the real deal. We're making it really simple. That's why we're making such a big deal out of it. So you can see how it's been done. Financial services organizations, consumer goods companies, and digital natives they've used KCP and they've made it happen. Check out the QR code to learn more about exactly how it works. And when you make the switch, you get the benefits. We see it at our customers from Indeed to SAS and beyond. Customers that move over save between 40% and 70% on their, on their streaming costs, but they also unlock streaming for all the different data use cases, so they get better outcomes. And we can do it for you too. We can make the migration simple, and we can make the economics undeniable. All right, I said ubiquity, and we also have to acknowledge that about 50% of the world's infrastructure exists on-prem or in private clouds. So today, we're excited to introduce a cloud-native streaming environment for those deployment models too. Introducing Confluent Private Cloud.
Cloud.
It lets your platform teams centrally manage a streaming environment on behalf of other teams or business units on your own infrastructure, in your VPC, or in your data center. Now, what do we mean by cloud-like? Well, think multi-tenancy, dramatically higher performance, easy upgrade, scaling, and patching, and much more. We've taken the very best of Confluent Platform and brought in the key building blocks of Confluent Cloud. And with Confluent— sorry, and the key building blocks of Confluent Cloud. And with Confluent Private Cloud, organizations can save up to 50% across their operational and infrastructure costs for their streaming environments. How exactly? Well, with our 10 times faster broker, intelligent replication, and bin packing, you can drive up your infrastructure density while still achieving great flexibility, reducing your infrastructure costs by up to 70%. And with Conformant Private Cloud Gateway, you can centrally enforce policies and governance that enable you to safely share infrastructure and data across teams. And soon we'll be introducing logical multi-tenancy into Conformant Private Cloud, which will deliver even greater infrastructure density and flexibility. So we have deployment models that meet your data wherever it happens to be. On-prem, in the cloud, or in your data, or in a virtual private cloud, or in our cloud. We can go anywhere, and many of you will have your data in all of those places. You will be hybrid. So how can you manage and monitor all of this environment? Well, last year we shared with you our vision for Unified Stream Manager, and we're excited to announce that Unified Stream Manager is now GA. It lets you see all of your Confluent infrastructure. All of your clusters, all your topics, your schemas, your lineage, your metrics, all of it from one pane of glass, all your data, wherever it is. And across all these different deployment options, configure— configuration options and controls, we take your data sovereignty extremely seriously. We offer the right combination of deployment model, configuration, and controls to meet, to help you meet any of your regulatory or governance requirements, including DORA, NIST 2, the Cloud Act, GDPR, all of it. So, okay, onto our next step. Once streaming is ubiquitous, we can shift governance and processing left to stream once, use everywhere. And that is how we unlock contextualized, rich, governed, AI-ready data for all of our use cases. Now, processing data with Flink is the key to having data be rich and contextualized before it lands in a database, data lake, or AI system. And in Confluent Cloud, we offer a totally serverless Flink that just auto-scales to the needs of your workloads. And on-prem, we offer a Flink that is easy to scale, upgrade, and manage. And we're always working to make Flink more powerful, whether you prefer to work in code or in SQL. Let's start with Flink for those who prefer to work in code. Over the past several years, we've been working with the Flink community to improve the various different Flink programmatic APIs. We've been working to improve the equivalency between the Java APIs and the Python APIs and make it idiomatic. And recently into the table API, we've added support for processed table functions, timers, and anonymous user-defined functions. Now the good news for all of you Flink developers in the audience is that this means that the table API is now capable of almost anything you may have used the data stream API to do before, but it is easier to use and it's much simpler to scale and upgrade over time. This is a massive improvement. In our cloud, this is totally serverless, You create the Flink job, we run it for you and scale it and upgrade it. Now, the other major audience for, for who need to work with data or streaming it and enriching it are data engineers, and they typically prefer to work with SQL. Now, the Flink SQL API is actually incredibly powerful for joining, aggregating, filtering, and enriching data as it moves. And we've recently made Flink SQL even more powerful by introducing the concept of materialized tables. Materialized tables let you express the table that is produced by a SQL statement and then safely evolve and improve the SQL to produce the table without breaking downstream consumers of that table. So preserving the model. This is actually a very powerful building block. And the next step is connecting it to the tools and infrastructure that your data, that your data practitioners are using for their pipelines today. And that's why we are super excited to announce our brand new Flink adapter for dbt, which is GA today. Now, your data practitioners are already using tools like dbt to build declarative pipelines in the batch and data lake world. dbt is actually incredibly powerful because it lets you express the shape of data, separate from how that data is produced, and then use configuration as code, CI/CD, and modern engineering practices to safely evolve and improve the pipelines, even at scale. And now, with the dbt adapter for Apache Flink, you can upgrade those pipelines into real-time with very little change. You can produce fast AI-ready data using the same tools and infrastructure your teams are literally already using all day. And then once we have this AI-ready data, we can then use Confluent Intelligence to feed it directly to our agents through MCP. Let's check out how that works at River Racing. River needs to get information from the car, including the tire temperatures, battery level, fuel level, and they need that in alongside race standing data as well to power their AI pit strategy. Let's see how it works. Sean, show us some AI agents.
Sounds great, Sean. So we're going to start off by detecting anomalies against the car telemetry topic that we looked at earlier. And this is how we're going to be able to tell whether our tire pressure is getting low or we're starting to see brake wear. We'll use these as signals to trigger agents. And then we're gonna also have those agents take into account the current race standings. Through Confluence Real-Time Context Engine so that they can make decisions essentially based on what's happening right now. We're gonna set up and deploy this all using Confluence DBT adapter. So let's take a look. Cue the demo, please. All right, so we're starting off by enabling Confluence Real-Time Context Engine against the race standings topic. When we enable the context engine, data's instantly materialized into a low-latency data serving layer accessible via MCP. By any AI agent or application. In our case, our Pit agent is receiving fresh materialized context so it can make decisions based on what's happening right now. And next, we need to actually define our agent. Within Confluence Flink, agents are first-class constructs. I simply define their role, the goal, the models they're using, and the tools that they have available, and they can run them natively within the stream. Your streaming agent is always on, it's always evaluating, it's always ready to take action. But of course, we don't want it taking action against raw telemetry data. We want it taking action based around clear structured snapshots that represent decisions that we can actually, uh, understand. So this is where our pipeline comes into play, and we're going to use the Confluent DBT adapter to set this up. It consists of two different pipelines. The first is anomaly detection with our car metrics. We're gonna first enrich the car metrics with driver profile and lap information. This way, instead of a fire hose of raw events, we're getting a clean structured snapshot representing the car's performance every 10 seconds. And then we're gonna use this enriched stream to apply our built-in anomaly detection. So things like tire pressure and temperature, fuel levels, brake wear, all metrics simultaneously. And the great thing about this is you don't have to be an ML expert to use these built-in ML functions. They're easy to use and meant to be used out of the box. All right, so with our first pipeline, the anomaly detection pipeline, out of the way, we can turn our attention to the second pipeline, which is deploy our agent. Every event that gets written into the car state topic, we wanna trigger our agent, and the agent is going to consume these messages, understand the current situation, and then make a recommendation about best next best action. Simple command, dbt run, and I deploy this pipeline. dbt compiles the model and instantly saves it as a Flink job. Now, this is not a batch process. This is a real-time streaming table materialization running 24/7 and processing every telemetry metric the millisecond it arrives. And behind the scenes, the dbt adapter's doing the heavy lifting. Meeting data engineers where they are, converting your DBT models into live production-grade Flink jobs. So everything is deployed now and our car state topic is there. And this is real-time intelligence that we can actually take action on. So when we dive into the car state topic and see where we're at in the race, we see here that we're actually on lap 32. We've been on the same soft tires the entire race. And as a result, we've moved from P3 to P8. Because we have not boxed the car yet. I don't know what we were thinking, but we can see here that our tire temperatures actually spiked, kickstarting our anomaly detection pipeline for the first time in the race. This triggers our agent. The agent assesses the current situation, understands that we're on worn softs, that we're losing positions, and that the other cars have already pitted. It recommends that we box now, change the mediums, and then race to the end. Now, the result of all this is that we get real-time proactive decision-making with clear intelligence behind every decision before we take action. And with that, let's head back to the slides. All right, so we just saw how Confluent Intelligence can help you power a fully automated racing intelligence system. And that's, that's really powerful and is great for our team at River. But conceptually, this is how we should be thinking about building enterprise AI system. We should be enriching data in real time. We should be using anomaly detection and other means to detect key business events, and then using those as signals to autonomously trigger agents, and then feeding those agents real-time context so that they can make decisions about what's happening right now and be able to do this all at the same time. And with that, I'll hand this back to Sean.
Thank you, Sean. So our agents are in motion making decisions, but obviously River wants to know about the quality of those decisions too, and that's something they would typically do using analytics in the data lake or data warehouse. The problem is that traditionally feeding real-time high-quality data to a data lake has honestly been just as hard as feeding it to an AI system. But it doesn't have to be that way anymore, thanks to the emergence of open table formats like Iceberg and Delta Lake. And that's because of Tableflow. With Tableflow, in just a few clicks, you can set it up so that a stream is constantly presented as an Iceberg or Delta Lake table and is immediately usable in all of your analytical toolsets, all of the stuff you are already using. There's no need for, ETL, ELT, endless data prep. This is just seamless data flow directly from the stream into your analytics estate. Now, we went GA with Tableflow a little while ago on support for AWS and Iceberg, but since then we've been hard at work making it even better. We've shipped support for Delta Lake and Unity Catalog so that the tables are easy to use in Databricks as well. We added support for Azure, and we've added critical capabilities like upserts, BYOK encryption, dead letter queues, and observability. So let's see Tableflow in action as well. Sean, take it away.
All right, thanks, Sean. So we're now going to take those PIT recommendations that we looked at before, and we're going to use Databricks to perform some analytics with that using Confluence Tableflow. All right, so please cue the demo. So Confluence Tableflow makes it push-button simple to turn your Kafka topics into Delta Lake or Iceberg tables. And we wanna move our Pit recommendations into our data lake. It's pretty simple to do that with Tableflow. We choose our format, Delta Lake in this case, set up our credentials and choose our S3 bucket. And then we go ahead and launch Tableflow. And just like that, we're live. There's no custom code to write, there's no pipelines to manage. Essentially, with Tableflow, your streams become your tables, and then you can use any BI or SQL tool to query them instantly. Now, in order for us to actually understand what's happening, we need one more piece of the puzzle: historical race context. And this data lives in a Postgres database. So 22 drivers, 11 teams, 9 Grand Prixs, starting positions, ending positions, tire strategies, all in this database, and we wanna move it into our data lake. And we're gonna use Confluence CDC to Tableau Agent skill to set this up. So when I go ahead and invoke the skill, it will automatically create a CDC connector to read the database transaction logs and move those into Confluence in real time. And it also automatically enables Tableau on the output topic so we can move the data into our data lake seamlessly. And because this is CDC, every wind, every point change is instantly available within Copilot. And the great thing about this is you don't have to be a DBA or Kafka expert or Databricks specialist to set this up. You just use the skill and essentially now we've deployed our pipeline and we're ready to go. So over in Databricks, we're gonna use Genie, Genie AI to actually interact with our data using natural language. So I can ask a question like, how many positions did we gain? You know, following essentially the agent's pit recommendations so far. Genie is able to figure out the intent, execute a SQL query behind the scenes. We can even inspect the query that executed, bring back raw results, and then reinterpret those back to me in natural language. So I can see here that we've actually gained 11 positions over the last 4 races. Including two wins in Bahrain and Spain following our pit agent recommendations. That's a pretty good ROI for this pipeline that we built. And then we can continue to explore and interact with this data. So for example, I can ask a follow-up question of what is the best performing tire strategy so far this season? And again, it's going to execute this and it shows me visually that actually the best strategy has been moving from softs to mediums so far. So I continue to interact with the data like this because once we have this continuously refreshed contextual data available in Databricks in near real time, it opens up all kinds of different use cases for our team at River. All right, so this ends the Sean and Sean portion of our time for the keynote. So F squared is going back to S to the power of 1. And thank you so much, and I'll leave it to Sean to take you home.
All right, thank you, Sean. And as you just saw, Tableflow demonstrates yet again the value of shifting left, moving governance and processing closer to the stream. Tableflow gave us the silver materialized tables that we needed in our data lake so that we could immediately use them, and we got to reuse the exact same data for our AI use case, for our analytics, and for our real-time experiences. Now, the best part is that not only does this end up being faster and better in landing data into your data lake, it actually also ends up being less expensive as well. We've seen it again and again with our customers, Notion, Siemens, and many more, that your data quality issues can be reduced by up to 60% Your data processing costs can be reduced by 30% and more, and your engineering teams can go a lot faster because they have this reliable, reusable data ready to go off the shelf. But Tableflow actually does more than just feed data lakes. It also makes Flink more powerful too. Earlier you heard Jay Kreps describe the fact that Flink can work on streams or in tables in real time or in batch. And that means that with the Tableflow-backed topics, Flink can run snapshot queries, which are point-in-time batch-like queries advantaged by the Tableflow data that go 50 to 100 times faster than they would just on the stream itself. Now, these queries are perfect for when you are developing, iterating, or debugging a particular Flink job or pipeline. But I hope you can see where all of this is going. With Flink, Tableflow, snapshot queries, dbt, and the other capabilities we're talking about, you can unify batch and streaming. Anything that you can do in batch, you can do in streaming, and the result is faster, better, and cost-effective. That's what we mean about the unification of batch and streaming that we have been talking about for so long. All right, so we solved River's use case, but you should give it a go yourself. You should check out our booth on the show floor where you can build your own racing intelligence system and see if you can help the River Racing do even better. But before I close up today, I wanted to take a brief moment to highlight the more than 90 major innovations that have been delivered since we were last here in London a year ago. 90 different major innovations. I couldn't possibly walk you through all of them, but if you'd like to learn more, please check out the QR code on the screen. And with that, thank you so much for joining us here this morning. We hope you have a fantastic show.
Appreciate it.
More from Jay

How to build a company when you’re not an optimist | Jay Kreps (Co-founder and CEO, Confluent)

We're the 'central nervous system' for data streaming, says Confluent CEO Jay Kreps

Beyond Speed: Kafka Summit London 2023 Keynote | Jay Kreps, Co-founder & CEO, Confluent

From Open Source to a Global Business - (Confluent) Jay Kreps & (Sequoia) Matt Miller | Slush 2022

Keynote: Jay Kreps, Confluent | Modern Data Flow: Data Pipelines Done Right | RTA Summit '22

The Evolution of Apache Kafka: From In-House Infrastructure to Managed Cloud Service ft. Jay Kreps

Event-Driven Architectures | Jay Kreps, CEO, Confluent (Overview for Technical Leaders & Executives)

Confluent CEO: Cloud services are a new way of delivering capabilities to companies

Data Mess to Data Mesh | Jay Kreps, CEO, Confluent | Kafka Summit Americas 2021 Keynote

