Robert Ta describes Booking.com's AI observability system, which collects telemetry, redacts personally identifiable information, and streams data to Arise for real-time monitoring. The setup caught latency spikes and context bloat in AI agents before users noticed them, preventing incidents that would have caused outages.
This is how Booking.com scales AI observability with Arise, which is an observability and evals tooling vendor, AI infrastructure. So why Booking.com needs AI observability at scale. To those that are interested in enterprise architectures and responsible for figuring out what is a scalable architecture to meet our AI adoption and AI-native development needs going forward as a technology team, I found this to be a very interesting use case. I'll just skip ahead to The diagrams that matter. I thought this is a really interesting diagram to kind of ground on in terms of their architecture. So on the left, you have n number of GenAI applications. They're using an AI telemetry collector. They pull that in, they normalize it, and then they have traces going into a secure message bus. In this case, it looks like they are using a managed Kafka service. So that's a durable buffer that absorbs spikes and decouples producers from consumers. And then they have a PII redaction service. I thought this was really interesting for anybody in enterprise software that has to deal with compliance and regulation and privacy and things like that. So they specifically call that out in their architecture, that they have a service that detects and masks PII in span attributes and text fields. And then they go and export to Arise and a warehouse. So redacted traces are sent to Arise in real time and stored in the warehouse for analytics. Arise will then enrich with space project metadata, Et cetera. And then on the warehouse ingestion side, it looks like these persist to use for offline analytics, long-term trends, and backfills. So it's interesting that they have Kafka right in the middle there. I wonder if other teams are going to this event management kind of architecture from all of their telemetry. This makes it so you could do things like this PII redaction service. And here's another diagram for their ML prediction log pipeline for Booking.com. from their data warehouse all the way to Arise, and then getting to specific datasets that they could use for GenAI applications such as ranking or fraud detection or any other model that they could then fine-tune for domain-specific problems. So model gets the prediction logs, gets to warehouse, does some level of normalization, unification, sampling, enrichment of data per model transforms, and then they use Arise-specific tech to import that data in For further fine-tuning. One of the biggest takeaways from reading this to me was everybody that is deploying AI architectures at scale, you really need to be thinking about chunking out their services so that you could do this at scale. That's one of the things the article goes through, and they even built out their telemetry collectors pretty thoughtfully. And I'm assuming, like, anybody who's ran big organizations and engineering organizations, like I'm sure Booking.com has, all of these teams, these applications are different teams. So what's actually going on is they seem to have very strong architectural governance to be like, hey, everybody use OpenTelemetry, or maybe it's used by default. And that way you could enable this sort of thing. Yeah. What are your thoughts on this, Jonathan, from a CTO perspective? I don't know if you have strong thoughts on this article. I thought it was just cool to see a use case out in the wild that actually works for a product that's used globally and with a large, large volume of data.
Yeah. I mean, the 2 use cases they put here that, that seemed useful. One was latency in the agent. So detecting latency. And the other was context bloat. So that's where you use this approach. We actually had one of our agents just had a latency problem. Nature of the problem where the configuration had been in place in production since like December, but the gradual volume increase caused a latency problem that was noticeable 3 days ago, but we didn't catch it until our, you know, basically the people who use the application reported problems with like slowness. In theory, like if we'd had this pipeline set up, we could have identified using these observability tools before our customers felt the pain. We could have like seen that this latency number was going up with the scale that it was having to endure and solve that. So that seems generally useful. You know, we don't have to worry about scaling across our organization. It's just the two of us. But I like how they use Kafka and other tools to show that. I mean, Arise is a cool toolset. And I think like if you think about using it for context bloat, or you think about using it for reducing latency in agents and detecting latency in agents, that's the kind of thing that is becoming best practice.
