Hamel Husain defines a trace as the complete record of a user session in LLM systems, from initial request to final response, including every query, tool call, and agent interaction. Errors should be annotated at the trace level because mistakes cascade: an early wrong tool choice can break everything downstream, and many failures only make sense when you see the whole session.
was what is a trace? Can we have a formal definition of a trace? I think it's a good question. We've been saying that this is the full record of everything that happens from the user's initial query to the most final response. That is our opinion of like what is a good trace definition. It should include everything possible, tool calls, you know, data retrievals across every agent and system component. But you might be wondering, okay, why should we annotate errors at this trace level? Like why isn't a trace specifically a single turn? Why isn't it a single request-response for an agent? When we've done error analysis, many errors only emerge or make sense in the context of the entire trace, the end-to-end flow. You kind of want to see all of that in front of you. We find that mistakes often cascade. This is a big one. The agent chose the wrong tool, then it has a very low likelihood of actually giving you the correct answer at the end. You might as well just only label that first failure in your open coding and then move on. There's no need to spend all of your time reviewing every single Hamel, do you have anything you want to add to why define a trace this way as maybe like the union of all of the little traces that people might consider?
Yeah, I mean, you just need a way to have all the information in context. When you're talking about a user session where you might have multiple tool calls, multiple turns back and forth between the AI retrieval, any other kinds of things going on, you want to log all of those things so you can debug them. That's the general idea behind this.
