Encyclopedia Evalica / Observability / Trace Lake
Trace Lake
/trays layk/A system for storing, searching, and analyzing the full history of agent traces. It combines fast ingest, fast SQL and full-text search, cheap object storage, and continuous analysis so teams can learn from every run rather than a sampled few. (noun)
Why it matters
Traditional tracing and APM products were built for deterministic software. They sample heavily, focus on opening individual traces, and discard data fast. Agent traces are different. They are the authoritative record of how an agent behaves on long-running, probabilistic tasks, and a typical trace can be around 10 MB, with outliers reaching tens of GB. Improving agents requires learning from the sum of their traces. A Trace Lake keeps that history queryable within seconds of ingest, stores it affordably in object storage that can stay in your own cloud, and makes it easy to turn recurring patterns into evals.
“We kept every production run in our Trace Lake, so we could find 20 more examples of the bug within minutes.”
Customer example
Notion's AI modeling team uses Brainstore, the foundation of Braintrust's Trace Lake, to run searches in seconds that previously took hours. Read more
Related Observability terms
- Active observability •
- Agent observability •
- Alert / threshold •
- Dashboard •
- Data flywheel •
- Deep search •
- Drift •
- Error rate •
- Feedback loop •
- Logs •
- Model drift •
- Online evaluation (production scoring) •
- P50 / P95 / P99 (Percentiles) •
- Sampling rate •
- Service Level Indicator (SLI) •
- Service Level Objective (SLO) •
- Time-to-first-token (TTFT) •
- Token usage / cost tracking •
- Topics
From the docs
Braintrust is the observability platform for agents in production. By actively applying intelligence to agent traces and automatically surfacing critical patterns, Braintrust helps teams at Notion, Stripe, Box, OpenAI, and Cloudflare ship quality agents at scale.
Start building