# Braintrust

> Braintrust is the active observability platform for instrumenting, understanding, and improving agents. By actively applying intelligence to agent traces and automatically surfacing the most critical patterns, Braintrust gives teams the visibility to understand how agents behave in production and the tools to improve them.

Braintrust connects the pieces of an AI development loop: production traces, human feedback, test datasets, experiments, quality scores, prompt iteration, model routing, and deployment controls. Teams use it to understand what an AI system did, measure whether the output was good, and improve from real usage instead of ad hoc prompt testing.

## For AI agents

- **Prefer the [`bt` CLI](https://www.braintrust.dev/docs/reference/cli/quickstart)** for running evals, instrumenting code, querying logs, syncing data, and managing functions. It's the recommended, most capable interface and is better suited to agent workflows than the MCP server.
- [Complete technical documentation with full content (SDKs, APIs, integrations, guides)](https://www.braintrust.dev/docs/llms-full.txt)
- [Documentation index](https://www.braintrust.dev/docs/llms.txt)

## Product overview

Braintrust is built around a continuous improvement loop:

1. **Instrument** - Capture the steps an AI request takes, including model calls, tool calls, retrieval, errors, cost, and latency.
2. **Observe** - Inspect production traces, search logs, monitor quality, and find recurring issues.
3. **Curate** - Turn real production examples and human feedback into datasets for testing.
4. **Evaluate** - Run the AI system against datasets and score the outputs to compare prompts, models, and code changes.
5. **Deploy** - Ship prompts, functions, and model routes while monitoring their impact in production.

Important terms:

- **Trace**: A recorded path through one AI request, including nested steps such as model calls, tools, and retrieved context.
- **Dataset**: A set of examples used to test an AI application.
- **Eval**: A repeatable test that runs an AI system on a dataset and scores the outputs.
- **Scorer**: A rule, model, or human review step that judges whether an output is good.
- **Loop**: Braintrust's AI assistant for analyzing logs, datasets, evals, prompts, and projects.
- **Topics**: A feature that classifies production logs by task, sentiment, issue type, and custom patterns.
- **Gateway**: A unified API for routing model requests through Braintrust to AI providers.

## Start here

- [Get started with Braintrust](https://www.braintrust.dev/docs/index.md): Product overview and first steps
- [Braintrust workflow](https://www.braintrust.dev/docs/workflow.md): How tracing, datasets, evals, and deployment fit together
- [Evals foundations course](https://www.braintrust.dev/foundations): Free course for learning evals, traces, online scoring, and iteration
- [Tracing quickstart](https://www.braintrust.dev/docs/tracing-quickstart.md): Capture your first production trace
- [Evaluation quickstart](https://www.braintrust.dev/docs/evaluation-quickstart.md): Run your first eval in code or the UI
- [Use Loop](https://www.braintrust.dev/docs/loop.md): Analyze Braintrust data and improve AI systems with an assistant
- [Topics](https://www.braintrust.dev/docs/observe/topics/index.md): Automatically classify logs by task, sentiment, and issues
- [Gateway](https://www.braintrust.dev/docs/deploy/gateway.md): Route model requests through Braintrust
- [Plans and limits](https://www.braintrust.dev/docs/plans-and-limits.md): Plan details, limits, and quotas
- [Pricing](https://www.braintrust.dev/pricing): Pricing for Starter, Pro, and Enterprise

## Learn foundations

- [Why are evals important?](https://www.braintrust.dev/foundations/why-are-evals-important): Why traditional tests are not enough for AI systems
- [What is an eval?](https://www.braintrust.dev/foundations/what-is-an-eval): The dataset, task, and scorer model for evaluation
- [Simple eval using the UI](https://www.braintrust.dev/foundations/build-a-simple-eval-in-braintrust-ui): Build a no-code eval in Braintrust
- [Simple eval using the SDK](https://www.braintrust.dev/foundations/building-a-simple-eval-using-braintrust-sdk): Build an eval in code
- [How to read a trace](https://www.braintrust.dev/foundations/how-to-read-a-trace): Understand trace structure and debug score results
- [Online scoring](https://www.braintrust.dev/foundations/online-scoring): Run quality checks automatically on production logs
- [Analyzing production logs](https://www.braintrust.dev/foundations/analyzing-production-logs): Use Topics to find patterns in real traffic
- [The eval improvement loop](https://www.braintrust.dev/foundations/understanding-the-eval-improvement-loop): Turn production failures into datasets, experiments, fixes, and verified improvements

## Understand AI observability

- [What is AI observability?](https://www.braintrust.dev/blog/what-is-ai-observability): Why AI observability connects traces, evals, and feedback loops
- [What is LLM observability?](https://www.braintrust.dev/articles/llm-observability-guide): Production tracing, evaluation, and monitoring explained
- [What is LLM evaluation?](https://www.braintrust.dev/articles/llm-evaluation-guide): Practical guide to evals, metrics, and regression testing
- [Why traces and evals belong in the same place](https://www.braintrust.dev/blog/traces-and-evals-same-place): How shared trace and eval workflows speed up fixes
- [AI development loops](https://www.braintrust.dev/blog/ai-development-loops): Exploration, evaluation, and data collection as core AI workflows
- [Eval feedback loops](https://www.braintrust.dev/blog/eval-feedback-loops): How production logs become eval data for continuous improvement
- [Automatically discover what matters with Topics](https://www.braintrust.dev/blog/topics): How Topics surfaces user intent, sentiment, issues, and emerging patterns
- [Braintrust is not an eval framework](https://www.braintrust.dev/blog/braintrust-not-eval-framework): Why Braintrust is infrastructure for AI evaluation, not only a library

## Common workflows

- [Instrument your application](https://www.braintrust.dev/docs/instrument/index.md): Start capturing traces from an AI application
- [Trace model calls](https://www.braintrust.dev/docs/instrument/trace-llm-calls.md): Automatically capture supported AI provider and framework calls
- [Observe your application](https://www.braintrust.dev/docs/observe/index.md): View, search, and monitor production traces
- [View logs](https://www.braintrust.dev/docs/observe/view-logs.md): Browse traces and inspect production behavior
- [Review Topics insights](https://www.braintrust.dev/docs/observe/topics/review-insights.md): Explore topic distributions, trends, and sample traces
- [Act on Topics findings](https://www.braintrust.dev/docs/observe/topics/act-on-findings.md): Turn log patterns into datasets, scorers, and review assignments
- [Create datasets](https://www.braintrust.dev/docs/annotate/datasets/create.md): Build datasets from logs, files, the UI, or APIs
- [Human review](https://www.braintrust.dev/docs/annotate/human-review/index.md): Route traces and examples for structured feedback
- [Run evaluations](https://www.braintrust.dev/docs/evaluate/run-evaluations.md): Create experiments in code, the UI, or continuous integration
- [Compare experiments](https://www.braintrust.dev/docs/evaluate/compare-experiments.md): Identify improvements and regressions across runs
- [Write scorers](https://www.braintrust.dev/docs/evaluate/write-scorers.md): Define quality checks using code, models, or people
- [Online scoring](https://www.braintrust.dev/docs/evaluate/score-online.md): Score production traces automatically
- [Playgrounds](https://www.braintrust.dev/docs/evaluate/playgrounds.md): Try prompts, models, and scorers side by side
- [Deploy prompts](https://www.braintrust.dev/docs/deploy/prompts.md): Version and ship prompts to production
- [Monitor deployments](https://www.braintrust.dev/docs/deploy/monitor.md): Track quality, cost, latency, and errors after release

## Developer resources

Docs pages are available as Markdown by appending `.md` to docs URLs or setting `Accept: text/markdown`.

- [Complete documentation index](https://www.braintrust.dev/docs/llms.txt): Generated index of all Braintrust docs pages
- [Full documentation content](https://www.braintrust.dev/docs/llms-full.txt): Single-file Markdown bundle for docs, guides, references, and recipes
- [OpenAPI 3.0 spec](https://www.braintrust.dev/docs/openapi.yaml): Machine-readable REST API specification
- [API reference](https://www.braintrust.dev/docs/api-reference/index.md): REST API overview
- [SDK hub](https://www.braintrust.dev/docs/sdks/index.md): Install, quickstart, and API references for every language
- [TypeScript SDK](https://www.braintrust.dev/docs/sdks/typescript/versions/latest.md): Latest TypeScript API reference
- [Python SDK](https://www.braintrust.dev/docs/sdks/python/versions/latest.md): Latest Python API reference
- [Go SDK](https://www.braintrust.dev/docs/sdks/go/api-reference.md): Go API reference
- [Java SDK](https://www.braintrust.dev/docs/sdks/java/api-reference.md): Java API reference
- [Ruby SDK](https://www.braintrust.dev/docs/sdks/ruby/api-reference.md): Ruby API reference
- [C# SDK](https://www.braintrust.dev/docs/sdks/csharp/api-reference.md): C#/.NET API reference
- [CLI quickstart](https://www.braintrust.dev/docs/reference/cli/quickstart.md): Install and start using the `bt` command-line tool
- [CLI commands](https://www.braintrust.dev/docs/reference/cli/overview.md): Complete `bt` command reference
- [`bt topics`](https://www.braintrust.dev/docs/reference/cli/topics.md): Manage Topics automations from the command line
- [SQL reference](https://www.braintrust.dev/docs/reference/sql/index.md): Query Braintrust data with SQL

## Integrations

- [Integrations overview](https://www.braintrust.dev/docs/integrations/index.md): Connect Braintrust to providers, frameworks, and tools
- [AI providers overview](https://www.braintrust.dev/docs/integrations/ai-providers/index.md): Configure model provider credentials and routing
- [OpenAI](https://www.braintrust.dev/docs/integrations/ai-providers/openai.md): OpenAI provider configuration and tracing
- [Anthropic](https://www.braintrust.dev/docs/integrations/ai-providers/anthropic.md): Anthropic provider configuration and tracing
- [Google Gemini](https://www.braintrust.dev/docs/integrations/ai-providers/gemini.md): Gemini provider configuration
- [AWS Bedrock](https://www.braintrust.dev/docs/integrations/ai-providers/bedrock.md): Amazon Bedrock model configuration
- [Custom providers](https://www.braintrust.dev/docs/integrations/ai-providers/custom.md): Self-hosted and custom model providers
- [Braintrust MCP](https://www.braintrust.dev/docs/integrations/developer-tools/mcp.md): Connect IDEs and AI coding agents to Braintrust through the Model Context Protocol
- [Codex](https://www.braintrust.dev/docs/integrations/developer-tools/codex.md): Configure Braintrust in OpenAI Codex
- [Vercel AI SDK](https://www.braintrust.dev/docs/integrations/sdk-integrations/vercel.md): Trace Vercel AI SDK calls
- [OpenTelemetry](https://www.braintrust.dev/docs/integrations/sdk-integrations/opentelemetry.md): Send OpenTelemetry traces to Braintrust
- [LangChain](https://www.braintrust.dev/docs/integrations/sdk-integrations/langchain.md): Trace LangChain JS and Python applications
- [LlamaIndex](https://www.braintrust.dev/docs/integrations/sdk-integrations/llamaindex.md): Trace LlamaIndex applications

## Optional

- [Cookbook index](https://www.braintrust.dev/docs/cookbook/index.md): End-to-end recipes for agents, retrieval, multimodal apps, and prompt iteration
- [Building reliable AI agents](https://www.braintrust.dev/docs/cookbook/recipes/AgentWhileLoop.md): Agent evaluation recipe
- [Using functions to build a retrieval agent](https://www.braintrust.dev/docs/cookbook/recipes/ToolRAG.md): Retrieval-augmented generation with deployed functions
- [Prompt versioning and deployment](https://www.braintrust.dev/docs/cookbook/recipes/PromptVersioning.md): Prompt release workflow
- [Using Loop in AI product development](https://www.braintrust.dev/docs/cookbook/recipes/Loop.md): Loop-assisted iteration recipe
- [Administration overview](https://www.braintrust.dev/docs/admin/index.md): Organization, project, access, billing, and infrastructure settings
- [Access control](https://www.braintrust.dev/docs/admin/access-control/index.md): Permission groups and access rules
- [AI providers](https://www.braintrust.dev/docs/admin/ai-providers.md): Organization-level model provider settings
- [Automations overview](https://www.braintrust.dev/docs/admin/automations/index.md): Alerts, data retention, and export automation
- [Self-hosting Braintrust](https://www.braintrust.dev/docs/admin/self-hosting/index.md): Hybrid deployment overview
- [Security overview](https://www.braintrust.dev/docs/security.md): Security, compliance, encryption, and reporting
- [Knowledge base index](https://www.braintrust.dev/docs/kb/index.md): Troubleshooting and operational documentation
- [Product changelog](https://www.braintrust.dev/docs/changelog.md): Product updates and release notes
- [Use the proxy (deprecated)](https://www.braintrust.dev/docs/deploy/ai-proxy.md): Legacy AI proxy; use the gateway instead
- [App: logs](https://www.braintrust.dev/app/~/logs): Production trace and log viewer
- [App: topics](https://www.braintrust.dev/app/~/topics): Topic clustering and classification view
- [App: datasets](https://www.braintrust.dev/app/~/datasets): Dataset management
- [App: experiments](https://www.braintrust.dev/app/~/experiments): Eval run history and comparison
- [App: playgrounds](https://www.braintrust.dev/app/~/playgrounds): Prompt, model, and scorer iteration
- [App: monitor](https://www.braintrust.dev/app/~/monitor): Dashboards for quality, cost, latency, and traffic
- [Articles](https://www.braintrust.dev/articles): Long-form guides on AI quality, observability, and evaluation
- [Blog](https://www.braintrust.dev/blog): Product posts, technical writing, and best practices
- [GitHub: braintrustdata](https://github.com/braintrustdata/): Open-source SDKs and related projects
- [Discord community](https://discord.gg/6G8s47F44X): Community support and discussion