Skip to main content
Contributed by Max Kern on 2026-08-19
Coding agent insights is in public preview and can change before reaching general availability.
Coding agent insights shows what your team’s coding agents cost, who uses them, and where they could work more effectively. A dashboard covers usage and cost, and a daily Loop automation finds recurring problems and inefficiencies across your team’s sessions. This cookbook sets up both for your team. The dashboard at Dashboards > Coding agent insights shows usage by user, API key, model, tool, and branch, with cost breakdowns. It also shows which teammates are sending traces, to help you track adoption across your team. Coding agent insights dashboard showing a Usage over time chart of cost, and a table grouped by user. The User column header reads 4 / 12 (33%), and each row lists a teammate's cost, cost change, tokens, sessions, LLM calls, and tool calls. A daily Loop automation reviews your team’s sessions and records recurring problems as patterns with supporting evidence. For example, the automation might find:
  • Agents repeatedly resending large amounts of context, increasing model usage.
  • File operations failing because of stale paths or imprecise replacement text.
  • Agents repeatedly checking whether long-running commands have finished.
  • Tracing issues that combine unrelated work or reprocess the same sessions.

​
Prerequisites

Before you begin, you’ll need:

​
1. Create a project for your traces

First, create a dedicated project for your coding-agent traces.
  1. In Braintrust, open the project dropdown and click Create project. Creating a project requires organization-level Create permission.
  2. Enter a name for your project, such as coding-agent-insights. This cookbook uses that name in its examples, so if you choose a different one, use it wherever coding-agent-insights appears.
  3. Optionally add a description, then click Create.
  4. Make sure every teammate whose sessions you want to collect has Read permission on the project to view it and Update permission on its logs to send traces.

​
2. Protect sensitive data (optional)

Ingestion redaction is in private preview, available to a limited set of customers. Behavior can change during the preview. To request access, contact Braintrust.
Coding-agent traces can contain source code, file paths, prompts, command output, and any credentials that appear during a session. Ingestion redaction detects sensitive text, such as names, email addresses, and credentials, and replaces it before Braintrust stores your traces. Detection is model-based and can miss sensitive values. Ingestion redaction applies only to traces that arrive after you create a protected field, so create one for your project before teammates start sending traces. For instructions, see Create a protected field.
To remove values before they’re sent to Braintrust, add a span plugin when configuring your coding agents in the next step.

​
3. Configure your coding agents

Next, configure your coding agents to send traces to the project. You and your teammates must complete this setup on each computer where you use a coding agent.
1

Install or update bt

If you already have bt installed, update to the latest version. Otherwise, install it:
After installing, open a new terminal so that bt is on your PATH. Then run bt --version. You should see bt v0.19.3 or later.
2

Authenticate with Braintrust

If bt is already signed in with OAuth and can access coding-agent-insights, skip this step. Otherwise, run the following command and complete the browser flow:
bt saves an OAuth profile that the tracing integrations use to authenticate.
Each teammate should sign in with their own Braintrust login. The dashboard at Dashboards > Coding agent insights attributes each session to the Braintrust user whose credentials sent it. If teammates share one API key, the dashboard can’t tell their sessions apart.
If your deployment uses custom authentication URLs, sign in using this command:
  • --api-url specifies the API URL for OAuth authentication, which defaults to https://api.braintrust.dev.
  • --app-url specifies the Braintrust app URL used to find your organizations and their data planes, which defaults to https://www.braintrust.dev.
Omit either flag if your deployment uses its default URL. bt saves these settings and automatically resolves where to send traces for the selected organization.
3

Enable tracing for each agent

Run the commands for the coding agents you use on this computer. Each command enables tracing for one agent and sends its sessions to coding-agent-insights.If you have several saved profiles, bt uses the one that can access the organization you pass with --org, and asks you to choose if more than one can.
To remove sensitive values on each computer before bt sends them, write a span plugin and add --plugin <PATH> to the command shown below. Span plugins require bt v0.21.0 or later.
After setup:
  • Restart Claude Code to load the tracing configuration.
  • Run bt trace doctor claude.
  • Confirm that Enabled is true, Auth is ready, and the displayed organization and destination project are correct.
  • If the organization or project is incorrect, rerun bt trace enable claude with the correct --org and --project values.
4

Verify tracing

Confirm that each configured agent sends sessions to the shared project:
  • Start a new session in each configured coding agent and ask it to complete a small task. Ask participating teammates to do the same.
  • Go to Logs, select coding-agent-insights, and confirm that each new session appears as a root trace.
If a trace does not appear:
  • For any agent, run the agent’s bt trace doctor command and resolve any authentication or routing problems it reports.
  • For Codex, run /hooks and confirm that the Braintrust hooks are trusted.
  • For Grok, run /reload-plugins before sending the test prompt.
For more troubleshooting, see the guide for Claude Code, Codex, OpenCode, Pi, Antigravity, or Grok.

​
4. Configure Topics and Loop

With tracing configured, enable Topics, then use a prebuilt configuration file to add the coding agent insights facets and daily Loop automation to your project.
1

Enable Topics

If Topics is already enabled for coding-agent-insights, skip this step. Otherwise, ask an organization owner to:
  1. In coding-agent-insights, go to Topics.
  2. Under Get started with topics, make sure Task is selected. You can leave the other built-in facets off.
  3. Choose whether to Apply to existing traces.
  4. Click Enable topics.
2

Download the configuration file

The configuration adds three Topics facets that classify each coding-agent session:
  • Coding agent context debt: Identifies reusable knowledge or access the agent lacked that you had to supply during the task, such as a repository convention it missed.
  • Primary failure mode: Labels the main problem in a session, such as an unresolved test failure, editing the wrong files, or getting stuck in repeated attempts.
  • Skills analysis: Labels problems caused by a skill the agent loaded, such as choosing an irrelevant skill or ignoring its instructions.
It also adds Coding Agent Improvements Discovery, a daily Loop automation that reviews the past 30 days of traces, checks facet classifications against raw evidence, and creates or updates patterns.
3

Apply the configuration file to your project

Applying the configuration file requires project-level Read, Create, and Update permissions. To add its Topics facets and Loop automation to your project, run this command:
--prefer-profile uses your saved login even when BRAINTRUST_API_KEY is set. An explicit --profile also takes precedence.
At the confirmation prompt, verify the organization, project, three facets, one Loop automation, and Topics wiring. Then confirm the push. The command should report that it pushed 3 facets and 1 Loop automation.
4

Review the installed resources

Confirm that the template added the facets to the project and attached them to the Topics automation:
  1. In coding-agent-insights, go to Topics. Confirm that cards appear for Coding agent context debt, Primary failure mode, and Skills analysis.
  2. Click Automation and select Topics. Expand Active facets and confirm that it includes the same three custom facets.
Confirm that the template created an active daily automation with the expected analysis window and pattern permissions:
  1. Go to Settings > Automations. Confirm that Coding Agent Improvements Discovery shows Every 24 hours and Active.
  2. Click Coding Agent Improvements Discovery to open it. On the Edit tab, confirm that:
    • Agent configuration shows GPT-6.1 Sol with Extra high reasoning effort.
    • Default query range shows Custom, 30 days.
    • Write tool permissions shows 2 allowed. Open it and confirm that new_pattern and update_pattern are selected.
These checks confirm that Topics will label coding-agent sessions and Loop will review the most recent 30 days for recurring problems.

​
5. Collect coding-agent traces

Now, collect traces as you and your teammates use your coding agents. Each session gives the automation more evidence to distinguish recurring problems from one-off difficulties.
You can also import saved sessions from Claude Code, Codex, or Antigravity.

​
6. Verify the automation

Once you’ve collected some traces, run the Loop automation manually to check that it’s working:
  1. Go to Settings > Automations and open Coding Agent Improvements Discovery.
  2. Click Run now, then Open Loop in the confirmation notification to view the run’s thread. You can also open the latest run from the automation’s Past runs tab.
  3. Confirm that the run completes without reported errors and describes the traces it reviewed.
A successful run does not need to create a pattern. If no traces were found, check that your project contains sessions from the past 30 days. For model errors, see Troubleshooting. The automation continues running daily, so further manual runs are optional.

​
7. Review your team’s usage

The Coding agent insights dashboard shows usage and cost as soon as traces arrive, without waiting for the daily automation.
  1. In coding-agent-insights, go to Dashboards > Coding agent insights.
  2. Check which teammates are sending traces. With the table grouped by Users, the User column header counts members with recorded model calls matching the selected period and filters. Click the count to see who is and isn’t sending traces.
  3. Use the grouping selector to break down usage and cost by user, API key, model, tool, or branch: Coding agent insights dashboard toolbar below the Usage over time chart, with the pointer on the Users grouping selector. Next to it are the Past 7 days time range selector and an Add filter field.
To learn more, read Coding agent insights dashboard.

​
8. Review patterns

As findings appear, review them in Patterns to decide what to change in your agents’ instructions, skills, or tools. To review a pattern:
  1. Go to Patterns in coding-agent-insights.
  2. Select a pattern and read its summary and suggested fix, if one is provided.
  3. Open the evidence traces to check that they support the finding. Review any attached monitor charts to see how the behavior changes over time.
  4. Choose an action: click Continue in Loop to investigate further, create a scorer or classifier to track the behavior, or click Copy for agent to copy the pattern as a prompt for a coding agent to work on a fix. See Review and act on patterns for instructions.
If the list is empty, continue collecting sessions and check later runs.

​
9. Refine the analysis and share reports (optional)

Finally, you can change what the automation looks for by editing its Instruction prompt, and send its reports to Slack or a webhook.
To change what the automation analyzes, start in an interactive Loop thread. You can try different analyses against recent sessions before updating the automation.
Loop asks before it changes anything in your project, such as the automation. While you experiment, click Skip on those requests and leave Always allow off.
Ask Loop to suggest changes. Go to Loop and enter this prompt:
Continue the thread to refine and test its suggestions.Try your own analysis. If you already know what you want to try, describe the revised analysis in a new Loop thread and ask Loop to run it against recent sessions. Adjust your instructions and rerun the analysis until you’re satisfied with the results.Apply a change. When you’re ready to apply a change, ask Loop to make it and click Accept, or edit the automation yourself. To edit it, go to Settings > Automations, open Coding Agent Improvements Discovery, make the change, and click Save.
To send the result of each successful automation run to Slack or a webhook, add a destination to Coding Agent Improvements Discovery.
1

Connect Slack

Skip this step if Slack is already connected to your organization. Only members of the Owners permission group, or a custom permission group with the Manage settings organization permission, can connect Slack.Go to Settings > Integrations, click Enable Slack, and authorize Braintrust to access your Slack workspace. For details, see Enable Slack integration.
2

Add the destination

Go to Settings > Automations and click Coding Agent Improvements Discovery to open it. Under Destinations, click Destination and select Send to Slack. Then choose the workspace and channel.Braintrust prepopulates a suggested Formatting prompt. Review it and adjust it as needed for your team.
Test the destination. Save the automation and click Run now. When the run finishes, confirm that the Slack channel or webhook endpoint received the formatted report.Adding a destination does not change the automation’s daily schedule or 30-day lookback window.

​
Troubleshooting

If the curl command for downloading the configuration file returns an HTTP error:
  • Confirm that the URL is https://www.braintrust.dev/docs/assets/active-observability-config.json.
  • Retry from a network that can reach www.braintrust.dev.
If bt observability template is not recognized when you push the template:
  • Run bt --version to check the installed version.
  • Install or update to bt v0.19.3 or later. If the installed version already meets this requirement, reinstall it.
  • After the installation succeeds, return to Apply the configuration file to your project and run the template command shown there.
If bt observability template push reports that a matching facet or Loop automation already exists, it stops before the confirmation prompt:
  • If you intend to restore the template configuration, rerun the same command with --force and confirm the replacements.
  • A new project does not need --force.
Rerunning the command with --force resets the three facets and Coding Agent Improvements Discovery to the versions in the JSON file. Any edits you made to them are lost. Slack and webhook destinations already added to the automation are preserved.
If bt trace enable codex reports a marketplace clone timeout:
  • Run the command again.
  • If it repeatedly reports fatal: early EOF, troubleshoot the network connection rather than the Braintrust project configuration.
If a test session does not appear in Logs after you configure tracing:
  • Restart the coding agent and run another test task.
  • For a self-hosted deployment, confirm that the saved OAuth profile uses the correct authentication URLs. If BRAINTRUST_API_URL is set in the shell where you start the agent, check that it points to the intended data plane or unset it to let bt resolve the selected organization’s endpoint. Restart the agent after changing its environment.
  • If the trace still does not appear, run bt trace doctor <AGENT>, replacing <AGENT> with claude, codex, opencode, pi, antigravity, or grok.
  • Confirm that Enabled is true, Auth is ready, and the displayed organization and destination project are correct.
  • If Auth is not ready, run bt login --oauth, then run the doctor command again.
  • If the organization or destination project is wrong, rerun bt trace enable for that agent with the correct --org and --project values.
If bt trace import cannot find a Codex, Claude Code, or Antigravity session:
  • Confirm that the command names the agent that created the session. Use bt trace import codex for Codex, bt trace import claude for Claude Code, or bt trace import antigravity for Antigravity.
  • Run the command from the same user account and computer where that agent saved the session.
If Send to Slack does not appear in the Destination menu:
If Coding Agent Improvements Discovery fails because it cannot call gpt-6.1-sol:
  • Configure gpt-6.1-sol for the OpenAI provider under AI providers, or select an available model in the automation editor.
  • Run the automation again.

​
Cleanup

No action is required to keep collecting traces. Choose any cleanup actions you need:
After pushing the template, run rm /tmp/active-observability-config.json.
Rerun the appropriate bt trace enable command for each configured coding agent with the new --org and --project values.
Run bt trace disable <AGENT> for each configured agent, replacing <AGENT> with claude, codex, opencode, pi, antigravity, or grok. This removes the Braintrust tracing integration and its saved configuration.
First stop tracing or send future traces to another project. Then follow the instructions below if you no longer need the project’s data.
Deleting the project permanently removes its traces, facets, automations, and other project data. This action cannot be undone.
  1. Go to Settings > General.
  2. Click Delete project.
  3. Enter coding-agent-insights and click Delete.

​
Recap

You’ve set up coding agent insights for your team. The dashboard at Dashboards > Coding agent insights shows who is sending traces and where usage and cost go. Topics classifies each session, and Loop reviews the evidence each day and creates or updates patterns automatically. Use those findings as a starting point for improving your agent instructions, skills, tools, and configuration. As your team’s needs change, you can refine the analysis and share reports in Slack or through a webhook.

​
Next steps