Skip to main content
Loop has its own tools for driving playgrounds, custom views, dataset snapshots, and the workspace, plus the read and write tools that the Braintrust MCP server exposes to coding agents. It can chain several of these in a single turn: find a failure in your logs, write a scorer for it, and open that scorer in the workspace for you to review. This page catalogs what Loop can do, grouped by what it’s doing on your behalf: investigating your data, building objects from what it finds, running and automating the result, and helping you find your way around. Each area includes example prompts.

​
Investigate

Ask questions about your data, from a single trace up to every project in your organization.

​
Explore your data

Loop can query your logs, experiments, and datasets with SQL and inspect the schema of a data source. To dig through large or complex traces, it can also read, search, and list files in a sandbox, or run commands there. When it resolves a trace or generates a permalink, you can open that trace in the workspace and inspect it alongside the thread. Loop can also:
  • Hand off to the SQL sandbox. Loop can write a query into the SQL sandbox and run it there, then refine or fix it on request. The query stays in the sandbox so you can edit it by hand.
  • Start from the logs table. Loop can turn a question into a SQL filter without you writing the query, and find traces similar to ones you’ve selected.
  • Resolve identities. Loop resolves span_attributes.created_by_user_id to organization member names and emails, and span_attributes.created_by_api_key_id to API key and service token names, so you can ask about people and keys by name instead of by ID.
Fields you define in metadata, including ones named like metadata.user_id, describe your own application’s users. Loop doesn’t read them as Braintrust members.
Example prompts:

​
Analyze traces

Loop can work through a complex trace and explain what happened across many LLM and tool calls, either diagnosing a failure or summarizing the work performed. You start both from the trace view rather than from a Loop thread. See Debug a trace. You can also ask about a trace from inside a Loop thread. Paste the trace’s permalink or give Loop its ID, and Loop resolves it and pulls in the spans. Example prompts:

​
Find patterns

Loop can search your logs and raw trace data for recurring problems and trends, regardless of whether they’re already measured by a scorer, and record them as patterns. Each pattern includes the evidence Loop found when creating it. When new evidence points at a pattern Loop already recorded, it updates that pattern rather than adding a second one. Example prompts:
Loop can also find patterns on a schedule, without being asked. See Enable Patterns.

​
Analyze projects

Loop can answer questions that span a whole project, rather than a single trace or experiment, and it can compare across every project in your organization. To analyze a project in depth, open the Loop agent in the bottom right corner of a project’s Overview page, and provide a prompt. Loop has access to the project’s logs, recent experiments, datasets, prompts, and the score progress chart on the overview page. Example prompts:
To compare metrics across all your projects, select Loop agent from the organization-level projects list. Loop can compare project metrics, identify trends in usage and performance, and show how different projects are being used. Example prompts:

​
Build

Turn what you find into saved Braintrust objects. Each of these pauses for your approval before anything is created or changed.

​
Author prompts and evaluators

Loop can suggest evaluators grounded in your application’s real behavior, then create and save them rather than just describing them. An evaluator is either a scorer, which returns a number, or a classifier, which returns a label. Loop can also create and update prompts directly. New and updated evaluators open in the workspace for review. In a playground, Loop can turn your annotations into prompt revisions, so the feedback you leave on individual outputs becomes a concrete edit. Example prompts:

​
Curate datasets

Loop can generate datasets from log patterns and edit rows in an existing dataset. When Loop edits dataset rows, the dataset opens as a read-only table in the workspace, so that you can review the changes. Select a row to inspect its trace inline, and the table refreshes as Loop makes further edits. To undo a round of edits, restore an earlier snapshot, which Loop can also do for you. Loop can also manage dataset versions and the environments that point at them:
  • Snapshots. Loop can save a snapshot before it changes anything, list existing snapshots along with their environment tags and current head version, and restore the dataset to a snapshot. Restoring uses the same confirmation flow as the dataset UI, and can save the current state first.
  • Environments. Loop can point an environment such as production or staging at a specific dataset snapshot, or clear that mapping. It can list the organization’s environments to check available slugs, and create one that doesn’t exist yet.
Restoring a dataset to a snapshot replaces its current rows, and retagging an environment changes which dataset state your applications resolve. Review what Loop proposes before you approve either action.
Example prompts:

​
Build views and dashboards

Loop can create and edit custom views for both trace and dataset row data, and it can build charts for a dashboard. When Loop creates or updates a dashboard, you can open it in the workspace to review the result alongside the thread. Example prompts:

​
Configure Topics

Loop can configure the whole Topics pipeline, not just test individual facets. Given a goal, Loop works through the pipeline in stages:
  • Check the preprocessor. Loop inspects your project’s effective default preprocessor and runs it on real traces to confirm it returns usable conversation text. If the default returns nothing, Loop writes a custom preprocessor for your trace shape, tests it, and can set it as the project default. Setting the project default changes the preprocessor every function that takes one uses, including the built-in Topics facets, so Loop asks for approval first.
  • Confirm the scope. Loop looks for a grouping key such as metadata.conversation_id and recommends whether to analyze a span, a trace, or a group of related traces.
  • Validate the facets. Loop runs a facet on up to ten representative traces and shows you the results inline, so you can correct its reading before it commits to a prompt. Build up a set of traces worth checking, then have Loop re-run the facet against that same set as you refine it. For a larger set, Loop runs the facet as an eval instead.
  • Evaluate candidates. Loop runs evals over candidate facet prompts to measure quality before anything is saved.
  • Enable the automation. Loop creates the facets and turns on the Topics automation, then offers to rewind it over a historical window.
Rewinding a Topics automation processes historical traces and draws from your monthly model credits. Loop estimates the volume and asks for approval before it rewinds.
Example prompts:

​
Run and automate

Test a change with an evaluation, then turn the check that caught it into a scheduled run.

​
Run evaluations

Loop runs evaluations two ways:
  • In a playground or experiment. Loop can edit the task prompt or code, add or modify scorers, and re-run the evaluation to show the effect of a change.
  • As a new experiment. Loop can run an experiment from a dataset, from rows it supplies inline, or from a prior experiment, using a saved or inline task and any scorers you choose. When a prior experiment supplies the data, its outputs become the expected values.
When Loop starts an evaluation, the runs open in an Experiments panel in the workspace listing each run’s progress and scores, so you can compare results without leaving the chat. Example prompts:
Loop runs evaluations in Braintrust rather than by writing an Eval() script. To keep evaluations in your own repository and run them in CI, see Run evaluations in code.

​
Manage automations and alerts

Loop can set up recurring work and notifications from what it finds, so a problem you just investigated becomes something you get told about, or something Loop rechecks on a schedule.
  • Alerts. Loop can create an alert on individual matching logs, on an aggregate that crosses a threshold over a recent window, or on environment updates, generating the filter from your description.
  • Automations. Loop can create a scheduled Loop job with its own instruction and schedule. It can also list the project’s existing automations, including online scoring rules, alerts, exports, retention policies, and Topics automations, and return each one’s full configuration.
  • Status. Loop can pause an automation or set it back to active.
  • Updates. Loop can change an alert or a Loop automation by rewriting it under its existing name. A rewrite replaces the whole configuration rather than patching a single field, so ask Loop to read the current configuration first, then check what it proposes before you approve. Scheduled exports and retention policies are inspect-only.
You can also edit any automation directly in Settings > Automations. Example prompts:

​
Get help

Loop can help you find your way around Braintrust, and reach out to support when you can’t resolve something.

​
Search docs and find objects

Loop can search the Braintrust documentation, resolve object names and URLs, list recent objects, summarize an experiment’s metrics against a baseline, and generate shareable links to objects it creates. Example prompts:

​
Request support

Loop can file a support ticket with the Braintrust team when you encounter something it can’t resolve. Ask for help, or type / and choose the support command, and Loop guides you through creating the request. It includes relevant context from your current page and conversation so the support team can understand the issue.

​
Next steps