Skip to main content
Once you’re capturing traces in Braintrust, enable Patterns to schedule a Loop automation that finds recurring problems and trends in your project.

​
Before you start

To enable Patterns, you need:
  • When built-in models are available, Patterns selects them automatically. On Enterprise, you can choose a compatible configured provider instead.
  • When built-in models are unavailable, Patterns selects your only compatible configured provider automatically. If you have several, you choose one.
You can change the provider after setup in the automation’s configuration.
To receive notifications, connect a Slack workspace, then choose a workspace and channel during Patterns setup. You can also add a notification destination after setup.Loop sends a separate Slack alert for each pattern it creates or reactivates during an automation run, up to 20 alerts per Slack destination per run. Alerts include the pattern’s description and available evidence links and frequency estimates. Updates to an active pattern do not trigger another alert.By default, each Slack digest summarizes up to two new or reactivated patterns and one existing active pattern, with available frequency estimates and links to additional findings. It excludes closed patterns. If no active patterns remain, the digest says “No current pattern outcomes.”Existing automations using the default Formatting prompt also use this format. Custom prompts retain their formatting and include the patterns selected for the run that remain active, plus patterns closed during that run. They exclude patterns that were already closed and stayed closed.

​
Enable pattern discovery

  1. Go to Patterns. The Get started with Patterns panel guides you through setup.
  2. If the panel shows Add provider, configure an organization-level AI provider before continuing. See Common issues for provider and model credit requirements.
  3. If the panel shows Continue, click it to open Configure Patterns. Choose a provider under Select a model provider if that field appears. If a Slack workspace is connected, you can also choose a workspace and channel for notifications.
  4. Click Enable Patterns. If no provider choice or Slack configuration is needed, this button appears directly on the initial panel.
Braintrust creates a Pattern discovery automation with these defaults:
  • Schedule: Daily at 9:00 AM in your default timezone.
  • Query range: The last 30 days.
  • Permissions: Create and update patterns.
  • Instruction: A default prompt telling Loop what to look for. You can edit it.
The first run is queued immediately. The page shows Analyzing traces… while a run is in progress, and patterns appear as it finishes.
If a project has no patterns yet and no run is in progress, click Analyze patterns to queue another run without waiting for the schedule.

​
Tune the automation

You can change the automation’s instruction, schedule, model, query range, and destinations. To edit it, either click Settings on the Patterns page and select the automation, or go to Settings > Automations. Start with these fields:
  • Frequency. How often the run happens. A project with steady traffic rarely needs more than daily.
  • Default query range. Each run inspects the full range, including traces inspected by previous runs. A wider range costs more.
  • Instruction. What Loop should look at, how deeply, and when to stop.
  • Destinations. Where run digests are delivered.
Click Save to apply your changes. For all fields, see Create a Loop automation. When you resume a paused automation, Patterns keeps its saved model and provider unless you select a different one.
Keep the instruction framed around finding or updating patterns, even when you narrow the automation to one concern. An instruction that asks for a summary produces a summary, not a pattern record.

​
Add a focused automation

Add an automation when a concern needs its own scope or schedule, such as tool-call errors, cost growth, latency for specific customers, or quality regressions. All automations write to the same patterns list. Select Settings > New automation on the Patterns page. Review the prefilled automation, adjust its settings, and click Create.

​
Common issues

When neither a compatible organization-level AI provider nor usable built-in models are available, the setup panel shows Add provider instead of Enable Patterns. Click Add provider to configure a provider without leaving setup.
  • Built-in models are disabled. To use them instead of your own provider, enable them in Settings > AI providers. Your organization must meet the built-in model requirements.
  • Your model credits are exhausted. Add your own provider key with Add provider, or select Upgrade to raise your plan’s limits. See model credits for how usage is counted.
If the latest run of an enabled Patterns automation fails, the page header shows Last run failed:
  1. Click it to see automation names and error messages.
  2. Select an entry to open the failed run in Loop in a new tab.
Automation errors can also appear in a banner below the page header. Enabling Patterns does not test your provider’s connection before creating the automation.For provider errors, check that the configured credentials are valid and the provider can serve the automation’s selected model. After resolving the error, run the automation again with Run now.
Loop records a pattern only when it can attach supporting evidence, so a quiet run is a normal outcome rather than a failure.To work out what happened, open the automation, select the Past runs tab, and select the run to read the transcript. Select Fork to chat to fork it into an editable thread and ask Loop directly why it recorded nothing.If runs are consistently empty, the query range or the instruction is usually too narrow.
Destinations belong to automation runs. Patterns that Loop records during an interactive chat appear in the list but do not trigger Slack or webhook destinations.Confirm the destination is configured on the automation itself, under Destinations in the configuration panel, and that the Slack workspace is still connected.
Patterns requires data plane v2.13 or later. To upgrade, see the routine upgrade guide.

​
Next steps