Skip to main content
Your costs come from two sources:
  • Braintrust bill: Charges based on your usage, such as the data you send and the scores you record.
  • LLM spend: The cost of the model calls that your application, evaluations, and scorers make.
Each approach below reduces one or both. Evaluations show whether a lower-cost option still meets your quality bar.

​
Optimize your Braintrust usage

Processed data is measured at ingestion, so deleting logs or shortening retention doesn’t lower it. Your bill goes down when you send less duplicate data and record fewer scores.

​
Reduce duplicate logging

Duplicate logging sends the same data more than once, and every copy counts toward processed data. It can come from overlapping instrumentation or from repeating a call’s details on more than one span. Spans unrelated to your LLM calls add to processed data the same way.
When more than one tracing layer instruments the same LLM call and each sends spans to Braintrust, the call is logged once per layer. Keeping one layer per call avoids the duplicates.For example, LiveKit Agents can send traces to Braintrust through LiveKit’s built-in OpenTelemetry instrumentation or through Braintrust’s auto-instrumentation. If you use LiveKit’s OpenTelemetry pipeline and also run auto-instrumentation for other libraries, turn off Braintrust’s LiveKit integration so each session is traced once.How you turn off an integration depends on the SDK:
  • TypeScript: List the integrations to skip in the BRAINTRUST_DISABLE_INSTRUMENTATION environment variable. For LiveKit, this also disables wrapLiveKit(). See Disabling specific integrations.
  • Python: Pass False for the integration’s keyword to auto_instrument(). See Disabling specific integrations.
Copying an LLM call’s prompt, response, or token metrics onto its parent span sends the same data twice, and both copies count toward processed data. Preset cost charts also add up the token metrics on every span, so they show the call’s cost twice, even though the provider billed it once.Log the token metrics and the model only on the span that made the LLM call. See Option 1: Log model on token-reporting spans.
General-purpose OpenTelemetry instrumentation also exports spans unrelated to LLM calls, such as HTTP requests and database queries. The Braintrust span processor can send only AI-related spans. See Braintrust span processor.How you configure the filter depends on the SDK:
  • TypeScript: Set filterAISpans to true. For finer control, pass a customFilter function.
  • Python: Set filter_ai_spans to True. For finer control, pass a custom_filter function.
Braintrust removes OpenTelemetry attributes from metadata once it maps them to fields such as input and output, so the same payload isn’t stored twice. This is automatic on Braintrust-hosted deployments, and self-hosted deployments need to opt in. See Strip OTel attributes from metadata.Setting braintrust.otel.preserve_attributes to true on a span keeps the raw attributes in metadata alongside the parsed fields, so the span stores them twice. Set it only on spans you’re debugging:
  • TypeScript: Call setAttribute() on the span.
  • Python: Call set_attribute() on the span.
To confirm that a change removed duplicates, compare spans per trace before and after the change. See Braintrust usage.

​
Control online scoring volume

Each score you record counts toward your plan’s score usage. Online scoring rules run their scorers on each sampled production trace or span, so a rule’s Sampling rate controls both the number of scores you record and the model spend of its LLM-as-a-judge scorers.
  • Lower the sampling rate for high-volume traffic. High-volume applications can use rates of 1-10% to manage costs. Low-volume or critical applications can use 50-100% for comprehensive coverage.
  • Score only the traces and spans you need. Use a rule’s Filter setting to choose which traces or spans it scores, such as spans with specific names. See Configuration parameters.
  • Match the scope to the judgment. A span-scoped rule scores every matching span, so one trace can produce many scores. A trace-scoped rule scores each sampled trace once. New instrumentation can add spans that match an existing span-scoped rule and raise scorer volume without any change to the rule.
  • Pause rules you don’t need. A paused rule stops scoring new traces and keeps its scorers, filters, and scope. Traces logged during the pause aren’t scored later. See Pause an automation.
To change a rule’s sampling rate, go to Settings > Automations, open the rule under Online scoring, and update Sampling rate. To lower the model cost of each judge call, see Optimize scorer calls.

​
Optimize LLM costs

Evaluations show whether a lower-cost model or scorer holds quality on your own data before you change production. The evaluations themselves can also run for less.

​
Choose lower-cost models

A smaller model often handles a well-defined task as well as a frontier model, at a lower per-token price. Braintrust serves several open-weight models, including GLM, Kimi, and DeepSeek models, as built-in models that you can use without configuring your own AI provider. Their usage draws down your organization’s model credits. Per-token rates vary widely, and not every open-weight model costs less than a comparable proprietary model. Model credits lists the rate for each built-in model. Built-in models also have plan and deployment requirements. A playground compares models side by side on the same dataset and scorers. To compare a candidate model with your current one:
  1. In a playground, add a prompt task for your current model and another for the candidate, with the same prompt, dataset, and scorers. Built-in models appear under the Braintrust provider.
  2. Select Run. Each output in the grid shows its estimated LLM cost.
  3. Turn on diff mode with your current model as the base task. Each candidate shows score changes, timing and token usage differences, and a comparison grade of Improvement, Regression, Tradeoff, or Tie.
  4. To compare estimated cost across the whole dataset, select + Experiment. Each task becomes its own experiment, so you can compare the experiments and review scores, estimated cost, and latency together.

​
Optimize scorer calls

Scorers that call an LLM add a model call for every case they grade, in both experiments and online scoring.
Scorers that don’t call a model, such as custom code checks and heuristic autoevals like Levenshtein, add no model cost and run faster. They suit checks with a deterministic answer, such as exact matches, valid JSON, required fields, or numeric tolerances. LLM-as-a-judge scorers are better for criteria that need judgment.When a check needs a judge, the judge model sets its cost. For frequent, focused checks with a clear answer, a structured decision model costs less per call than a general-purpose LLM. TypeSafe Jev selects one of the answers you define instead of generating text, and Braintrust turns the selection into a score or a classification label.Built-in Jev draws down your model credits at a low per-token rate, listed in Model credits. It must be enabled for your organization and isn’t available for BYOC or self-hosted deployments. See Configure access.For broad or nuanced criteria, such as tone or helpfulness, a general-purpose LLM is the better judge. Comparing candidate judges against human-assigned scores shows whether a lower-cost judge is accurate enough. See Choose a judge model.
A trace-level LLM-as-a-judge scorer reads the trace through template variables such as {{thread}}, and everything in them adds input tokens to each judge call. The scorer’s preprocessor builds these variables, so a custom preprocessor can skip spans the judge doesn’t need, such as middleware, or shorten large tool call payloads.To set a scorer’s preprocessor, select one under Preprocessor when you edit the scorer, or use bt scorers create --preprocessor. Scorers without their own preprocessor use the project’s default preprocessor, which Topics facets also use.

​
Run evaluations through batch APIs

Workflow evaluations are in public preview and can change before reaching general availability.
Model providers such as OpenAI and Anthropic offer batch APIs that process requests asynchronously, often at a discount. Workflow evaluations send an evaluation’s task or scorer requests through these APIs, collect the results by polling or through a webhook, and record the outputs and scores in one Braintrust experiment. Batch processing suits large regression suites, model comparisons, and scoring jobs that don’t need immediate results. Results arrive when the provider completes the batch, which can take hours. Workflow evaluations require Python SDK v0.39.0 or later, TypeScript SDK v3.37.0 or later, or Go SDK v0.18.0 or later. The Python, TypeScript, and Go guides walk through a complete OpenAI Batch API example.

​
Measure your savings

Usage and cost rise and fall with traffic. Comparing per-trace numbers across equivalent periods, such as the same days of the week, separates real savings from changes in volume. See Calculate per-trace averages in dashboards.

​
Braintrust usage

Usage analytics on the Settings > Billing page charts daily logs in GB, scores, and Topics tokens for your whole organization or by project. See Monitor usage. Unlike total usage, spans per trace doesn’t rise with traffic, so an increase points to a change in your application or its tracing, such as two layers tracing the same calls. To track it, use the average spans per trace query.

​
LLM spend

Braintrust estimates the cost of each LLM call from its token counts and the model’s pricing, so you can see the effect of each change:
  • Experiments: The Estimated cost column on the Experiments page and the comparison summary layouts show cost next to scores. See Assess overall impact.
  • Dashboards: The built-in Cost and quality dashboard tracks cost and scores over time across your logs and experiments. See Dashboards.
  • Scheduled reports: The Weekly cost report Loop automation recipe posts estimated cost by user to Slack each week, compared with the previous week.
  • Traces: Each span shows its estimated cost, and parent spans include the cost of their children. See Examine traces.
If the same call’s token metrics are logged on more than one span, preset cost charts show its cost more than once, even though the provider billed the call once. See Reduce duplicate logging.
The Estimated cost column on the Experiments page and the preset cost charts on dashboards exclude scorer spans. To see what your scorers cost, query scorer spans with SQL.

​
Next steps