AI observability pricing: compare cost and features at 2 million requests

The workload and why headline prices don't answer the budget question

A headline subscription price does not show what this workload will cost because each platform meters different usage. The shared estimate assumes 2 million application requests per month, with five spans per request. Each request produces one root span, two LLM spans, and two other spans, for 10 million spans in total. Assuming an average of 10,000 bytes per span, including allocated metadata and score payload, the workload produces 10 million × 10,000 bytes = 100 decimal GB of telemetry. Five people need access to at least 30 days of trace history, and one externally executed code score runs on 10% of requests, or 200,000 scores.

The comparison excludes model inference and external scorer compute for every vendor. It also excludes offline experiment traffic, so the figures isolate the modeled production observability bill. The meters still differ. Braintrust charges for processed data and scores beyond its allowances, while Langfuse Cloud counts traces, observations, and scores as usage units. LangSmith prices root traces and seats, and Datadog Agent Observability bills LLM spans at a per-span rate that varies with the retention you select.

Cost comparison at 2 million requests per month

These computed estimates use published monthly list prices for 2 million requests, 10 million spans, 200,000 scores, five users, and at least 30 days of trace history. Rates checked November 12, 2026. They are not vendor quotes; annual discounts, startup offers, and negotiated agreements are excluded. Weave's figure covers ingestion alone, so it is not an all-in total comparable with the other rows.

Braintrust

Best for: Teams that want production failures to become repeatable tests, with a shared workspace for deciding whether a model or prompt change is safe to release.

Pros: Braintrust combines interactive prompt testing with detailed experiment review. The value becomes concrete when a support specialist and an engineer need to agree on a changed refund decision.

Test a prompt in the browser

Playgrounds let a reviewer try a candidate prompt against a dataset without writing an evaluation script. Tasks can use different models, and scorers make their outputs comparable. The team can share the configuration rather than passing around screenshots of isolated responses. Engineers can also run experiments in code or CI/CD, keeping evaluations alongside application changes.

Review regressions and inconsistent behavior

Experiment comparison aligns cases against a baseline and highlights score changes. A reviewer can sort for regressions, then inspect the output diff for a specific policy failure. When an experiment includes repeated trials, grouping by input exposes aggregate results and expandable individual attempts. That helps distinguish a consistently correct answer from a candidate that happened to succeed once; it does not by itself prove reliability.

Pro adds dataset snapshots and unlimited human review scorers. A team can hold the test data fixed while comparing versions, and record human judgments where a code scorer misses a policy nuance. Custom dashboards and environment tags connect that review work with ongoing production monitoring.

Use Loop to turn a production failure into a model test

The work between finding a bad answer and testing a better model matters to the buying decision. Loop, Braintrust's built-in agent, can help prepare that test from the production evidence already in your project.

Suppose a support agent keeps giving the wrong refund answer. Ask Loop to investigate similar failures in your logs. It can inspect the underlying traces and help identify the recurring problem. Once you have checked that explanation, ask it to copy relevant log rows into a regression dataset; the rows retain links to their sources. Loop can also draft a scorer for the failure so your team has a concrete proposal to review. That gives the reviewer a starting point for comparing the current model with a cheaper candidate, rather than an empty evaluation dataset.

The benefit to test is less manual investigation and test preparation. A five-person team handling 500,000 requests with this article's span and scoring assumptions would pay approximately $309 for Braintrust Pro versus $244.50 for Langfuse Core. The $64.50 difference is equivalent to about 39 minutes of staff time at an illustrative $100 per hour. If Loop saves more than that each month after accounting for review time and additional usage costs, the higher platform bill can still mean lower total operating cost. This is a break-even illustration, not a measured time-saving claim.

During a pilot, use a real failure and compare the time needed to reach a reviewed regression test in each platform. Check whether the selected examples and scorer are useful, not just whether an agent produced them. Loop requests approval before creating or editing objects by default. Its model usage consumes credits or is billed through a configured provider; additional evaluation usage also belongs in the budget. Langfuse and LangSmith support evaluations too. Braintrust's case is the value of having this investigation assistance inside the evaluation workspace, which the trial should establish for your team.

Start with the free tier

Braintrust's Starter plan has a $0 platform fee and unlimited users, projects, datasets, playgrounds, and experiments. A five-person team can therefore try the core workflow together without buying seats. It includes 1 GB of processed data and 10,000 scores per month, plus $10 in monthly model credits and 14-day trace retention. That is generous access for a focused evaluation pilot, although unlimited objects do not mean unlimited usage.

For example, the 1,200-output evaluation illustrated later fits within Starter's score allowance if no other scores have consumed it. To remain within free allowances, its telemetry must also fit within the included data, and model usage must stay within applicable credits or be budgeted separately. Starter overages are $4 per processed GB and $2.50 per 1,000 scores. Built-in model access on Starter requires an owner with a work email address or a payment method on file. See plans and limits for the full breakdown.

Starter is a way to evaluate the product before paying, not the plan used for this production comparison. Its 14-day history does not meet the 30-day requirement. Dataset snapshots, playground annotations, and the other Pro-specific features above also require Pro; Starter allows one human review scorer per project. Starter's only built-in permission group is Owners, so every invited teammate holds full organization access.

Cons: Enterprise SAML/OIDC SSO, audit logging, and custom retention policies require Enterprise. Extra experiment telemetry and scores add usage, while Loop and model-based judges can add inference costs. Competitors may offer a better fit for longer history or full-stack self-hosting.

Pricing: At the shared 2-million-request workload, Pro remains $759 per month: $249 base, $285 for 95 GB above the data allowance, and $225 for 150,000 scores above the score allowance. Five users and 30-day history are included. This is the lowest modeled total among the four fully priced plans here; the free tier is not used to discount that estimate.

Langfuse Cloud

Best for: Langfuse Cloud suits teams that want a managed version of an open-source platform, experiment comparisons, and 90 days of trace history.

Pros: Langfuse supports experiment comparisons against a baseline, including quality, cost, and latency results. Its Core plan meets the workload's 30-day history requirement without a retention add-on.

Cons: Langfuse counts traces, observations, and scores as billable units. Deeper tracing or more scoring therefore raises usage even when request volume stays fixed.

Pricing: For this workload, 2 million traces, 10 million observations, and 200,000 scores total 12.2 million units. After Core's 100,000 included units, 12.1 million units remain billable. The modeled total is $874 per month, comprising the $29 plan charge and $845 in usage charges calculated using Langfuse's published graduated tiers. The paid bands are 900,000 units at $8 per 100,000 ($72), 9 million at $7 per 100,000 ($630), and 2.2 million at $6.50 per 100,000 ($143). Braintrust costs $115 less, a 13.2% reduction.

LangSmith

Best for: LangSmith suits teams that value its tracing and evaluation workflows, including teams using LangChain. LangSmith does not require LangChain.

Pros: LangSmith supports baseline comparisons, output diffs, and regression filters, so you can inspect changes across experiment runs.

Cons: Base traces retain for 14 days. Meeting this workload's 30-day minimum requires the extended-retention upgrade, which raises the modeled bill.

Pricing: At published monthly rates, five Plus seats cost $195. LangSmith meters usage in LangChain Standard Units at $1 per LSU, where a base trace consumes 0.005 LSU and an extended-retention upgrade adds 0.0025 LSU. For 2 million root traces, base-trace usage costs (2,000,000 − 10,000) × $0.005, or $9,950. Extended retention adds 2,000,000 × $0.0025, or $5,000. The modeled estimate is $15,145 per month, not a vendor quote. The LSU-per-trace conversion is not stated on the pricing page itself, and the estimate assumes LangSmith applies the 10,000 included traces to base usage. Confirm both before using the figure as a purchasing budget.

Datadog Agent Observability

Best for: Teams already using Datadog for application and infrastructure monitoring who want to connect LLM traces with their existing observability data.

Pros: Datadog Agent Observability includes tracing, evaluations, datasets, and experiments. It bills only the workload's 4 million LLM spans, not its 6 million non-LLM spans. A separate APM subscription is not required.

Cons: Retention is priced into the per-span rate rather than included, so a 30-day requirement costs more per span than Datadog's 15-day default. Charges for other Datadog products are excluded.

Pricing: Datadog's published rates produce a modeled monthly estimate of $2,423. The $200 month-to-month plan includes 100,000 LLM spans. The remaining 3.9 million spans are billed at the month-to-month 30-day retention rate of $5.70 per 10,000 spans, or $2,223.

Datadog prices retention as alternative per-span rates rather than as an add-on: you select one retention period and pay that rate, rather than a base rate plus a retention surcharge. The rate also varies with billing commitment. At month-to-month rates the published list shows $4.20 per 10,000 spans at 15-day retention, $5.70 at 30 days, $7.20 at 60 days, and $8.20 at 90 days; committing annually lowers those to $3.50, $5.00, $6.50, and $7.50, and the annual plan price drops from $200 to $160. This estimate uses month-to-month rates throughout, consistent with excluding annual discounts. Datadog also publishes a free allowance of 40,000 LLM spans separate from the paid plan, so confirm your plan tier and commitment before using this figure.

Weights & Biases Weave

Best for: Weights & Biases Weave merits consideration if you want tracing, evaluations, and production monitoring, but its published rates do not establish a full cost for this workload.

Pros: Weave includes tracing and evaluations, with production monitoring and LLM-as-a-judge metrics.

Cons: The published page does not confirm seat pricing for this five-user team or whether traces remain accessible for the required 30 days. The ingestion figure therefore cannot be compared as an all-in total with the fully modeled platforms.

Pricing, ingestion subtotal only: Weave includes 1.5 GB of ingestion, then charges $0.10 per MB. At 100,000 decimal MB, the calculation is (100,000 − 1,500) × $0.10, or $9,850 per month before any other applicable charges. CoreWeave lists Pro starting at $60 per month for companies with fewer than 50 employees, but that starting price does not resolve the full plan cost here.

How plan-specific features change the real decision

Braintrust Pro connects production tracing to release decisions. Suppose a support agent gives the wrong refund answer. Add that trace to a dataset alongside representative successful requests, so fixing the failure does not come at the expense of common cases. Keep the dataset version and scoring definitions fixed while comparing the current model with a cheaper candidate. In experiment comparison, choose the current model as the baseline and sort by score regressions. Open the output diff for a refund-policy failure to see whether the candidate changed the decision or only the wording.

This is the workflow to assess in a trial: can the reviewer explain the failed case and decide whether to ship the candidate? A lower average model bill is useful only if the candidate meets the team's quality requirements. Langfuse and LangSmith also support experiment comparison; Braintrust's case here is that its review workflow comes with the lower modeled platform bill, not that competitors cannot evaluate models.

Braintrust Pro's dashboards can track the support agent after release, and environment tags identify versions intended for staging or production. Sandbox evals and human review scorers offer ways to check the proposed fix, though additional evaluations can increase usage charges.

What changes the estimate

Measure the bytes your application actually sends before using this budget. The 100 GB figure includes metadata and score payload; additional payload raises a byte-based bill. Running a scorer externally does not remove the modeled charge for storing its scores. Use a representative production sample to reconcile recorded spans, processed bytes, and scores with the billing view before committing.

At the same 2 million requests, two boundaries help interpret the $115 advantage over Langfuse:

These are calculations from the Braintrust and Langfuse rates used above, not limits or recommended sampling policies. Added score payload can move the second crossover earlier. Keep the evaluation coverage your application needs and recalculate the bill; reducing useful evaluations solely to preserve a price ranking would defeat the purpose.

Langfuse Core also includes 90-day history. If you need more than the stated 30 days, compare the retained-data charge before choosing. Full-stack self-hosting, automated exports, or enterprise identity requirements likewise call for a separate plan comparison. A lower bill for the stated workload does not establish feature superiority for every team.

Offline experiments are additional workload. For example, 200 cases tested against two candidate models over three trials produce 1,200 outputs. With one score per output, Braintrust adds $1.80 in scoring charges at the modeled usage level, plus processed telemetry and any model or execution costs. Feature access does not mean unlimited evaluation usage.

Why Braintrust fits teams scaling production evals

Braintrust Pro pairs the lowest modeled total among Braintrust, Langfuse Cloud, LangSmith, and Datadog with a workflow for testing changes against failures found in production. The $759 monthly estimate applies only to the shared workload; more evaluation runs or different telemetry volumes can change the bill.

Trial Braintrust first if this workload and managed evaluation workflow match your needs. Confirm the production meter against the $759 estimate, then use the fixed evaluation dataset to judge whether its experiment-review workflow helps your team make a reliable model decision. Budget offline experiments separately using the example above; any contingency is a planning choice, not part of the published price. Check current pricing before purchase. This is a selected comparison, not a claim that Braintrust is cheapest across the market. Other platforms and negotiated contracts may cost less.

Start free with Braintrust and reconcile the meter against your own traffic before committing to a plan.