> ## Documentation Index
> Fetch the complete documentation index at: https://www.braintrust.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Braintrust Python SDK

> Python API reference for Braintrust SDK v0.47.0

This page provides a complete, auto-generated reference for the Braintrust Python SDK. For usage guidance on important APIs, see the [API Reference](https://www.braintrust.dev/docs/sdks/python/api-reference.md). Also see the [Braintrust Python SDK](https://github.com/braintrustdata/braintrust-sdk-python) on GitHub.

## Requirements

* Python 3.10 or higher

## Installation

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
pip install braintrust
```

## Functions

### DatasetPipeline

Create a runnable dataset pipeline.

**name** `str | None`

**source** `DatasetPipelineSource`

**transform** `DatasetPipelineTransform[DatasetPipelineRow]`

**target** `DatasetPipelineTarget`

### Eval

A function you can use to define an evaluator. This is a convenience wrapper around the `Evaluator` class.

**name** `str`

The name of the evaluator. This corresponds to a project name in Braintrust.

**data** `EvalData[Input, Expected]`

Returns an iterator over the evaluation dataset. Each element of the iterator should be a `EvalCase`.

**task** `EvalTask[Input, Output, Expected]`

Runs the evaluation task on a single input. The `hooks` object can be used to add metadata to the evaluation.

**scores** `Sequence[EvalScorer[Input, Output, Expected]] | None`

A list of scorers to evaluate the results of the task. Each scorer can be a Scorer object or a function that returns a number, a `Score` object, a `ScoreDict`, or a sequence of named scores. A single dict can omit `name` to use the scorer's name; each score in a sequence must include a name.

**classifiers** `Sequence[EvalClassifier[Input, Output, Expected]] | None`

**experiment\_name** `str | None`

(Optional) Experiment name. If not specified, a name will be generated automatically.

**trial\_count** `int`

The number of times to run the evaluator per input. This is useful for evaluating applications that have non-deterministic behavior and gives you both a stronger aggregate measure and a sense of the variance in the results.

**metadata** `Metadata | None`

(Optional) A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the `prompt`, example's `id`, or anything else that would be useful to slice/dice later. The values in `metadata` can be any JSON-serializable type, but its keys must be strings.

**tags** `Sequence[str] | None`

(Optional) A list of tags to associate with the experiment.

**is\_public** `bool`

(Optional) Whether the experiment should be public. Defaults to false.

**update** `bool`

**reporter** `ReporterDef[Input, Output, Expected, EvalReport] | None`

(Optional) A reporter that takes an evaluator and its result and returns a report.

**reporter.name** `str` *required*

**reporter.report\_eval** `Callable[[Evaluator[Input, Output, Expected], EvalResultWithSummary[Input, Output, Expected], bool, bool], EvalReport | Awaitable[EvalReport]]` *required*

**reporter.report\_run** `Callable[[list[EvalReport], bool, bool], bool | Awaitable[bool]]` *required*

**timeout** `float | None`

(Optional) The duration, in seconds, after which to time out the evaluation. Defaults to None, in which case there is no timeout.

**max\_concurrency** `int | None`

**project\_id** `str | None`

(Optional) If specified, uses the given project ID instead of the evaluator's name to identify the project.

**base\_experiment\_name** `str | None`

An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.

**base\_experiment\_id** `str | None`

An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment. This takes precedence over `base_experiment_name` if specified.

**git\_metadata\_settings** `GitMetadataSettings | None`

Optional settings for collecting git metadata. By default, defers to org-level settings returned by the control plane.

**git\_metadata\_settings.collect** `Literal['all', 'some', 'none']` *required*

**git\_metadata\_settings.fields** `list[str] | None` *required*

**repo\_info** `RepoInfo | None`

Optionally explicitly specify the git metadata for this experiment. This takes precedence over `git_metadata_settings` if specified.

**repo\_info.commit** `str | None` *required*

**repo\_info.branch** `str | None` *required*

**repo\_info.tag** `str | None` *required*

**repo\_info.dirty** `bool | None` *required*

**repo\_info.author\_name** `str | None` *required*

**repo\_info.author\_email** `str | None` *required*

**repo\_info.commit\_message** `str | None` *required*

**repo\_info.commit\_time** `str | None` *required*

**repo\_info.git\_diff** `str | None` *required*

**error\_score\_handler** `ErrorScoreHandler[Input, Expected] | None`

Optionally supply a custom function to specifically handle score values when tasks or scoring functions have errored.

**description** `str | None`

An optional description for the experiment.

**summarize\_scores** `bool`

Whether to summarize the scores of the experiment after it has run.

**no\_send\_logs** `bool`

Do not send logs to Braintrust. When True, the evaluation runs locally and builds a local summary instead of creating an experiment. Defaults to False.

**parameters** `EvalParameters | RemoteEvalParameters | None`

A set of parameters that will be passed to the evaluator.

**on\_start** `Callable[[ExperimentSummary], None] | None`

An optional callback that will be called when the evaluation starts. It receives the `ExperimentSummary` object, which can be used to display metadata about the experiment.

**stream** `Callable[[SSEProgressEvent], None] | None`

A function that will be called with progress events, which can be used to display intermediate progress.

**parent** `str | None`

If specified, instead of creating a new experiment object, the Eval() will populate the object or span specified by this parent.

**state** `BraintrustState | None`

Optional BraintrustState to use for the evaluation. If not specified, the global login state will be used.

**enable\_cache** `bool`

Whether to enable the span cache for this evaluation. Defaults to True. The span cache stores span data on disk to minimize memory usage and allow scorers to read spans without server round-trips.

**cancel\_event** `threading.Event | None`

Optional threading.Event. Setting it stops the evaluation and cancels running async tasks. A timeout also sets this event. Running synchronous tasks must cooperate or finish before the eval returns.

### EvalAsync

A function you can use to define an evaluator. This is a convenience wrapper around the `Evaluator` class.

**name** `str`

The name of the evaluator. This corresponds to a project name in Braintrust.

**data** `EvalData[Input, Expected]`

Returns an iterator over the evaluation dataset. Each element of the iterator should be a `EvalCase`.

**task** `EvalTask[Input, Output, Expected]`

Runs the evaluation task on a single input. The `hooks` object can be used to add metadata to the evaluation.

**scores** `Sequence[EvalScorer[Input, Output, Expected]] | None`

A list of scorers to evaluate the results of the task. Each scorer can be a Scorer object or a function that returns a number, a `Score` object, a `ScoreDict`, or a sequence of named scores. A single dict can omit `name` to use the scorer's name; each score in a sequence must include a name.

**classifiers** `Sequence[EvalClassifier[Input, Output, Expected]] | None`

**experiment\_name** `str | None`

(Optional) Experiment name. If not specified, a name will be generated automatically.

**trial\_count** `int`

The number of times to run the evaluator per input. This is useful for evaluating applications that have non-deterministic behavior and gives you both a stronger aggregate measure and a sense of the variance in the results.

**metadata** `Metadata | None`

(Optional) A dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the `prompt`, example's `id`, or anything else that would be useful to slice/dice later. The values in `metadata` can be any JSON-serializable type, but its keys must be strings.

**tags** `Sequence[str] | None`

(Optional) A list of tags to associate with the experiment.

**is\_public** `bool`

(Optional) Whether the experiment should be public. Defaults to false.

**update** `bool`

**reporter** `ReporterDef[Input, Output, Expected, EvalReport] | None`

(Optional) A reporter that takes an evaluator and its result and returns a report.

**reporter.name** `str` *required*

**reporter.report\_eval** `Callable[[Evaluator[Input, Output, Expected], EvalResultWithSummary[Input, Output, Expected], bool, bool], EvalReport | Awaitable[EvalReport]]` *required*

**reporter.report\_run** `Callable[[list[EvalReport], bool, bool], bool | Awaitable[bool]]` *required*

**timeout** `float | None`

(Optional) The duration, in seconds, after which to time out the evaluation. Defaults to None, in which case there is no timeout.

**max\_concurrency** `int | None`

**project\_id** `str | None`

(Optional) If specified, uses the given project ID instead of the evaluator's name to identify the project.

**base\_experiment\_name** `str | None`

An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment.

**base\_experiment\_id** `str | None`

An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this experiment. This takes precedence over `base_experiment_name` if specified.

**git\_metadata\_settings** `GitMetadataSettings | None`

Optional settings for collecting git metadata. By default, defers to org-level settings returned by the control plane.

**git\_metadata\_settings.collect** `Literal['all', 'some', 'none']` *required*

**git\_metadata\_settings.fields** `list[str] | None` *required*

**repo\_info** `RepoInfo | None`

Optionally explicitly specify the git metadata for this experiment. This takes precedence over `git_metadata_settings` if specified.

**repo\_info.commit** `str | None` *required*

**repo\_info.branch** `str | None` *required*

**repo\_info.tag** `str | None` *required*

**repo\_info.dirty** `bool | None` *required*

**repo\_info.author\_name** `str | None` *required*

**repo\_info.author\_email** `str | None` *required*

**repo\_info.commit\_message** `str | None` *required*

**repo\_info.commit\_time** `str | None` *required*

**repo\_info.git\_diff** `str | None` *required*

**error\_score\_handler** `ErrorScoreHandler[Input, Expected] | None`

Optionally supply a custom function to specifically handle score values when tasks or scoring functions have errored.

**description** `str | None`

An optional description for the experiment.

**summarize\_scores** `bool`

Whether to summarize the scores of the experiment after it has run.

**no\_send\_logs** `bool`

Do not send logs to Braintrust. When True, the evaluation runs locally and builds a local summary instead of creating an experiment. Defaults to False.

**parameters** `EvalParameters | RemoteEvalParameters | None`

A set of parameters that will be passed to the evaluator.

**on\_start** `Callable[[ExperimentSummary], None] | None`

An optional callback that will be called when the evaluation starts. It receives the `ExperimentSummary` object, which can be used to display metadata about the experiment.

**stream** `Callable[[SSEProgressEvent], None] | None`

A function that will be called with progress events, which can be used to display intermediate progress.

**parent** `str | None`

If specified, instead of creating a new experiment object, the Eval() will populate the object or span specified by this parent.

**state** `BraintrustState | None`

Optional BraintrustState to use for the evaluation. If not specified, the global login state will be used.

**enable\_cache** `bool`

Whether to enable the span cache for this evaluation. Defaults to True. The span cache stores span data on disk to minimize memory usage and allow scorers to read spans without server round-trips.

**cancel\_event** `threading.Event | None`

Optional threading.Event. Setting it stops the evaluation and cancels running async tasks. A timeout also sets this event. Running synchronous tasks must cooperate or finish before the eval returns.

### Reporter

A function you can use to define a reporter. This is a convenience wrapper around the `ReporterDef` class.

**name** `str`

The name of the reporter.

**report\_eval** `Callable[[Evaluator[Input, Output, Expected], EvalResultWithSummary[Input, Output, Expected], bool, bool], EvalReport | Awaitable[EvalReport]]`

A function that takes an evaluator and its result and returns a report.

**report\_run** `Callable[[list[EvalReport], bool, bool], bool | Awaitable[bool]]`

A function that takes all evaluator results and returns a boolean indicating whether the run was successful.

### auto\_instrument

Auto-instrument supported AI/ML libraries for Braintrust tracing.

**openai** `bool`

Enable OpenAI instrumentation (default: True)

**anthropic** `bool`

Enable Anthropic instrumentation (default: True)

**litellm** `bool`

Enable LiteLLM instrumentation (default: True)

**ai\_sdk** `bool`

Enable Vercel AI SDK for Python instrumentation (default: True)

**pydantic\_ai** `bool`

Enable Pydantic AI instrumentation (default: True)

**google\_genai** `bool`

Enable Google GenAI instrumentation (default: True)

**google\_discoveryengine** `bool`

Enable Google Discovery Engine v1 instrumentation (default: True)

**instructor** `bool`

Enable Instructor (structured-output) instrumentation (default: True)

**openrouter** `bool`

Enable OpenRouter instrumentation (default: True)

**mistral** `bool`

Enable Mistral instrumentation (default: True)

**huggingface\_hub** `bool`

Enable HuggingFace Hub instrumentation (default: True)

**transformers** `bool`

Enable local Hugging Face Transformers pipeline instrumentation (default: True)

**agno** `bool`

Enable Agno instrumentation (default: True)

**agentscope** `bool`

Enable AgentScope instrumentation (default: True)

**claude\_agent\_sdk** `bool`

Enable Claude Agent SDK instrumentation (default: True)

**cursor\_sdk** `bool`

Enable Cursor SDK instrumentation (default: True)

**dspy** `bool`

Enable DSPy instrumentation (default: True)

**adk** `bool`

Enable Google ADK instrumentation (default: True)

**langchain** `bool`

Enable LangChain instrumentation (default: True)

**llamaindex** `bool`

Enable LlamaIndex instrumentation (default: True)

**openai\_agents** `bool`

Enable OpenAI Agents SDK instrumentation (default: True)

**cohere** `bool`

Enable Cohere instrumentation (default: True)

**autogen** `bool`

Enable AutoGen instrumentation (default: True)

**bedrock** `bool`

Enable boto3 Bedrock Runtime instrumentation (default: True)

**crewai** `bool`

Enable CrewAI instrumentation (default: True)

**strands** `bool`

Enable Strands Agents instrumentation (default: True)

**temporal** `bool`

Enable Temporal instrumentation (default: True)

**livekit\_agents** `bool`

Enable LiveKit Agents instrumentation (default: True)

**pipecat** `bool`

Enable Pipecat AI instrumentation (default: True)

**typesafe** `bool`

Enable TypeSafe instrumentation (default: True)

**span\_customizers** `Sequence[SpanCustomizer] | None`

Ordered synchronous export customizers for native SDK spans
and OTel spans sent through OtelExporter or BraintrustSpanProcessor,
including manual spans; generic OTLP exporters, dataset rows, and
feedback are excluded.
Copies and replaces the global list when provided; None leaves existing
configuration unchanged. Pass \[] to disable.

### current\_experiment

Returns the currently-active experiment (set by `braintrust.init(...)`). Returns None if no current experiment has been set.

### current\_logger

Returns the currently-active logger (set by `braintrust.init_logger(...)`). Returns None if no current logger has been set.

### current\_span

Return the currently-active span for logging (set by running a span under a context manager). If there is no active span, returns a no-op span object, which supports the same interface as spans but does no logging.

### extract\_trace\_context

Extract an opaque W3C trace-context from inbound request headers.

**headers** `dict`

Inbound request headers (e.g. an HTTP framework's headers).

### flush

Flush any pending rows to the server.

### get\_prompt\_versions

Get the versions for a specific prompt.

**project\_id** `str`

The ID of the project to query

**prompt\_id** `str`

The ID of the prompt to get versions for

### get\_span\_parent\_object

Mainly for internal use. Return the parent object for starting a span in a global context. Applies precedence: current span > propagated parent > experiment > logger.

**parent** `str | dict | None`

**state** `BraintrustState | None`

### init

Log in, and then initialize a new experiment in a specified project. If the project does not exist, it will be created.

**project** `str | None`

The name of the project to create the experiment in. Must specify at least one of `project` or `project_id`.

**experiment** `str | None`

The name of the experiment to create. If not specified, a name will be generated automatically.

**description** `str | None`

(Optional) An optional description of the experiment.

**dataset** `Dataset | None | DatasetRef`

(Optional) A dataset to associate with the experiment. The dataset must be initialized with `braintrust.init_dataset` before passing it into the experiment.

**parameters** `RemoteEvalParameters | ParametersRef | None`

(Optional) Saved parameters to associate with the experiment. Pass either a `RemoteEvalParameters` object or a dictionary containing an `id` and optional `version`.

**parameters.id** `str | None` *required*

**parameters.project\_id** `str | None` *required*

**parameters.name** `str` *required*

**parameters.slug** `str` *required*

**parameters.version** `str | None` *required*

**parameters.schema** `ParametersSchema` *required*

**parameters.data** `dict[str, Any]` *required*

**open** `bool`

If the experiment already exists, open it in read-only mode. Throws an error if the experiment does not already exist.

**base\_experiment** `str | None`

An optional experiment name to use as a base. If specified, the new experiment will be summarized and compared to this experiment. Otherwise, it will pick an experiment by finding the closest ancestor on the default (e.g. main) branch.

**is\_public** `bool`

An optional parameter to control whether the experiment is publicly visible to anybody with the link or privately visible to only members of the organization. Defaults to private.

**app\_url** `str | None`

The URL of the Braintrust App. Defaults to [https://www.braintrust.dev](https://www.braintrust.dev).

**api\_key** `str | None`

The API key to use. If the parameter is not specified, will try to use the `BRAINTRUST_API_KEY` environment variable. If no API key is specified, will prompt the user to login.

**org\_name** `str | None`

(Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.

**metadata** `Metadata | None`

(Optional) a dictionary with additional data about the test example, model outputs, or just about anything else that's relevant, that you can use to help find and analyze examples later. For example, you could log the `prompt`, example's `id`, or anything else that would be useful to slice/dice later. The values in `metadata` can be any JSON-serializable type, but its keys must be strings.

**tags** `Sequence[str] | None`

(Optional) a list of strings to tag the experiment with. Tags can be used to filter and organize experiments.

**git\_metadata\_settings** `GitMetadataSettings | None`

(Optional) Settings for collecting git metadata. By default, will collect git metadata fields allowed in org-level settings, excluding diff content unless the org opts in.

**git\_metadata\_settings.collect** `Literal['all', 'some', 'none']` *required*

**git\_metadata\_settings.fields** `list[str] | None` *required*

**set\_current** `bool`

If true (the default), set the global current-experiment to the newly-created one.

**update** `bool | None`

If the experiment already exists, continue logging to it. If it does not exist, creates the experiment with the specified arguments.

**project\_id** `str | None`

The id of the project to create the experiment in. This takes precedence over `project` if specified.

**base\_experiment\_id** `str | None`

An optional experiment id to use as a base. If specified, the new experiment will be summarized and compared to this. This takes precedence over `base_experiment` if specified.

**repo\_info** `RepoInfo | None`

(Optional) Explicitly specify the git metadata for this experiment. This takes precedence over `git_metadata_settings` if specified.

**repo\_info.commit** `str | None` *required*

**repo\_info.branch** `str | None` *required*

**repo\_info.tag** `str | None` *required*

**repo\_info.dirty** `bool | None` *required*

**repo\_info.author\_name** `str | None` *required*

**repo\_info.author\_email** `str | None` *required*

**repo\_info.commit\_message** `str | None` *required*

**repo\_info.commit\_time** `str | None` *required*

**repo\_info.git\_diff** `str | None` *required*

**state** `BraintrustState | None`

(Optional) A BraintrustState object to use. If not specified, will use the global state. This is for advanced use only.

### init\_dataset

Create or load a dataset. When creating a dataset, its project will be created if it does not exist.

**project** `str | None`

The name of the project to create the dataset in.

**name** `str | None`

The name of the dataset to create. Defaults to `logs` if not specified.

**description** `str | None`

An optional description of the dataset.

**version** `str | int | None`

An optional version of the dataset (to read). If not specified, the latest version will be used.

**app\_url** `str | None`

The URL of the Braintrust App. Defaults to [https://www.braintrust.dev](https://www.braintrust.dev).

**api\_key** `str | None`

The API key to use. If the parameter is not specified, will try to use the `BRAINTRUST_API_KEY` environment variable. If no API key is specified, will prompt the user to login.

**org\_name** `str | None`

(Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.

**project\_id** `str | None`

The id of the project to create the dataset in. This takes precedence over `project` if specified.

**metadata** `Metadata | None`

(Optional) a dictionary, or an object that serializes to a dictionary (such as a Pydantic model), with additional data about the dataset. The values in `metadata` can be any JSON-serializable type, but its keys must be strings.

**use\_output** `bool`

(Deprecated) If True, records will be fetched from this dataset in the legacy format, with the "expected" field renamed to "output". This option will be removed in a future version of Braintrust.

**\_internal\_btql** `dict[str, Any] | None`

(Internal) If specified, the dataset will be created with the given BTQL filters.

**state** `BraintrustState | None`

(Internal) The Braintrust state to use. If not specified, will use the global state. For advanced use only.

**environment** `str | None`

The environment to load the dataset from. If both `version` and `environment` are provided, `version` takes precedence.

**dataset\_id** `str | None`

The id of an existing dataset to load. If specified, this takes precedence over `project`, `project_id`, and `name`.

### init\_experiment

Alias for `init`

**args** `Any`

**kwargs** `Any`

### init\_function

Creates a function that can be used as either a task or scorer in the Eval framework. When used as a task, it will invoke the specified Braintrust function with the input. When used as a scorer, it will invoke the function with the scorer arguments.

**project\_name** `str`

The name of the project containing the function.

**slug** `str`

The slug of the function to invoke.

**version** `str | None`

Optional version of the function to use. Defaults to latest.

### init\_logger

Create a new logger in a specified project. If the project does not exist, it will be created.

**project** `str | None`

The name of the project to log into. If unspecified, will default to the Global project.

**project\_id** `str | None`

The id of the project to log into. This takes precedence over project if specified.

**async\_flush** `bool`

If true (the default), log events will be batched and sent asynchronously in a background thread. If false, log events will be sent synchronously. Set to false in serverless environments.

**app\_url** `str | None`

The URL of the Braintrust API. Defaults to [https://www.braintrust.dev](https://www.braintrust.dev).

**api\_key** `str | None`

The API key to use. If the parameter is not specified, will try to use the `BRAINTRUST_API_KEY` environment variable. If no API key is specified, will prompt the user to login.

**org\_name** `str | None`

(Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.

**force\_login** `bool`

Login again, even if you have already logged in (by default, the logger will not login if you are already logged in)

**set\_current** `bool`

If true (the default), set the global current-experiment to the newly-created one.

**state** `BraintrustState | None`

**environment** `SpanOriginEnvironment | None`

**\_create\_in\_project\_group** `str | None`

Experimental: the name of an existing project group to create the project in. Existing projects must already belong to the group, otherwise registration fails with a 409. Ignored when project\_id is provided.

### inject\_trace\_context

Inject W3C trace-context headers for the current (or given) span into a carrier.

**carrier** `dict | None`

Optional carrier dict (e.g. outbound HTTP headers) to mutate.

**span** `Span | None`

Optional span to inject. Defaults to the current span.

### invoke

Invoke a Braintrust function, returning a `BraintrustStream` or the value as a plain Python object.

**function\_id** `str | None`

The ID of the function to invoke.

**version** `str | None`

The version of the function to invoke.

**prompt\_session\_id** `str | None`

The ID of the prompt session to invoke the function from.

**prompt\_session\_function\_id** `str | None`

The ID of the function in the prompt session to invoke.

**project\_name** `str | None`

The name of the project containing the function to invoke.

**project\_id** `str | None`

The ID of the project to use for execution context (API keys, project defaults, etc.).
This is not the project the function belongs to, but the project context for the invocation.

**slug** `str | None`

The slug of the function to invoke.

**global\_function** `str | None`

The name of the global function to invoke.

**function\_type** `FunctionTypeEnum | None`

The type of the global function to invoke. If unspecified, defaults to 'scorer'
for backward compatibility.

**input** `Any`

The input to the function. This will be logged as the `input` field in the span.

**messages** `Sequence[Any] | None`

Additional OpenAI-style messages to add to the prompt (only works for llm functions).

**metadata** `Metadata | None`

Additional metadata to add to the span. This will be logged as the `metadata` field in the span.
It will also be available as the \{\{metadata}} field in the prompt and as the `metadata` argument
to the function.

**tags** `Sequence[str] | None`

Tags to add to the span. This will be logged as the `tags` field in the span.

**parent** `Exportable | str | None`

The parent of the function. This can be an existing span, logger, or experiment, or
the output of `.export()` if you are distributed tracing. If unspecified, will use
the same semantics as `traced()` to determine the parent and no-op if not in a tracing
context.

**stream** `bool`

Whether to stream the function's output. If True, the function will return a
`BraintrustStream`, otherwise it will return the output of the function as a JSON
object.

**mode** `ModeType | None`

The response shape of the function if returning tool calls. If "auto", will return
a string if the function returns a string, and a JSON object otherwise. If "parallel",
will return an array of JSON objects with one object per tool call.

**strict** `bool | None`

Whether to use strict mode for the function. If true, the function will throw an
error if the variable names in the prompt do not match the input keys.

**overrides** `dict[str, Any] | None`

Per-call deep-merge into the resolved function data server-side. Useful for
facet, code, global, and remote\_eval functions (for example, overriding a facet's
`model` or a global function's `config`). Has no effect on prompt functions,
whose parameters live on a separate field that the override path does not touch.

**org\_name** `str | None`

The name of the Braintrust organization to use.

**api\_key** `str | None`

The API key to use for authentication.

**app\_url** `str | None`

The URL of the Braintrust application.

**force\_login** `bool`

Whether to force a new login even if already logged in.

### invoke\_async

Asynchronously invoke a Braintrust function without blocking the event loop.

**function\_id** `str | None`

**version** `str | None`

**prompt\_session\_id** `str | None`

**prompt\_session\_function\_id** `str | None`

**project\_name** `str | None`

**project\_id** `str | None`

**slug** `str | None`

**global\_function** `str | None`

**function\_type** `FunctionTypeEnum | None`

**input** `Any`

**messages** `Sequence[Any] | None`

**metadata** `Metadata | None`

**tags** `Sequence[str] | None`

**parent** `Exportable | str | None`

**stream** `bool`

**mode** `ModeType | None`

**strict** `bool | None`

**overrides** `dict[str, Any] | None`

**org\_name** `str | None`

**api\_key** `str | None`

**app\_url** `str | None`

**force\_login** `bool`

### load\_parameters

Load saved parameters from Braintrust.

**project** `str | None`

The name of the project to load the parameters from. Must specify at least one of `project` or `project_id`.

**slug** `str | None`

The slug of the parameters to load.

**version** `str | int | None`

An optional version of the parameters to read. If not specified, the latest version will be used.

**project\_id** `str | None`

The ID of the project to load the parameters from. This takes precedence over `project`.

**id** `str | None`

The ID of a specific parameters object to load. If specified, this takes precedence over project and slug.

**environment** `str | None`

The environment to load the parameters from. If both `version` and `environment` are provided, `version` takes precedence.

**app\_url** `str | None`

The URL of the Braintrust App. Defaults to [https://www.braintrust.dev](https://www.braintrust.dev).

**api\_key** `str | None`

The API key to use for this request, independently of any existing global login. If the parameter is not specified, will use an existing login or try the `BRAINTRUST_API_KEY` environment variable.

**org\_name** `str | None`

The name of a specific organization to connect to.

### load\_prompt

Loads a prompt from the specified project.

**project** `str | None`

The name of the project to load the prompt from. Must specify at least one of `project` or `project_id`.

**slug** `str | None`

The slug of the prompt to load.

**version** `str | int | None`

An optional version of the prompt (to read). If not specified, the latest version will be used.

**project\_id** `str | None`

The id of the project to load the prompt from. This takes precedence over `project` if specified.

**id** `str | None`

The id of a specific prompt to load. If specified, this takes precedence over all other parameters (project, slug, version).

**defaults** `Mapping[str, Any] | None`

(Optional) A dictionary of default values to use when rendering the prompt. Prompt values will override these defaults.

**no\_trace** `bool`

If true, do not include logging metadata for this prompt when build() is called.

**environment** `str | None`

The environment to load the prompt from. If both `version` and `environment` are provided, `version` takes precedence.

**app\_url** `str | None`

The URL of the Braintrust App. Defaults to [https://www.braintrust.dev](https://www.braintrust.dev).

**api\_key** `str | None`

The API key to use for this request, independently of any existing global login. If the parameter is not specified, will use an existing login or try the `BRAINTRUST_API_KEY` environment variable. If no API key is specified, will prompt the user to login.

**org\_name** `str | None`

(Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.

### load\_prompt\_async

Asynchronously loads a prompt from the specified project.

**project** `str | None`

**slug** `str | None`

**version** `str | int | None`

**project\_id** `str | None`

**id** `str | None`

**defaults** `Mapping[str, Any] | None`

**no\_trace** `bool`

**environment** `str | None`

**app\_url** `str | None`

**api\_key** `str | None`

**org\_name** `str | None`

### log

Log a single event to the current experiment. The event will be batched and uploaded behind the scenes.

**event** `Any`

### login

Log into Braintrust. This will prompt you for your API token, which you can find at [https://www.braintrust.dev/app/token](https://www.braintrust.dev/app/token). This method is called automatically by `init()`.

**app\_url** `str | None`

The URL of the Braintrust App. Defaults to [https://www.braintrust.dev](https://www.braintrust.dev).

**api\_key** `str | None`

The API key to use. If the parameter is not specified, will try to use the `BRAINTRUST_API_KEY` environment variable. If no API key is specified, will prompt the user to login.

**org\_name** `str | None`

(Optional) The name of a specific organization to connect to. This is useful if you belong to multiple.

**force\_login** `bool`

Login again, even if you have already logged in (by default, this function will exit quickly if you have already logged in)

### parent\_context

Context manager to temporarily set the parent context for spans.

**parent** `str | dict | None`

The parent to set during the context. May be an exported slug
string or an opaque W3C trace-context dict (from
`extract_trace_context`).

**state** `BraintrustState | None`

Optional BraintrustState to use. If not provided, uses the global state.

### parse\_stream

Parse a BraintrustStream into its final value.

**stream** `BraintrustStream`

The BraintrustStream to parse.

### patch\_openai

Patch OpenAI globally for Braintrust tracing.

### permalink

Format a permalink to the Braintrust application for viewing the span represented by the provided `slug`.

**slug** `str`

The identifier generated from `Span.export`.

**org\_name** `str | None`

The org name to use. If not provided, the org name will be inferred from the global login state.

**app\_url** `str | None`

The app URL to use. If not provided, the app URL will be inferred from the global login state.

### register\_otel\_flush

Register a callback to flush OTEL spans. This is called by the OTEL integration when it initializes a span processor/exporter.

**callback** `Any`

The async callback function to flush OTEL spans.

### register\_sandbox

Register a sandbox function with Braintrust.

**name** `str`

Group name for the sandbox functions.

**project** `str`

Name of the project to register the sandbox in.

**sandbox** `SandboxConfig`

Sandbox configuration (provider and snapshot reference).

**sandbox.provider** `Literal['modal']` *required*

**sandbox.snapshot\_ref** `str` *required*

**entrypoints** `Sequence[str] | None`

Optional list of entrypoints available in the sandbox.

**description** `str | None`

Optional description.

**metadata** `dict[str, Any] | None`

Optional metadata dict.

**if\_exists** `IfExists | None`

What to do if function already exists. Defaults to "replace".

**api\_key** `str | None`

Braintrust API key. Uses BRAINTRUST\_API\_KEY env var if not provided.

**app\_url** `str | None`

Braintrust app URL. Uses default if not provided.

**org\_name** `str | None`

Organization name.

**state** `BraintrustState | None`

Optional BraintrustState instance. Defaults to the global state.

### run\_evaluator

Run an evaluator, cancelling its work on timeout or when cancel\_event is set.

**experiment** `Experiment | None`

**evaluator** `Evaluator[Input, Output, Expected]`

**evaluator.project\_name** `str` *required*

**evaluator.eval\_name** `str` *required*

**evaluator.data** `EvalData[Input, Expected]` *required*

**evaluator.task** `EvalTask[Input, Output, Expected]` *required*

**evaluator.scores** `Sequence[EvalScorer[Input, Output, Expected]]` *required*

**evaluator.experiment\_name** `str | None` *required*

**evaluator.metadata** `Metadata | None` *required*

**evaluator.tags** `Sequence[str] | None` *required*

**evaluator.trial\_count** `int` *required*

**evaluator.is\_public** `bool` *required*

**evaluator.update** `bool` *required*

**evaluator.timeout** `float | None` *required*

**evaluator.max\_concurrency** `int | None` *required*

**evaluator.project\_id** `str | None` *required*

**evaluator.base\_experiment\_name** `str | None` *required*

**evaluator.base\_experiment\_id** `str | None` *required*

**evaluator.git\_metadata\_settings** `GitMetadataSettings | None` *required*

**evaluator.repo\_info** `RepoInfo | None` *required*

**evaluator.error\_score\_handler** `ErrorScoreHandler[Input, Expected] | None` *required*

**evaluator.description** `str | None` *required*

**evaluator.summarize\_scores** `bool` *required*

**evaluator.parameters** `EvalParameters | RemoteEvalParameters | None` *required*

**evaluator.classifiers** `list[EvalClassifier[Input, Output, Expected]] | None` *required*

**evaluator.parameter\_values** `dict[str, Any] | None` *required*

**evaluator.cancel\_event** `threading.Event | None` *required*

**position** `int | None`

**filters** `list[Filter]`

**stream** `Callable[[SSEProgressEvent], None] | None`

**state** `BraintrustState | None`

**enable\_cache** `bool`

### set\_http\_adapter

Specify a custom HTTP adapter to use for all network requests. This is useful for setting custom retry policies, timeouts, etc. Braintrust uses the `requests` library, so the adapter should be an instance of `requests.adapters.HTTPAdapter`. Alternatively, consider sub-classing our `RetryRequestExceptionsAdapter` to get automatic retries on network-related exceptions.

**adapter** `HTTPAdapter`

The adapter to use.

### set\_masking\_function

Set a global masking function that will be applied to all logged data before sending to Braintrust. The masking function will be applied after records are merged but before they are sent to the backend. Internally, masking is a logger-local export customizer that runs after span customizers and also covers non-span records such as datasets and feedback.

**masking\_function** `Callable[[Any], Any] | None`

A function that takes a JSON-serializable object and returns a masked version. Set to None to disable masking.

### set\_span\_customizers

Replace the process-wide ordered customizer list with a snapshot.

**customizers** `Sequence[SpanCustomizer] | None`

### set\_thread\_pool\_max\_workers

Set the maximum number of threads to use for running evaluators. By default, this is the number of CPUs on the machine.

**max\_workers** `Any`

### span\_components\_to\_object\_id

Utility function to resolve the object ID of a SpanComponentsV4 object. This function may trigger a login to braintrust if the object ID is encoded lazily.

**components** `SpanComponentsV4`

**components.object\_type** `SpanObjectTypeV3` *required*

**components.object\_id** `str | None` *required*

**components.compute\_object\_metadata\_args** `dict | None` *required*

**components.row\_id** `str | None` *required*

**components.span\_id** `str | None` *required*

**components.root\_span\_id** `str | None` *required*

**components.propagated\_event** `dict | None` *required*

### start\_span

Lower-level alternative to `@traced` for starting a span at the toplevel. It creates a span under the first active object (using the same precedence order as `@traced`), or if `parent` is specified, under the specified parent row, or returns a no-op span object.

**name** `str | None`

**type** `SpanTypeAttribute | None`

**span\_attributes** `SpanAttributes | Mapping[str, Any] | None`

**start\_time** `float | None`

**set\_current** `bool | None`

**parent** `str | dict[str, str] | None`

**propagated\_event** `dict[str, Any] | None`

**state** `BraintrustState | None`

**internal** `SpanInternalOptions | None`

**event** `Any`

### summarize

Summarize the current experiment, including the scores (compared to the closest reference experiment) and metadata.

**summarize\_scores** `bool`

Whether to summarize the scores. If False, only the metadata will be returned.

**comparison\_experiment\_id** `str | None`

The experiment to compare against. If None, the backend uses the experiment's stored base experiment, then the most recent experiment in the same project.

### traced

Decorator to trace the wrapped function. Can either be applied bare (`@traced`) or by providing arguments (`@traced(*span_args, **span_kwargs)`), which will be forwarded to the created span. See `Span.start_span` for full details on the span arguments.

**span\_args** `Any`

**span\_kwargs** `Any`

### update\_span

Update a span using the output of `span.export()`. It is important that you only resume updating to a span once the original span has been fully written and flushed, since otherwise updates to the span may conflict with the original span.

**exported** `str`

The output of `span.export()`.

**event** `Any`

## Classes

### AsyncScorerLike

Protocol for asynchronous scorers that implement the eval\_async interface. The framework will prefer this interface if available.

Methods

`eval_async()`

### Attachment

Represents an attachment to be uploaded and the associated metadata.

Properties

**reference** `AttachmentReference`

The object that replaces this `Attachment` at upload time.

**data** `bytes`

The attachment contents. This is a lazy value that will read the attachment contents from disk or memory on first access.

Methods

`__init__()`, `upload()`, `debug_info()`

### BaseExperiment

Use this to specify that the dataset should actually be the data from a previous (base) experiment. If you do not specify a name, Braintrust will automatically figure out the best base experiment to use based on your git history (or fall back to timestamps).

Properties

**name** `str | None`

The name of the base experiment to use. If unspecified, Braintrust will automatically figure out the best base
using your git history (or fall back to timestamps).

Methods

`__init__()`

### BraintrustConsoleChunk

A console chunk from a Braintrust stream.

Properties

**message** `str`

**stream** `Literal['stderr', 'stdout']`

**type** `Literal['console']`

Methods

`__init__()`

### BraintrustErrorChunk

An error chunk from a Braintrust stream.

Properties

**data** `str`

**type** `Literal['error']`

Methods

`__init__()`

### BraintrustInvokeError

An error that occurs during a Braintrust stream.

### BraintrustJsonChunk

A chunk of JSON data from a Braintrust stream.

Properties

**data** `str`

**type** `Literal['json_delta']`

Methods

`__init__()`

### BraintrustLogHandler

Forward Python `logging` records to a Braintrust logger.

Methods

`__init__()`, `emit()`, `flush()`

### BraintrustProgressChunk

A progress chunk from a Braintrust stream.

Properties

**data** `str`

**id** `str`

**object\_type** `str`

**format** `str`

**output\_type** `str`

**name** `str`

**event** `Literal['json_delta', 'text_delta', 'reasoning_delta']`

**type** `Literal['progress']`

Methods

`__init__()`

### BraintrustStream

A Braintrust stream. This is a wrapper around a generator of `BraintrustStreamChunk`, with synchronous and asynchronous iteration utilities for logging and conversion.

Properties

**stream** `Iterable[BraintrustStreamChunk]`

Methods

`__init__()`, `copy()`, `final_value()`, `final_value_async()`

### BraintrustTextChunk

A chunk of text data from a Braintrust stream.

Properties

**data** `str`

**type** `Literal['text_delta']`

Methods

`__init__()`

### ClassifierBuilder

Builder to create a classifier in Braintrust.

Properties

**project** `Any`

Methods

`__init__()`, `create()`

### CodeFunction

A generic callable, with metadata.

Properties

**project** `Project`

**handler** `Callable[..., Any]`

**name** `str`

**slug** `str`

**type\_** `str`

**description** `str | None`

**parameters** `Any`

**returns** `Any`

**if\_exists** `IfExists | None`

**metadata** `Metadata | None`

**tags** `Sequence[str] | None`

Methods

`__init__()`

### CodePrompt

A prompt defined in code, with metadata.

Properties

**project** `Project`

**name** `str`

**slug** `str`

**prompt** `PromptData`

**tool\_functions** `list[CodeFunction | SavedFunctionId]`

**description** `str | None`

**function\_type** `str | None`

**id** `str | None`

**if\_exists** `IfExists | None`

**metadata** `Metadata | None`

**tags** `Sequence[str] | None`

Methods

`to_function_definition()`, `__init__()`

### DataSummary

Summary of a dataset's data.

Properties

**new\_records** `int`

New or updated records added in this session.

**total\_records** `int`

Total records in the dataset.

Methods

`__init__()`

### Dataset

A dataset is a collection of records, such as model inputs and outputs, which represent data you can use to evaluate and fine-tune models. You can log production data to datasets, curate them with interesting examples, edit/delete records, and run evaluations against them.

Properties

**new\_records** `Any`

**state** `Any`

**id** `str`

**name** `str`

**data** `Any`

**project** `Any`

**logging\_state** `BraintrustState`

Methods

`__init__()`, `insert()`, `update()`, `delete()`, `summarize()`, `close()`, `flush()`

### DatasetPipelineDefinition

A registered dataset pipeline definition consumed by the bt CLI.

Properties

**source** `DatasetPipelineSource`

**transform** `DatasetPipelineTransform[Row]`

**target** `DatasetPipelineTarget`

**name** `str | None`

Methods

`__init__()`

### DatasetPipelineRow

A row returned by a dataset pipeline transform.

Properties

**id** `str`

Stable row ID for the target dataset. Defaults to the source span or trace ID.

**input** `Any | None`

Input value for the target dataset row.

**expected** `Any | None`

Expected value for the target dataset row.

**tags** `Sequence[str] | None`

Tags for the target dataset row.

**metadata** `Metadata | None`

Metadata for the target dataset row.

### DatasetPipelineSource

Information about what spans or traces should be passed into the dataset pipeline.

Properties

**project\_id** `str`

Project ID to take spans or traces from. Takes precedence over project\_name.

**project\_name** `str`

Project name to take spans or traces from.

**org\_name** `str`

Organization name to take spans or traces from.

**filter** `str`

Optional BTQL filter. When omitted, all spans or traces are eligible.

**scope** `DatasetPipelineScope`

Whether to pass spans or entire traces to the pipeline. Defaults to "span".

### DatasetPipelineTarget

Information about the target dataset.

Properties

**dataset\_name** `str`

Dataset name. This can be an existing dataset name or a name to create.

**project\_id** `NotRequired[str]`

Project ID where the dataset lives or should be created.

**project\_name** `NotRequired[str]`

Project name where the dataset lives or should be created.

**org\_name** `NotRequired[str]`

Organization name where the dataset lives or should be created.

**description** `NotRequired[str]`

Dataset description to use when creating the dataset.

**metadata** `NotRequired[Metadata]`

Dataset metadata to use when creating the dataset.

### DatasetPipelineTransformArgs

Arguments passed to a dataset pipeline transform.

Properties

**id** `str`

Source span row ID for span-scoped transforms.

**input** `Any | None`

Source span input for span-scoped transforms.

**output** `Any | None`

Source span output for span-scoped transforms.

**metadata** `Metadata | None`

Source span metadata for span-scoped transforms.

**expected** `Any | None`

Source span expected value for span-scoped transforms.

**trace** `Trace`

Source trace. This is always available.

### DatasetRef

Reference to a dataset by ID and optional version.

Properties

**id** `str`

**version** `str`

### DatasetSummary

Summary of a dataset's scores and metadata.

Properties

**project\_name** `str`

Name of the project that the dataset belongs to.

**dataset\_name** `str`

Name of the dataset.

**project\_url** `str`

URL to the project's page in the Braintrust app.

**dataset\_url** `str`

URL to the experiment's page in the Braintrust app.

**data\_summary** `DataSummary | None`

Summary of the dataset's data.

Methods

`__init__()`

### EvalCase

An evaluation case. This is a single input to the evaluation task, along with an optional expected output, metadata, and tags.

Properties

**input** `Input`

**expected** `Expected | None`

**metadata** `Metadata | None`

**tags** `Sequence[str] | None`

**trial\_count** `int | None`

**id** `str | None`

**created** `str | None`

**origin** `ObjectReference | None`

**upsert\_id** `str | None`

Methods

`__init__()`

### EvalHooks

An object that can be used to add metadata to an evaluation. This is passed to the `task` function.

Properties

**cancel\_event** `threading.Event`

Set when the evaluation is cancelled or times out.

**metadata** `Metadata | None`

The metadata object for the current evaluation. You can mutate this object to add or remove metadata.

**expected** `Expected | None`

The expected output for the current evaluation.

**span** `Span`

Access the span under which the task is run. Also accessible via braintrust.current\_span()

**trial\_index** `int`

The index of the current trial (0-based). This is useful when trial\_count > 1.

**tags** `Sequence[str]`

The tags for the current evaluation. You can mutate this object to add or remove tags.

**parameters** `ValidatedParameters | None`

The parameters for the current evaluation. These are the validated parameter values
that were passed to the evaluator.

Methods

`report_progress()`, `meta()`

### EvalResult

The result of an evaluation. This includes the input, expected output, actual output, and metadata.

Properties

**input** `Input`

**output** `Output`

**scores** `Mapping[str, float | None]`

**classifications** `dict[str, list[ClassificationItem]] | None`

**expected** `Expected | None`

**metadata** `Metadata | None`

**tags** `list[str] | None`

**error** `Exception | None`

**exc\_info** `str | None`

**origin** `ObjectReference | None`

Methods

`__init__()`

### EvalScorerArgs

Arguments passed to an evaluator scorer. This includes the input, expected output, actual output, metadata, tags, and evaluation case ID.

Properties

**input** `Input`

**output** `Output`

**expected** `Expected | None`

**metadata** `Metadata | None`

**id** `str | None`

**tags** `Sequence[str] | None`

### Evaluator

An evaluator is an abstraction that defines an evaluation dataset, a task to run on the dataset, and a set of scorers to evaluate the results of the task. Each method attribute can be synchronous or asynchronous (for optimal performance, it is recommended to provide asynchronous implementations).

Properties

**project\_name** `str`

The name of the project the eval falls under.

**eval\_name** `str`

A name that describes the experiment. You do not need to change it each time the experiment runs.

**data** `EvalData[Input, Expected]`

Returns an iterator over the evaluation dataset. Each element of the iterator should be an `EvalCase` or a dict
with the same fields as an `EvalCase` (`input`, `expected`, `metadata`).

**task** `EvalTask[Input, Output, Expected]`

Runs the evaluation task on a single input. The `hooks` object can be used to add metadata to the evaluation.

**scores** `Sequence[EvalScorer[Input, Output, Expected]]`

A list of scorers to evaluate the results of the task. Each scorer can be a Scorer object or a function
that takes `input`, `output`, and `expected` arguments. It can return a number, a `Score` object,
a `ScoreDict`, or a sequence of named scores. A single dict can omit `name` to use the scorer's name;
each score in a sequence must include a name. The function can be async.

**experiment\_name** `str | None`

Optional experiment name. If not specified, a name will be generated automatically.

**metadata** `Metadata | None`

A dictionary with additional data about the test example, model outputs, or just about anything else that's
relevant, that you can use to help find and analyze examples later. For example, you could log the `prompt`,
example's `id`, or anything else that would be useful to slice/dice later. The values in `metadata` can be any
JSON-serializable type, but its keys must be strings.

**tags** `Sequence[str] | None`

Optional list of tags for the experiment

**trial\_count** `int`

The number of times to run the evaluator per input. This is useful for evaluating applications that
have non-deterministic behavior and gives you both a stronger aggregate measure and a sense of the
variance in the results.

**is\_public** `bool`

Whether the experiment should be public. Defaults to false.

**update** `bool`

Whether to update an existing experiment with `experiment_name` if one exists. Defaults to false.

**timeout** `float | None`

The duration, in seconds, after which to time out the evaluation.
Defaults to None, in which case there is no timeout.

**max\_concurrency** `int | None`

The maximum number of tasks/scorers that will be run concurrently.
Defaults to None, in which case there is no max concurrency.

**project\_id** `str | None`

If specified, uses the given project ID instead of the evaluator's name to identify the project.

**base\_experiment\_name** `str | None`

An optional experiment name to use as a base. If specified, the new experiment will be summarized and
compared to this experiment.

**base\_experiment\_id** `str | None`

An optional experiment id to use as a base. If specified, the new experiment will be summarized and
compared to this experiment. This takes precedence over `base_experiment_name` if specified.

**git\_metadata\_settings** `GitMetadataSettings | None`

Optional settings for collecting git metadata. By default, defers entirely
to org-level settings returned by the control plane; no git metadata is
collected if the org has not configured any.

**repo\_info** `RepoInfo | None`

Optionally explicitly specify the git metadata for this experiment. This
takes precedence over `git_metadata_settings` if specified.

**error\_score\_handler** `ErrorScoreHandler[Input, Expected] | None`

Optionally supply a custom function to specifically handle score values when tasks or scoring functions have errored.
A default implementation is exported as `default_error_score_handler` which will log a 0 score to the root span for any scorer that was not run.

**description** `str | None`

An optional description for the experiment.

**summarize\_scores** `bool`

Whether to summarize the scores of the experiment after it has run.

**parameters** `EvalParameters | RemoteEvalParameters | None`

A set of parameters that will be passed to the evaluator.
Can be used to define prompts or other configurable values.

**classifiers** `list[EvalClassifier[Input, Output, Expected]] | None`

Optional list of classifiers to evaluate the task output. Classifier results are
recorded under the `classifications` field instead of `scores`.

**parameter\_values** `dict[str, Any] | None`

**cancel\_event** `threading.Event | None`

Optional event that cancels the evaluation when set. Also set on timeout.

Methods

`__init__()`

### Experiment

An experiment is a collection of logged events, such as model inputs and outputs, which represent a snapshot of your application at a particular point in time. An experiment is meant to capture more than just the model you use, and includes the data you use to test, pre- and post- processing code, comparison metrics (scores), and any other metadata you want to include.

Properties

**dataset** `Any`

**last\_start\_time** `Any`

**state** `Any`

**id** `str`

**name** `str`

**data** `Mapping[str, Any]`

**project** `ObjectMetadata`

**logging\_state** `BraintrustState`

Methods

`__init__()`, `log()`, `log_feedback()`, `start_span()`, `update_span()`, `fetch_base_experiment()`, `summarize()`, `export()`, `close()`, `flush()`

### ExperimentSummary

Summary of an experiment's metadata and comparison result.

Properties

**project\_name** `str`

Name of the project that the experiment belongs to.

**project\_id** `str | None`

ID of the project. May be `None` if the eval was run locally.

**experiment\_id** `str | None`

ID of the experiment. May be `None` if the eval was run locally.

**experiment\_name** `str`

Name of the experiment.

**project\_url** `str | None`

URL to the project's page in the Braintrust app.

**experiment\_url** `str | None`

URL to the experiment's page in the Braintrust app.

**comparison\_experiment\_name** `str | None`

The experiment scores are baselined against.

**comparison** `SummaryResult`

The primary result to inspect when consuming an experiment summary.

A successful comparison contains the score and metric maps. A skipped
comparison explains why comparison data is unavailable, so callers do not
mistake it for a successful summary with no scores.

**scores** `dict[str, ScoreSummary]`

Deprecated. Use `comparison.scores` after checking its status.

**metrics** `dict[str, MetricSummary]`

Deprecated. Use `comparison.metrics` after checking its status.

Methods

`as_dict()`, `from_dict_deep()`, `__init__()`

### ExternalAttachment

Represents an attachment that resides in an external object store and the associated metadata.

Properties

**reference** `AttachmentReference`

The object that replaces this `Attachment` at upload time.

**data** `bytes`

The attachment contents. This is a lazy value that will read the attachment contents from the external object store on first access.

Methods

`__init__()`, `upload()`, `debug_info()`

### JSONAttachment

A convenience class for creating attachments from JSON-serializable objects.

Methods

`__init__()`

### MetricSummary

Summary of a metric's performance.

Properties

**name** `str`

Name of the metric.

**metric** `float | int`

Average metric across all examples.

**unit** `str`

Unit label for the metric.

**improvements** `int | None`

Number of improvements in the metric.

**regressions** `int | None`

Number of regressions in the metric.

**diff** `float | None`

Difference in metric between the current and reference experiment.

Methods

`__init__()`

### ParametersBuilder

Builder to create saved parameters in Braintrust.

Properties

**project** `Any`

Methods

`__init__()`, `create()`

### ParametersRef

Reference to saved parameters by ID and optional version.

Properties

**id** `str`

**version** `str`

### Project

A handle to a Braintrust project.

Properties

**name** `Any`

**project\_group\_name** `Any`

**tools** `Any`

**prompts** `Any`

**parameters** `Any`

**scorers** `Any`

**classifiers** `Any`

Methods

`__init__()`, `add_code_function()`, `add_prompt()`, `add_parameters()`, `publish()`

### ProjectBuilder

Creates handles to Braintrust projects.

Methods

`create()`

### Prompt

A prompt object consists of prompt text, a model, and model parameters (such as temperature), which can be used to generate completions or chat messages. The prompt object supports calling `.build()` which uses mustache templating to build the prompt with the given formatting options and returns a plain dictionary that includes the built prompt and arguments. The dictionary can be passed as kwargs to the OpenAI client or modified as you see fit.

Properties

**defaults** `Any`

**no\_trace** `Any`

**id** `str`

**name** `str`

**slug** `str`

**prompt** `PromptBlockData | None`

**version** `str`

**options** `PromptOptions`

Methods

`__init__()`, `from_prompt_data()`, `build()`

### PromptBuilder

Builder to create a prompt in Braintrust.

Properties

**project** `Any`

Methods

`__init__()`, `create()`

### ReadonlyAttachment

A readonly alternative to `Attachment`, which can be used for fetching already-uploaded Attachments.

Properties

**reference** `Any`

**data** `bytes`

The attachment contents. This is a lazy value that will read the attachment contents from the object store on first access.

Methods

`__init__()`, `metadata()`, `status()`

### ReadonlyExperiment

A read-only view of an experiment, initialized by passing `open=True` to `init()`.

Properties

**state** `Any`

**id** `str`

**logging\_state** `BraintrustState`

Methods

`__init__()`, `as_dataset()`

### RegisterSandboxResult

Result of registering a sandbox.

Properties

**project\_id** `str`

Project ID the sandbox is registered in.

**functions** `list[RegisteredSandboxFunction]`

Registered eval functions discovered from this sandbox.

Methods

`__init__()`

### RegisteredSandboxFunction

Registered eval function discovered from sandbox list endpoint.

Properties

**eval\_name** `str`

Eval name discovered in the sandbox.

**id** `str`

Unique identifier for the function.

**name** `str`

Function name.

**slug** `str`

URL-friendly identifier.

Methods

`__init__()`

### RepoInfo

Information about the current HEAD of the repo.

Properties

**commit** `str | None`

**branch** `str | None`

**tag** `str | None`

**dirty** `bool | None`

**author\_name** `str | None`

**author\_email** `str | None`

**commit\_message** `str | None`

**commit\_time** `str | None`

**git\_diff** `str | None`

Methods

`__init__()`

### ReporterDef

A reporter takes an evaluator and its result and returns a report.

Properties

**name** `str`

The name of the reporter.

**report\_eval** `Callable[[Evaluator[Input, Output, Expected], EvalResultWithSummary[Input, Output, Expected], bool, bool], EvalReport | Awaitable[EvalReport]]`

A function that takes an evaluator and its result and returns a report.

**report\_run** `Callable[[list[EvalReport], bool, bool], bool | Awaitable[bool]]`

A function that takes all evaluator results and returns a boolean indicating whether the run was successful.
If you return false, the `braintrust eval` command will exit with a non-zero status code.

Methods

`__init__()`

### SSEProgressEvent

A progress event that can be reported during task execution, specifically for SSE (Server-Sent Events) streams. This is a subclass of TaskProgressEvent with additional fields for SSE-specific metadata.

Properties

**id** `str`

**object\_type** `str`

**origin** `NotRequired[ObjectReference]`

**name** `str`

### SandboxConfig

Configuration for a sandbox runtime.

Properties

**provider** `Literal['modal']`

The sandbox provider. Currently only "modal" is supported.

**snapshot\_ref** `str`

Reference to the sandbox snapshot.

Methods

`__init__()`

### ScoreSummary

Summary of a score's performance.

Properties

**name** `str`

Name of the score.

**score** `float`

Average score across all examples.

**improvements** `int | None`

Number of improvements in the score.

**regressions** `int | None`

Number of regressions in the score.

**diff** `float | None`

Difference in score between the current and reference experiment.

Methods

`__init__()`

### ScorerBuilder

Builder to create a scorer in Braintrust.

Properties

**project** `Any`

Methods

`__init__()`, `create()`

### Span

A Span encapsulates logged data and metrics for a unit of work. This interface is shared by all span implementations.

Properties

**id** `str`

Row ID of the span.

**name** `str`

Name of the span, for display purposes only.

Methods

`log()`, `log_feedback()`, `start_span()`, `export()`, `inject()`, `link()`, `permalink()`, `end()`, `flush()`, `close()`, `set_attributes()`, `set_current()`, `unset_current()`

### SpanCustomizer

Extensible hooks for all SDK spans. Omitted hooks are no-ops.

Methods

`on_span_export()`

### SpanIds

The three IDs that define a span's position in the trace tree.

Properties

**span\_id** `str`

**root\_span\_id** `str`

**span\_parents** `list[str] | None`

Methods

`__init__()`

### SpanImpl

Primary implementation of the `Span` interface. See the `Span` interface for full details on each method.

Properties

**can\_set\_current** `bool`

**state** `Any`

**parent\_object\_type** `Any`

**parent\_object\_id** `Any`

**parent\_compute\_object\_metadata\_args** `Any`

**propagated\_event** `Any`

**propagated\_state** `Any`

**span\_id** `Any`

**root\_span\_id** `Any`

**span\_parents** `Any`

**id** `str`

**name** `str`

Methods

`__init__()`, `set_attributes()`, `log()`, `log_internal()`, `log_feedback()`, `start_span()`, `end()`, `export()`, `inject()`, `link()`, `permalink()`, `close()`, `flush()`, `set_current()`, `unset_current()`

### SpanInternalOptions

Options reserved for Braintrust SDK internals.

Properties

**instrumentation** `str`

Identifier for the instrumentation code creating the span. Stamped into
`context.span_origin.instrumentation.name`. Set by SDK integrations
(`openai-auto`, `anthropic-auto`, etc.).

**initial\_span\_write\_as\_merge** `bool`

Emit the initial row as a merge for an SDK-owned deterministic span.

**skip\_initial\_span\_write** `bool`

Rehydrate an existing span without emitting an initial row.

**span\_id** `str`

SDK-controlled span ID.

**root\_span\_id** `str`

SDK-controlled root span ID.

### SpanScope

Scope for operating on a single span.

Properties

**type** `Literal['span']`

**id** `str`

**root\_span\_id** `str`

### SummarySkipped

An experiment comparison that was skipped or could not be retrieved.

Properties

**reason** `str`

**status** `Literal['skipped']`

Methods

`__init__()`

### SummarySuccess

A successfully retrieved experiment comparison.

Properties

**scores** `dict[str, ScoreSummary]`

**metrics** `dict[str, MetricSummary]`

**status** `Literal['success']`

Methods

`__init__()`

### SyncScorerLike

Protocol for synchronous scorers that implement the callable interface. This is the most common interface and is used when no async version is available.

Methods

`__call__()`

### TaskProgressEvent

Progress event that can be reported during task execution.

Properties

**format** `FunctionFormat`

**output\_type** `FunctionOutputType`

**event** `Literal['reasoning_delta', 'text_delta', 'json_delta', 'error', 'console', 'start', 'done', 'progress']`

**data** `str`

### ToolBuilder

Builder to create a tool in Braintrust.

Properties

**project** `Any`

Methods

`__init__()`, `create()`

### TraceScope

Scope for operating on an entire trace.

Properties

**type** `Literal['trace']`

**root\_span\_id** `str`

## Integration modules

### braintrust.integrations.adk

Braintrust integration for Google ADK.

Import from `braintrust.integrations.adk`.

#### adk.ADKIntegration

Braintrust instrumentation for Google ADK (Agent Development Kit).

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### adk.setup\_adk

Setup Braintrust integration with Google ADK. Will automatically patch Google ADK agents, runners, flows, and MCP tools for automatic tracing.

**api\_key** `str | None`

Braintrust API key.

**project\_id** `str | None`

Braintrust project ID.

**project\_name** `str | None`

Braintrust project name.

**SpanProcessor** `type | None`

Deprecated parameter.

#### adk.setup\_braintrust

**args** `Any`

**kwargs** `Any`

#### adk.wrap\_agent

Manually patch an agent class for tracing.

**Agent** `Any`

#### adk.wrap\_runner

Manually patch a runner class for tracing.

**Runner** `Any`

#### adk.wrap\_workflow

Manually patch ADK 2.x workflow nodes for tracing.

**Workflow** `Any`

#### adk.wrap\_flow

Manually patch a flow class for tracing.

**Flow** `Any`

#### adk.wrap\_mcp\_tool

Manually patch an MCP tool class for tracing.

**McpTool** `Any`

### braintrust.integrations.agentscope

Braintrust integration for AgentScope.

Import from `braintrust.integrations.agentscope`.

#### agentscope.AgentScopeIntegration

Braintrust instrumentation for AgentScope. Requires AgentScope v1.0.0 or higher.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### agentscope.setup\_agentscope

Setup Braintrust integration with AgentScope.

**api\_key** `str | None`

**project\_id** `str | None`

**project\_name** `str | None`

### braintrust.integrations.agno

Braintrust integration for Agno.

Import from `braintrust.integrations.agno`.

#### agno.AgnoIntegration

Braintrust instrumentation for Agno.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### agno.setup\_agno

Setup Braintrust integration with Agno. Will automatically patch Agno agents, models, function calls, and evals (`agno.eval`) for tracing.

**api\_key** `str | None`

Braintrust API key (optional, can use env var BRAINTRUST\_API\_KEY)

**project\_id** `str | None`

Braintrust project ID (optional)

**project\_name** `str | None`

Braintrust project name (optional; defaults to the Global project)

**eval\_experiments** `bool | None`

Whether an eval suite run should open a Braintrust experiment,
so its cases land as experiment rows rather than logs. Defaults to the
BRAINTRUST\_AGNO\_EVAL\_EXPERIMENTS env var, which itself defaults to true.
Individual evals (AccuracyEval and friends) always log to whatever is
current, so pass eval\_experiments=False to keep suite runs in logs too.

#### agno.wrap\_accuracy\_eval

Manually patch an AccuracyEval class for tracing.

**AccuracyEval** `Any`

#### agno.wrap\_agent

Manually patch an Agent class for tracing.

**Agent** `Any`

#### agno.wrap\_agent\_as\_judge\_eval

Manually patch an AgentAsJudgeEval class for tracing.

**AgentAsJudgeEval** `Any`

#### agno.wrap\_eval\_suite

Manually patch the `agno.eval.suite` module for tracing.

**suite\_module** `Any`

#### agno.wrap\_function\_call

Manually patch a FunctionCall class for tracing.

**FunctionCall** `Any`

#### agno.wrap\_model

Manually patch a Model class for tracing.

**Model** `Any`

#### agno.wrap\_performance\_eval

Manually patch a PerformanceEval class for tracing.

**PerformanceEval** `Any`

#### agno.wrap\_reliability\_eval

Manually patch a ReliabilityEval class for tracing.

**ReliabilityEval** `Any`

#### agno.wrap\_team

Manually patch a Team class for tracing.

**Team** `Any`

#### agno.wrap\_workflow

Manually patch a Workflow class for tracing.

**Workflow** `Any`

### braintrust.integrations.ai\_sdk

Braintrust integration for Vercel AI SDK for Python.

Import from `braintrust.integrations.ai_sdk`.

#### ai\_sdk.AISDKIntegration

Braintrust instrumentation for Vercel AI SDK for Python.

Properties

**name** `Any`

**import\_names** `Any`

**distribution\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### ai\_sdk.BraintrustAISDKAdapter

Adapter for `ai.experimental_telemetry`'s callback protocol.

Methods

`__init__()`, `on_span_start()`, `on_span_event()`, `on_span_end()`

#### ai\_sdk.patch\_ai\_sdk

Register the telemetry adapter without initializing a Braintrust logger.

#### ai\_sdk.setup\_ai\_sdk

Initialize Braintrust and register the Vercel AI SDK telemetry adapter.

**api\_key** `str | None`

**project\_id** `str | None`

**project\_name** `str | None`

#### ai\_sdk.unpatch\_ai\_sdk

Unregister the Braintrust telemetry adapter. Safe to call repeatedly.

### braintrust.integrations.anthropic

Import from `braintrust.integrations.anthropic`.

#### anthropic.AnthropicIntegration

Braintrust instrumentation for the Anthropic Python SDK on anthropic>=0.48.0.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### anthropic.wrap\_anthropic\_client

**client** `Any`

### braintrust.integrations.autogen

Braintrust AutoGen integration.

Import from `braintrust.integrations.autogen`.

#### autogen.AutoGenIntegration

Braintrust instrumentation for Microsoft AutoGen AgentChat.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### autogen.setup\_autogen

Setup Braintrust integration with AutoGen.

**api\_key** `str | None`

**project\_id** `str | None`

**project\_name** `str | None`

### braintrust.integrations.bedrock\_runtime

Public entry points for the boto3 Bedrock Runtime integration.

Import from `braintrust.integrations.bedrock_runtime`.

#### bedrock\_runtime.BedrockRuntimeIntegration

Braintrust instrumentation for boto3 Bedrock Runtime clients.

Properties

**name** `Any`

**import\_names** `Any`

**distribution\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### bedrock\_runtime.setup\_bedrock

Patch botocore client creation to auto-wrap Bedrock Runtime clients.

#### bedrock\_runtime.wrap\_bedrock

Instrument a boto3 Bedrock Runtime client instance in place.

**client** `Any`

### braintrust.integrations.claude\_agent\_sdk

Braintrust integration for Claude Agent SDK with automatic tracing.

Import from `braintrust.integrations.claude_agent_sdk`.

#### claude\_agent\_sdk.ClaudeAgentSDKIntegration

Braintrust instrumentation for the Claude Agent SDK.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### claude\_agent\_sdk.setup\_claude\_agent\_sdk

Setup Braintrust integration with Claude Agent SDK. Will automatically patch the SDK for automatic tracing.

**api\_key** `str | None`

Braintrust API key.

**project\_id** `str | None`

Braintrust project ID.

**project** `str | None`

Braintrust project name.

### braintrust.integrations.cohere

Braintrust integration for the Cohere Python SDK.

Import from `braintrust.integrations.cohere`.

#### cohere.CohereIntegration

Braintrust instrumentation for the Cohere Python SDK.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### cohere.wrap\_cohere

Wrap a Cohere client instance for Braintrust tracing.

**client** `Any`

### braintrust.integrations.crewai

Braintrust integration for CrewAI.

Import from `braintrust.integrations.crewai`.

#### crewai.BraintrustCrewAIListener

CrewAI event-bus listener that maps events into Braintrust spans.

#### crewai.CrewAIIntegration

Braintrust instrumentation for CrewAI.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### crewai.patch\_crewai

Register the Braintrust CrewAI listener without initializing a logger.

#### crewai.setup\_crewai

Set up Braintrust tracing for CrewAI.

**api\_key** `str | None`

Braintrust API key (optional, `BRAINTRUST_API_KEY` env var works too).

**project\_id** `str | None`

Braintrust project id (optional).

**project\_name** `str | None`

Braintrust project name (optional, `BRAINTRUST_PROJECT` env var works too).

### braintrust.integrations.cursor\_sdk

Braintrust tracing integration for the Cursor Python SDK.

Import from `braintrust.integrations.cursor_sdk`.

#### cursor\_sdk.CursorSDKIntegration

Braintrust instrumentation for Cursor's Python agent SDK.

Properties

**name** `Any`

**import\_names** `Any`

**distribution\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### cursor\_sdk.setup\_cursor\_sdk

Patch Cursor SDK agent runs for Braintrust tracing.

**api\_key** `str | None`

**project\_id** `str | None`

**project** `str | None`

#### cursor\_sdk.wrap\_cursor\_sdk\_agent

Instrument a Cursor `Agent` or `AsyncAgent` class in place.

**agent\_class** `Any`

### braintrust.integrations.dspy

Braintrust integration for DSPy.

Import from `braintrust.integrations.dspy`.

#### dspy.BraintrustDSpyCallback

Callback handler that logs DSPy execution traces to Braintrust.

Methods

`__init__()`, `on_lm_start()`, `on_lm_end()`, `on_module_start()`, `on_module_end()`, `on_adapter_format_start()`, `on_adapter_format_end()`, `on_adapter_parse_start()`, `on_adapter_parse_end()`, `on_tool_start()`, `on_tool_end()`, `on_evaluate_start()`, `on_evaluate_end()`

#### dspy.DSPyIntegration

Braintrust instrumentation for DSPy.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### dspy.patch\_dspy

Patch DSPy to automatically add Braintrust tracing callback.

### braintrust.integrations.google\_discoveryengine

Braintrust integration for google-cloud-discoveryengine v1.

Import from `braintrust.integrations.google_discoveryengine`.

#### google\_discoveryengine.GoogleDiscoveryEngineIntegration

Properties

**name** `Any`

**import\_names** `Any`

**distribution\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### google\_discoveryengine.setup\_google\_discoveryengine

Instrument supported v1 clients in this process.

#### google\_discoveryengine.wrap\_google\_discoveryengine

Instrument one v1 client instance, returning the same client.

**client** `Any`

### braintrust.integrations.google\_genai

Braintrust integration for Google GenAI.

Import from `braintrust.integrations.google_genai`.

#### google\_genai.GoogleGenAIIntegration

Braintrust instrumentation for the Google GenAI Python SDK.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### google\_genai.setup\_genai

Setup Braintrust integration with Google GenAI.

**api\_key** `str | None`

Braintrust API key.

**project\_id** `str | None`

Braintrust project ID.

**project\_name** `str | None`

Braintrust project name.

### braintrust.integrations.harbor

Braintrust's native Harbor job plugin.

Import from `braintrust.integrations.harbor`.

#### harbor.HarborPlugin

Harbor plugin that reconciles final trials into Braintrust experiments.

Properties

**config** `Any`

Methods

`__init__()`, `on_job_start()`, `on_job_end()`, `sync_job_directory()`

#### harbor.backfill\_job

Backfill a Harbor job directory into Braintrust.

**job\_dir** `str | Path`

**plugin\_options** `Any`

### braintrust.integrations.huggingface\_hub

Braintrust integration for the HuggingFace Hub Python SDK.

Import from `braintrust.integrations.huggingface_hub`.

#### huggingface\_hub.HuggingFaceHubIntegration

Braintrust instrumentation for the HuggingFace Hub Python SDK.

Properties

**name** `Any`

**import\_names** `Any`

**distribution\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### huggingface\_hub.wrap\_huggingface\_hub

Wrap an `InferenceClient` or `AsyncInferenceClient` for Braintrust tracing.

**client** `Any`

### braintrust.integrations.instructor

Braintrust integration for the Instructor structured-output library.

Import from `braintrust.integrations.instructor`.

#### instructor.InstructorIntegration

Braintrust instrumentation for the Instructor structured-output library.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### instructor.wrap\_instructor

Instrument an `instructor.Instructor` / `AsyncInstructor` client in place.

**client** `Any`

### braintrust.integrations.langchain

Braintrust integration for LangChain.

Import from `braintrust.integrations.langchain`.

#### langchain.BraintrustCallbackHandler

Methods

`__init__()`

#### langchain.LangChainIntegration

Braintrust instrumentation for LangChain via a global callback handler.

Properties

**name** `Any`

**import\_names** `Any`

**patchers** `Any`

#### langchain.set\_global\_handler

**handler** `Any`

#### langchain.setup\_langchain

Setup Braintrust integration with LangChain.

**api\_key** `str | None`

**project\_id** `str | None`

**project\_name** `str | None`

### braintrust.integrations.litellm

Braintrust LiteLLM integration.

Import from `braintrust.integrations.litellm`.

#### litellm.LiteLLMIntegration

Braintrust instrumentation for the LiteLLM Python SDK.

Properties

**name** `Any`

**import\_names** `Any`

**patchers** `Any`

#### litellm.patch\_litellm

Patch LiteLLM top-level callables to emit Braintrust spans.

#### litellm.wrap\_litellm

Wrap a LiteLLM module-like object in-place with Braintrust tracing.

**litellm** `Any`

### braintrust.integrations.livekit\_agents

Import from `braintrust.integrations.livekit_agents`.

#### livekit\_agents.LiveKitAgentsIntegration

Properties

**name** `Any`

**import\_names** `Any`

**distribution\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### livekit\_agents.setup\_livekit\_agents

Set up LiveKit Agents tracing.

#### livekit\_agents.wrap\_livekit\_agents

**target** `Any`

### braintrust.integrations.llamaindex

Braintrust integration for LlamaIndex.

Import from `braintrust.integrations.llamaindex`.

#### llamaindex.BraintrustSpanHandler

Methods

`__init__()`

#### llamaindex.LlamaIndexIntegration

Braintrust instrumentation for LlamaIndex via dispatcher handlers.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### llamaindex.setup\_llamaindex

**api\_key** `str | None`

**project\_id** `str | None`

**project\_name** `str | None`

### braintrust.integrations.mistral

Braintrust integration for the Mistral Python SDK.

Import from `braintrust.integrations.mistral`.

#### mistral.MistralIntegration

Braintrust instrumentation for the Mistral Python SDK.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### mistral.wrap\_mistral

Wrap a single Mistral client instance for tracing.

**client** `Any`

### braintrust.integrations.openai

Braintrust integration for the OpenAI Python SDK and OpenAI-compatible gateways.

Import from `braintrust.integrations.openai`.

#### openai.OpenAIIntegration

Braintrust instrumentation for the OpenAI Python SDK.

Properties

**name** `Any`

**import\_names** `Any`

**patchers** `Any`

#### openai.setup\_openai

Setup Braintrust integration with OpenAI.

**api\_key** `str | None`

Braintrust API key (optional, can use env var BRAINTRUST\_API\_KEY)

**project\_id** `str | None`

Braintrust project ID (optional)

**project\_name** `str | None`

Braintrust project name (optional, can use env var BRAINTRUST\_PROJECT)

#### openai.wrap\_openai

Manually wrap an OpenAI client instance for tracing.

**client** `Any`

### braintrust.integrations.openai\_agents

Braintrust integration for the OpenAI Agents SDK.

Import from `braintrust.integrations.openai_agents`.

#### openai\_agents.BraintrustTracingProcessor

Methods

`__init__()`

#### openai\_agents.OpenAIAgentsIntegration

Braintrust instrumentation for the OpenAI Agents SDK.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### openai\_agents.setup\_openai\_agents

Setup Braintrust tracing for the OpenAI Agents SDK.

**api\_key** `str | None`

**project\_id** `str | None`

**project** `str | None`

**project\_name** `str | None`

### braintrust.integrations.openrouter

Braintrust integration for the OpenRouter Python SDK.

Import from `braintrust.integrations.openrouter`.

#### openrouter.OpenRouterIntegration

Braintrust instrumentation for the OpenRouter Python SDK.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### openrouter.wrap\_openrouter

Wrap a single OpenRouter client instance for tracing.

**client** `Any`

### braintrust.integrations.pipecat

Braintrust integration for Pipecat AI.

Import from `braintrust.integrations.pipecat`.

#### pipecat.BraintrustPipecatObserver

Pipecat observer that emits Braintrust task, llm, and tool spans.

Properties

**capture\_audio\_attachments** `Any`

**trace\_turns** `Any`

Methods

`__init__()`, `on_pipeline_started()`, `on_process_frame()`, `on_push_frame()`, `cleanup()`

#### pipecat.PipecatIntegration

Braintrust instrumentation for Pipecat AI pipelines.

Properties

**name** `Any`

**import\_names** `Any`

**distribution\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### pipecat.setup\_pipecat

Set up Braintrust tracing for Pipecat `PipelineWorker` instances.

**api\_key** `str | None`

**project\_id** `str | None`

**project\_name** `str | None`

**capture\_audio\_attachments** `bool | None`

**capture\_user\_audio\_attachments** `bool | None`

**capture\_agent\_audio\_attachments** `bool | None`

**trace\_turns** `bool`

#### pipecat.wrap\_pipeline\_worker

Instrument a Pipecat `PipelineWorker` class directly.

**PipelineWorker** `Any`

### braintrust.integrations.pydantic\_ai

Braintrust integration for Pydantic AI.

Import from `braintrust.integrations.pydantic_ai`.

#### pydantic\_ai.PydanticAIIntegration

Braintrust instrumentation for Pydantic AI.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### pydantic\_ai.setup\_pydantic\_ai

Setup Braintrust integration with Pydantic AI. Will automatically patch Pydantic AI agents and direct API functions for automatic tracing.

**api\_key** `str | None`

Braintrust API key.

**project\_id** `str | None`

Braintrust project ID.

**project\_name** `str | None`

Braintrust project name.

#### pydantic\_ai.wrap\_agent

**Agent** `Any`

#### pydantic\_ai.wrap\_model\_classes

Deprecated compatibility shim for scanning currently loaded model subclasses.

#### pydantic\_ai.wrap\_model\_request

**original\_func** `Any`

#### pydantic\_ai.wrap\_model\_request\_sync

**original\_func** `Any`

#### pydantic\_ai.wrap\_model\_request\_stream

**original\_func** `Any`

#### pydantic\_ai.wrap\_model\_request\_stream\_sync

**original\_func** `Any`

### braintrust.integrations.strands

Braintrust integration for Strands Agents.

Import from `braintrust.integrations.strands`.

#### strands.StrandsIntegration

Braintrust instrumentation for Strands Agents.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### strands.setup\_strands

Set up Braintrust tracing for Strands Agents.

**api\_key** `str | None`

**project\_id** `str | None`

**project\_name** `str | None`

#### strands.wrap\_strands\_tracer

Manually patch a Strands `Tracer` class for Braintrust tracing.

**Tracer** `Any`

### braintrust.integrations.temporal

Braintrust integration for Temporal workflows and activities.

Import from `braintrust.integrations.temporal`.

#### temporal.BraintrustInterceptor

Braintrust interceptor for tracing Temporal workflows and activities.

Properties

**payload\_converter** `Any`

Methods

`__init__()`, `intercept_client()`, `intercept_activity()`, `workflow_interceptor_class()`

#### temporal.BraintrustPlugin

Braintrust plugin for Temporal that automatically configures tracing.

Methods

`__init__()`

#### temporal.TemporalIntegration

Braintrust instrumentation for Temporal workflows and activities.

Properties

**name** `Any`

**import\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### temporal.setup\_temporal

Set up Braintrust auto-instrumentation for Temporal.

**api\_key** `str | None`

**project\_id** `str | None`

**project\_name** `str | None`

### braintrust.integrations.transformers

Braintrust integration for local Hugging Face Transformers pipelines.

Import from `braintrust.integrations.transformers`.

#### transformers.TransformersIntegration

Instrument supported local `transformers` text pipelines.

Properties

**name** `Any`

**import\_names** `Any`

**distribution\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### transformers.setup\_transformers

Instrument supported Transformers pipeline classes.

#### transformers.wrap\_transformers

Manually instrument one supported pipeline instance or class.

**target** `_T`

### braintrust.integrations.typesafe

Braintrust integration for the TypeSafe Python SDK.

Import from `braintrust.integrations.typesafe`.

#### typesafe.TypeSafeIntegration

Braintrust instrumentation for the TypeSafe Python SDK.

Properties

**name** `Any`

**import\_names** `Any`

**distribution\_names** `Any`

**min\_version** `Any`

**patchers** `Any`

#### typesafe.setup\_typesafe

Instrument installed TypeSafe sync and async clients.

#### typesafe.wrap\_typesafe

Instrument a TypeSafe sync or async client in place.

**client** `Any`


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.