ReferenceAPI

GET

/v1/project

List projects

List out all projects. The projects are sorted by creation date, with the most recently-created projects coming first

Query Parameters

limitinteger

Limit the number of objects to return

Minimum: 0

starting_afterstring

Pagination cursor id.

For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before

Format: "uuid"

ending_beforestring

Pagination cursor id.

For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before

Format: "uuid"

project_namestring

Name of the project to search for

org_namestring

Filter search results to within a particular organization

idsAny properties in string, array of string

Filter search results to a particular set of object IDs. To specify a list of IDs, include the query param multiple times

Status codeDescription
200Returns a list of project objects
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X GET "https://api.braintrustdata.com/v1/project"
Example Response
{
  "objects": [
    {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
      "name": "string",
      "created": "2019-08-24T14:15:22Z",
      "deleted_at": "2019-08-24T14:15:22Z",
      "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
      "settings": {
        "comparison_key": "string"
      }
    }
  ]
}

POST

/v1/project

Create project

Create a new project. If there is an existing project with the same name as the one specified in the request, will return the existing project unmodified

Request Body

Any desired information about the new project object

name
Required
string

Name of the project

org_namestring | null

For nearly all users, this parameter should be unnecessary. But in the rare case that your API key belongs to multiple organizations, you may specify the name of the organization the project belongs in.

Status codeDescription
200Returns the new project object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/project" \
  -d '{
  "name": "string",
  "org_name": "string"
}'
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
  "name": "string",
  "created": "2019-08-24T14:15:22Z",
  "deleted_at": "2019-08-24T14:15:22Z",
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
  "settings": {
    "comparison_key": "string"
  }
}

PUT

/v1/project

DEPRECATED. Create or replace project

NOTE: This operation is deprecated and will be removed in a future revision of the API. Create or replace a new project. If there is an existing project with the same name as the one specified in the request, will return the existing project unmodified

Request Body

Any desired information about the new project object

name
Required
string

Name of the project

org_namestring | null

For nearly all users, this parameter should be unnecessary. But in the rare case that your API key belongs to multiple organizations, you may specify the name of the organization the project belongs in.

Status codeDescription
200Returns the new project object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X PUT "https://api.braintrustdata.com/v1/project" \
  -d '{
  "name": "string",
  "org_name": "string"
}'
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
  "name": "string",
  "created": "2019-08-24T14:15:22Z",
  "deleted_at": "2019-08-24T14:15:22Z",
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
  "settings": {
    "comparison_key": "string"
  }
}

GET

/v1/project/{project_id}

Get project

Get a project object by its id

Path Parameters

project_id
Required
string

Project id

Format: "uuid"
Status codeDescription
200Returns the project object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X GET "https://api.braintrustdata.com/v1/project/497f6eca-6276-4993-bfeb-53cbbbba6f08"
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
  "name": "string",
  "created": "2019-08-24T14:15:22Z",
  "deleted_at": "2019-08-24T14:15:22Z",
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
  "settings": {
    "comparison_key": "string"
  }
}

PATCH

/v1/project/{project_id}

Partially update project

Partially update a project object. Specify the fields to update in the payload. Any object-type fields will be deep-merged with existing content. Currently we do not support removing fields or setting them to null.

Request Body (Optional)

Fields to update

namestring | null

Name of the project

settingsobject | null

Project settings. Patch operations replace all settings, so make sure you include all settings you want to keep.

Path Parameters

project_id
Required
string

Project id

Format: "uuid"
Status codeDescription
200Returns the project object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X PATCH "https://api.braintrustdata.com/v1/project/497f6eca-6276-4993-bfeb-53cbbbba6f08" \
  -d '{
  "name": "string",
  "settings": {
    "comparison_key": "string"
  }
}'
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
  "name": "string",
  "created": "2019-08-24T14:15:22Z",
  "deleted_at": "2019-08-24T14:15:22Z",
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
  "settings": {
    "comparison_key": "string"
  }
}

DELETE

/v1/project/{project_id}

Delete project

Delete a project object by its id

Path Parameters

project_id
Required
string

Project id

Format: "uuid"
Status codeDescription
200Returns the deleted project object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X DELETE "https://api.braintrustdata.com/v1/project/497f6eca-6276-4993-bfeb-53cbbbba6f08"
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
  "name": "string",
  "created": "2019-08-24T14:15:22Z",
  "deleted_at": "2019-08-24T14:15:22Z",
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
  "settings": {
    "comparison_key": "string"
  }
}

POST

/v1/project_logs/{project_id}/insert

Insert project logs events

Insert a set of events into the project logs

Request Body

An array of project logs events to insert

events
Required
array of Any properties in object, object

A list of project logs events to insert

Path Parameters

project_id
Required
string

Project id

Format: "uuid"
Status codeDescription
200Returns the inserted row ids
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/project_logs/497f6eca-6276-4993-bfeb-53cbbbba6f08/insert" \
  -d '{
  "events": [
    {
      "input": null,
      "output": null,
      "expected": null,
      "scores": {
        "property1": 1,
        "property2": 1
      },
      "metadata": {
        "property1": null,
        "property2": null
      },
      "tags": [
        "string"
      ],
      "metrics": {
        "start": 0,
        "end": 0,
        "prompt_tokens": 0,
        "completion_tokens": 0,
        "tokens": 0,
        "property1": null,
        "property2": null
      },
      "context": {
        "caller_functionname": "string",
        "caller_filename": "string",
        "caller_lineno": 0,
        "property1": null,
        "property2": null
      },
      "span_attributes": {
        "name": "string",
        "type": "llm",
        "property1": null,
        "property2": null
      },
      "id": "string",
      "_object_delete": true,
      "_is_merge": false,
      "_parent_id": "string"
    }
  ]
}'
Example Response
{
  "row_ids": [
    "string"
  ]
}

GET

/v1/project_logs/{project_id}/fetch

Fetch project logs (GET form)

Fetch the events in a project logs. Equivalent to the POST form of the same path, but with the parameters in the URL query rather than in the request body

Path Parameters

project_id
Required
string

Project id

Format: "uuid"

Query Parameters

limitinteger

limit the number of traces fetched

Fetch queries may be paginated if the total result size is expected to be large (e.g. project_logs which accumulate over a long time). Note that fetch queries only support pagination in descending time order (from latest to earliest _xact_id. Furthermore, later pages may return rows which showed up in earlier pages, except with an earlier _xact_id. This happens because pagination occurs over the whole version history of the event log. You will most likely want to exclude any such duplicate, outdated rows (by id) from your combined result set.

The limit parameter controls the number of full traces to return. So you may end up with more individual rows than the specified limit if you are fetching events containing traces.

Minimum: 0

max_xact_idstring

DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.

Together, max_xact_id and max_root_span_id form a pagination cursor

Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.

max_root_span_idstring

DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.

Together, max_xact_id and max_root_span_id form a pagination cursor

Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.

versionstring

Retrieve a snapshot of events from a past time

The version id is essentially a filter on the latest event transaction id. You can use the max_xact_id returned by a past fetch as the version to reproduce that exact fetch.

Status codeDescription
200Returns the fetched rows
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X GET "https://api.braintrustdata.com/v1/project_logs/497f6eca-6276-4993-bfeb-53cbbbba6f08/fetch"
Example Response
{
  "events": [
    {
      "id": "string",
      "_xact_id": "string",
      "created": "2019-08-24T14:15:22Z",
      "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
      "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
      "log_id": "g",
      "input": null,
      "output": null,
      "expected": null,
      "scores": {
        "property1": 1,
        "property2": 1
      },
      "metadata": {
        "property1": null,
        "property2": null
      },
      "tags": [
        "string"
      ],
      "metrics": {
        "start": 0,
        "end": 0,
        "prompt_tokens": 0,
        "completion_tokens": 0,
        "tokens": 0,
        "property1": null,
        "property2": null
      },
      "context": {
        "caller_functionname": "string",
        "caller_filename": "string",
        "caller_lineno": 0,
        "property1": null,
        "property2": null
      },
      "span_id": "f8e1ad1a-a94f-4999-9759-e8b20e66def2",
      "span_parents": [
        "string"
      ],
      "root_span_id": "7de2028e-5c6e-4582-a52e-53684fd07f7d",
      "span_attributes": {
        "name": "string",
        "type": "llm",
        "property1": null,
        "property2": null
      }
    }
  ],
  "cursor": "string"
}

POST

/v1/project_logs/{project_id}/fetch

Fetch project logs (POST form)

Fetch the events in a project logs. Equivalent to the GET form of the same path, but with the parameters in the request body rather than in the URL query

Request Body (Optional)

Filters for the fetch query

limitinteger | null

limit the number of traces fetched

Fetch queries may be paginated if the total result size is expected to be large (e.g. project_logs which accumulate over a long time). Note that fetch queries only support pagination in descending time order (from latest to earliest _xact_id. Furthermore, later pages may return rows which showed up in earlier pages, except with an earlier _xact_id. This happens because pagination occurs over the whole version history of the event log. You will most likely want to exclude any such duplicate, outdated rows (by id) from your combined result set.

The limit parameter controls the number of full traces to return. So you may end up with more individual rows than the specified limit if you are fetching events containing traces.

Minimum: 0

cursorstring | null

An opaque string to be used as a cursor for the next page of results, in order from latest to earliest.

The string can be obtained directly from the cursor property of the previous fetch query

max_xact_idstring | null

DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.

Together, max_xact_id and max_root_span_id form a pagination cursor

Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.

max_root_span_idstring | null

DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.

Together, max_xact_id and max_root_span_id form a pagination cursor

Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.

filtersarray of object | null

A list of filters on the events to fetch. Currently, only path-lookup type filters are supported, but we may add more in the future

versionstring | null

Retrieve a snapshot of events from a past time

The version id is essentially a filter on the latest event transaction id. You can use the max_xact_id returned by a past fetch as the version to reproduce that exact fetch.

Path Parameters

project_id
Required
string

Project id

Format: "uuid"
Status codeDescription
200Returns the fetched rows
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/project_logs/497f6eca-6276-4993-bfeb-53cbbbba6f08/fetch" \
  -d '{
  "limit": 0,
  "cursor": "string",
  "max_xact_id": "string",
  "max_root_span_id": "string",
  "filters": [
    {
      "type": "path_lookup",
      "path": [
        "string"
      ],
      "value": null
    }
  ],
  "version": "string"
}'
Example Response
{
  "events": [
    {
      "id": "string",
      "_xact_id": "string",
      "created": "2019-08-24T14:15:22Z",
      "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
      "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
      "log_id": "g",
      "input": null,
      "output": null,
      "expected": null,
      "scores": {
        "property1": 1,
        "property2": 1
      },
      "metadata": {
        "property1": null,
        "property2": null
      },
      "tags": [
        "string"
      ],
      "metrics": {
        "start": 0,
        "end": 0,
        "prompt_tokens": 0,
        "completion_tokens": 0,
        "tokens": 0,
        "property1": null,
        "property2": null
      },
      "context": {
        "caller_functionname": "string",
        "caller_filename": "string",
        "caller_lineno": 0,
        "property1": null,
        "property2": null
      },
      "span_id": "f8e1ad1a-a94f-4999-9759-e8b20e66def2",
      "span_parents": [
        "string"
      ],
      "root_span_id": "7de2028e-5c6e-4582-a52e-53684fd07f7d",
      "span_attributes": {
        "name": "string",
        "type": "llm",
        "property1": null,
        "property2": null
      }
    }
  ],
  "cursor": "string"
}

POST

/v1/project_logs/{project_id}/feedback

Feedback for project logs events

Log feedback for a set of project logs events

Request Body

An array of feedback objects

feedback
Required
array of object

A list of project logs feedback items

Path Parameters

project_id
Required
string

Project id

Format: "uuid"
Status codeDescription
200No return value
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/project_logs/497f6eca-6276-4993-bfeb-53cbbbba6f08/feedback" \
  -d '{
  "feedback": [
    {
      "id": "string",
      "scores": {
        "property1": 1,
        "property2": 1
      },
      "expected": null,
      "comment": "string",
      "metadata": {
        "property1": null,
        "property2": null
      },
      "source": "app"
    }
  ]
}'
Example Response
null

GET

/v1/experiment

List experiments

List out all experiments. The experiments are sorted by creation date, with the most recently-created experiments coming first

Query Parameters

limitinteger

Limit the number of objects to return

Minimum: 0

starting_afterstring

Pagination cursor id.

For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before

Format: "uuid"

ending_beforestring

Pagination cursor id.

For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before

Format: "uuid"

experiment_namestring

Name of the experiment to search for

project_namestring

Name of the project to search for

org_namestring

Filter search results to within a particular organization

idsAny properties in string, array of string

Filter search results to a particular set of object IDs. To specify a list of IDs, include the query param multiple times

Status codeDescription
200Returns a list of experiment objects
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X GET "https://api.braintrustdata.com/v1/experiment"
Example Response
{
  "objects": [
    {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
      "name": "string",
      "description": "string",
      "created": "2019-08-24T14:15:22Z",
      "repo_info": {
        "commit": "string",
        "branch": "string",
        "tag": "string",
        "dirty": true,
        "author_name": "string",
        "author_email": "string",
        "commit_message": "string",
        "commit_time": "string",
        "git_diff": "string"
      },
      "commit": "string",
      "base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
      "deleted_at": "2019-08-24T14:15:22Z",
      "dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
      "dataset_version": "string",
      "public": true,
      "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
      "metadata": {
        "property1": null,
        "property2": null
      }
    }
  ]
}

POST

/v1/experiment

Create experiment

Create a new experiment. If there is an existing experiment in the project with the same name as the one specified in the request, will return the existing experiment unmodified

Request Body

Any desired information about the new experiment object

project_id
Required
string

Unique identifier for the project that the experiment belongs under

Format: "uuid"

namestring | null

Name of the experiment. Within a project, experiment names are unique

descriptionstring | null

Textual description of the experiment

repo_infoobject | null

Metadata about the state of the repo when the experiment was created

base_exp_idstring | null

Id of default base experiment to compare against when viewing this experiment

Format: "uuid"

dataset_idstring | null

Identifier of the linked dataset, or null if the experiment is not linked to a dataset

Format: "uuid"

dataset_versionstring | null

Version number of the linked dataset the experiment was run against. This can be used to reproduce the experiment after the dataset has been modified.

publicboolean | null

Whether or not the experiment is public. Public experiments can be viewed by anybody inside or outside the organization

metadataobject | null

User-controlled metadata about the experiment

ensure_newboolean | null

Normally, creating an experiment with the same name as an existing experiment will return the existing one un-modified. But if ensure_new is true, registration will generate a new experiment with a unique name in case of a conflict.

Status codeDescription
200Returns the new experiment object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/experiment" \
  -d '{
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "description": "string",
  "repo_info": {
    "commit": "string",
    "branch": "string",
    "tag": "string",
    "dirty": true,
    "author_name": "string",
    "author_email": "string",
    "commit_message": "string",
    "commit_time": "string",
    "git_diff": "string"
  },
  "base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
  "dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
  "dataset_version": "string",
  "public": true,
  "metadata": {
    "property1": null,
    "property2": null
  },
  "ensure_new": true
}'
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "repo_info": {
    "commit": "string",
    "branch": "string",
    "tag": "string",
    "dirty": true,
    "author_name": "string",
    "author_email": "string",
    "commit_message": "string",
    "commit_time": "string",
    "git_diff": "string"
  },
  "commit": "string",
  "base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
  "deleted_at": "2019-08-24T14:15:22Z",
  "dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
  "dataset_version": "string",
  "public": true,
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
  "metadata": {
    "property1": null,
    "property2": null
  }
}

PUT

/v1/experiment

DEPRECATED. Create or replace experiment

NOTE: This operation is deprecated and will be removed in a future revision of the API. Create or replace a new experiment. If there is an existing experiment in the project with the same name as the one specified in the request, will return the existing experiment unmodified

Request Body

Any desired information about the new experiment object

project_id
Required
string

Unique identifier for the project that the experiment belongs under

Format: "uuid"

namestring | null

Name of the experiment. Within a project, experiment names are unique

descriptionstring | null

Textual description of the experiment

repo_infoobject | null

Metadata about the state of the repo when the experiment was created

base_exp_idstring | null

Id of default base experiment to compare against when viewing this experiment

Format: "uuid"

dataset_idstring | null

Identifier of the linked dataset, or null if the experiment is not linked to a dataset

Format: "uuid"

dataset_versionstring | null

Version number of the linked dataset the experiment was run against. This can be used to reproduce the experiment after the dataset has been modified.

publicboolean | null

Whether or not the experiment is public. Public experiments can be viewed by anybody inside or outside the organization

metadataobject | null

User-controlled metadata about the experiment

ensure_newboolean | null

Normally, creating an experiment with the same name as an existing experiment will return the existing one un-modified. But if ensure_new is true, registration will generate a new experiment with a unique name in case of a conflict.

Status codeDescription
200Returns the new experiment object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X PUT "https://api.braintrustdata.com/v1/experiment" \
  -d '{
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "description": "string",
  "repo_info": {
    "commit": "string",
    "branch": "string",
    "tag": "string",
    "dirty": true,
    "author_name": "string",
    "author_email": "string",
    "commit_message": "string",
    "commit_time": "string",
    "git_diff": "string"
  },
  "base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
  "dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
  "dataset_version": "string",
  "public": true,
  "metadata": {
    "property1": null,
    "property2": null
  },
  "ensure_new": true
}'
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "repo_info": {
    "commit": "string",
    "branch": "string",
    "tag": "string",
    "dirty": true,
    "author_name": "string",
    "author_email": "string",
    "commit_message": "string",
    "commit_time": "string",
    "git_diff": "string"
  },
  "commit": "string",
  "base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
  "deleted_at": "2019-08-24T14:15:22Z",
  "dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
  "dataset_version": "string",
  "public": true,
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
  "metadata": {
    "property1": null,
    "property2": null
  }
}

GET

/v1/experiment/{experiment_id}

Get experiment

Get an experiment object by its id

Path Parameters

experiment_id
Required
string

Experiment id

Format: "uuid"
Status codeDescription
200Returns the experiment object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X GET "https://api.braintrustdata.com/v1/experiment/497f6eca-6276-4993-bfeb-53cbbbba6f08"
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "repo_info": {
    "commit": "string",
    "branch": "string",
    "tag": "string",
    "dirty": true,
    "author_name": "string",
    "author_email": "string",
    "commit_message": "string",
    "commit_time": "string",
    "git_diff": "string"
  },
  "commit": "string",
  "base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
  "deleted_at": "2019-08-24T14:15:22Z",
  "dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
  "dataset_version": "string",
  "public": true,
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
  "metadata": {
    "property1": null,
    "property2": null
  }
}

PATCH

/v1/experiment/{experiment_id}

Partially update experiment

Partially update an experiment object. Specify the fields to update in the payload. Any object-type fields will be deep-merged with existing content. Currently we do not support removing fields or setting them to null.

Request Body (Optional)

Fields to update

namestring | null

Name of the experiment. Within a project, experiment names are unique

descriptionstring | null

Textual description of the experiment

repo_infoobject | null

Metadata about the state of the repo when the experiment was created

base_exp_idstring | null

Id of default base experiment to compare against when viewing this experiment

Format: "uuid"

dataset_idstring | null

Identifier of the linked dataset, or null if the experiment is not linked to a dataset

Format: "uuid"

dataset_versionstring | null

Version number of the linked dataset the experiment was run against. This can be used to reproduce the experiment after the dataset has been modified.

publicboolean | null

Whether or not the experiment is public. Public experiments can be viewed by anybody inside or outside the organization

metadataobject | null

User-controlled metadata about the experiment

Path Parameters

experiment_id
Required
string

Experiment id

Format: "uuid"
Status codeDescription
200Returns the experiment object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X PATCH "https://api.braintrustdata.com/v1/experiment/497f6eca-6276-4993-bfeb-53cbbbba6f08" \
  -d '{
  "name": "string",
  "description": "string",
  "repo_info": {
    "commit": "string",
    "branch": "string",
    "tag": "string",
    "dirty": true,
    "author_name": "string",
    "author_email": "string",
    "commit_message": "string",
    "commit_time": "string",
    "git_diff": "string"
  },
  "base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
  "dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
  "dataset_version": "string",
  "public": true,
  "metadata": {
    "property1": null,
    "property2": null
  }
}'
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "repo_info": {
    "commit": "string",
    "branch": "string",
    "tag": "string",
    "dirty": true,
    "author_name": "string",
    "author_email": "string",
    "commit_message": "string",
    "commit_time": "string",
    "git_diff": "string"
  },
  "commit": "string",
  "base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
  "deleted_at": "2019-08-24T14:15:22Z",
  "dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
  "dataset_version": "string",
  "public": true,
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
  "metadata": {
    "property1": null,
    "property2": null
  }
}

DELETE

/v1/experiment/{experiment_id}

Delete experiment

Delete an experiment object by its id

Path Parameters

experiment_id
Required
string

Experiment id

Format: "uuid"
Status codeDescription
200Returns the deleted experiment object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X DELETE "https://api.braintrustdata.com/v1/experiment/497f6eca-6276-4993-bfeb-53cbbbba6f08"
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "repo_info": {
    "commit": "string",
    "branch": "string",
    "tag": "string",
    "dirty": true,
    "author_name": "string",
    "author_email": "string",
    "commit_message": "string",
    "commit_time": "string",
    "git_diff": "string"
  },
  "commit": "string",
  "base_exp_id": "4838cee2-a665-4545-aa9f-483678c01a6b",
  "deleted_at": "2019-08-24T14:15:22Z",
  "dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
  "dataset_version": "string",
  "public": true,
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
  "metadata": {
    "property1": null,
    "property2": null
  }
}

POST

/v1/experiment/{experiment_id}/insert

Insert experiment events

Insert a set of events into the experiment

Request Body

An array of experiment events to insert

events
Required
array of Any properties in object, object

A list of experiment events to insert

Path Parameters

experiment_id
Required
string

Experiment id

Format: "uuid"
Status codeDescription
200Returns the inserted row ids
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/experiment/497f6eca-6276-4993-bfeb-53cbbbba6f08/insert" \
  -d '{
  "events": [
    {
      "input": null,
      "output": null,
      "expected": null,
      "scores": {
        "property1": 1,
        "property2": 1
      },
      "metadata": {
        "property1": null,
        "property2": null
      },
      "tags": [
        "string"
      ],
      "metrics": {
        "start": 0,
        "end": 0,
        "prompt_tokens": 0,
        "completion_tokens": 0,
        "tokens": 0,
        "property1": null,
        "property2": null
      },
      "context": {
        "caller_functionname": "string",
        "caller_filename": "string",
        "caller_lineno": 0,
        "property1": null,
        "property2": null
      },
      "span_attributes": {
        "name": "string",
        "type": "llm",
        "property1": null,
        "property2": null
      },
      "id": "string",
      "dataset_record_id": "string",
      "_object_delete": true,
      "_is_merge": false,
      "_parent_id": "string"
    }
  ]
}'
Example Response
{
  "row_ids": [
    "string"
  ]
}

GET

/v1/experiment/{experiment_id}/fetch

Fetch experiment (GET form)

Fetch the events in an experiment. Equivalent to the POST form of the same path, but with the parameters in the URL query rather than in the request body

Path Parameters

experiment_id
Required
string

Experiment id

Format: "uuid"

Query Parameters

limitinteger

limit the number of traces fetched

Fetch queries may be paginated if the total result size is expected to be large (e.g. project_logs which accumulate over a long time). Note that fetch queries only support pagination in descending time order (from latest to earliest _xact_id. Furthermore, later pages may return rows which showed up in earlier pages, except with an earlier _xact_id. This happens because pagination occurs over the whole version history of the event log. You will most likely want to exclude any such duplicate, outdated rows (by id) from your combined result set.

The limit parameter controls the number of full traces to return. So you may end up with more individual rows than the specified limit if you are fetching events containing traces.

Minimum: 0

max_xact_idstring

DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.

Together, max_xact_id and max_root_span_id form a pagination cursor

Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.

max_root_span_idstring

DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.

Together, max_xact_id and max_root_span_id form a pagination cursor

Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.

versionstring

Retrieve a snapshot of events from a past time

The version id is essentially a filter on the latest event transaction id. You can use the max_xact_id returned by a past fetch as the version to reproduce that exact fetch.

Status codeDescription
200Returns the fetched rows
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X GET "https://api.braintrustdata.com/v1/experiment/497f6eca-6276-4993-bfeb-53cbbbba6f08/fetch"
Example Response
{
  "events": [
    {
      "id": "string",
      "dataset_record_id": "string",
      "_xact_id": "string",
      "created": "2019-08-24T14:15:22Z",
      "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
      "experiment_id": "916afd89-cac5-4339-9c59-dd068abdfa69",
      "input": null,
      "output": null,
      "expected": null,
      "scores": {
        "property1": 1,
        "property2": 1
      },
      "metadata": {
        "property1": null,
        "property2": null
      },
      "tags": [
        "string"
      ],
      "metrics": {
        "start": 0,
        "end": 0,
        "prompt_tokens": 0,
        "completion_tokens": 0,
        "tokens": 0,
        "property1": null,
        "property2": null
      },
      "context": {
        "caller_functionname": "string",
        "caller_filename": "string",
        "caller_lineno": 0,
        "property1": null,
        "property2": null
      },
      "span_id": "f8e1ad1a-a94f-4999-9759-e8b20e66def2",
      "span_parents": [
        "string"
      ],
      "root_span_id": "7de2028e-5c6e-4582-a52e-53684fd07f7d",
      "span_attributes": {
        "name": "string",
        "type": "llm",
        "property1": null,
        "property2": null
      }
    }
  ],
  "cursor": "string"
}

POST

/v1/experiment/{experiment_id}/fetch

Fetch experiment (POST form)

Fetch the events in an experiment. Equivalent to the GET form of the same path, but with the parameters in the request body rather than in the URL query

Request Body (Optional)

Filters for the fetch query

limitinteger | null

limit the number of traces fetched

Fetch queries may be paginated if the total result size is expected to be large (e.g. project_logs which accumulate over a long time). Note that fetch queries only support pagination in descending time order (from latest to earliest _xact_id. Furthermore, later pages may return rows which showed up in earlier pages, except with an earlier _xact_id. This happens because pagination occurs over the whole version history of the event log. You will most likely want to exclude any such duplicate, outdated rows (by id) from your combined result set.

The limit parameter controls the number of full traces to return. So you may end up with more individual rows than the specified limit if you are fetching events containing traces.

Minimum: 0

cursorstring | null

An opaque string to be used as a cursor for the next page of results, in order from latest to earliest.

The string can be obtained directly from the cursor property of the previous fetch query

max_xact_idstring | null

DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.

Together, max_xact_id and max_root_span_id form a pagination cursor

Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.

max_root_span_idstring | null

DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.

Together, max_xact_id and max_root_span_id form a pagination cursor

Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.

filtersarray of object | null

A list of filters on the events to fetch. Currently, only path-lookup type filters are supported, but we may add more in the future

versionstring | null

Retrieve a snapshot of events from a past time

The version id is essentially a filter on the latest event transaction id. You can use the max_xact_id returned by a past fetch as the version to reproduce that exact fetch.

Path Parameters

experiment_id
Required
string

Experiment id

Format: "uuid"
Status codeDescription
200Returns the fetched rows
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/experiment/497f6eca-6276-4993-bfeb-53cbbbba6f08/fetch" \
  -d '{
  "limit": 0,
  "cursor": "string",
  "max_xact_id": "string",
  "max_root_span_id": "string",
  "filters": [
    {
      "type": "path_lookup",
      "path": [
        "string"
      ],
      "value": null
    }
  ],
  "version": "string"
}'
Example Response
{
  "events": [
    {
      "id": "string",
      "dataset_record_id": "string",
      "_xact_id": "string",
      "created": "2019-08-24T14:15:22Z",
      "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
      "experiment_id": "916afd89-cac5-4339-9c59-dd068abdfa69",
      "input": null,
      "output": null,
      "expected": null,
      "scores": {
        "property1": 1,
        "property2": 1
      },
      "metadata": {
        "property1": null,
        "property2": null
      },
      "tags": [
        "string"
      ],
      "metrics": {
        "start": 0,
        "end": 0,
        "prompt_tokens": 0,
        "completion_tokens": 0,
        "tokens": 0,
        "property1": null,
        "property2": null
      },
      "context": {
        "caller_functionname": "string",
        "caller_filename": "string",
        "caller_lineno": 0,
        "property1": null,
        "property2": null
      },
      "span_id": "f8e1ad1a-a94f-4999-9759-e8b20e66def2",
      "span_parents": [
        "string"
      ],
      "root_span_id": "7de2028e-5c6e-4582-a52e-53684fd07f7d",
      "span_attributes": {
        "name": "string",
        "type": "llm",
        "property1": null,
        "property2": null
      }
    }
  ],
  "cursor": "string"
}

POST

/v1/experiment/{experiment_id}/feedback

Feedback for experiment events

Log feedback for a set of experiment events

Request Body

An array of feedback objects

feedback
Required
array of object

A list of experiment feedback items

Path Parameters

experiment_id
Required
string

Experiment id

Format: "uuid"
Status codeDescription
200No return value
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/experiment/497f6eca-6276-4993-bfeb-53cbbbba6f08/feedback" \
  -d '{
  "feedback": [
    {
      "id": "string",
      "scores": {
        "property1": 1,
        "property2": 1
      },
      "expected": null,
      "comment": "string",
      "metadata": {
        "property1": null,
        "property2": null
      },
      "source": "app"
    }
  ]
}'
Example Response
null

GET

/v1/experiment/{experiment_id}/summarize

Summarize experiment

Summarize experiment

Path Parameters

experiment_id
Required
string

Experiment id

Format: "uuid"

Query Parameters

summarize_scoresboolean

Whether to summarize the scores and metrics. If false (or omitted), only the metadata will be returned.

comparison_experiment_idstring

The experiment to compare against, if summarizing scores and metrics. If omitted, will fall back to the base_exp_id stored in the experiment metadata, and then to the most recent experiment run in the same project. Must pass summarize_scores=true for this id to be used

Format: "uuid"
Status codeDescription
200Experiment summary
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X GET "https://api.braintrustdata.com/v1/experiment/497f6eca-6276-4993-bfeb-53cbbbba6f08/summarize"

Summary of an experiment

Example Response
{
  "project_name": "string",
  "experiment_name": "string",
  "project_url": "http://example.com",
  "experiment_url": "http://example.com",
  "comparison_experiment_name": "string",
  "scores": {
    "property1": {
      "name": "string",
      "score": 1,
      "diff": -1,
      "improvements": 0,
      "regressions": 0
    },
    "property2": {
      "name": "string",
      "score": 1,
      "diff": -1,
      "improvements": 0,
      "regressions": 0
    }
  },
  "metrics": {
    "property1": {
      "name": "string",
      "metric": 0,
      "unit": "string",
      "diff": 0,
      "improvements": 0,
      "regressions": 0
    },
    "property2": {
      "name": "string",
      "metric": 0,
      "unit": "string",
      "diff": 0,
      "improvements": 0,
      "regressions": 0
    }
  }
}

GET

/v1/dataset

List datasets

List out all datasets. The datasets are sorted by creation date, with the most recently-created datasets coming first

Query Parameters

limitinteger

Limit the number of objects to return

Minimum: 0

starting_afterstring

Pagination cursor id.

For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before

Format: "uuid"

ending_beforestring

Pagination cursor id.

For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before

Format: "uuid"

dataset_namestring

Name of the dataset to search for

project_namestring

Name of the project to search for

org_namestring

Filter search results to within a particular organization

idsAny properties in string, array of string

Filter search results to a particular set of object IDs. To specify a list of IDs, include the query param multiple times

Status codeDescription
200Returns a list of dataset objects
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X GET "https://api.braintrustdata.com/v1/dataset"
Example Response
{
  "objects": [
    {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
      "name": "string",
      "description": "string",
      "created": "2019-08-24T14:15:22Z",
      "deleted_at": "2019-08-24T14:15:22Z",
      "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
    }
  ]
}

POST

/v1/dataset

Create dataset

Create a new dataset. If there is an existing dataset in the project with the same name as the one specified in the request, will return the existing dataset unmodified

Request Body

Any desired information about the new dataset object

project_idstring | null

Unique identifier for the project that the dataset belongs under

Format: "uuid"

name
Required
string

Name of the dataset. Within a project, dataset names are unique

descriptionstring | null

Textual description of the dataset

Status codeDescription
200Returns the new dataset object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/dataset" \
  -d '{
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "description": "string"
}'
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "deleted_at": "2019-08-24T14:15:22Z",
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}

PUT

/v1/dataset

DEPRECATED. Create or replace dataset

NOTE: This operation is deprecated and will be removed in a future revision of the API. Create or replace a new dataset. If there is an existing dataset in the project with the same name as the one specified in the request, will return the existing dataset unmodified

Request Body

Any desired information about the new dataset object

project_idstring | null

Unique identifier for the project that the dataset belongs under

Format: "uuid"

name
Required
string

Name of the dataset. Within a project, dataset names are unique

descriptionstring | null

Textual description of the dataset

Status codeDescription
200Returns the new dataset object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X PUT "https://api.braintrustdata.com/v1/dataset" \
  -d '{
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "description": "string"
}'
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "deleted_at": "2019-08-24T14:15:22Z",
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}

GET

/v1/dataset/{dataset_id}

Get dataset

Get a dataset object by its id

Path Parameters

dataset_id
Required
string

Dataset id

Format: "uuid"
Status codeDescription
200Returns the dataset object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X GET "https://api.braintrustdata.com/v1/dataset/497f6eca-6276-4993-bfeb-53cbbbba6f08"
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "deleted_at": "2019-08-24T14:15:22Z",
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}

PATCH

/v1/dataset/{dataset_id}

Partially update dataset

Partially update a dataset object. Specify the fields to update in the payload. Any object-type fields will be deep-merged with existing content. Currently we do not support removing fields or setting them to null.

Request Body (Optional)

Fields to update

namestring | null

Name of the dataset. Within a project, dataset names are unique

descriptionstring | null

Textual description of the dataset

Path Parameters

dataset_id
Required
string

Dataset id

Format: "uuid"
Status codeDescription
200Returns the dataset object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X PATCH "https://api.braintrustdata.com/v1/dataset/497f6eca-6276-4993-bfeb-53cbbbba6f08" \
  -d '{
  "name": "string",
  "description": "string"
}'
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "deleted_at": "2019-08-24T14:15:22Z",
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}

DELETE

/v1/dataset/{dataset_id}

Delete dataset

Delete a dataset object by its id

Path Parameters

dataset_id
Required
string

Dataset id

Format: "uuid"
Status codeDescription
200Returns the deleted dataset object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X DELETE "https://api.braintrustdata.com/v1/dataset/497f6eca-6276-4993-bfeb-53cbbbba6f08"
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "deleted_at": "2019-08-24T14:15:22Z",
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5"
}

POST

/v1/dataset/{dataset_id}/insert

Insert dataset events

Insert a set of events into the dataset

Request Body

An array of dataset events to insert

events
Required
array of Any properties in object, object

A list of dataset events to insert

Path Parameters

dataset_id
Required
string

Dataset id

Format: "uuid"
Status codeDescription
200Returns the inserted row ids
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/dataset/497f6eca-6276-4993-bfeb-53cbbbba6f08/insert" \
  -d '{
  "events": [
    {
      "input": null,
      "expected": null,
      "metadata": {
        "property1": null,
        "property2": null
      },
      "tags": [
        "string"
      ],
      "id": "string",
      "_object_delete": true,
      "_is_merge": false,
      "_parent_id": "string"
    }
  ]
}'
Example Response
{
  "row_ids": [
    "string"
  ]
}

GET

/v1/dataset/{dataset_id}/fetch

Fetch dataset (GET form)

Fetch the events in a dataset. Equivalent to the POST form of the same path, but with the parameters in the URL query rather than in the request body

Path Parameters

dataset_id
Required
string

Dataset id

Format: "uuid"

Query Parameters

limitinteger

limit the number of traces fetched

Fetch queries may be paginated if the total result size is expected to be large (e.g. project_logs which accumulate over a long time). Note that fetch queries only support pagination in descending time order (from latest to earliest _xact_id. Furthermore, later pages may return rows which showed up in earlier pages, except with an earlier _xact_id. This happens because pagination occurs over the whole version history of the event log. You will most likely want to exclude any such duplicate, outdated rows (by id) from your combined result set.

The limit parameter controls the number of full traces to return. So you may end up with more individual rows than the specified limit if you are fetching events containing traces.

Minimum: 0

max_xact_idstring

DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.

Together, max_xact_id and max_root_span_id form a pagination cursor

Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.

max_root_span_idstring

DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.

Together, max_xact_id and max_root_span_id form a pagination cursor

Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.

versionstring

Retrieve a snapshot of events from a past time

The version id is essentially a filter on the latest event transaction id. You can use the max_xact_id returned by a past fetch as the version to reproduce that exact fetch.

Status codeDescription
200Returns the fetched rows
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X GET "https://api.braintrustdata.com/v1/dataset/497f6eca-6276-4993-bfeb-53cbbbba6f08/fetch"
Example Response
{
  "events": [
    {
      "id": "string",
      "_xact_id": "string",
      "created": "2019-08-24T14:15:22Z",
      "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
      "dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
      "input": null,
      "expected": null,
      "metadata": {
        "property1": null,
        "property2": null
      },
      "tags": [
        "string"
      ],
      "span_id": "f8e1ad1a-a94f-4999-9759-e8b20e66def2",
      "root_span_id": "7de2028e-5c6e-4582-a52e-53684fd07f7d"
    }
  ],
  "cursor": "string"
}

POST

/v1/dataset/{dataset_id}/fetch

Fetch dataset (POST form)

Fetch the events in a dataset. Equivalent to the GET form of the same path, but with the parameters in the request body rather than in the URL query

Request Body (Optional)

Filters for the fetch query

limitinteger | null

limit the number of traces fetched

Fetch queries may be paginated if the total result size is expected to be large (e.g. project_logs which accumulate over a long time). Note that fetch queries only support pagination in descending time order (from latest to earliest _xact_id. Furthermore, later pages may return rows which showed up in earlier pages, except with an earlier _xact_id. This happens because pagination occurs over the whole version history of the event log. You will most likely want to exclude any such duplicate, outdated rows (by id) from your combined result set.

The limit parameter controls the number of full traces to return. So you may end up with more individual rows than the specified limit if you are fetching events containing traces.

Minimum: 0

cursorstring | null

An opaque string to be used as a cursor for the next page of results, in order from latest to earliest.

The string can be obtained directly from the cursor property of the previous fetch query

max_xact_idstring | null

DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.

Together, max_xact_id and max_root_span_id form a pagination cursor

Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.

max_root_span_idstring | null

DEPRECATION NOTICE: The manually-constructed pagination cursor is deprecated in favor of the explicit 'cursor' returned by object fetch requests. Please prefer the 'cursor' argument going forwards.

Together, max_xact_id and max_root_span_id form a pagination cursor

Since a paginated fetch query returns results in order from latest to earliest, the cursor for the next page can be found as the row with the minimum (earliest) value of the tuple (_xact_id, root_span_id). See the documentation of limit for an overview of paginating fetch queries.

filtersarray of object | null

A list of filters on the events to fetch. Currently, only path-lookup type filters are supported, but we may add more in the future

versionstring | null

Retrieve a snapshot of events from a past time

The version id is essentially a filter on the latest event transaction id. You can use the max_xact_id returned by a past fetch as the version to reproduce that exact fetch.

Path Parameters

dataset_id
Required
string

Dataset id

Format: "uuid"
Status codeDescription
200Returns the fetched rows
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/dataset/497f6eca-6276-4993-bfeb-53cbbbba6f08/fetch" \
  -d '{
  "limit": 0,
  "cursor": "string",
  "max_xact_id": "string",
  "max_root_span_id": "string",
  "filters": [
    {
      "type": "path_lookup",
      "path": [
        "string"
      ],
      "value": null
    }
  ],
  "version": "string"
}'
Example Response
{
  "events": [
    {
      "id": "string",
      "_xact_id": "string",
      "created": "2019-08-24T14:15:22Z",
      "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
      "dataset_id": "8c4c51f1-f6f3-43bc-b65d-7415e8ef22c0",
      "input": null,
      "expected": null,
      "metadata": {
        "property1": null,
        "property2": null
      },
      "tags": [
        "string"
      ],
      "span_id": "f8e1ad1a-a94f-4999-9759-e8b20e66def2",
      "root_span_id": "7de2028e-5c6e-4582-a52e-53684fd07f7d"
    }
  ],
  "cursor": "string"
}

POST

/v1/dataset/{dataset_id}/feedback

Feedback for dataset events

Log feedback for a set of dataset events

Request Body

An array of feedback objects

feedback
Required
array of object

A list of dataset feedback items

Path Parameters

dataset_id
Required
string

Dataset id

Format: "uuid"
Status codeDescription
200No return value
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/dataset/497f6eca-6276-4993-bfeb-53cbbbba6f08/feedback" \
  -d '{
  "feedback": [
    {
      "id": "string",
      "comment": "string",
      "metadata": {
        "property1": null,
        "property2": null
      },
      "source": "app"
    }
  ]
}'
Example Response
null

GET

/v1/dataset/{dataset_id}/summarize

Summarize dataset

Summarize dataset

Path Parameters

dataset_id
Required
string

Dataset id

Format: "uuid"

Query Parameters

summarize_databoolean

Whether to summarize the data. If false (or omitted), only the metadata will be returned.

Status codeDescription
200Dataset summary
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X GET "https://api.braintrustdata.com/v1/dataset/497f6eca-6276-4993-bfeb-53cbbbba6f08/summarize"

Summary of a dataset

Example Response
{
  "project_name": "string",
  "dataset_name": "string",
  "project_url": "http://example.com",
  "dataset_url": "http://example.com",
  "data_summary": {
    "total_records": 0
  }
}

GET

/v1/prompt

List prompts

List out all prompts. The prompts are sorted by creation date, with the most recently-created prompts coming first

Query Parameters

limitinteger

Limit the number of objects to return

Minimum: 0

starting_afterstring

Pagination cursor id.

For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before

Format: "uuid"

ending_beforestring

Pagination cursor id.

For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before

Format: "uuid"

prompt_namestring

Name of the prompt to search for

project_namestring

Name of the project to search for

slugstring

Retrieve prompt with a specific slug

versionstring

Retrieve prompt at a specific version.

The version id can either be a transaction id (e.g. '1000192656880881099') or a version identifier (e.g. '81cd05ee665fdfb3').

org_namestring

Filter search results to within a particular organization

idsAny properties in string, array of string

Filter search results to a particular set of object IDs. To specify a list of IDs, include the query param multiple times

Status codeDescription
200Returns a list of prompt objects
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X GET "https://api.braintrustdata.com/v1/prompt"
Example Response
{
  "objects": [
    {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "_xact_id": "string",
      "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
      "log_id": "p",
      "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
      "name": "string",
      "slug": "string",
      "description": "string",
      "created": "2019-08-24T14:15:22Z",
      "prompt_data": {
        "prompt": {
          "type": "completion",
          "content": "string"
        },
        "options": {
          "model": "string",
          "params": {
            "use_cache": true,
            "temperature": 0,
            "top_p": 0,
            "max_tokens": 0,
            "frequency_penalty": 0,
            "presence_penalty": 0,
            "response_format": {
              "type": "json_object"
            },
            "tool_choice": "auto",
            "function_call": "auto",
            "n": 0,
            "stop": [
              "string"
            ]
          },
          "position": "string"
        },
        "origin": {
          "prompt_id": "string",
          "project_id": "string",
          "prompt_version": "string"
        }
      },
      "tags": [
        "string"
      ],
      "metadata": {
        "property1": null,
        "property2": null
      }
    }
  ]
}

POST

/v1/prompt

Create prompt

Create a new prompt. If there is an existing prompt in the project with the same slug as the one specified in the request, will return the existing prompt unmodified

Request Body

Any desired information about the new prompt object

project_id
Required
string

Unique identifier for the project that the prompt belongs under

Format: "uuid"

name
Required
string

Name of the prompt

slug
Required
string

Unique identifier for the prompt

descriptionstring | null

Textual description of the prompt

prompt_dataobject | null

The prompt, model, and its parameters

tagsarray of string | null

A list of tags for the prompt

Status codeDescription
200Returns the new prompt object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/prompt" \
  -d '{
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "slug": "string",
  "description": "string",
  "prompt_data": {
    "prompt": {
      "type": "completion",
      "content": "string"
    },
    "options": {
      "model": "string",
      "params": {
        "use_cache": true,
        "temperature": 0,
        "top_p": 0,
        "max_tokens": 0,
        "frequency_penalty": 0,
        "presence_penalty": 0,
        "response_format": {
          "type": "json_object"
        },
        "tool_choice": "auto",
        "function_call": "auto",
        "n": 0,
        "stop": [
          "string"
        ]
      },
      "position": "string"
    },
    "origin": {
      "prompt_id": "string",
      "project_id": "string",
      "prompt_version": "string"
    }
  },
  "tags": [
    "string"
  ]
}'
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "_xact_id": "string",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "log_id": "p",
  "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
  "name": "string",
  "slug": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "prompt_data": {
    "prompt": {
      "type": "completion",
      "content": "string"
    },
    "options": {
      "model": "string",
      "params": {
        "use_cache": true,
        "temperature": 0,
        "top_p": 0,
        "max_tokens": 0,
        "frequency_penalty": 0,
        "presence_penalty": 0,
        "response_format": {
          "type": "json_object"
        },
        "tool_choice": "auto",
        "function_call": "auto",
        "n": 0,
        "stop": [
          "string"
        ]
      },
      "position": "string"
    },
    "origin": {
      "prompt_id": "string",
      "project_id": "string",
      "prompt_version": "string"
    }
  },
  "tags": [
    "string"
  ],
  "metadata": {
    "property1": null,
    "property2": null
  }
}

PUT

/v1/prompt

Create or replace prompt

Create or replace prompt. If there is an existing prompt in the project with the same slug as the one specified in the request, will replace the existing prompt with the provided fields

Request Body

Any desired information about the new prompt object

project_id
Required
string

Unique identifier for the project that the prompt belongs under

Format: "uuid"

name
Required
string

Name of the prompt

slug
Required
string

Unique identifier for the prompt

descriptionstring | null

Textual description of the prompt

prompt_dataobject | null

The prompt, model, and its parameters

tagsarray of string | null

A list of tags for the prompt

Status codeDescription
200Returns the new prompt object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X PUT "https://api.braintrustdata.com/v1/prompt" \
  -d '{
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "name": "string",
  "slug": "string",
  "description": "string",
  "prompt_data": {
    "prompt": {
      "type": "completion",
      "content": "string"
    },
    "options": {
      "model": "string",
      "params": {
        "use_cache": true,
        "temperature": 0,
        "top_p": 0,
        "max_tokens": 0,
        "frequency_penalty": 0,
        "presence_penalty": 0,
        "response_format": {
          "type": "json_object"
        },
        "tool_choice": "auto",
        "function_call": "auto",
        "n": 0,
        "stop": [
          "string"
        ]
      },
      "position": "string"
    },
    "origin": {
      "prompt_id": "string",
      "project_id": "string",
      "prompt_version": "string"
    }
  },
  "tags": [
    "string"
  ]
}'
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "_xact_id": "string",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "log_id": "p",
  "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
  "name": "string",
  "slug": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "prompt_data": {
    "prompt": {
      "type": "completion",
      "content": "string"
    },
    "options": {
      "model": "string",
      "params": {
        "use_cache": true,
        "temperature": 0,
        "top_p": 0,
        "max_tokens": 0,
        "frequency_penalty": 0,
        "presence_penalty": 0,
        "response_format": {
          "type": "json_object"
        },
        "tool_choice": "auto",
        "function_call": "auto",
        "n": 0,
        "stop": [
          "string"
        ]
      },
      "position": "string"
    },
    "origin": {
      "prompt_id": "string",
      "project_id": "string",
      "prompt_version": "string"
    }
  },
  "tags": [
    "string"
  ],
  "metadata": {
    "property1": null,
    "property2": null
  }
}

GET

/v1/prompt/{prompt_id}

Get prompt

Get a prompt object by its id

Path Parameters

prompt_id
Required
string

Prompt id

Format: "uuid"
Status codeDescription
200Returns the prompt object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X GET "https://api.braintrustdata.com/v1/prompt/497f6eca-6276-4993-bfeb-53cbbbba6f08"
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "_xact_id": "string",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "log_id": "p",
  "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
  "name": "string",
  "slug": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "prompt_data": {
    "prompt": {
      "type": "completion",
      "content": "string"
    },
    "options": {
      "model": "string",
      "params": {
        "use_cache": true,
        "temperature": 0,
        "top_p": 0,
        "max_tokens": 0,
        "frequency_penalty": 0,
        "presence_penalty": 0,
        "response_format": {
          "type": "json_object"
        },
        "tool_choice": "auto",
        "function_call": "auto",
        "n": 0,
        "stop": [
          "string"
        ]
      },
      "position": "string"
    },
    "origin": {
      "prompt_id": "string",
      "project_id": "string",
      "prompt_version": "string"
    }
  },
  "tags": [
    "string"
  ],
  "metadata": {
    "property1": null,
    "property2": null
  }
}

PATCH

/v1/prompt/{prompt_id}

Partially update prompt

Partially update a prompt object. Specify the fields to update in the payload. Any object-type fields will be deep-merged with existing content. Currently we do not support removing fields or setting them to null.

Request Body (Optional)

Fields to update

namestring | null

Name of the prompt

descriptionstring | null

Textual description of the prompt

prompt_dataobject | null

The prompt, model, and its parameters

tagsarray of string | null

A list of tags for the prompt

Path Parameters

prompt_id
Required
string

Prompt id

Format: "uuid"
Status codeDescription
200Returns the prompt object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X PATCH "https://api.braintrustdata.com/v1/prompt/497f6eca-6276-4993-bfeb-53cbbbba6f08" \
  -d '{
  "name": "string",
  "description": "string",
  "prompt_data": {
    "prompt": {
      "type": "completion",
      "content": "string"
    },
    "options": {
      "model": "string",
      "params": {
        "use_cache": true,
        "temperature": 0,
        "top_p": 0,
        "max_tokens": 0,
        "frequency_penalty": 0,
        "presence_penalty": 0,
        "response_format": {
          "type": "json_object"
        },
        "tool_choice": "auto",
        "function_call": "auto",
        "n": 0,
        "stop": [
          "string"
        ]
      },
      "position": "string"
    },
    "origin": {
      "prompt_id": "string",
      "project_id": "string",
      "prompt_version": "string"
    }
  },
  "tags": [
    "string"
  ]
}'
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "_xact_id": "string",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "log_id": "p",
  "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
  "name": "string",
  "slug": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "prompt_data": {
    "prompt": {
      "type": "completion",
      "content": "string"
    },
    "options": {
      "model": "string",
      "params": {
        "use_cache": true,
        "temperature": 0,
        "top_p": 0,
        "max_tokens": 0,
        "frequency_penalty": 0,
        "presence_penalty": 0,
        "response_format": {
          "type": "json_object"
        },
        "tool_choice": "auto",
        "function_call": "auto",
        "n": 0,
        "stop": [
          "string"
        ]
      },
      "position": "string"
    },
    "origin": {
      "prompt_id": "string",
      "project_id": "string",
      "prompt_version": "string"
    }
  },
  "tags": [
    "string"
  ],
  "metadata": {
    "property1": null,
    "property2": null
  }
}

DELETE

/v1/prompt/{prompt_id}

Delete prompt

Delete a prompt object by its id

Path Parameters

prompt_id
Required
string

Prompt id

Format: "uuid"
Status codeDescription
200Returns the deleted prompt object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X DELETE "https://api.braintrustdata.com/v1/prompt/497f6eca-6276-4993-bfeb-53cbbbba6f08"
Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "_xact_id": "string",
  "project_id": "405d8375-3514-403b-8c43-83ae74cfe0e9",
  "log_id": "p",
  "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
  "name": "string",
  "slug": "string",
  "description": "string",
  "created": "2019-08-24T14:15:22Z",
  "prompt_data": {
    "prompt": {
      "type": "completion",
      "content": "string"
    },
    "options": {
      "model": "string",
      "params": {
        "use_cache": true,
        "temperature": 0,
        "top_p": 0,
        "max_tokens": 0,
        "frequency_penalty": 0,
        "presence_penalty": 0,
        "response_format": {
          "type": "json_object"
        },
        "tool_choice": "auto",
        "function_call": "auto",
        "n": 0,
        "stop": [
          "string"
        ]
      },
      "position": "string"
    },
    "origin": {
      "prompt_id": "string",
      "project_id": "string",
      "prompt_version": "string"
    }
  },
  "tags": [
    "string"
  ],
  "metadata": {
    "property1": null,
    "property2": null
  }
}

POST

/v1/prompt/{prompt_id}/feedback

Feedback for prompt events

Log feedback for a set of prompt events

Request Body

An array of feedback objects

feedback
Required
array of object

A list of prompt feedback items

Path Parameters

prompt_id
Required
string

Prompt id

Format: "uuid"
Status codeDescription
200No return value
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/prompt/497f6eca-6276-4993-bfeb-53cbbbba6f08/feedback" \
  -d '{
  "feedback": [
    {
      "id": "string",
      "comment": "string",
      "metadata": {
        "property1": null,
        "property2": null
      },
      "source": "app"
    }
  ]
}'
Example Response
null

GET

/v1/role

List roles

List out all roles. The roles are sorted by creation date, with the most recently-created roles coming first

Query Parameters

limitinteger

Limit the number of objects to return

Minimum: 0

starting_afterstring

Pagination cursor id.

For example, if the final item in the last page you fetched had an id of foo, pass starting_after=foo to fetch the next page. Note: you may only pass one of starting_after and ending_before

Format: "uuid"

ending_beforestring

Pagination cursor id.

For example, if the initial item in the last page you fetched had an id of foo, pass ending_before=foo to fetch the previous page. Note: you may only pass one of starting_after and ending_before

Format: "uuid"

role_namestring

Name of the role to search for

org_namestring

Filter search results to within a particular organization

idsAny properties in string, array of string

Filter search results to a particular set of object IDs. To specify a list of IDs, include the query param multiple times

Status codeDescription
200Returns a list of role objects
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X GET "https://api.braintrustdata.com/v1/role"
Example Response
{
  "objects": [
    {
      "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
      "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
      "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
      "created": "2019-08-24T14:15:22Z",
      "name": "string",
      "description": "string",
      "deleted_at": "2019-08-24T14:15:22Z",
      "member_permissions": [
        {
          "permission": "create",
          "restrict_object_type": "organization"
        }
      ],
      "member_roles": [
        "497f6eca-6276-4993-bfeb-53cbbbba6f08"
      ]
    }
  ]
}

POST

/v1/role

Create role

Create a new role. If there is an existing role with the same name as the one specified in the request, will return the existing role unmodified

Request Body

Any desired information about the new role object

name
Required
string

Name of the role

descriptionstring | null

Textual description of the role

member_permissionsarray of object | null

(permission, restrict_object_type) tuples which belong to this role

member_rolesarray of string | null

Ids of the roles this role inherits from

An inheriting role has all the permissions contained in its member roles, as well as all of their inherited permissions

org_namestring | null

For nearly all users, this parameter should be unnecessary. But in the rare case that your API key belongs to multiple organizations, you may specify the name of the organization the role belongs in.

Status codeDescription
200Returns the new role object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X POST "https://api.braintrustdata.com/v1/role" \
  -d '{
  "name": "string",
  "description": "string",
  "member_permissions": [
    {
      "permission": "create",
      "restrict_object_type": "organization"
    }
  ],
  "member_roles": [
    "497f6eca-6276-4993-bfeb-53cbbbba6f08"
  ],
  "org_name": "string"
}'

A role is a collection of permissions which can be granted as part of an ACL

Roles can consist of individual permissions, as well as a set of roles they inherit from

Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
  "created": "2019-08-24T14:15:22Z",
  "name": "string",
  "description": "string",
  "deleted_at": "2019-08-24T14:15:22Z",
  "member_permissions": [
    {
      "permission": "create",
      "restrict_object_type": "organization"
    }
  ],
  "member_roles": [
    "497f6eca-6276-4993-bfeb-53cbbbba6f08"
  ]
}

PUT

/v1/role

DEPRECATED. Create or replace role

NOTE: This operation is deprecated and will be removed in a future revision of the API. Create or replace a new role. If there is an existing role with the same name as the one specified in the request, will return the existing role unmodified

Request Body

Any desired information about the new role object

name
Required
string

Name of the role

descriptionstring | null

Textual description of the role

member_permissionsarray of object | null

(permission, restrict_object_type) tuples which belong to this role

member_rolesarray of string | null

Ids of the roles this role inherits from

An inheriting role has all the permissions contained in its member roles, as well as all of their inherited permissions

org_namestring | null

For nearly all users, this parameter should be unnecessary. But in the rare case that your API key belongs to multiple organizations, you may specify the name of the organization the role belongs in.

Status codeDescription
200Returns the new role object
400The request was unacceptable, often due to missing a required parameter
401No valid API key provided
403The API key doesn’t have permissions to perform the request
429Too many requests hit the API too quickly. We recommend an exponential backoff of your requests
500Something went wrong on Braintrust's end. (These are rare.)
curl
curl -X PUT "https://api.braintrustdata.com/v1/role" \
  -d '{
  "name": "string",
  "description": "string",
  "member_permissions": [
    {
      "permission": "create",
      "restrict_object_type": "organization"
    }
  ],
  "member_roles": [
    "497f6eca-6276-4993-bfeb-53cbbbba6f08"
  ],
  "org_name": "string"
}'

A role is a collection of permissions which can be granted as part of an ACL

Roles can consist of individual permissions, as well as a set of roles they inherit from

Example Response
{
  "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
  "org_id": "a40f5d1f-d889-42e9-94ea-b9b33585fc6b",
  "user_id": "a169451c-8525-4352-b8ca-070dd449a1a5",
  "created": "2019-08-24T14:15:22Z",
  "name": "string",
  "description": "string",
  "deleted_at": "2019-08-24T14:15:22Z",
  "member_permissions": [
    {
      "permission": "create",
      "restrict_object_type": "organization"
    }
  ],
  "member_roles": [
    "497f6eca-6276-4993-bfeb-53cbbbba6f08"
  ]
}

GET

/v1/role/{role_id}

Get role