The OpenAI Agents SDK is OpenAI’s official framework for building AI agents. Braintrust integrates with the OpenAI Agents SDK using trace processors to capture agent execution, tool calls, and LLM interactions.
Setup
Install the Braintrust trace processor for OpenAI Agents:
# pnpm
pnpm add braintrust @braintrust/openai-agents @openai/agents
# npm
npm install braintrust @braintrust/openai-agents @openai/agents
Trace with OpenAI Agents SDK
Configure Braintrust’s trace processor to automatically send agent traces to Braintrust:
import asyncio
from agents import Agent, Runner, set_trace_processors
from braintrust import init_logger
from braintrust.wrappers.openai import BraintrustTracingProcessor
async def main():
agent = Agent(
name="Assistant",
instructions="You only respond in haikus.",
)
result = await Runner.run(agent, "Tell me about recursion in programming.")
print(result.final_output)
if __name__ == "__main__":
set_trace_processors([BraintrustTracingProcessor(init_logger("openai-agent"))])
asyncio.run(main())
The trace processor constructor can accept a braintrust.Span, braintrust.Experiment, or braintrust.Logger as the root for logging. If not provided, it automatically selects the current span, experiment, or logger.
Evaluate with OpenAI Agents SDK
Use OpenAI Agents SDK as the task in a Braintrust Eval to build and evaluate agentic workflows:
from agents import Agent, Runner, set_trace_processors
from autoevals import ClosedQA
from braintrust import Eval
from braintrust.wrappers.openai import BraintrustTracingProcessor
set_trace_processors([BraintrustTracingProcessor()])
async def task(input: str):
agent = Agent(
name="Assistant",
instructions="You only respond in haikus.",
)
result = await Runner.run(agent, input)
return result.final_output
Eval(
name="openai-agent",
data=[
{
"input": "Tell me about recursion in programming.",
}
],
task=task,
scores=[
ClosedQA.partial(
criteria="The response should respond to the prompt and be a haiku.",
)
],
)
Resources
