OpenAI Agents SDK

When installed with the openai-agents extra, the Braintrust SDK provides a tracing.TracingProcessor implementation that sends the traces and spans from the OpenAI Agents SDK to Braintrust.

pip install braintrust[openai-agents]

import asyncio
 
from agents import Agent, Runner, set_trace_processors
from braintrust import init_logger
from braintrust.wrappers.openai import BraintrustTracingProcessor
 
 
async def main():
    agent = Agent(
        name="Assistant",
        instructions="You only respond in haikus.",
    )
 
    result = await Runner.run(agent, "Tell me about recursion in programming.")
    print(result.final_output)
 
 
if __name__ == "__main__":
    set_trace_processors([BraintrustTracingProcessor(init_logger("openai-agent"))])
    asyncio.run(main())

The constructor of BraintrustTracingProcessor can take a braintrust.Span, braintrust.Experiment, or braintrust.Logger that serves as the root under which all spans will be logged. If None is passed, the current span, experiment, or logger will be selected exactly as in braintrust.start_span.

OpenAI Agents SDK Logs

The Agents SDK can also be used to implement a task in an Eval, making it straightforward to build and evaluate agentic workflows:

from agents import Agent, Runner, set_trace_processors
from autoevals import ClosedQA
from braintrust import Eval
from braintrust.wrappers.openai import BraintrustTracingProcessor
 
set_trace_processors([BraintrustTracingProcessor()])
 
 
async def task(input: str):
    agent = Agent(
        name="Assistant",
        instructions="You only respond in haikus.",
    )
 
    result = await Runner.run(agent, input)
    return result.final_output
 
 
Eval(
    name="openai-agent",
    data=[
        {
            "input": "Tell me about recursion in programming.",
        }
    ],
    task=task,
    scores=[
        ClosedQA.partial(
            criteria="The response should respond to the prompt and be a haiku.",
        )
    ],
)

OpenAI Agents SDK Eval

OpenAI Agents SDK

On this page