> ## Documentation Index
> Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Roast

> Trace Roast AI workflows and individual cogs in Braintrust to debug step-by-step execution, inspect LLM calls, and monitor workflow failures

[Roast](https://github.com/Shopify/roast) is a Ruby framework for building structured AI workflows from steps called cogs, such as `chat`, `agent`, `ruby`, and `cmd`. Braintrust traces each workflow run and each cog in it, including the LLM calls that `chat` cogs make.

<View title="Ruby" icon="/images/sdk-icons/ruby.svg">
  <h2 id="setup-ruby">
    Setup
  </h2>

  Install the Braintrust and Roast gems, then configure your API keys. Roast integration requires Braintrust Ruby SDK v0.6.0 or later and `roast-ai` v1.0.0 or later, which requires Ruby 3.3 or later.

  <Steps>
    <Step title="Install gems">
      Add the gems to your Gemfile:

      ```ruby title="Gemfile" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
      gem "roast-ai"
      gem "braintrust"
      ```

      Then install them:

      ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
      bundle install
      ```
    </Step>

    <Step title="Set environment variables">
      ```bash title=".env" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
      BRAINTRUST_API_KEY=your-api-key
      BRAINTRUST_DEFAULT_PROJECT=your-project-name  # Project that spans are logged to

      # API key for the provider your chat cogs use (OpenAI by default)
      OPENAI_API_KEY=your-openai-key
      ```
    </Step>
  </Steps>

  <h2 id="auto-instrumentation-ruby">
    Auto-instrumentation
  </h2>

  To trace Roast workflows without modifying your workflow files, run the `roast` command through `braintrust exec`. Braintrust patches Roast when it loads, along with [RubyLLM](/docs/integrations/sdk-integrations/ruby-llm), which Roast's `chat` cog uses to call models.

  <Steps>
    <Step title="Create workflow.rb">
      This workflow passes a question from a `ruby` cog to a `chat` cog:

      ```ruby title="workflow.rb" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
      config do
        chat(:answer) { no_show_stats! }
      end

      execute do
        ruby(:question) { "What is the capital of France? Answer in three words or fewer." }
        chat(:answer) { ruby!(:question).value }
      end
      ```
    </Step>

    <Step title="Run the workflow">
      ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
      bundle exec braintrust exec -- roast execute workflow.rb
      ```

      `braintrust exec` initializes Braintrust and logs to the project set by `BRAINTRUST_DEFAULT_PROJECT`.
    </Step>
  </Steps>

  To run workflows from your own Ruby code instead, load `braintrust/setup` before you require `roast`. When you call `Roast::Workflow.from_file` inside a span, the workflow's span nests under it.

  ```ruby title="run_workflow.rb" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
  require "braintrust/setup"
  require "roast"

  Roast::Workflow.from_file("workflow.rb", Roast::WorkflowParams.new([], [], {}))
  ```

  <h2 id="manual-instrumentation-ruby">
    Manual instrumentation
  </h2>

  To trace Roast manually, initialize Braintrust with `auto_instrument: false` and enable instrumentation yourself with `Braintrust.instrument!(:roast)`. Also call `Braintrust.instrument!(:ruby_llm)` to trace the LLM calls that `chat` cogs make.

  ```ruby title="run_workflow.rb" theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
  require "braintrust"
  require "roast"

  Braintrust.init(default_project: "My Project", auto_instrument: false)

  Braintrust.instrument!(:roast)
  Braintrust.instrument!(:ruby_llm)

  Roast::Workflow.from_file("workflow.rb", Roast::WorkflowParams.new([], [], {}))
  ```

  <h2 id="what-traced-ruby">
    What Braintrust traces
  </h2>

  Braintrust captures:

  * Workflow spans (`roast.workflow`), with the workflow file name in metadata (`contrib.roast.workflow.name`), plus the workflow's targets, positional arguments, and keyword arguments as input when you pass them.
  * Cog spans (`roast.cog.<name>`), nested under the workflow span, with the cog's type, name, and outcome (`completed`, `failed`, or `skipped`) in metadata. Cogs in a scope that a `call` cog runs nest under that cog's span.
  * Cog output for `ruby` cogs that return a string, and the model response for `chat` cogs.
  * LLM call spans (`ruby_llm.chat`) for `chat` cogs, nested under the cog span, with the messages, model, response, and token usage that the [RubyLLM integration](/docs/integrations/sdk-integrations/ruby-llm#what-traced-ruby) records.
  * Errors, recorded on the cog span when a cog fails.

  <h2 id="resources-ruby">
    Resources
  </h2>

  * [Roast on GitHub](https://github.com/Shopify/roast)
  * [Roast tracing example](https://github.com/braintrustdata/braintrust-sdk-ruby/blob/main/examples/contrib/roast/basic.rb)
  * [Braintrust Ruby SDK](https://github.com/braintrustdata/braintrust-sdk-ruby)
</View>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.