Skip to main content
Applies to:
  • Plan -
  • Deployment -

​
Summary

Issue: A trace-level scorer automation fails with Failed to fetch trace spans: ForbiddenError: Missing read access to project_log when the scorer lives in a different project than the traces it is scoring. Cause: Trace-level scorers fetch span data using the async scoring service principal of the project where the scorer is defined, not the project where the traces live — so that identity must have read access to the target project’s logs. Resolution: Grant the scorer’s async scoring service principal read access to the target project, either org-wide or per-project.

​
Resolution steps

​
Option 1: Grant org-wide Project:Read (shared scorer registry pattern)

Use this when a central scorer project scores traces across many feature projects and the scorer project is tightly access-controlled.

​
Step 1: Identify the async scoring service principal

Find the principal in the ForbiddenError log. It follows this pattern: 
bt::sp::service::project::<scorer_project_id>::async_scoring
You can also locate it in your org’s Members or service users list.

​
Step 2: Create an org permission group with Project:Read

In the Braintrust UI, create a new permission group scoped to Project:Read at the org level.

​
Step 3: Add the service principal to the group

Add only the async scoring service principal to this group. Do not add regular users. This allows the scorer runtime to read logs across all projects in the org.

​
Option 2: Grant per-project read access

Use this to limit blast radius when only specific feature projects need to be scored.

​
Step 1: Identify the scorer’s async scoring service principal

Same as Option 1, Step 1.

​
Step 2: Grant read access on the target project

In the target feature project’s settings, add the scorer’s service principal with Project:Read permission. This access inherits to project_log. Repeat for each additional feature project that uses the shared scorer.

​
Option 3: Copy a wrapper scorer into each feature project

Use this to avoid cross-project identity issues entirely.

​
Step 1: Create a local scorer in each feature project

Copy or wrap the shared scorer logic into a scorer defined directly in each feature project. The automation then runs under that feature project’s own async scoring identity, which already has read access to its own logs.

​
Security considerations

Granting org-wide Project:Read to a shared scorer’s async identity means any code executing under that principal can read logs across all projects while running. This is acceptable when:
  • Write access to the scorer registry project is restricted
  • Scorer code changes are reviewed before deployment
  • Scorer deployment is controlled by a locked-down service account
If the scorer project has looser controls, use Option 2 or Option 3 to limit access scope.