Skip to main content
Applies to:
  • Plan -
  • Deployment -

​
Summary

Issue: Using facets.* fields (for example, facets.Sentiment) in an online scoring automation SQL filter returns Error: Unknown field: facets, or the filter never matches traces. Cause: Online scoring automation SQL filters run when spans complete, but Topics writes facets later on a separate span. Online scoring does not re-run when facets are populated. Resolution: Use programmatic batch scoring with SQL and log_feedback() after Topics has populated facets.

​
Resolution steps

Real-time chaining (Topics automation to scoring automation) is not currently supported. Use the batch scoring approach below instead. Topics writes facets on the Topics automation span, not the root span. Use ANY_SPAN() to find matching traces and FILTER_SPANS(is_root) to return the root span id for log_feedback(). Adjust FILTER_SPANS() to return the span that has the fields your scorer needs.

​
Step 1: Query traces by facet value

SELECT id
FROM project_logs('<PROJECT_ID>', shape => 'traces')
WHERE ANY_SPAN(facets."Feature_category" = 'A')
  AND FILTER_SPANS(is_root)
LIMIT 100

​
Step 2: Score and attach results

import braintrust

logger = braintrust.init_logger(project_id="<PROJECT_ID>")

query = """
SELECT id
FROM project_logs('<PROJECT_ID>', shape => 'traces')
WHERE ANY_SPAN(facets."Feature_category" = 'A')
  AND FILTER_SPANS(is_root)
LIMIT 100
"""

for trace in braintrust.api_conn().post_json("btql", {"query": query})["data"]:
    score = my_scorer(trace)
    logger.log_feedback(id=trace["id"], scores={"my_score": score})

braintrust.flush()
Use the span id from the query for log_feedback() rather than the root_span_id or span_id.