Skip to main content

​
Summary

Search timeouts on log queries can occur when custom columns use CAST() functions in their definitions. The CAST() operation forces the database to perform a full table scan instead of using optimized inverted indexes, resulting in significantly degraded query performance and potential timeouts. In most cases, the cast operation is unnecessary for display purposes in custom columns and can be safely removed to restore normal query performance.

​
Symptoms

  • Search queries timing out when filtering or querying logs with custom columns
  • Queries that previously completed quickly now taking excessively long to execute
  • Timeout errors appearing specifically when custom columns containing CAST() operations are involved in the query
  • Performance degradation correlating with the addition or use of custom columns that include type casting

​
Workarounds

Update your custom column definitions to remove unnecessary CAST() operations. This allows Braintrust to use inverted indexes for efficient queries. In most cases, type casting is not required for custom columns to display correctly in the UI.

​
Example: Before (Slow)

-- Custom column with CAST (causes full table scan)
CAST(metadata.user_id AS VARCHAR)

​
Example: After (Fast)

-- Custom column without CAST (uses inverted index)
metadata.user_id

​
Steps to Update Custom Columns

  1. Navigate to your project’s log view in the Braintrust UI
  2. Locate the custom columns that are using CAST() operations
  3. Edit each custom column definition to remove the CAST() function
  4. Save the updated custom column definition
  5. Test your queries to verify performance has improved

​
Option 2: Use Type Casting Only When Strictly Required

If type casting is genuinely necessary for your use case (such as performing type-specific operations or comparisons), consider applying the cast only in specific queries rather than in the custom column definition itself. This allows the base custom column to remain performant while enabling type conversion when needed.

​
Example: Apply CAST in Query Instead of Column Definition

-- In your query (not in the custom column definition)
SELECT * FROM logs
WHERE CAST(custom_column_name AS INTEGER) > 100

​
Technical Details

When a CAST() function is used in a custom column definition, the database query planner cannot utilize inverted indexes that would normally accelerate searches on JSON fields and metadata. Instead, the query engine must scan every row in the table and apply the cast operation, which becomes increasingly expensive as your log volume grows. This behavior is consistent with standard database query optimization principles where operations that transform indexed values prevent index usage.

​
Notes

  • Custom columns without CAST() operations will automatically display values in their native format, which is typically sufficient for most use cases
  • If you need to perform numeric comparisons or other type-specific operations, consider whether the cast can be applied at query time rather than in the column definition
  • This performance consideration applies to all custom columns across datasets, experiments, and log queries in Braintrust
  • After removing CAST() operations from custom columns, you should see immediate improvements in query performance without needing to wait for index rebuilds or other maintenance operations
  • Review all custom columns in your projects to identify and update any that may be using unnecessary type casting operations

​
References