> ## Documentation Index
> Fetch the complete documentation index at: https://braintrust.dev/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Transient JSONL.GZ files in S3 export bucket

export const plans_0 = "Plans: Any"

export const deployments_0 = "Deployments: Any"

export const data_plane_version_0 = "Data plane - v2.0 and earlier"

export const use_case_0 = "Use case - S3 export automation where users observe unexpected create-then-delete file events in their S3 bucket notification pipeline"

<Note>
  **Applies to:**

  * Plan - {plans_0}
  * Deployment - {deployments_0}
  * {data_plane_version_0}
  * {use_case_0}
</Note>

## Summary

**Issue:** The S3 exporter produces transient `jsonl.gz` files that are created and immediately deleted, generating noise in downstream S3 notification pipelines.

**Cause:** The exporter uses a streaming architecture that uploads files to S3 before the row count is known — if the final batch of an export cycle contains zero rows, the already-uploaded empty file is deleted.

**Resolution:** This is expected behavior; filter out transient files in your downstream pipeline using one of the approaches below.

***

## When this occurs

Two code paths produce transient files:

1. **End of export cycle** — The exporter iterates in batches. The final batch, when the exporter has caught up to current data, returns zero rows. The empty file is deleted after upload.
2. **Test automation button** — Creates and immediately deletes a test file to verify S3 access permissions.

No data is lost. Only empty (0-row) files and test files are deleted. Files containing actual exported data are never deleted by the exporter.

***

## Resolution steps

### Filter by object size

Empty gzipped JSONL files are approximately 20 bytes. Skip processing for files below a safe threshold.

```python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
def should_process(s3_event):
    size = s3_event["object"]["size"]
    return size > 100  # bytes
```

***

### Check file existence before processing

The delete follows the create almost immediately. A `HEAD` request confirms the file still exists before triggering downstream logic.

```python theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
import boto3
from botocore.exceptions import ClientError

def file_exists(bucket, key):
    s3 = boto3.client("s3")
    try:
        s3.head_object(Bucket=bucket, Key=key)
        return True
    except ClientError:
        return False
```

***

### Filter S3 event types

If your notification listener handles both `ObjectCreated` and `ObjectRemoved` events, restrict it to `ObjectCreated` only, then add the existence check above.

In your S3 event notification configuration, set the event filter to:

```
s3:ObjectCreated:*
```

Remove any `s3:ObjectRemoved:*` triggers.

***

### Use the export status API instead of S3 notifications

Poll the automation run history to confirm a successful export with actual rows before processing files. This avoids reacting to transient S3 events entirely. See the [Export to cloud storage documentation](/admin/automations/export-to-cloud-storage) for run result details.