Bucket layout

Every key in the bucket follows the same shape — once you can read one path, you can read all of them.

s3://<YOUR_BUCKET>/v1/datamart_claudia/claudia_conversation/snapshot_date=2026-08-06/0000_part_00.parquet
                   │  │                │                    │                        │
                   │  │                │                    │                        └── file
                   │  │                │                    └── daily partition
                   │  │                └── table
                   │  └── source schema
                   └── data contract version

Path segment	What it is
`v1/`	Contract version. An incompatible layout change becomes `v2/`, and `v1/` stays in place
`datamart_claudia/`	Source schema. There are two today, `datamart_claudia` and `datamart_cloudchat_saas` — the examples on this page use the first one, but treat the schema as something to discover, not a constant
`<TABLE>/`	One folder per exported table. See the table catalog for every table delivered and what one row of it means
`snapshot_date=<DATE>/`	Hive-style partition, with `<DATE>` in `YYYY-MM-DD` format. Athena, Spark, and DuckDB recognize it with no extra configuration
`NNNN_part_MM.parquet`	`NNNN` is the UNLOAD slice, `MM` is the size-based split. Files are split at a maximum of 256 MB
`_SUCCESS`	Written last. Only exists once the partition is complete
`_STATE`	Internal control for our ingestion, at the table root. Present only in incremental tables. Safe to ignore

Beyond the tables, there’s a manifest prefix:

s3://<YOUR_BUCKET>/v1/_manifests/snapshot_date=<DATE>/manifest.json

One manifest per snapshot date, recording what that day’s delivery holds. Its fields are:

Field	What it tells you
`complete`	`false` means the day is partial. Fall back to the latest `snapshot_date` holding `_SUCCESS`
`tables_delivered`, `tables_missing`	Which tables arrived and which did not. Every missing entry carries a `reason`
`mode`	How the table is delivered: `full_refresh` or `incremental_append`
`read`	How to read it: `latest_snapshot`, or `union_all_partitions` for an incremental table
`reseed`	`true` on an incremental table rebuilt from scratch. Drop your copy and ingest it again instead of appending, which would count every row twice

How to tell if a partition is ready

Always check for _SUCCESS before processing the partition. It’s written last, after every Parquet file in that partition has been closed. A partition without _SUCCESS is either still in progress or failed.

aws s3 ls s3://<YOUR_BUCKET>/v1/datamart_claudia/<TABLE>/snapshot_date=<DATE>/_SUCCESS \
  --profile cloudhumans-export

Empty output, exit code 1: not ready yet.

Getting Started

Integrations

Data Export

How to tell if a partition is ready

​How to tell if a partition is ready

How to tell if a partition is ready