> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cloudhumans.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Storage classes and restoring archived data

> How file age maps to S3 storage class, and how to restore files archived in Glacier Flexible Retrieval.

## File age and storage class

As files age, they change storage class automatically. **Nothing is deleted.** What
changes is how you read them — and only at the last step.

| File age             | Class                             | How you read it                    |
| -------------------- | --------------------------------- | ---------------------------------- |
| 0 to 29 days         | S3 Standard                       | Direct `GET`                       |
| 30 to 89 days        | S3 Standard-IA                    | Direct `GET`, same latency         |
| 90 to 179 days       | S3 Glacier Instant Retrieval      | Direct `GET`, same latency         |
| **180 days or more** | **S3 Glacier Flexible Retrieval** | **Requires `RestoreObject` first** |

In other words: **nothing changes for you until day 179.** The only thing to watch in
this whole guide is day 180.

Two details that make your life easier:

* **Small files are never archived.** The transition only applies to objects of 128 KB
  or more, so `manifest.json` and `_SUCCESS` stay permanently in S3 Standard — you can
  always list, browse, and find out what exists in the bucket without restoring
  anything, no matter how old the period is.
* **Age is per file, not per partition.** A table with 2 years of history has the last
  6 months directly readable and the rest archived, and that changes file by file,
  every day.

Find out what's archived before you try to download it:

```bash theme={null}
aws s3api list-objects-v2 \
  --bucket <YOUR_BUCKET> \
  --prefix v1/datamart_claudia/<TABLE>/ \
  --query 'Contents[?StorageClass==`GLACIER`].Key' \
  --output text \
  --profile cloudhumans-export
```

## Restoring archived data

If you try to download a file in S3 Glacier Flexible Retrieval without restoring it
first, the API responds with `InvalidObjectState` (HTTP 403), and `aws s3 sync` skips
the file with a warning and exits with code 2. It isn't a permission failure — it's
the object telling you it's asleep.

### Request the restore

```bash theme={null}
aws s3api restore-object \
  --bucket <YOUR_BUCKET> \
  --key v1/datamart_claudia/<TABLE>/snapshot_date=<DATE>/0000_part_00.parquet \
  --restore-request '{"Days":7,"GlacierJobParameters":{"Tier":"Bulk"}}' \
  --profile cloudhumans-export
```

`Days` sets how many days the restored copy stays available for download. Seven days
is a comfortable value for a reprocessing job; choose whatever makes sense for you.

Three details worth knowing before you automate this:

* **`GlacierJobParameters` is optional.** `--restore-request '{"Days":7}'` works and
  uses the `Standard` tier. The CLI reference marks `Tier` as required, but that only
  applies *inside* the block: if you omit the whole block, AWS applies the default.
* **Repeating the restore of an already-restored object doesn't error** and doesn't
  restart the queue. You can call it inside a retry without worrying about
  idempotency.
* **Restoring an object that isn't archived errors out**, rather than no-op-ing:
  `InvalidObjectState: Restore is not allowed for the object's current storage class`.
  That applies to S3 Standard, S3 Standard-IA, and S3 Glacier Instant Retrieval. In
  other words, don't restore an entire partition blindly — filter by
  `StorageClass == GLACIER` first, as in the
  [restoring many files](/data-export/storage-classes#restoring-many-files) example.

### Choose the tier

| Tier        | Typical time   |
| ----------- | -------------- |
| `Bulk`      | 5 to 12 hours  |
| `Standard`  | 3 to 5 hours   |
| `Expedited` | 1 to 5 minutes |

<Tip>
  **Use `Bulk` by default.** Data older than 180 days is almost never urgent, and
  historical reprocessing is easy to plan a day ahead.

  `Expedited` exists for the legitimate exception — an incident, an audit, a regulatory
  deadline. If you need it, use it without hesitation.
</Tip>

Now, if fast restores become routine in your workflow, the problem isn't the tier —
it's that slice of data being cold when it should be hot.
[Contact your Cloud Humans account team](/data-export/troubleshooting#support-and-contract-changes):
this gets resolved on our side, and that's a lot better than living with a restore
step in every pipeline.

### Track progress

```bash theme={null}
aws s3api head-object \
  --bucket <YOUR_BUCKET> \
  --key v1/datamart_claudia/<TABLE>/snapshot_date=<DATE>/0000_part_00.parquet \
  --profile cloudhumans-export
```

In progress:

```json theme={null}
{ "Restore": "ongoing-request=\"true\"", "StorageClass": "GLACIER" }
```

Completed:

```json theme={null}
{
  "Restore": "ongoing-request=\"false\", expiry-date=\"Thu, 13 Aug 2026 00:00:00 GMT\"",
  "StorageClass": "GLACIER"
}
```

Notice that **`StorageClass` stays `GLACIER`**. That isn't a bug. The restore creates
a temporary accessible copy; the archived object stays where it was. After the
`expiry-date`, the copy disappears and the download fails again. If you need the data
permanently hot, copy it somewhere else while the copy exists.

To automate this instead of polling, S3 emits the `s3:ObjectRestore:Completed` event.
If you want to receive it,
[contact your Cloud Humans account team](/data-export/troubleshooting#support-and-contract-changes)
— the notification is configured on the bucket, and that's ours.

### Restoring many files

An entire partition, with one restore per object:

```bash theme={null}
aws s3api list-objects-v2 \
  --bucket <YOUR_BUCKET> \
  --prefix v1/datamart_claudia/<TABLE>/snapshot_date=<DATE>/ \
  --query 'Contents[?StorageClass==`GLACIER`].Key' \
  --output text \
  --profile cloudhumans-export \
| tr '\t' '\n' \
| while read -r key; do
    aws s3api restore-object \
      --bucket <YOUR_BUCKET> \
      --key "$key" \
      --restore-request '{"Days":7,"GlacierJobParameters":{"Tier":"Bulk"}}' \
      --profile cloudhumans-export
  done
```

This works fine for up to a few hundred objects. Above that, or if you need whole
months of history,
[contact your Cloud Humans account team](/data-export/troubleshooting#support-and-contract-changes)
first — S3 Batch Operations handles this far more efficiently, and the operation runs
on the bucket side.

### Download after the restore

This is where the gotcha lives — the one behind most "I restored it and it still
won't download" reports. **`cp` and `sync` behave differently**, and we measured both:

| Command                | Restored object      | Needs `--force-glacier-transfer`? |
| ---------------------- | -------------------- | --------------------------------- |
| `aws s3 cp <key>`      | downloads normally   | **no**                            |
| `aws s3 sync <prefix>` | **silently skipped** | **yes**                           |

The reason is mechanical: `sync` decides what to download from the prefix listing,
and the listing only reports the storage class — it doesn't carry the `Restore`
header. From `sync`'s point of view, a restored object is still "an object in
Glacier". A `cp` of a specific key, on the other hand, runs a `HeadObject` first,
sees the completed restore, and downloads without complaining.

So after the restore, use the flag on `sync`:

```bash theme={null}
aws s3 sync \
  s3://<YOUR_BUCKET>/v1/datamart_claudia/<TABLE>/snapshot_date=<DATE>/ \
  ./data/<TABLE>/snapshot_date=<DATE>/ \
  --force-glacier-transfer \
  --profile cloudhumans-export
```

If any object in the prefix still **isn't** restored, that `sync` fails on it with
`InvalidObjectState` and exits with code 1 — the rest download normally. Restore the
entire partition before syncing, or accept the exit code and handle it in your
orchestrator.
