Skip to main content

​
File age and storage class

As files age, they change storage class automatically. Nothing is deleted. What changes is how you read them — and only at the last step. In other words: nothing changes for you until day 179. The only thing to watch in this whole guide is day 180. Two details that make your life easier:
  • Small files are never archived. The transition only applies to objects of 128 KB or more, so manifest.json and _SUCCESS stay permanently in S3 Standard — you can always list, browse, and find out what exists in the bucket without restoring anything, no matter how old the period is.
  • Age is per file, not per partition. A table with 2 years of history has the last 6 months directly readable and the rest archived, and that changes file by file, every day.
Find out what’s archived before you try to download it:

​
Restoring archived data

If you try to download a file in S3 Glacier Flexible Retrieval without restoring it first, the API responds with InvalidObjectState (HTTP 403), and aws s3 sync skips the file with a warning and exits with code 2. It isn’t a permission failure — it’s the object telling you it’s asleep.

​
Request the restore

Days sets how many days the restored copy stays available for download. Seven days is a comfortable value for a reprocessing job; choose whatever makes sense for you. Three details worth knowing before you automate this:
  • GlacierJobParameters is optional. --restore-request '{"Days":7}' works and uses the Standard tier. The CLI reference marks Tier as required, but that only applies inside the block: if you omit the whole block, AWS applies the default.
  • Repeating the restore of an already-restored object doesn’t error and doesn’t restart the queue. You can call it inside a retry without worrying about idempotency.
  • Restoring an object that isn’t archived errors out, rather than no-op-ing: InvalidObjectState: Restore is not allowed for the object's current storage class. That applies to S3 Standard, S3 Standard-IA, and S3 Glacier Instant Retrieval. In other words, don’t restore an entire partition blindly — filter by StorageClass == GLACIER first, as in the restoring many files example.

​
Choose the tier

Use Bulk by default. Data older than 180 days is almost never urgent, and historical reprocessing is easy to plan a day ahead.Expedited exists for the legitimate exception — an incident, an audit, a regulatory deadline. If you need it, use it without hesitation.
Now, if fast restores become routine in your workflow, the problem isn’t the tier — it’s that slice of data being cold when it should be hot. Contact your Cloud Humans account team: this gets resolved on our side, and that’s a lot better than living with a restore step in every pipeline.

​
Track progress

In progress:
Completed:
Notice that StorageClass stays GLACIER. That isn’t a bug. The restore creates a temporary accessible copy; the archived object stays where it was. After the expiry-date, the copy disappears and the download fails again. If you need the data permanently hot, copy it somewhere else while the copy exists. To automate this instead of polling, S3 emits the s3:ObjectRestore:Completed event. If you want to receive it, contact your Cloud Humans account team — the notification is configured on the bucket, and that’s ours.

​
Restoring many files

An entire partition, with one restore per object:
This works fine for up to a few hundred objects. Above that, or if you need whole months of history, contact your Cloud Humans account team first — S3 Batch Operations handles this far more efficiently, and the operation runs on the bucket side.

​
Download after the restore

This is where the gotcha lives — the one behind most “I restored it and it still won’t download” reports. cp and sync behave differently, and we measured both: The reason is mechanical: sync decides what to download from the prefix listing, and the listing only reports the storage class — it doesn’t carry the Restore header. From sync’s point of view, a restored object is still “an object in Glacier”. A cp of a specific key, on the other hand, runs a HeadObject first, sees the completed restore, and downloads without complaining. So after the restore, use the flag on sync:
If any object in the prefix still isn’t restored, that sync fails on it with InvalidObjectState and exits with code 1 — the rest download normally. Restore the entire partition before syncing, or accept the exit code and handle it in your orchestrator.