Overview and access

This guide walks you through extracting your data with the data export service, step by step. You’ll start by setting up access to the S3 bucket that holds your operation’s data. S3 is the file storage service AWS provides. Think of it as a drive in the cloud, where your data is kept. We load your data every day. Delivery is D-1: today, you have access to everything up to yesterday.

Requirements

To use data export, you need:

Data export enabled on your account. Contact the support team to request it.
Working knowledge of a few AWS services — S3, IAM, and the AWS CLI. If you’re not familiar with them, make sure your technical team is on hand to help.
An AWS account.

No AWS account? You can still receive your data, through a different setup. Talk to your Cloud Humans support team to explore the alternatives.

What you receive

Aspect	Details
Where	A dedicated S3 bucket for your company, in Cloud Humans’ account, `us-east-1` region
What	Parquet files, partitioned by date
When	Daily delivery. The pipeline runs at 07:00 UTC
How you access it	Directly, with your own AWS account’s credentials. No credential from us, no portal, no SFTP
Retention	Nothing is deleted on a timer. The full history stays available, though older files move to colder S3 storage classes

You will receive read-only access, meaning you cannot write, overwrite, or delete anything.

Granting access

To get access to your bucket: we don’t issue credentials to you. There’s no Cloud Humans access key for you to store, rotate, or explain in an audit. Instead, you tell us the ARN of an IAM principal in your own account, and we authorize that ARN on the bucket. An ARN is an Amazon Resource Name, the unique identifier AWS gives every resource — a role, a user, a bucket. See Amazon Resource Names (ARNs) in the AWS documentation for the full format. In practice: the credential is yours, the control is yours, and revoking access on your side is one line in your own policy. Nothing on our end needs to be undone.

Send us your ARN

We accept any of these formats:

arn:aws:iam::<YOUR_AWS_ACCOUNT>:role/<YOUR_ROLE>
arn:aws:iam::<YOUR_AWS_ACCOUNT>:user/<YOUR_USER>
arn:aws:iam::<YOUR_AWS_ACCOUNT>:root

We recommend a dedicated role, not :root and not a user with a static key. A role lets you change who assumes it without talking to us and without generating a long-lived credential.

What we apply on our side

As soon as we have the ARN, this is the exact policy we attach to the bucket:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DataExportClientRead",
      "Effect": "Allow",
      "Principal": { "AWS": "arn:aws:iam::<YOUR_AWS_ACCOUNT>:role/<YOUR_ROLE>" },
      "Action": ["s3:ListBucket", "s3:GetObject", "s3:RestoreObject"],
      "Resource": [
        "arn:aws:s3:::<YOUR_BUCKET>",
        "arn:aws:s3:::<YOUR_BUCKET>/*"
      ]
    }
  ]
}

s3:RestoreObject is there because files older than 180 days are archived; see file age and storage class.

What you need to configure on your side

Watch out! In cross-account access, the bucket policy is not enough on its own.AWS documentation is explicit: “When the principal and the resource are in separate AWS accounts, you must also use an identity-based policy to grant the principal access to the resource.” (IAM User Guide)In short: we grant access on our end, but your role also needs explicit permission. If AccessDenied shows up on the first command, start here.

Attach this to the role you gave us — or, if you use IAM Identity Center, to the permission set that provisions that role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CloudHumansDataExportRead",
      "Effect": "Allow",
      "Action": ["s3:ListBucket", "s3:GetObject", "s3:RestoreObject"],
      "Resource": [
        "arn:aws:s3:::<YOUR_BUCKET>",
        "arn:aws:s3:::<YOUR_BUCKET>/*"
      ]
    }
  ]
}

Don’t know your bucket name? Ask the Cloud Humans support team — the bucket is in our account, so the name comes from us.

Local profile

The policies above say which ARN may read the bucket. The local profile is how your machine becomes that ARN when you run a command — without it, the AWS CLI acts as a different identity and you get AccessDenied even with both policies correct. If your ingestion runs on AWS compute, you can skip this section. Send us the ARN of the role that compute already runs as — an EC2 instance profile, an ECS task role, a Lambda or Glue execution role — and attach the identity-based policy to that role. The SDK picks the credentials up on its own, with nothing for you to store or rotate: boto3.Session(), no profile_name. Otherwise, which of the three setups below fits depends on who runs the command:

A person at a terminal — IAM Identity Center. Nothing long-lived stays on the machine.
A scheduled ingestion job — a role assumed from a base profile. IAM Identity Center cannot serve this: aws sso login needs a browser, and the session expires within hours.
Static keys — only when neither of the above is available to you.

The three are mutually exclusive: set up one.

IAM Identity Center (SSO)
Assume a role from a base profile
IAM user with static keys

This setup uses two blocks: the profile’s sso_session is only a reference, and without the matching [sso-session] block, the CLI responds with The specified sso-session does not exist.

~/.aws/config

[profile cloudhumans-export]
sso_session    = cloudhumans-export
sso_account_id = <YOUR_AWS_ACCOUNT>
sso_role_name  = <YOUR_ROLE>
region         = us-east-1
output         = json

[sso-session cloudhumans-export]
sso_start_url = https://<YOUR_ORGANIZATION>.awsapps.com/start
sso_region    = <YOUR_IDENTITY_CENTER_REGION>

Before your first command, authenticate the session:

aws sso login --profile cloudhumans-export

This profile assumes a role, using your base profile for the underlying credentials:

~/.aws/config

[profile cloudhumans-export]
role_arn       = arn:aws:iam::<YOUR_AWS_ACCOUNT>:role/<YOUR_ROLE>
source_profile = default
region         = us-east-1

This setup uses long-lived static keys directly:

[cloudhumans-export]
aws_access_key_id     = <YOUR_ACCESS_KEY>
aws_secret_access_key = <YOUR_SECRET_KEY>

[profile cloudhumans-export]
region = us-east-1

Security: the credentials in all three examples are yours, issued by your own account. Cloud Humans never asks for, receives, or stores a customer’s key. If anyone asks you for yours on Cloud Humans’ behalf, it isn’t Cloud Humans.

Smoke test

A smoke test is a minimal command that confirms the access you just configured works. Before you pull 40 GB, spend three seconds confirming the door opens:

aws s3 ls s3://<YOUR_BUCKET>/v1/ --profile cloudhumans-export

Expected output:

                           PRE _manifests/
                           PRE datamart_claudia/
                           PRE datamart_cloudchat_saas/

One prefix per source schema, plus the manifests. For the tables inside each one, see the table catalog. This command only needs s3:ListBucket. If it works and the download fails later, the problem is s3:GetObject in your policy — not the network, not the credential. That already rules out half the possible causes.

Getting Started

Integrations

Data Export

Requirements

What you receive

Granting access

Send us your ARN

What we apply on our side

What you need to configure on your side

Local profile

Smoke test

​Requirements

​What you receive

​Granting access

​Send us your ARN

​What we apply on our side

​What you need to configure on your side

​Local profile

​Smoke test

Requirements

What you receive

Granting access

Send us your ARN

What we apply on our side

What you need to configure on your side

Local profile

Smoke test