Access from Google Cloud

Your data always lands in an S3 bucket on AWS. If your company runs on Google Cloud and has no AWS account, you don’t need to create one: you read the bucket with a service account from your own Google Cloud project. It works like this:

You tell us which service account will read the data.
We create a read-only role for you on our side, and it trusts only that service account.
Your service account asks Google for an ID token and exchanges it with AWS for temporary credentials to that role.
You use those temporary credentials to read the bucket, with the same commands as the rest of this guide.

As in the AWS setup, we don’t issue credentials to you. No Cloud Humans key is stored on your side, and none of yours is sent to us. Every credential in this flow is short-lived and generated on demand.

Requirements

Data export enabled on your account. Contact the support team to request it.
A Google Cloud project where you can create a service account, and the gcloud CLI.
The AWS CLI or boto3 on the machine that downloads the files. Both read the bucket without any AWS account of your own.

Step 1: Create a dedicated service account

We recommend a service account used only for this export. It keeps the access easy to audit and to revoke.

gcloud iam service-accounts create cloudhumans-export \
  --project <YOUR_PROJECT> \
  --display-name "Cloud Humans data export"

The service account needs no IAM role in your Google Cloud project. Its only job is to prove its identity to AWS.

Step 2: Send us the service account’s unique ID

Every service account has a unique ID: a long number that never changes, even if the service account is renamed. This number, and nothing else, decides who can assume your role.

gcloud iam service-accounts describe \
  cloudhumans-export@<YOUR_PROJECT>.iam.gserviceaccount.com \
  --format "value(uniqueId)"

Send the support team:

What	Example
Unique ID (required)	`<YOUR_SERVICE_ACCOUNT_UNIQUE_ID>`, digits only
Service account email (for reference)	`cloudhumans-export@<YOUR_PROJECT>.iam.gserviceaccount.com`

The email does not replace the unique ID. If you delete the service account and create another one with the same name, the new one gets a different unique ID, and it cannot assume the role until you send us the new number.

Step 3: Receive your access details

Once the role is ready, we send you three values:

Value	What it is
`<ROLE_ARN>`	The ARN of the read-only role created for you. Your service account assumes it
`<AUDIENCE>`	The audience your ID token must carry, exactly as we send it
`<YOUR_BUCKET>`	Your bucket name, in the `us-east-1` region

The role can only list, download, and restore files in your bucket: s3:ListBucket, s3:GetObject, and s3:RestoreObject. You don’t need to attach any policy on your side, which is a difference from the AWS setup.

The audience is not a secret. It labels the token as meant for this export. What protects the role is the unique ID: AWS only accepts a token issued to your service account.

Step 4: Get a Google ID token

Your service account requests an ID token from Google with the audience we sent you. How it does that depends on where the download runs.

On Google Cloud compute
From your own machine

On Compute Engine, Cloud Run, Cloud Functions, or GKE with Workload Identity, run the job as the service account from step 1. The metadata server issues the token, with no key to store:

curl -s -H "Metadata-Flavor: Google" \
  "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/identity?audience=<AUDIENCE>&format=full" \
  > /tmp/cloudhumans-export-token

Impersonate the service account with your own Google login. Your user needs the Service Account Token Creator role (roles/iam.serviceAccountTokenCreator) on that service account.

gcloud auth print-identity-token \
  --impersonate-service-account cloudhumans-export@<YOUR_PROJECT>.iam.gserviceaccount.com \
  --audiences "<AUDIENCE>" \
  --include-email \
  > /tmp/cloudhumans-export-token

Don’t drop --impersonate-service-account. A token issued to your own user account is valid for Google, but it is not the service account we authorized, and AWS refuses it.

Avoid service account keys. Both options above work without downloading a JSON key, so there is nothing long-lived to leak or rotate.

Step 5: Configure the AWS CLI profile

Point the cloudhumans-export profile at the token file. The AWS CLI and boto3 exchange the token for temporary credentials on their own, and every command in this guide works unchanged.

~/.aws/config

[profile cloudhumans-export]
role_arn                = <ROLE_ARN>
web_identity_token_file = /tmp/cloudhumans-export-token
role_session_name       = cloudhumans-export
region                  = us-east-1

You don’t need a ~/.aws/credentials file, and you don’t need any AWS account.

The Google ID token expires after about an hour. Generate a new one (step 4) at the start of every run, before the first AWS command. If a run lasts longer than the token, regenerate it during the run: the CLI and boto3 read the file again whenever they renew the credentials.

Smoke test

Generate the token, then list the bucket:

aws s3 ls s3://<YOUR_BUCKET>/v1/ --profile cloudhumans-export

The expected output is the same as in the AWS smoke test. From here on, downloading works exactly as described for AWS customers: use the cloudhumans-export profile.

In a Python job

If you’d rather not write a token file, fetch the token and exchange it in code. On Google Cloud compute, fetch_id_token uses the metadata server; elsewhere, it uses the service account in GOOGLE_APPLICATION_CREDENTIALS.

import boto3
import google.auth.transport.requests
import google.oauth2.id_token

ROLE_ARN = "<ROLE_ARN>"
AUDIENCE = "<AUDIENCE>"
BUCKET = "<YOUR_BUCKET>"

token = google.oauth2.id_token.fetch_id_token(
    google.auth.transport.requests.Request(), AUDIENCE
)

credentials = boto3.client("sts", region_name="us-east-1").assume_role_with_web_identity(
    RoleArn=ROLE_ARN,
    RoleSessionName="cloudhumans-export",
    WebIdentityToken=token,
)["Credentials"]

session = boto3.Session(
    aws_access_key_id=credentials["AccessKeyId"],
    aws_secret_access_key=credentials["SecretAccessKey"],
    aws_session_token=credentials["SessionToken"],
    region_name="us-east-1",
)
s3 = session.client("s3")
print(s3.list_objects_v2(Bucket=BUCKET, Prefix="v1/", Delimiter="/")["CommonPrefixes"])

The call to AWS needs no AWS credentials: the Google token is the credential. The temporary credentials expire too, and credentials["Expiration"] tells you when. For a job that runs past it, fetch a new token and call assume_role_with_web_identity again.

Revoking access

The control stays with you. Disable or delete the service account in your Google Cloud project, and it can no longer get tokens, so it can no longer assume the role. Nothing on our end needs to change. To move the access to another service account, send the support team its unique ID. The old service account stops working as soon as we update the role.

Troubleshooting

`AccessDenied` on `AssumeRoleWithWebIdentity`

AWS checked the token and it doesn’t match the role. Decode the token’s payload and compare it with what you sent us:

cut -d. -f2 /tmp/cloudhumans-export-token | tr '_-' '/+' | base64 -d 2>/dev/null; echo

aud must be exactly <AUDIENCE>, with no extra slash or space.
sub must be the unique ID you sent us. A different number means the token came from another identity: usually your own user, when impersonation was left out.

`InvalidIdentityToken` or `ExpiredTokenException`

The token is malformed or older than about an hour. Generate a new one (step 4) and run the command again.

`AccessDenied` on `s3:ListBucket` or `s3:GetObject`

The role was assumed, so your identity is correct. Check the bucket name and the region (us-east-1). For files older than 180 days, see InvalidObjectState on download.

Getting Started

Integrations

Data Export

Access from Google Cloud

Requirements

Step 1: Create a dedicated service account

Step 2: Send us the service account’s unique ID

Step 3: Receive your access details

Step 4: Get a Google ID token

Step 5: Configure the AWS CLI profile

Smoke test

In a Python job

Revoking access

Troubleshooting

`AccessDenied` on `AssumeRoleWithWebIdentity`

`InvalidIdentityToken` or `ExpiredTokenException`

`AccessDenied` on `s3:ListBucket` or `s3:GetObject`

​Requirements

​Step 1: Create a dedicated service account

​Step 2: Send us the service account’s unique ID

​Step 3: Receive your access details

​Step 4: Get a Google ID token

​Step 5: Configure the AWS CLI profile

​Smoke test

​In a Python job

​Revoking access

​Troubleshooting

​AccessDenied on AssumeRoleWithWebIdentity

​InvalidIdentityToken or ExpiredTokenException

​AccessDenied on s3:ListBucket or s3:GetObject

Requirements

Step 1: Create a dedicated service account

Step 2: Send us the service account’s unique ID

Step 3: Receive your access details

Step 4: Get a Google ID token

Step 5: Configure the AWS CLI profile

Smoke test

In a Python job

Revoking access

Troubleshooting

`AccessDenied` on `AssumeRoleWithWebIdentity`

`InvalidIdentityToken` or `ExpiredTokenException`

`AccessDenied` on `s3:ListBucket` or `s3:GetObject`