Quickstart: Google Cloud to AWS

Keep the Google Cloud Storage client and request model while Cloud Adapter stores objects in Amazon S3.

Google Cloud API requests pass through Tensor9 Cloud Adapter to AWS services.

Compatibility snapshot

The Cloud Storage to AWS profile supports core JSON object CRUD and listing, XML request re-signing, lifecycle, and versioning behavior. Google retention policies and notifications are outside the profile. Generation identifiers adapt to S3 version or ETag behavior. ARCHIVE and COLDLINE semantics differ because some S3 objects require restore before read, and compose rejects non-final parts smaller than 5 MiB. Existing object bodies, versions, and shared links are not copied automatically.

Prerequisites

an installed Google Cloud-origin Cloud Adapter endpoint;
a Cloud Storage to S3 service adapter;
an origin-side Google credential accepted by the endpoint;
an AWS runtime identity with access to one test bucket;
a separate read-only AWS verification profile;
Python 3 with google-cloud-storage and AWS CLI v2.

Set values

export ADAPTER_STORAGE_ENDPOINT="https://adapter.example.test"
export GOOGLE_CLOUD_PROJECT="origin-project"
export ORIGIN_BUCKET="adapter-smoke-test"
export TARGET_BUCKET="adapter-smoke-test-aws"
export AWS_VERIFY_PROFILE="aws-target-verify"
export AWS_TARGET_REGION="us-east-1"
export TEST_OBJECT="quickstart/gcp-aws-$(date +%s).txt"

The API endpoint shape and origin authentication in this sample are deployment-specific placeholders. Use the exact endpoint and caller credential mode supplied by your Cloud Adapter deployment. Target AWS credentials remain adapter-owned.

Write through the Cloud Storage client

import os
import google.auth
from google.api_core.client_options import ClientOptions
from google.cloud import storage

credentials, _ = google.auth.default()
client = storage.Client(
    project=os.environ["GOOGLE_CLOUD_PROJECT"],
    credentials=credentials,
    client_options=ClientOptions(api_endpoint=os.environ["ADAPTER_STORAGE_ENDPOINT"]),
)

bucket = client.bucket(os.environ["ORIGIN_BUCKET"])
blob = bucket.blob(os.environ["TEST_OBJECT"])
expected = b"hello from Cloud Storage to S3\n"

blob.upload_from_string(expected, content_type="text/plain")
blob.reload()
actual = blob.download_as_bytes()
assert actual == expected
print({"generation": blob.generation, "etag": blob.etag, "size": blob.size})

Keep endpoint construction in one client factory. Target AWS credentials do not belong in the Google Cloud application.

Verify S3 natively

aws s3api head-object \
  --bucket "$TARGET_BUCKET" --key "$TEST_OBJECT" \
  --profile "$AWS_VERIFY_PROFILE" --region "$AWS_TARGET_REGION"

aws s3api get-object \
  --bucket "$TARGET_BUCKET" --key "$TEST_OBJECT" \
  --profile "$AWS_VERIFY_PROFILE" --region "$AWS_TARGET_REGION" \
  /tmp/cloud-adapter-target.txt

printf 'hello from Cloud Storage to S3\n' | cmp - /tmp/cloud-adapter-target.txt

Confirm content type and bytes. Do not copy a Google generation into code that expects an S3 version ID without following the documented identity translation.

Exercise a failure

Request an object name that does not exist and retain the Google API error code, HTTP status, request ID, and diagnostic headers. Also test a stale generation precondition if your application uses optimistic concurrency.

Clean up

blob.delete()

Confirm that the current target object is no longer readable. Versioned or soft-delete-enabled targets can retain prior versions or recovery state; use an unversioned disposable target for this quickstart or remove the created version explicitly. Bucket deletion and retained target data have separate ownership.

Before production

Test listing and pagination, generation preconditions, compose part sizes, lifecycle, versioning, storage classes, retries, and uncertain writes. Plan existing data and shared-link migration separately. If the workload uses retention or notifications, do not assume this route supplies them. Read the full Cloud Storage profile and add the lifecycle to CI/CD.

Tensor9 Cloud Adapter

How It Works

Getting Started

Configuration

Deployment

Operations

Tuning

Debugging

Testing

AWS

Google Cloud

Azure

Reference

Guides

Quickstart: Google Cloud to AWS

Compatibility snapshot

Prerequisites

Set values

Write through the Cloud Storage client

Verify S3 natively

Exercise a failure

Clean up

Before production

​Compatibility snapshot

​Prerequisites

​Set values

​Write through the Cloud Storage client

​Verify S3 natively

​Exercise a failure

​Clean up

​Before production

Compatibility snapshot

Prerequisites

Set values

Write through the Cloud Storage client

Verify S3 natively

Exercise a failure

Clean up

Before production