# S3 gateway > How to serve the cluster's encrypted, replicated storage as an S3-compatible HTTPS endpoint from your nodes, manage its access keys, and use it from the AWS CLI and boto3. The S3 gateway gives the whole cluster one S3-compatible endpoint, served over HTTPS by the nodes you enable it on, so backups, media tools and programs written for S3 can keep their data in PowerCloud's encrypted, replicated storage. Every request must be signed with an access key you create with `pcctl`, each key can be limited to certain buckets or to reading, and storage providers still see only ciphertext. In outline, any gateway node can take a request: it checks the signature against the access keys in the nodes' consensus state, encrypts the object and writes it through the storage it serves. ```mermaid %% alt: S3 clients send signed HTTPS requests to any gateway node; each node checks the access key against its copy of the nodes' consensus state, encrypts the object and writes ciphertext to the storage locations it serves. flowchart TB C["S3 client
AWS CLI, boto3"] subgraph gw["Gateway nodes"] G1["home-1 :9443
keys from
consensus"] G2["colo-1 :9443
keys from
consensus"] end C -->|"HTTPS, SigV4"| G1 C -->|"or"| G2 ST[("Storage locations
ciphertext only")] G1 -->|"encrypted chunks"| ST G2 --> ST ``` ## Before you start The gateway builds on two things a cluster may not have yet: 1. **Nodes that serve the storage.** Each gateway node reads and writes through the storage locations it serves, set with `pcctl --repo ... node storage NAME` (see [Storage](https://powercloud.unidatum.com/docs/storage.md)). A node that serves no storage can be enabled, but `s3 enable` warns and its gateway answers every request with `ServiceUnavailable`. 2. **The nodes' consensus group.** Access keys live in the nodes' replicated consensus state, so `pcctl --repo ... consensus init` must have been run (see [Managing nodes](https://powercloud.unidatum.com/docs/nodes.md)). A gateway node whose consensus isn't running refuses requests with `ServiceUnavailable`. You also need the repository (`--repo` or `POWERCLOUD_REPO`) and the passphrase to enable the gateway and to create access keys, because the gateway's own storage key is kept in the cluster's encrypted store. ## Enable the gateway ```sh pcctl --repo LOC,... s3 enable [--port 9443] [--name DNS]... [--ip ADDR]... NODE... ``` | Flag | Meaning | Default | |---|---|---| | `--port` | Port the gateway listens on (HTTPS, all interfaces) | `9443` | | `--name` | A DNS name clients use to reach the gateway; put in its certificate. Repeatable. | none | | `--ip` | An address clients use, besides the node's own addresses; put in its certificate. Repeatable. Useful behind NAT or a load balancer. | none | For example, two nodes behind one DNS name: ```sh pcctl --repo "$POWERCLOUD_REPO" s3 enable --name s3.example.com home-1 colo-1 ``` ```text home-1: S3 gateway on port 9443, for s3.example.com, 192.0.2.10, ... (certificate until 2027-10-28) colo-1: S3 gateway on port 9443, for s3.example.com, 198.51.100.4, ... (certificate until 2027-10-28) Clients trust the cluster CA ('pcctl s3 ca > ca.pem', then AWS_CA_BUNDLE=ca.pem) and sign with an access key ('pcctl --repo ... s3 key add NAME'). ``` `s3 enable` issues each node a TLS server certificate from the cluster CA, valid for one year, for the node's own addresses plus the `--name`s and `--ip`s given. The certificate is good for serving TLS only: it grants nothing on the node API. Each node gets the gateway's storage key, which only gateway nodes receive. - **Renewing:** run `s3 enable` again on the same nodes before the certificate expires (`pcctl s3 status` shows days left). Give the same `--name` and `--ip` flags again, since the new certificate covers only what this run names. - **After `pcctl rotate`:** the cluster CA changes, so run `s3 enable` again and give clients the new CA certificate. - Any gateway node serves the same objects, so clients may use whichever node is up, or one DNS name pointing at several. To stop the gateway on nodes: ```sh pcctl s3 disable home-1 colo-1 ``` ```text home-1: S3 gateway stopped colo-1: S3 gateway stopped ``` ## Access keys ### Create a key ```sh pcctl --repo LOC,... s3 key add [--bucket B]... [--read-only] NAME ``` | Flag | Meaning | |---|---| | `--bucket` | A bucket the key may use. Repeatable. Without it, the key may use every bucket. | | `--read-only` | The key may only read (GET and HEAD requests). | `NAME` is a label for you, unique among the keys. For example, a read-write key for one bucket: ```sh pcctl --repo "$POWERCLOUD_REPO" s3 key add --bucket photos laptop ``` ```text access key PCXXXXXXXXXXXXXXXXXX for laptop; its secret isn't shown again: AWS_ACCESS_KEY_ID=PCXXXXXXXXXXXXXXXXXX AWS_SECRET_ACCESS_KEY=... ``` The secret (40 characters) is shown once; store it where the client will use it. The key is written to the nodes' consensus state with its secret sealed under a key derived from the gateway's, so every node keeps the entry but only gateway nodes can open it. It works on every gateway node as soon as that node applies the change, without `pcctl` reaching each one. A key limited to some buckets sees only those buckets when it lists, and gets `AccessDenied` for others. A read-only key gets `AccessDenied` for any write. ### List and remove keys ```sh pcctl s3 key ls pcctl s3 key rm laptop # by name or by access key ID ``` ```text NAME ACCESS KEY BUCKETS ACCESS CREATED backup PCYYYYYYYYYYYYYYYYYY all read-only 2026-10-20 laptop PCXXXXXXXXXXXXXXXXXX photos read-write 2026-10-28 ``` Removal takes effect on every gateway node as soon as it applies the change. A node that was down gets the change when it rejoins its peers. These commands need only your admin certificate, not the repository. ## Check the gateway ```sh pcctl s3 status ``` ```text NODE GATEWAY NAMES KEYS CERT colo-1 port 9443 s3.example.com 2 364d home-1 port 9443 s3.example.com 2 364d web-1 - ``` `GATEWAY` shows `-` for nodes without a gateway, `unreachable` for nodes that didn't answer, and `failed: ...` when a node's gateway couldn't start (for example, the port is taken). `KEYS` counts the access keys usable on that node. `CERT` shows the days until its certificate expires, or `expired`. ## Trust the CA certificate The gateway's certificate comes from your cluster CA, not a public authority, so clients must be told to trust it: ```sh pcctl s3 ca > ca.pem export AWS_CA_BUNDLE=$PWD/ca.pem ``` `pcctl s3 ca` prints the cluster CA certificate (PEM) from your admin credentials. `AWS_CA_BUNDLE` is read by the AWS CLI and boto3; most other tools take a CA file option. The CA certificate is public and safe to copy to client machines. After `pcctl rotate` it changes, so fetch it again. ## Use it from clients The endpoint is `https://HOST:9443`, where `HOST` is one of the names or addresses in the gateway's certificate (a `--name`, an `--ip`, or one of the node's own addresses). Requests are path-style: `https://HOST:9443/BUCKET/KEY`. The region can be any value; `us-east-1` is a safe choice. Requests must be within 15 minutes of the node's clock. ### AWS CLI ```sh export AWS_ACCESS_KEY_ID=PCXXXXXXXXXXXXXXXXXX AWS_SECRET_ACCESS_KEY=... export AWS_CA_BUNDLE=$PWD/ca.pem AWS_DEFAULT_REGION=us-east-1 aws --endpoint-url https://s3.example.com:9443 s3 cp --recursive ~/Pictures s3://photos/ aws --endpoint-url https://s3.example.com:9443 s3 ls s3://photos/ --recursive aws --endpoint-url https://s3.example.com:9443 s3 cp s3://photos/cat.jpg ./cat.jpg aws --endpoint-url https://s3.example.com:9443 s3 presign s3://photos/cat.jpg --expires-in 3600 ``` Uploads above the CLI's multipart threshold (8 MiB) go as multipart uploads, which the gateway supports. `--ca-bundle ca.pem` can replace `AWS_CA_BUNDLE`. ### boto3 ```python import boto3 s3 = boto3.client( "s3", endpoint_url="https://s3.example.com:9443", region_name="us-east-1", verify="ca.pem", aws_access_key_id="PCXXXXXXXXXXXXXXXXXX", aws_secret_access_key="...", ) s3.put_object(Bucket="photos", Key="a.jpg", Body=open("a.jpg", "rb")) s3.upload_file("big.mov", "photos", "big.mov") # multipart for large files data = s3.get_object(Bucket="photos", Key="a.jpg")["Body"].read() keys = [o["Key"] for o in s3.list_objects_v2(Bucket="photos").get("Contents", [])] s3.delete_object(Bucket="photos", Key="a.jpg") ``` ### Other clients The AWS CLI and boto3 are the clients checked in automated tests; others are untested. Any client you try must sign with AWS Signature Version 4 (presigned GET and HEAD may also use Signature Version 2) and use path-style addressing. The AWS CLI and boto3 use path style for a custom endpoint by themselves; other SDKs need it set, for example `UsePathStyle` in the Go SDK or `forcePathStyle` in the JavaScript SDK. Virtual-hosted style (`https://BUCKET.HOST/KEY`) is not supported. ## What is supported | Operation | Supported | |---|---| | ListBuckets | Yes: the buckets that hold objects, limited to a key's buckets | | Create bucket (PUT), HEAD bucket | Accepted; a bucket exists once an object is in it | | Delete bucket | Accepted, but does nothing; a bucket disappears when its last object is deleted | | PutObject, GetObject, HeadObject, DeleteObject | Yes | | ListObjectsV2 | Yes, with `prefix`, `delimiter`, `start-after`, `continuation-token` and `max-keys` (up to 1000 per page) | | Ranged GET | Yes: `bytes=A-B`, `bytes=A-` and `bytes=-N`. A request for several ranges gets the whole object | | Multipart uploads | Yes: create, upload part (parts 1 to 10000), complete, abort | | Presigned URLs | Yes, valid for up to 7 days; for GET and HEAD, also as version 1 of the AWS CLI presigns them (Signature Version 2) | | CopyObject | No (`NotImplemented`) | | DeleteObjects (several keys in one request) | No | | Versioning, ACLs, bucket policies, listing in-progress multipart uploads | No | ### Integrity checks on uploads The gateway checks every body as the client declared it, and the object exists only if it matches: its signed SHA-256, an unsigned payload, or `aws-chunked` uploads with each chunk signed or not and checksums (CRC32, CRC32C, CRC64NVME, SHA-1, SHA-256) in trailing headers, which is how current AWS SDKs upload. `Content-MD5` and `x-amz-checksum-*` headers are checked too. A body that doesn't match fails: its encrypted chunks, written as it arrived, are never part of an object, and the nodes' sync reclaims them. A request without a valid signature gets `403` and nothing else, not even whether a bucket exists. A wrong secret gets `SignatureDoesNotMatch`. A PutObject, step by step: ```mermaid %% alt: For a PutObject the gateway node checks the signature with the access key from its copy of the consensus state, checks the key's buckets and write access, encrypts the body as it arrives, writes the manifest only if the body matched what the client declared, and answers with its ETag. sequenceDiagram participant C as Client participant G as Gateway node participant K as Consensus participant S as Storage C->>G: PUT /photos/a.jpg, SigV4 G->>K: look up access key K-->>G: sealed secret, buckets G->>G: check signature, clock G->>G: key may write here? G->>S: encrypted chunks G->>G: body as declared? G->>S: manifest, if it was G->>K: record version G-->>C: 200 OK, ETag ``` ## How the data is stored - **A namespace of its own.** Gateway objects are encrypted under the gateway's own key and stored under blob IDs of their own. They don't appear in `pcctl ls`, and files written with `pcctl put` don't appear in the gateway. The cluster's own `check` and `gc`, and the purge of old blobs after `pcctl rotate`, leave them alone. The gateway's key itself is kept in the cluster's store. - **Same storage, same guarantees.** Objects go through the storage the gateway node serves, so they are replicated or erasure-coded over the same locations, with rollback protection through the nodes' consensus. - **Providers see ciphertext only**, with neither bucket nor object names. - Chunks of deleted or overwritten gateway objects are reclaimed by the gateway nodes during their storage sync. ## Limits - **Path-style requests only.** Virtual-hosted style isn't supported. - **No copy, multi-object delete, versioning, ACLs, policies, or listing of in-progress multipart uploads.** Abort an unfinished multipart upload from the client that started it. - **Revocation follows the consensus.** A gateway node cut off from a majority of the other nodes keeps the access keys it last had, a removed one included, until it rejoins. If a key leaked and a node may be cut off, stop that node's gateway with `pcctl s3 disable NODE`. - **Last writer wins.** Two gateways writing the same key at the same moment leave one of the two versions. - **Certificates last a year.** Run `s3 enable` again to renew, and after every `pcctl rotate`. - **Clock skew.** Signed requests must be within 15 minutes of the node's clock. For node setup, consensus and the mesh, see [Managing nodes](https://powercloud.unidatum.com/docs/nodes.md). For an S3 endpoint that an app gets privately on its own node, see [Cluster apps](https://powercloud.unidatum.com/docs/cluster-apps.md).