S3 Buckets & Object Storage

How the Polycrate API manages S3 clusters and S3 buckets: creation via Web UI, REST API or Kubernetes Custom Resource, region resolution, capacity tracking, credentials, presigned uploads and billing.

S3 Buckets & Object Storage

Overview

The Polycrate API manages S3 clusters (technical storage clusters, e.g. MinIO, Ceph RadosGW) and S3 buckets (logical units for application data) with a complete lifecycle (create, provision, credentials, capacity, delete). Buckets can be created via the Web UI, the API or – since CLI 0.35.0 – declaratively via Kubernetes Custom Resources.

graph LR
    User[User / K8sApp] --> Bucket[S3Bucket]
    Bucket --> Cluster[S3Cluster<br/>MinIO / RadosGW]
    Bucket --> Creds[Credentials<br/>Access/Secret Key]
    Bucket --> Capacity[Capacity-Tracking]

Creating Buckets – Three Ways

1. Web UI

Storage → S3 Buckets → New

On save, Polycrate synchronously creates:

1. The bucket on the S3 cluster (MinIO or RadosGW)
2. An IAM user with access/secret key (only for this bucket)
3. A credentials record that is stored encrypted

2. REST API

POST /api/v1/s3/buckets/
{
  "name": "my-app-data",
  "region": "eu-central",
  "organization": "11111111-2222-3333-4444-555555555555",
  "workspace": "66666666-7777-8888-9999-000000000000",
  "cors_allow_all": false
}

If region is missing, the API uses the setting System Config → Default S3 Bucket Cluster. If this is not set either: HTTP 400 with a clear error message.

3. Kubernetes Custom Resource (new with CLI 0.35.0)

apiVersion: polycrate.io/v1alpha1
kind: S3Bucket
metadata:
  name: my-app-data
  namespace: my-app
spec:
  region: eu-central            # optional; falls back to the default
  corsAllowAll: false
  credentialsSecretName: my-app-s3   # optional; default: <cr-name>-s3

As soon as the CR is applied in the cluster:

1. The Polycrate Operator creates the bucket in the Polycrate API (including IAM user and credentials)

2. The access/secret key is stored as a Kubernetes Secret in the same namespace:

   apiVersion: v1
   kind: Secret
   metadata:
     name: my-app-s3
     namespace: my-app
   type: Opaque
   stringData:
     AWS_ACCESS_KEY_ID: "..."
     AWS_SECRET_ACCESS_KEY: "..."
     AWS_ENDPOINT_URL_S3: "https://minio.eu-central.example.com"
     BUCKET_NAME: "my-app-data"
   

3. The CR status shows the phases Pending → Provisioning → Ready

4. When the CR is deleted (kubectl delete s3bucket my-app-data), the bucket is deprovisioned in the API and the Secret is removed (finalizer pattern)

Region Resolution

The region in the request (region: eu-central) is resolved to an S3 cluster in this order:

1. Organization cluster: Does the organization have its own S3 cluster with this region? → use it
2. Platform service cluster: Is there a shared cluster with this region? → use it
3. First available cluster with this region
4. Default from SystemConfig (if region is missing entirely)
5. Otherwise: HTTP 400

If the resolved cluster has allow_new_buckets=false (e.g. because capacity is exhausted), the request is rejected with an understandable error message.

Capacity Tracking

Polycrate regularly fetches statistics from the provider per S3 cluster (MinIO Admin API or Ceph telemetry) and stores them per bucket:

These values feed into the pricing logic and are available via Prometheus metrics.

Credentials Access

• UI: Bucket detail → Credentials tab (shows access/secret only after an explicit "Reveal")
• API: GET /api/v1/s3/buckets/{id}/credentials/ — returns metadata without secrets
• Via CR: automatically as a Kubernetes Secret in the app namespace

Every bucket has its own IAM user – a compromise of one bucket does not affect other buckets.

Additional Access Keys (Ceph/RadosGW only)

For Ceph-based clusters (provider rook-ceph), multiple access keys with different permissions (read, write, readwrite) can be created per bucket:

POST /api/v1/s3/buckets/{id}/credentials/create
{
  "description": "CI/CD Read-only",
  "permissions": "read"
}

The response contains access_key and secret_key once:

{
  "id": "...",
  "description": "CI/CD Read-only",
  "permissions": "read",
  "access_key": "AKIAIOSFODNN7EXAMPLE",
  "secret_key": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
}

Presigned Browser Upload

For direct browser uploads into S3 buckets without going through a backend proxy, a short-lived presigned upload URL can be requested:

POST /api/v1/s3/buckets/{id}/objects/upload-url/
Authorization: Bearer <api-key>

{
  "key": "uploads/my-file.pdf",
  "content_type": "application/pdf",
  "size": 1048576,
  "overwrite": false
}

Response:

{
  "url": "https://s3.eu-central.example.com/my-bucket/uploads/my-file.pdf?...",
  "method": "PUT",
  "headers": {
    "Content-Type": "application/pdf"
  },
  "expires_at": "2026-07-15T14:30:00Z",
  "max_bytes": 5368709120
}

The browser then sends a PUT request directly (without a backend) to the returned URL. The URL is valid until expires_at and limited to the specified Content-Type and size.

Creating a Bucket Without a Workspace (via API)

Customers without their own Kubernetes cluster can create buckets via the API without specifying a workspace. In this case workspace can be null:

POST /api/v1/s3/buckets/
Authorization: Bearer <org-api-key>

{
  "name": "my-data",
  "region": "eu-central",
  "organization_id": "11111111-2222-3333-4444-555555555555",
  "workspace": null
}

The organization must always be specified. When using an Organization API Key, organization_id must belong to your own organization.

Productization & Billing

S3Bucket is a productized model: on creation, a default product (kind=object-storage) is assigned automatically, which comes from System Config → Default Products. Pricing follows the product's model (flat rate or volume-based via capacity tracking). Details: Pricing & Business Layer.

See Also

• Operator (CLI) – controller details for the S3Bucket CR
• Release CLI 0.35.0 – S3Bucket controller
• Release CLI 0.35.2 – agent token auto-assignment
• Release API 0.15.0 – capacity tracking, region default, filters
• Pricing & Business Layer