s3-encryption-gateway

S3 Encryption Gateway Helm Chart

A Helm chart for deploying the S3 Encryption Gateway - a transparent proxy that provides client-side encryption for S3-compatible storage services.

Description

The S3 Encryption Gateway sits between S3 clients and backend storage providers, encrypting/decrypting data transparently while maintaining full S3 API compatibility. This Helm chart simplifies deployment to Kubernetes clusters.

Bucket creation is disabled by default. Set config.allowBucketCreation.value to "true" only with matching credential scope and an explicit create grant. rw does not infer management access. Delete is independently granted with delete; backend IAM must permit the operation. Authorized CreateBucket requests preserve LocationConstraint bytes and backend responses.

Repository

This chart is available at: https://cloud37.github.io/s3-encryption-gateway

Prerequisites

Installation

OCI registry (GHCR)

Signed OCI releases are published at ghcr.io/cloud37/s3-encryption-gateway. Install a specific chart version with:

helm install my-gateway oci://ghcr.io/cloud37/s3-encryption-gateway \
  --version 0.11.10

For strongest artifact pinning, install by the immutable manifest digest:

helm install my-gateway \
  oci://ghcr.io/cloud37/s3-encryption-gateway@sha256:<manifest-digest>

Verify the digest-qualified chart with the GitHub Actions keyless identity before installation (replace <manifest-digest> with the release digest):

cosign verify \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  --certificate-identity-regexp '^https://github\.com/cloud37/s3-encryption-gateway/\.github/workflows/helm\.yml@refs/heads/(main|master)$' \
  ghcr.io/cloud37/s3-encryption-gateway@sha256:<manifest-digest>

Classic GitHub Pages repository

Add the Helm repository

helm repo add s3-encryption-gateway https://cloud37.github.io/s3-encryption-gateway
helm repo update

Install the chart

helm install my-gateway s3-encryption-gateway/s3-encryption-gateway \
  --set config.backend.accessKey.valueFrom.secretKeyRef.name=my-secrets \
  --set config.backend.accessKey.valueFrom.secretKeyRef.key=access-key \
  --set config.backend.secretKey.valueFrom.secretKeyRef.name=my-secrets \
  --set config.backend.secretKey.valueFrom.secretKeyRef.key=secret-key \
  --set config.encryption.password.valueFrom.secretKeyRef.name=my-secrets \
  --set config.encryption.password.valueFrom.secretKeyRef.key=encryption-password

Configuration

All configuration options support two methods:

  1. Direct values: Set a value directly in values.yaml or via --set
  2. valueFrom: Reference values from existing Secrets or ConfigMaps

Using valueFrom with Secrets

Most sensitive values should be stored in Kubernetes Secrets and referenced:

config:
  backend:
    accessKey:
      valueFrom:
        secretKeyRef:
          name: s3-encryption-gateway-secrets
          key: backend-access-key
    secretKey:
      valueFrom:
        secretKeyRef:
          name: s3-encryption-gateway-secrets
          key: backend-secret-key
  encryption:
    password:
      valueFrom:
        secretKeyRef:
          name: s3-encryption-gateway-secrets
          key: encryption-password

Using valueFrom with ConfigMaps

Non-sensitive configuration can be stored in ConfigMaps:

config:
  backend:
    endpoint:
      valueFrom:
        configMapKeyRef:
          name: s3-encryption-gateway-config
          key: backend-endpoint
    region:
      valueFrom:
        configMapKeyRef:
          name: s3-encryption-gateway-config
          key: backend-region

Configuration Options

Basic Configuration

Parameter Description Default
replicaCount Number of replicas 1
track Blue/green or canary track label ("", "blue", "green", "stable", "canary"). Empty = no label (backward-compatible). When set, a shared external Valkey address is required and valkey.enabled must be false. ""
config.listenAddr Listen address ":8080"
config.logLevel Log level (debug, info, warn, error) "info"
config.proxiedBucket Single bucket proxy mode — restricts gateway to one backend bucket (optional) ""
config.policies Glob path to per-bucket policy YAML files mounted in the container (optional) ""

Backend Configuration

| Parameter | Description | Default | |———–|————-|———| | config.backend.endpoint | S3 backend endpoint URL | "https://s3.amazonaws.com" | | config.backend.region | S3 backend region | "us-east-1" | | config.backend.accessKey | Backend access key (use valueFrom) | "" | | config.backend.secretKey | Backend secret key (use valueFrom) | "" | | config.backend.provider | Provider hint string (optional) | "" | | config.backend.useSSL | Selects HTTPS for scheme-less backend endpoints; explicit http:// or https:// endpoint schemes take precedence | "true" | | config.backend.usePathStyle | Use path-style bucket addressing | "false" | Gateway credentials are validated at the gateway and backend credentials are never forwarded from clients. Configure config.auth.credentials[].buckets with exact names or trailing-prefix scopes such as tenant-*; omit it or use ["*"] for unrestricted access, or use [] for deny-all. A bare * is broad authority, particularly with create, delete, or manage. permissions is ro or rw, while bucketPermissions independently grants create, delete, and manage (bucket configuration administration). Root ListBuckets is allowed for both ro and rw (with omitted permissions defaulting to rw): an absent buckets scope returns backend-visible buckets, while an explicit empty buckets: [] scope returns a successful empty inventory.

Credential Migration and Reload

Existing credentials remain unrestricted and read-write when buckets and permissions are omitted. To migrate safely, first add explicit bucket scopes, then use ro for readers and grant create or delete only where required. Exact bucket names, the explicit unrestricted *, and non-empty trailing-prefix patterns such as tenant-* are supported; other wildcard forms are rejected.

Credential policy files and the main configuration file can be reloaded with SIGHUP when the deployment is configured to watch them. A failed reload keeps the previous complete policy snapshot. Helm-provided environment variables are process environment, so changing a Secret or Helm value requires a pod restart; it is not a live environment reload. Roll out credential changes atomically and verify the rendered environment before removing old access.

Encryption Configuration

Parameter Description Default
config.encryption.password Master encryption password (use valueFrom for production) ""
config.encryption.keyFile Path to an encryption key file (optional) ""
config.encryption.preferredAlgorithm Preferred AEAD algorithm (AES256-GCM, ChaCha20-Poly1305) "AES256-GCM"
config.encryption.supportedAlgorithms Comma-separated list of algorithms accepted for decryption "AES256-GCM,ChaCha20-Poly1305"
config.encryption.keyManager.enabled Enable envelope encryption key-manager mode "false"
config.encryption.keyManager.provider KEK provider: self_contained (default, no external KMS), cosmian, memory "self_contained"
config.encryption.keyManager.dualReadWindow Number of previous key versions tried during rotation "1"
config.encryption.keyManager.selfContained.type Sub-type: rsa (recommended) or aes "rsa"
config.encryption.keyManager.selfContained.rsa.privateKeySource RSA PEM key source — prefix env:VAR, base64:DATA, or file:PATH ""
config.encryption.keyManager.selfContained.rsa.keyVersion RSA key version (integer ≥ 1; increment to rotate) "1"
config.encryption.keyManager.selfContained.aes.activeVersion Active AES key version for wrapping new DEKs "1"
config.encryption.keyManager.selfContained.aes.keys Comma-separated AES-256 key entries ("1=env:VAR,2=base64:DATA") ""
config.encryption.keyManager.cosmian.endpoint Cosmian KMIP endpoint (JSON/HTTP: http://host:9998/kmip/2_1; binary: host:5696) ""
config.encryption.keyManager.cosmian.timeout KMS operation timeout "10s"
config.encryption.keyManager.cosmian.keys Comma-separated wrapping keys ("key1:v1,key2:v2") ""
config.encryption.keyManager.cosmian.caCert CA certificate for TLS (use valueFrom) ""
config.encryption.keyManager.cosmian.clientCert Client certificate for mTLS (use valueFrom) ""
config.encryption.keyManager.cosmian.clientKey Client private key for mTLS (use valueFrom) ""
config.encryption.keyManager.cosmian.insecureSkipVerify Skip TLS verification (testing only) "false"

KDF Configuration (V1.0-CRYPTO-1)

Parameter Description Default
config.encryption.kdf.algorithm KDF algorithm: "pbkdf2-sha256" (default, FIPS-compatible) or "argon2id" "pbkdf2-sha256"
config.encryption.kdf.pbkdf2.iterations PBKDF2-SHA256 iteration count (≥ 600000 recommended per NIST SP 800-132) "600000"
config.encryption.kdf.argon2id.time Argon2id passes (range: 1–10) "2"
config.encryption.kdf.argon2id.memory Argon2id memory in KiB (range: 8192–1048576) "19456"
config.encryption.kdf.argon2id.threads Argon2id parallelism/threads (range: 1–256) "1"

FIPS note: When the binary is compiled with -tags=fips, Argon2id is rejected at startup.

Key Manager Configuration: When config.encryption.keyManager.enabled is set to "true", the gateway performs envelope encryption — each object gets a unique DEK that is wrapped by a Key Encryption Key (KEK). Three providers are available:

See the KMS Compatibility Guide for details.

Example — self-contained RSA envelope encryption (recommended):

# Create the RSA private key Secret:
#   openssl genrsa 2048 | kubectl create secret generic self-contained-kek \
#       --from-file=private-key.pem=/dev/stdin
config:
  encryption:
    password:
      valueFrom:
        secretKeyRef:
          name: s3-encryption-gateway-secrets
          key: encryption-password
    keyManager:
      enabled:
        value: "true"
      provider:
        value: "self_contained"
      selfContained:
        type:
          value: "rsa"
        rsa:
          privateKeySource:
            # "env:VAR" tells the gateway to read the PEM from the RSA_KEK_PRIVATE_KEY env var.
            # The valueFrom below injects the Secret data into that env var.
            value: "env:RSA_KEK_PRIVATE_KEY"
          keyVersion:
            value: "1"
# Inject the key material as an env var via extraEnv:
extraEnv:
  - name: RSA_KEK_PRIVATE_KEY
    valueFrom:
      secretKeyRef:
        name: self-contained-kek
        key: private-key.pem

Example — self-contained AES envelope encryption (multi-version rotation):

# Store base64-encoded 32-byte keys in a Secret:
#   KEY1=$(dd if=/dev/urandom bs=32 count=1 2>/dev/null | base64 -w0)
#   KEY2=$(dd if=/dev/urandom bs=32 count=1 2>/dev/null | base64 -w0)
#   kubectl create secret generic aes-kek --from-literal=key1="$KEY1" --from-literal=key2="$KEY2"
config:
  encryption:
    keyManager:
      enabled:
        value: "true"
      provider:
        value: "self_contained"
      selfContained:
        type:
          value: "aes"
        aes:
          activeVersion:
            value: "2"      # wraps new objects with key v2
          keys:
            # "1=env:AES_KEK_V1,2=env:AES_KEK_V2" — both versions available for unwrap
            value: "1=env:AES_KEK_V1,2=env:AES_KEK_V2"
extraEnv:
  - name: AES_KEK_V1
    valueFrom:
      secretKeyRef:
        name: aes-kek
        key: key1
  - name: AES_KEK_V2
    valueFrom:
      secretKeyRef:
        name: aes-kek
        key: key2

Example — Cosmian KMIP provider:

config:
  encryption:
    password:
      valueFrom:
        secretKeyRef:
          name: s3-encryption-gateway-secrets
          key: encryption-password
    keyManager:
      enabled:
        value: "true"
      provider:
        value: "cosmian"
      dualReadWindow:
        value: "1"
      cosmian:
        endpoint:
          value: "https://cosmian-kms:9998/kmip/2_1"
        insecureAllowPlaintextTransport:
          value: "false"
        timeout:
          value: "10s"
        keys:
          value: "wrapping-key-1:1"
        caCert:
          valueFrom:
            secretKeyRef:
              name: cosmian-kms-certs
              key: ca-cert
        clientCert:
          valueFrom:
            secretKeyRef:
              name: cosmian-kms-certs
              key: client-cert
        clientKey:
          valueFrom:
            secretKeyRef:
              name: cosmian-kms-certs
              key: client-key
        insecureSkipVerify:
          value: "false"

Cosmian Protocol Selection:

Valkey (Multipart Upload State + ListObjects Size Cache)

Valkey (or any Redis-protocol-compatible store) is the gateway’s shared state backend. As of v1.0 a single Valkey instance and one connection pool serve two features:

  1. Encrypted multipart-upload state — in-flight UploadState blobs (mpu:<id>, 7-day TTL). Required (fail-closed) when any policy sets encrypt_multipart_uploads: true.
  2. ListObjects plaintext-size cache (V1.0-S3-3) — per-bucket hash plainsize:<bucket> so ListObjects[i].Size == HeadObject(key).Content-Length without per-object HEAD calls. Strongly recommended; fail-soft to ciphertext sizes if Valkey is unavailable.

Use the built-in Valkey subchart for development or point at an external cluster for production.

Valkey subchart (development / single-release deployments):

Parameter Description Default
valkey.enabled Deploy the Valkey subchart in-cluster. Set to false when using an external Valkey or in blue/green topologies. false
valkey.architecture Valkey architecture (standalone, replication) standalone
valkey.auth.enabled Enable Valkey authentication false

Blue/green and canary deployments: valkey.enabled must be false. Both tracks must share a single external Valkey cluster. Set the address via config.multipartState.valkey.addr.

External Valkey connection (config.multipartState.valkey.*):

Parameter Description Default
config.multipartState.valkey.addr External Valkey address (host:port). Auto-wired when valkey.enabled: true. ""
config.multipartState.valkey.tls.enabled Enable TLS for Valkey connection ""
config.multipartState.valkey.tls.caFile CA certificate file for Valkey TLS ""
config.multipartState.valkey.tls.certFile Client certificate file for Valkey mTLS ""
config.multipartState.valkey.tls.keyFile Client key file for Valkey mTLS ""
config.multipartState.valkey.insecureAllowPlaintext Allow plaintext Valkey (development only) ""
config.multipartState.valkey.ttlSeconds TTL for in-flight MPU state records in Valkey (default: 604800 = 7 days) ""
config.multipartState.valkey.stateV2Writer.enabled.value Enable the fixed state-v2 encrypted-MPU writer capability required after the v0.12 rollout ""
config.multipartState.valkey.encryptionPassword Dedicated password for Valkey at-rest encryption (V1.0-CRYPTO-2). Provide via .value (plaintext) or .valueFrom (secret ref). When set, VALKEY_ENCRYPT_STATE is automatically "true" ""

CRYPTO-2 note: When encryptionPassword is set (either .value or .valueFrom), the gateway automatically enables at-rest encryption (VALKEY_ENCRYPT_STATE=true). If unset, the gateway falls back to the main ENCRYPTION_PASSWORD with a distinct HKDF salt.

v0.12 encrypted-MPU rollout: Enable config.multipartState.valkey.stateV2Writer.enabled.value: "true" on the second Helm upgrade. Helm renders VALKEY_MPU_STATE_V2_WRITER=true for every replica; the gateway derives the fixed internal protocol capability. Before that upgrade, complete a separate scale-down to one old replica with helm upgrade RELEASE CHART --reuse-values --set replicaCount=1, then wait for kubectl rollout status deployment/DEPLOYMENT. Do not combine this scale-down with the image upgrade. Drain or abort all in-flight encrypted MPUs before the second upgrade. The chart uses maxSurge: 0 and maxUnavailable: 1, so this single old writer is terminated before its v0.12 replacement starts. That first state-v2 writer atomically initializes Valkey mpu:writer-version; later replicas verify the same value. Do not use this upgrade path with multiple old replicas, because legacy writers cannot be detected by the new release.

ListObjects size cache (config.listSizeTranslate.*) — V1.0-S3-3. Reuses the Valkey instance above; no separate deployment. All fields use the configValue shape (.value / .valueFrom).

Parameter Description Default
config.listSizeTranslate.enabled Enable size-cache lookup on ListObjects. Automatically false when Valkey is not configured. true (when Valkey configured)
config.listSizeTranslate.fallbackHeadEnabled Issue bounded concurrent HeadObject calls for cache misses. Disabled by default to avoid per-API-call billing amplification on Wasabi / R2 / B2. false
config.listSizeTranslate.fallbackHeadConcurrency Max concurrent HeadObject goroutines per listing page (1–100). 10
config.listSizeTranslate.fallbackHeadTimeout Per-page deadline for the HEAD batch (100ms–60s). 5s

Operational note: The size cache has no TTL — entries persist until evicted by DeleteObject/DeleteObjects. Objects uploaded before enabling the feature are not auto-warmed; temporarily set fallbackHeadEnabled: true to warm them via normal listing traffic. See docs/RUNBOOK.md.

Compression Configuration

| Parameter | Description | Default | |———–|————-|———|

Server Configuration

Parameter Description Default
config.server.readTimeout HTTP read timeout (0 = disabled; use readHeaderTimeout for slow-loris protection) "0s"
config.server.writeTimeout HTTP write timeout (0 = disabled; required for large object streaming) "0s"
config.server.idleTimeout HTTP idle connection timeout "60s"
config.server.readHeaderTimeout HTTP read header timeout "10s"
config.server.maxHeaderBytes Maximum request header size (bytes) "1048576"

TLS Configuration

Parameter Description Default
config.tls.enabled Enable TLS on the gateway listener "false"
config.tls.useCertManager Provision certificates automatically via cert-manager "false"
config.tls.certFile TLS certificate file path (when not using cert-manager) ""
config.tls.keyFile TLS private key file path (when not using cert-manager) ""

Note: When config.tls.enabled is "true" and config.tls.useCertManager is "false", you must provide certFile and keyFile (validated by the values schema). When TLS is enabled the Service automatically uses port 443 with port name https.

cert-manager Configuration

Parameter Description Default
certManager.issuer.name Issuer resource name (defaults to chart fullname) ""
certManager.issuer.namespace Issuer namespace (defaults to release namespace) ""
certManager.issuer.selfSigned Self-signed issuer spec (set to {} to use self-signed) {}
certManager.issuer.clusterIssuer Name of a pre-existing ClusterIssuer (alternative to selfSigned) ""
certManager.certificate.extraDNSNames Additional DNS SANs for the certificate []
certManager.certificate.duration Certificate validity period "2160h"
certManager.certificate.renewBefore Renew this long before expiry "720h"

cert-manager Integration: When config.tls.useCertManager is enabled, the chart automatically creates Issuer and Certificate resources. The TLS certificate and key are automatically mounted into the pod.

Rate Limiting

Parameter Description Default
config.rateLimit.enabled Enable per-connection rate limiting "false"
config.rateLimit.limit Maximum requests per window "100"
config.rateLimit.window Rate limit time window "60s"

Cache Configuration

Parameter Description Default
config.cache.enabled Enable response caching "false"
config.cache.maxSize Maximum cache size (bytes) "104857600"
config.cache.maxItems Maximum number of cached items "1000"
config.cache.defaultTTL Default cache entry TTL "5m"

Audit Configuration

Parameter Description Default
config.audit.enabled Enable audit logging "false"
config.audit.maxEvents Maximum in-memory audit events "10000"
config.audit.redactMetadataKeys Comma-separated metadata key names to redact from audit log ""
config.audit.sink.type Sink type (stdout, file, http) "stdout"
config.audit.sink.endpoint HTTP sink endpoint URL (for type=http) ""
config.audit.sink.filePath Log file path (for type=file) ""
config.audit.sink.batchSize Maximum events per write batch "100"
config.audit.sink.flushInterval Maximum time between flushes "5s"
config.audit.sink.retryCount Retries on failed writes "3"
config.audit.sink.retryBackoff Initial retry backoff duration "1s"

Ingress Configuration

Parameter Description Default
ingress.enabled Create a standard networking.k8s.io/v1 Ingress false
ingress.className Ingress class name ""
ingress.annotations Additional ingress annotations {}
ingress.hosts List of ingress hosts and paths []
ingress.tls TLS configuration for the Ingress []

Traefik CRD Ingress (mutually exclusive with ingress.enabled):

Parameter Description Default
ingress.traefik.enabled Render a Traefik IngressRoute CRD (requires Traefik ≥ v3.0) false
ingress.traefik.entryPoints Traefik entrypoint names ["websecure"]
ingress.traefik.host Host() matcher — the S3 hostname clients will use ""
ingress.traefik.tls TLS stanza (secretName, certResolver) {}
ingress.traefik.middlewares List of Traefik Middleware references (name, namespace) []
ingress.traefik.weighted.enabled Render a kind: Weighted TraefikService for canary traffic splitting false
ingress.traefik.weighted.services List of backend services with weights summing to 100 (name, port, weight) []
ingress.traefik.weighted.sticky Sticky session cookie config (cookie.name, httpOnly, secure, sameSite) {}

Common Ingress Annotations:

Deployment Configuration

Parameter Description Default
image.repository Container image repository cloud37io/s3-encryption-gateway
image.tag Container image tag "0.11.10"
image.pullPolicy Image pull policy IfNotPresent
imagePullSecrets Image pull secrets []
nameOverride Override the chart name portion of resource names ""
fullnameOverride Override the full resource name ""
podAnnotations Extra annotations added to every pod {}
podSecurityContext Pod-level security context (runAsNonRoot, runAsUser, fsGroup) See values.yaml
securityContext Container-level security context (readOnlyRootFilesystem, allowPrivilegeEscalation, seccompProfile, etc.) See values.yaml
terminationGracePeriodSeconds Pod termination grace period. Increase for blue/green drain (≥ p99 request duration + 10 s). 30
lifecycle Container lifecycle hooks. Use a preStop sleep equal to the kube-proxy propagation tail for zero-downtime traffic flips. {}
resources CPU/memory resource requests and limits See values.yaml
nodeSelector Node selector labels {}
tolerations Pod tolerations []
affinity Pod affinity/anti-affinity rules {}
topologySpreadConstraints Pod topology spread constraints []

Health Probes

Parameter Description Default
livenessProbe.httpGet.path Liveness probe path /live
livenessProbe.initialDelaySeconds Seconds before first liveness check 10
livenessProbe.periodSeconds Liveness probe period 30
livenessProbe.timeoutSeconds Liveness probe timeout 3
livenessProbe.failureThreshold Failures before pod is restarted 3
readinessProbe.httpGet.path Readiness probe path /ready
readinessProbe.initialDelaySeconds Seconds before first readiness check 5
readinessProbe.periodSeconds Readiness probe period 10
readinessProbe.timeoutSeconds Readiness probe timeout 3
readinessProbe.failureThreshold Failures before pod is removed from endpoints 3

The /ready endpoint performs dependency health checks (KMS, Valkey) and returns 503 with a JSON checks map if any configured dependency is unhealthy. The aliases /readyz, /healthz, and /livez follow Kubernetes conventions.

Service Account

Parameter Description Default
serviceAccount.create Create a ServiceAccount for the gateway true
serviceAccount.annotations Annotations to add to the ServiceAccount (e.g. IRSA / Workload Identity) {}
serviceAccount.name Override the generated ServiceAccount name ""

Service

Parameter Description Default
service.enabled Create a Kubernetes Service true
service.type Service type ClusterIP
service.port Service port (overridden to 443 when TLS is enabled) 80
service.targetPort Container target port 8080

Note: When service.enabled is false, ServiceMonitor is also disabled automatically. Keep the Service enabled for stable DNS-based service discovery.

Autoscaling (HPA)

Parameter Description Default
autoscaling.enabled Enable Horizontal Pod Autoscaler false
autoscaling.minReplicas Minimum number of replicas 2
autoscaling.maxReplicas Maximum number of replicas 10
autoscaling.targetCPUUtilizationPercentage CPU utilization target (%) 70
autoscaling.targetMemoryUtilizationPercentage Memory utilization target (%) 80
autoscaling.behavior.scaleDown.stabilizationWindowSeconds Scale-down stabilisation window 300
autoscaling.behavior.scaleUp.stabilizationWindowSeconds Scale-up stabilisation window 0

Pod Disruption Budget

Parameter Description Default
podDisruptionBudget.enabled Create a PodDisruptionBudget false
podDisruptionBudget.minAvailable Minimum available pods during disruption (integer or percentage) ""
podDisruptionBudget.maxUnavailable Maximum unavailable pods during disruption (integer or percentage) ""

Monitoring

Parameter Description Default
serviceMonitor.enabled Create a Prometheus Operator ServiceMonitor false
serviceMonitor.interval Scrape interval 30s
serviceMonitor.scrapeTimeout Scrape timeout 10s
serviceMonitor.labels Extra labels for ServiceMonitor (e.g. prometheus: kube-prometheus) {}
podMonitor.enabled Create a Prometheus Operator PodMonitor (alternative to ServiceMonitor) false
podMonitor.interval Scrape interval 30s
podMonitor.scrapeTimeout Scrape timeout 10s
podMonitor.labels Extra labels for PodMonitor {}
metrics.port Dedicated unauthenticated /metrics listener port; 0 uses the admin or S3 port fallback 0
metrics.enableBucketLabel Preserve bucket names on bucket-labelled S3 metrics; disabled mode uses bucket="*" false

ServiceMonitor vs PodMonitor: ServiceMonitor targets the Service (recommended). PodMonitor targets pods directly — useful when the Service is disabled or for fine-grained pod-level metrics. Both emit a track relabel rule when track is set, enabling per-track PromQL queries in blue/green topologies.

Client Traffic Metrics

The gateway exposes the following Prometheus metrics for S3 client traffic:

Input bytes are counted after AWS streaming framing is decoded; output bytes are counted only when successfully written. Headers, TLS framing, backend retries, and internal copy traffic are excluded.

Set metrics.enableBucketLabel: true to preserve bucket names. With the default false, all bucket labels are collapsed to * to bound cardinality. Labels never contain object keys, upload IDs, credentials, paths, or error text.

Network Policy

Parameter Description Default
networkPolicy.enabled Create a NetworkPolicy false
networkPolicy.policyTypes Policy types to enforce [Ingress, Egress]
networkPolicy.namespaceIsolation Restrict ingress to pods in the same namespace only true
networkPolicy.namespaceLabel.key Namespace label key used for isolation matching "kubernetes.io/metadata.name"
networkPolicy.egress.awsS3 Allow egress to AWS S3 (adjust CIDRs per region) false
networkPolicy.egress.minioInternal Allow egress to an in-cluster MinIO service false
networkPolicy.egress.monitoring Allow egress to monitoring/logging services false
networkPolicy.ingress.ingressControllers Allow ingress from ingress controller pods false
networkPolicy.ingress.ingressNamespace Namespace of the ingress controller "ingress-nginx"
networkPolicy.ingress.monitoring Allow ingress from Prometheus scrape pods false
networkPolicy.ingress.monitoringNamespace Namespace of the Prometheus stack "monitoring"

Namespace Isolation: When namespaceIsolation is enabled (default), only pods in the same namespace can access the gateway. Most Kubernetes distributions auto-label namespaces with kubernetes.io/metadata.name. If yours does not, either label it or set a custom namespaceLabel.key.

Extending the Chart

The Helm chart supports several extension points to customize the deployment for advanced use cases.

Extra Environment Variables

Add custom environment variables to the main container:

extraEnv:
  - name: MY_CUSTOM_VAR
    value: "my-value"
  - name: MY_SECRET_VAR
    valueFrom:
      secretKeyRef:
        name: my-secret
        key: my-key

Extra Volumes and Volume Mounts

Mount additional volumes into the main container:

extraVolumes:
  - name: my-config
    configMap:
      name: my-configmap
  - name: my-secret-volume
    secret:
      secretName: my-secret

extraVolumeMounts:
  - name: my-config
    mountPath: /etc/my-config
    readOnly: true
  - name: my-secret-volume
    mountPath: /etc/my-secrets
    readOnly: true

Init Containers

Run initialization containers before the main gateway starts:

initContainers:
  - name: init-myservice
    image: busybox:1.35
    command: ['sh', '-c', 'echo "Initializing..." && sleep 5']
    volumeMounts:
      - name: shared-data
        mountPath: /data

Sidecar Containers

Run sidecar containers alongside the main gateway:

sidecars:
  - name: sidecar-logger
    image: fluent/fluent-bit:2.0
    ports:
      - containerPort: 2020
    volumeMounts:
      - name: varlogcontainers
        mountPath: /var/log/containers
        readOnly: true

Examples

Basic Installation with Secrets

# Create secrets first
kubectl create secret generic s3-encryption-gateway-secrets \
  --from-literal=backend-access-key='YOUR_ACCESS_KEY' \
  --from-literal=backend-secret-key='YOUR_SECRET_KEY' \
  --from-literal=encryption-password='YOUR_ENCRYPTION_PASSWORD'

# Install with secrets
helm install my-gateway s3-encryption-gateway/s3-encryption-gateway \
  --namespace default

Deploy with self-contained KEK — no external KMS service needed. Ideal for environments where you want envelope encryption without additional infrastructure.

# Generate a 2048-bit RSA key and store it as a Kubernetes Secret:
openssl genrsa 2048 | kubectl create secret generic self-contained-kek \
    --from-file=private-key.pem=/dev/stdin

kubectl create secret generic s3-encryption-gateway-secrets \
  --from-literal=backend-access-key='YOUR_ACCESS_KEY' \
  --from-literal=backend-secret-key='YOUR_SECRET_KEY' \
  --from-literal=encryption-password='fallback-password-123456'

config:
  backend:
    endpoint:
      value: "https://s3.amazonaws.com"
    region:
      value: "us-east-1"
    accessKey:
      valueFrom:
        secretKeyRef:
          name: s3-encryption-gateway-secrets
          key: backend-access-key
    secretKey:
      valueFrom:
        secretKeyRef:
          name: s3-encryption-gateway-secrets
          key: backend-secret-key
  encryption:
    password:
      valueFrom:
        secretKeyRef:
          name: s3-encryption-gateway-secrets
          key: encryption-password
    keyManager:
      enabled:
        value: "true"
      provider:
        value: "self_contained"
      selfContained:
        type:
          value: "rsa"
        rsa:
          privateKeySource:
            value: "env:RSA_KEK_PRIVATE_KEY"
          keyVersion:
            value: "1"

extraEnv:
  - name: RSA_KEK_PRIVATE_KEY
    valueFrom:
      secretKeyRef:
        name: self-contained-kek
        key: private-key.pem

Notes:

KMS Mode with Cosmian KMIP

Deploy the gateway with external KMS (Cosmian KMIP) for envelope encryption and key rotation:

kubectl create secret generic s3-encryption-gateway-secrets \
  --from-literal=backend-access-key='YOUR_ACCESS_KEY' \
  --from-literal=backend-secret-key='YOUR_SECRET_KEY' \
  --from-literal=encryption-password='fallback-password-123456'

# If using TLS, also create certificate secrets
kubectl create secret generic cosmian-kms-certs \
  --from-file=ca-cert=/path/to/ca.pem \
  --from-file=client-cert=/path/to/client.crt \
  --from-file=client-key=/path/to/client.key

config:
  backend:
    endpoint:
      value: "https://s3.amazonaws.com"
    region:
      value: "us-east-1"
    accessKey:
      valueFrom:
        secretKeyRef:
          name: s3-encryption-gateway-secrets
          key: backend-access-key
    secretKey:
      valueFrom:
        secretKeyRef:
          name: s3-encryption-gateway-secrets
          key: backend-secret-key
  encryption:
    password:
      valueFrom:
        secretKeyRef:
          name: s3-encryption-gateway-secrets
          key: encryption-password
    keyManager:
      enabled:
        value: "true"
      provider:
        value: "cosmian"
      dualReadWindow:
        value: "1"
      cosmian:
        endpoint:
          # JSON/HTTPS (recommended): full URL or base URL with auto-appended /kmip/2_1
          value: "https://cosmian-kms:9998/kmip/2_1"
        insecureAllowPlaintextTransport:
          value: "false"
        timeout:
          value: "10s"
        keys:
          value: "wrapping-key-1:1"
        caCert:
          valueFrom:
            secretKeyRef:
              name: cosmian-kms-certs
              key: ca-cert
        clientCert:
          valueFrom:
            secretKeyRef:
              name: cosmian-kms-certs
              key: client-cert
        clientKey:
          valueFrom:
            secretKeyRef:
              name: cosmian-kms-certs
              key: client-key
        insecureSkipVerify:
          value: "false"

Notes:

Encrypted Multipart Uploads & ListObjects Size Cache with Valkey

Define bucket policies inline in values.yaml using the top-level policies list. The chart renders a ConfigMap, mounts it, and sets POLICIES automatically — no manual extraVolumes / extraVolumeMounts required. The same Valkey instance also powers the ListObjects plaintext-size cache (V1.0-S3-3), which is enabled by default whenever Valkey is configured.

Development / staging (in-cluster Valkey subchart)

config:
  auth:
    # Deprecated SigV2 is disabled by default. Set true only during migration.
    allowLegacySignatureV2:
      value: "false"

valkey:
  enabled: true                  # auto-wires VALKEY_ADDR → <release>-valkey:6379
  architecture: standalone
  auth:
    enabled: false               # enable + use existingSecret in production

policies:
  - id: encrypted-uploads
    buckets:
      - "my-important-bucket"
      - "logs-*"
    encrypt_multipart_uploads: true
    require_encryption: true
  - id: public-assets
    buckets:
      - "public-*"
    require_encryption: false

Each entry maps 1:1 to PolicyConfig (internal/config/policy.go). All fields are supported: encrypt_multipart_uploads, require_encryption, disallow_lock_bypass, and per-bucket encryption / rate_limit overrides.

Production (external Valkey, TLS)

Disable the subchart and point at your shared Valkey cluster. The policies list works identically:

valkey:
  enabled: false

config:
  multipartState:
    valkey:
      addr:
        value: "valkey-shared.mpu-state.svc.cluster.local:6379"
      tls:
        enabled:
          value: "true"
        caFile:
          value: "/etc/valkey-tls/ca.crt"
      insecureAllowPlaintext:
        value: "false"

policies:
  - id: encrypted-uploads
    buckets:
      - "my-important-bucket"
    encrypt_multipart_uploads: true
    require_encryption: true

The ListObjects size cache is enabled by default with the Valkey config above. Override the fallback HEAD batch (e.g. to warm legacy objects) under config.listSizeTranslate:

config:
  listSizeTranslate:
    enabled:
      value: "true"
    fallbackHeadEnabled:
      value: "true"        # opt-in; watch billing on per-API-call backends
    fallbackHeadConcurrency:
      value: "10"
    fallbackHeadTimeout:
      value: "5s"

Note: policies (top-level list) and config.policies.value / config.policies.valueFrom are mutually exclusive. Using both causes a render-time error (Guard 6). Use config.policies only when you need to reference policy files that are mounted from an external source outside the chart (e.g. from a corporate Secret store).

Single Bucket Proxy Mode

Restrict the gateway to a single backend bucket, minimising IAM policy requirements:

config:
  proxiedBucket:
    value: "my-secure-bucket"
  backend:
    endpoint:
      value: "https://s3.amazonaws.com"
    region:
      value: "us-east-1"
    accessKey:
      valueFrom:
        secretKeyRef:
          name: s3-encryption-gateway-secrets
          key: backend-access-key
    secretKey:
      valueFrom:
        secretKeyRef:
          name: s3-encryption-gateway-secrets
          key: backend-secret-key
  encryption:
    password:
      valueFrom:
        secretKeyRef:
          name: s3-encryption-gateway-secrets
          key: encryption-password

Backend Identity

The gateway authenticates to the backend S3 provider using its own credentials (config.backend.accessKey / config.backend.secretKey). Caller credentials are validated at the gateway and are never forwarded to the backend (ADR-0012). Each auth.credentials entry controls client access independently of the backend identity.

With Pod Lifecycle Hooks (Progressive Delivery)

For blue/green and canary deployments, configure a preStop hook to drain in-flight connections before the pod receives a SIGTERM:

terminationGracePeriodSeconds: 60
lifecycle:
  preStop:
    exec:
      command: ["sh", "-c", "sleep 10"]

Custom Configuration with ConfigMap

config:
  backend:
    endpoint:
      valueFrom:
        configMapKeyRef:
          name: s3-gateway-config
          key: backend-endpoint
    region:
      valueFrom:
        configMapKeyRef:
          name: s3-gateway-config
          key: backend-region
  rateLimit:
    enabled:
      value: "true"
    limit:
      value: "200"
    window:
      value: "60s"

With Autoscaling

autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 10
  targetCPUUtilizationPercentage: 70
  targetMemoryUtilizationPercentage: 80
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300

With Prometheus Monitoring

serviceMonitor:
  enabled: true
  interval: 30s
  scrapeTimeout: 10s
  labels:
    prometheus: kube-prometheus

With Ingress

ingress:
  enabled: true
  className: nginx
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    cert-manager.io/cluster-issuer: letsencrypt-prod
  hosts:
    - host: s3-gateway.example.com
      paths:
        - path: /
          pathType: Prefix
  tls:
    - secretName: s3-gateway-tls
      hosts:
        - s3-gateway.example.com

With Traefik IngressRoute

ingress:
  traefik:
    enabled: true
    entryPoints: ["websecure"]
    host: "s3-gateway.example.com"
    tls:
      certResolver: "letsencrypt"
    middlewares:
      - name: my-auth-middleware
        namespace: default

With Pod Disruption Budget

podDisruptionBudget:
  enabled: true
  minAvailable: 1
  # Alternative: maxUnavailable: "50%"

With Topology Spread Constraints

topologySpreadConstraints:
  - maxSkew: 1
    topologyKey: kubernetes.io/hostname
    whenUnsatisfiable: DoNotSchedule
    labelSelector:
      matchLabels:
        app.kubernetes.io/name: s3-encryption-gateway
  - maxSkew: 1
    topologyKey: topology.kubernetes.io/zone
    whenUnsatisfiable: ScheduleAnyway
    labelSelector:
      matchLabels:
        app.kubernetes.io/name: s3-encryption-gateway

With cert-manager TLS

config:
  tls:
    enabled:
      value: "true"
    useCertManager:
      value: "true"

certManager:
  issuer:
    name: s3-gateway-issuer
    selfSigned: {}
    # Or use Let's Encrypt:
    # clusterIssuer: letsencrypt-prod
  certificate:
    extraDNSNames:
      - s3-gateway.internal.example.com
    duration: "2160h"
    renewBefore: "720h"

With Namespace Isolation

service:
  enabled: true  # Keep enabled for stable DNS resolution

networkPolicy:
  enabled: true
  namespaceIsolation: true
  namespaceLabel:
    key: "kubernetes.io/metadata.name"
  egress:
    awsS3: false      # set true if backend is AWS S3
    minioInternal: true
  ingress:
    ingressControllers: true
    ingressNamespace: "ingress-nginx"
    monitoring: true
    monitoringNamespace: "monitoring"

With IRSA / Workload Identity

Annotate the ServiceAccount for AWS IRSA or GKE Workload Identity:

serviceAccount:
  create: true
  annotations:
    eks.amazonaws.com/role-arn: "arn:aws:iam::123456789012:role/s3-gateway-role"

Progressive Delivery

The chart ships production-safe blue/green and canary deployment recipes for zero-downtime upgrades. See docs/OPS_DEPLOYMENT.md for the complete runbook.

Key points

Quick start

# Blue side:
helm install gw-blue . \
  --values examples/values-blue.yaml \
  --set config.multipartState.valkey.addr.value=valkey-shared.mpu-state.svc.cluster.local:6379

# Green side (new version):
helm install gw-green . \
  --set image.tag=v0.11.10 \
  --values examples/values-green.yaml \
  --set config.multipartState.valkey.addr.value=valkey-shared.mpu-state.svc.cluster.local:6379

# Cutover:
docs/examples/bluegreen/cutover.sh green
# Rollback:
docs/examples/bluegreen/cutover.sh blue

Values Validation

This chart ships a values.schema.json (JSON Schema draft-07) that validates all values client-side before the chart reaches the cluster.

Helm enforces the schema during helm lint, helm install, helm upgrade, and helm template. Errors appear with JSON-path-prefixed messages like:

at '/replicaCount': got string, want integer
at '/config/logLevel/value': Must be one of: debug, info, warn, error
at '': 'not' failed  (both ingress.enabled and ingress.traefik.enabled are true)

What the schema catches early (before template rendering):

Rule Description
Type mismatches replicaCount: "2" (string) is rejected; must be an integer
Enum violations logLevel: verbose rejected; must be one of debug/info/warn/error
I1 — track + Valkey Setting track: blue with valkey.enabled: true is rejected
I2 — ingress mutex ingress.enabled: true + ingress.traefik.enabled: true is rejected
I3 — weighted requires Traefik weighted.enabled: true without traefik.enabled: true is rejected
I5 — KeyManager provider keyManager.enabled=true with an unknown provider is rejected
I7 — TLS cert required tls.enabled=true + useCertManager=false without certFile/keyFile is rejected

The schema is intentionally permissive at the root (additionalProperties: true) so that overlays like values.fips.yaml can add arbitrary pod annotations and extraEnv entries without triggering false positives. Strict additionalProperties: false is applied only at well-structured sub-trees (config.backend.*, config.encryption.*, ingress.traefik.weighted.*).

If helm lint fails with a JSON-path error, consult the description in values.schema.json at that path — the description contains the fix. To bypass schema validation in an emergency (template guards still fire):

helm install ... --disable-openapi-validation

Schema source

See docs/plans/V0.6-OPS-2-plan.md for the full design document.

Upgrading

helm repo update
helm upgrade my-gateway s3-encryption-gateway/s3-encryption-gateway

Uninstalling

helm uninstall my-gateway

Backend TLS

Set config.backend.tls.caFile.value to the path of a mounted PEM private CA to augment system roots. Use valueFrom for secret/config-map references. config.backend.tls.insecureSkipVerify.value defaults to "false"; enabling it is unsafe and intended only for local diagnostics. Mount the CA with the chart’s extraVolumes and extraVolumeMounts, then restart the gateway after changes.

Security Best Practices

  1. Use Secrets for Sensitive Data: Always use valueFrom.secretKeyRef for:
    • Backend access keys and secret keys
    • Encryption passwords
    • KMS credentials and TLS certificates

  2. RBAC: The chart creates a ServiceAccount. Annotate it for IRSA / Workload Identity where applicable.

  3. Network Policies: Enable network policies for additional security: 
    networkPolicy:
  enabled: true
  namespaceIsolation: true

    When namespaceIsolation is enabled, only pods in the same namespace can access the gateway.

  4. Single Bucket Proxy: Use proxiedBucket to restrict access to a single bucket, minimising IAM policy requirements.

  5. TLS: Enable TLS on the gateway listener (config.tls.enabled) and use cert-manager for automatic certificate rotation.

  6. FIPS: Use image.tag: 0.11.10-fips and the values.fips.yaml overlay for FIPS-140-compliant deployments (AES-256-GCM only; ChaCha20-Poly1305 excluded).

Troubleshooting

Check Pod Logs

kubectl logs -l app.kubernetes.io/name=s3-encryption-gateway

Check Pod Status

kubectl get pods -l app.kubernetes.io/name=s3-encryption-gateway

Test Health Endpoints

# Via Service:
kubectl port-forward svc/s3-encryption-gateway 8080:80
curl http://localhost:8080/readyz

# Directly to pod (when Service is disabled):
kubectl port-forward <pod-name> 8080:8080
curl http://localhost:8080/readyz

The /readyz endpoint returns 200 OK when all dependencies (KMS, Valkey) are healthy, or 503 with a JSON checks map identifying the failing dependency. Aliases: /healthz, /livez, /ready, /live.

Support

For issues, feature requests, or questions:

License

MIT License — see LICENSE for details.

This site is open source. Improve this page.