Kubernetes Pod Certificates: A New mTLS Identity Path for API Gateways

Kubernetes workloads have long used service account JWTs as their native identity. Kubernetes 1.37 adds another production-grade path: Pod Certificates and ClusterTrustBundles are now stable, bringing X.509 certificate issuance and trust distribution into core Kubernetes APIs.

The release reached Hacker News through a discussion of the new identity primitives. For API platform teams, the practical question is not whether certificates replace every service account token. It is where proof-of-possession identity can strengthen the connection between a workload and an API gateway, and how certificate rotation changes gateway operations.

Pod Certificates make workload mTLS easier to automate. They do not automatically configure a certificate authority, choose authorization policy, reload a gateway's trust store, or decide which API routes a workload may call. Those responsibilities remain part of the platform design.

Key Takeaways

• Kubernetes 1.37 graduates Pod Certificates and ClusterTrustBundles to stable, but a signer controller is still required to issue certificates for a chosen signer.
• Kubelet generates the pod's private key and projects rotating credentials into the workload filesystem; the private key does not need to leave the node or pod boundary.
• ClusterTrustBundles distribute X.509 trust anchors through a validated, Kubernetes-native object and projected volume.
• A certificate proves possession of a private key and an asserted workload identity. The API gateway must still authorize that identity by route, method, tenant, and environment.
• Applications and gateways must notice projected-file updates and reload credentials or trust anchors without dropping traffic.
• Apache APISIX mTLS supports client-to-gateway and gateway-to-upstream authentication as two separate TLS relationships.

Why Kubernetes Added a Certificate Identity Path

Service account tokens are bearer credentials. A receiver validates the signed JWT and its audience, time, and object bindings, but any party holding a usable token can present it. Short lifetimes and audience restrictions reduce risk; they do not turn the credential into proof of private-key possession.

The Kubernetes 1.37 Pod Certificates announcement explains the alternative. In TLS, the workload holds a private key and presents a CA-signed certificate containing its public key and identity. Authentication proves that the caller possesses the corresponding private key without sending that private key to the peer.

This makes X.509 useful for mTLS connections to API gateways, databases, message brokers, and other systems that already understand certificate identities. It also changes the leakage model: copying a public certificate is harmless without the private key. An attacker that reads the projected credential bundle can still steal both, so filesystem isolation, pod security, and short lifetimes remain important.

Pod Certificates complement JWTs rather than universally replacing them. JWT federation is broadly supported by cloud IAM systems and is convenient for audience-scoped authentication. Certificates are valuable when the peer already operates a PKI trust model or when proof of possession and mTLS are required.

How Pod Certificates and Trust Bundles Work

The flow has four main actors:

sequenceDiagram
    participant P as Application Pod
    participant K as Kubelet
    participant S as Signer Controller
    participant G as API Gateway
    K->>K: Generate private key for the pod
    K->>S: Create PodCertificateRequest
    S-->>K: Return signed certificate chain
    K-->>P: Project rotating key and certificate files
    K-->>P: Project ClusterTrustBundle files
    P->>G: Connect with mTLS certificate
    G->>G: Verify signer trust and authorize identity

An administrator chooses a signer name and deploys a signer controller. The pod specification requests a podCertificate projected volume for that signer. After scheduling, Kubelet generates a private key, submits a PodCertificateRequest, receives the certificate chain, and writes the credential files into the pod.

Kubelet refreshes the files as the certificate approaches its beginRefreshAt time. The Kubernetes blog notes that future in-tree signers will issue certificates with a maximum lifetime of 24 hours, while other signers are limited to 91 days. Short-lived certificates reduce the useful life of stolen credentials, but only if consumers reload them correctly.

ClusterTrustBundles solve the other half of PKI. A certificate is meaningful only when the verifier has the relevant trust anchors. A ClusterTrustBundle is a cluster-scoped, validated object that can be associated with a signer and projected into workloads. Kubelet updates the projected file when the selected bundle changes.

The application must watch or poll both credential and trust files. Kubernetes rotates files; it cannot force a language runtime, connection pool, or gateway process to reconstruct its TLS configuration.

Where mTLS Fits Around an API Gateway

An API gateway commonly participates in two independent TLS relationships:

flowchart LR
    W[Workload with Pod Certificate] -->|client mTLS| G[Apache APISIX or API7 Gateway]
    G -->|separate upstream TLS or mTLS| U[Backend API]
    C[Signer and trust policy] -.-> W
    C -.-> G

Workload to Gateway

The workload presents its Pod Certificate as a client certificate. APISIX mTLS can verify the chain against the configured CA. That verification does not, by itself, provide a documented automatic mapping from certificate SANs to an APISIX Consumer or route policy. If authorization uses certificate identity, the platform must add a version-validated authentication plugin, policy engine, or custom integration that extracts approved SANs and maps them to an authorization decision.

Authentication is not authorization. A valid certificate for payments/reporting should not automatically grant access to a production refund endpoint. Route, method, tenant, data classification, and environment remain policy inputs. API7.ai's guide to mTLS and other API authentication methods provides the wider decision context.

Gateway to Upstream

The gateway may present a different client certificate when calling the upstream. Apache APISIX supports upstream mTLS through tls.client_cert and tls.client_key, or by referencing a client certificate object where supported. This identity represents the gateway or a gateway deployment, not necessarily the original workload.

If the backend needs caller-level identity, propagate an authenticated identity assertion in a protected header or token and make the backend trust only the gateway path. Do not reuse the caller's private key at the gateway; that would break the workload's proof-of-possession boundary.

The two legs can use different signers, lifetimes, and trust domains. Document them separately so rotation or revocation on one side does not create assumptions about the other.

Design the Signer and Identity Contract

Kubernetes supplies the request and projection mechanisms, not the organization's identity semantics. The signer profile must define:

• which namespaces, service accounts, and workloads may request certificates;
• how those attributes appear in certificate SANs;
• allowed key types, usages, and validity periods;
• which gateway environments trust the signer;
• how compromise, emergency rotation, and signer retirement work;
• which certificate identity maps to which API policy.

Avoid using a free-form Common Name as the authorization contract. Prefer a structured SAN scheme that is stable, documented, and validated by the signer. Keep development and production trust domains separate unless cross-environment access is deliberate.

ClusterTrustBundle write permissions are security-sensitive. Signer-linked bundles require authorization for the attest verb on the relevant signer name. A malicious trust-anchor update could make the gateway accept attacker-issued certificates, so changes need restricted RBAC, audit logs, and an emergency rollback path.

Trust distribution is not revocation. Short-lived certificates reduce dependence on certificate revocation lists, but a compromised credential may remain usable until expiration unless the gateway or signer has an additional deny mechanism. Keep gateway authorization capable of immediately disabling an identity or signer during an incident.

Plan Rotation Before Migration

Certificate rotation is where a sound architecture often fails in implementation. Test the complete path:

1. Kubelet writes a new credential bundle.
2. The client detects the file change and creates new TLS connections with the new key pair.
3. Existing connections drain according to policy.
4. The gateway accepts both old and new chains during the intended overlap.
5. Trust-anchor changes propagate to the gateway verifier.
6. Expired credentials stop working without causing a fleet-wide outage.

When Kubelet writes the key and certificate chain to one credential-bundle file, the application can reload one atomic projection. Separate key and certificate files require care to avoid reading them from different rotation generations.

Long-lived HTTP/2, gRPC, WebSocket, and pooled HTTP connections can outlive a certificate rotation. The new certificate is used only when a new TLS handshake occurs. Decide whether clients periodically recycle connections and ensure that connection churn cannot overwhelm the gateway.

If APISIX or another gateway consumes projected trust or client-credential files, verify how that exact deployment reloads them. Mounting an updated file does not prove that an already-running process has rebuilt its TLS context. Use a supported dynamic configuration path, controlled reload, or sidecar/controller integration, and test it under traffic.

A Migration Checklist for Platform Teams

• Confirm Kubernetes 1.37 feature availability across every target cluster.
• Select or deploy a signer controller and document its identity profile.
• Restrict PodCertificateRequest and signer-related permissions by workload and environment.
• Define SAN-to-gateway identity mapping before issuing production certificates.
• Project the required ClusterTrustBundles and protect their mutation with RBAC and audit logging.
• Configure gateway client-certificate trust and keep route authorization separate.
• Test atomic credential reload, trust-anchor rollover, and long-lived connections.
• Keep an immediate deny path for a compromised workload or signer.
• Record certificate identity, signer, route, policy result, and request ID without logging private material.
• Roll out to a narrow service first and compare authentication errors, handshake latency, and renewal behavior.

The API7 and APISIX Connection

Apache APISIX can protect HTTPS routes with client mTLS and can present a client certificate to an mTLS upstream. Certificate-to-authorization mapping remains a separate integration layer. Once that layer is explicitly configured and tested, APISIX can enforce the resulting route policy, rate limits, and observability controls.

API7 Enterprise adds centralized management for gateway environments and policies. In a multi-cluster deployment, keep certificate issuance local to each cluster or trust domain while managing consistent authorization intent at the gateway layer. The earlier API7.ai walkthrough for configuring mTLS with APISIX covers the gateway mechanics; Pod Certificates improve how Kubernetes workloads obtain and rotate the client identity supplied to that connection.

Do not describe this as automatic zero trust. Pod Certificates establish a stronger credential primitive. Zero-trust behavior still requires least-privilege issuance, explicit gateway authorization, protected trust anchors, short lifetimes, observable decisions, and tested rotation.

Conclusion

Kubernetes 1.37 gives platform teams a native way to request rotating X.509 identities and distribute their trust anchors. That reduces custom certificate plumbing and makes mTLS a more practical workload-to-gateway option.

The real work moves up a layer: define signer semantics, map certificate identity to API policy, reload credentials safely, and preserve an emergency deny path. When those pieces are explicit, Pod Certificates and an API gateway complement each other well. Kubernetes proves which workload holds the key; a separately configured gateway authorization policy decides what that workload may do with an API request.