Rotate LLM Provider Keys in AISIX Without Breaking Model Aliases

An application should not need a deployment because an upstream LLM provider issued a new credential. In AISIX, the application authenticates with a caller API key and requests a stable model alias. The gateway resolves that alias to a model and provider key. Rotating the upstream credential can therefore remain a platform operation, provided the team understands which models depend on the key and validates the change before revoking the old secret.

The workflow differs between AISIX Cloud and the open-source AISIX gateway. Cloud manages provider-key resources and projects them to gateways. Open-source AISIX loads operator-managed resources and environment variables. Treating those paths as identical creates avoidable outages.

Key Takeaways

Caller API keys identify applications or tenants; provider keys authenticate AISIX to upstream model services. Do not exchange one for the other.
An in-place rotation changes every model that references the provider key at once.
A replacement-key rotation moves models in stages and preserves a fast rollback path.
AISIX Cloud stores provider credentials as write-only material and manages provider-key resources through its control plane.
Open-source AISIX requires operators to update the gateway process environment or resources configuration and reload or restart the gateway deliberately.

Start with the AISIX Credential Model

Three resources participate in a normal model request:

The caller API key authenticates the application to AISIX and can restrict which model aliases it may use.
The model alias is the caller-facing name in the request, such as support-summary-prod.
The provider key contains or references the credential AISIX uses for the selected upstream provider.

This separation is the reason rotation can avoid application changes. The application continues to send the same base URL, caller key, and model name. The platform changes the provider key or the model-to-key reference behind that contract.

The separation also limits credential exposure. Provider secrets do not belong in browser code, prompts, notebooks, or every application deployment. AISIX still needs a secure source for those secrets, and the gateway host must be protected, but consumers do not receive the upstream credential. The OWASP Secrets Management Cheat Sheet provides a broader lifecycle baseline for creation, rotation, revocation, and auditing.

API7.ai's AI Gateway requirements checklist explains the broader resource model. This article focuses only on the rotation operation and its failure modes.

Cloud and Open-Source Rotation Are Different

Choose the workflow for the deployed management model.

Concern	AISIX Cloud	Open-source AISIX gateway
Configuration owner	AISIX Cloud control plane	Operator-managed `resources.yaml` and process environment
Secret update	Update a provider-key resource with new write-only credential material	Update the referenced environment variable or provider-key entry
Activation	Control plane projects the saved resource to the gateway	Operator reloads configuration or restarts/recreates the gateway process
Staged replacement	Create a new provider key and repoint models in cohorts	Declare both provider keys and change selected model references
Central audit and request logs	Available through AISIX Cloud features	Export logs and metrics to operator-owned systems

The official AISIX provider key rotation guide documents the Cloud and open-source paths separately. Keep that product boundary visible in runbooks, access reviews, and incident reports.

Inventory the Dependency Before Rotating

A provider key is often shared by several models. Before changing it, record:

provider and account or project ownership;
provider-key identifier or display name;
environments allowed to use the key;
every chat, embedding, rerank, image, audio, or other model that references it;
model aliases exposed to callers;
fallback routes that also depend on the credential;
owner, maintenance window, and rollback decision maker;
representative verification request for each affected endpoint type.

Do not verify an embedding model through a chat endpoint or assume that one successful completion covers streaming and regional credentials. Provider permissions, quotas, base URLs, and request formats may differ across model types.

The inventory determines whether in-place rotation is acceptable. If one key backs a single low-risk model, switching it at once may be simple. If it backs production chat, embeddings, semantic routing, and fallback targets across several environments, staged replacement is safer.

Option 1: Rotate the Provider Key in Place

In-place rotation preserves the provider-key identity and changes its credential material. Every model that references the key resolves the replacement credential without a model update.

Use this method when:

the provider key has a small, known dependency set;
the replacement credential has already been validated safely;
a short all-at-once cutover is acceptable;
the previous credential can be restored quickly if validation fails.

AISIX Cloud Procedure

Identify every model that references the provider key.
Update the existing provider-key resource with the complete replacement credential.
Wait for resource projection to reach the target gateway.
Send a representative request through each affected model alias.
Inspect request results and provider-side usage before revoking the old credential.

For providers with structured credentials, include every required field. Amazon Bedrock and Google Vertex AI, for example, use more than one value. A partial credential update can leave all dependent models unable to authenticate.

AISIX Cloud treats stored provider secrets as write-only material. The control plane does not return the current secret when the resource is read. An operator supplies a new value during rotation; viewing the resource is not a way to recover the old secret.

Open-Source AISIX Procedure

If the provider-key entry references ${OPENAI_API_KEY} or another environment variable, changing a shell variable outside the running process does not update the process environment. Replace the secret in the system that launches AISIX, then restart or recreate the gateway so it receives the new value. Keep the provider-key display name unchanged so model references remain stable.

After activation, inspect configuration status and send requests through every affected alias. Keep the prior secret available through the approved rollback mechanism until verification is complete, but do not leave it active indefinitely.

Option 2: Rotate with a Replacement Provider Key

A replacement-key rotation creates a second provider-key resource and moves models from the old key to the new key in stages.

Use this method when:

many models or environments share the current key;
the provider uses complex credentials or permissions;
traffic cannot tolerate an all-at-once authentication failure;
operators need model-by-model rollback;
a leak response still allows enough time for controlled migration.

The sequence is:

Create or declare the replacement provider key with the same provider adapter and required non-secret settings.
Make it available in the target environment.
Move one low-risk model to the replacement key.
Verify that model through its real endpoint and alias.
Move the remaining models in bounded cohorts.
Confirm no model still references the old key.
Revoke the provider credential and remove the old provider-key resource.

In AISIX Cloud, the provider-key selector only shows keys eligible for the model's environment. If a new key is missing, correct its allowed environments rather than creating duplicate keys blindly. After each model update, wait for projection before judging the credential.

In open-source AISIX, declare both provider keys in the resources file and make both secrets available to the gateway process. Change selected model references, validate and reload the configuration, then test the cohort. Remove the old entry only after every model has moved.

Verify the Rotation Through the Caller Contract

Management-plane success does not prove data-plane success. The verification request should use the same gateway endpoint, caller API key, model alias, protocol shape, and streaming mode that the application uses.

Check at least:

the gateway accepted the caller key and alias;
the selected provider accepted the new credential;
the expected upstream model served the request;
streaming began and completed normally where applicable;
embeddings or other non-chat endpoints returned the expected shape;
fallback did not silently hide a failed primary credential;
latency, error rate, token usage, and spend did not change unexpectedly.

Fallback deserves special care. A successful response can be misleading if the primary model failed authentication and AISIX routed to another target. Inspect request logs or exported telemetry to confirm which provider key and model actually served the verification call. The AISIX routing and failover guide explains how aliases and targets resolve.

Roll Back Without Reintroducing the Leak

Routine rotation and incident rotation have different rollback choices.

During routine rotation, restoring the previous credential or model reference may be acceptable while the replacement is corrected. During a confirmed credential leak, restoring the compromised key is not a safe rollback. The team may instead disable the affected route, move to a pre-approved replacement provider, or accept a controlled outage while issuing a clean credential.

Define the trigger before the change. Examples include a sustained authentication error rate, missing provider permissions, unexpected region selection, or failed streaming requests. Record who may execute rollback and how long the old key can remain valid.

Common Rotation Failures

Updating the Secret Without Activating It

An open-source process does not inherit a new environment value automatically. A Cloud resource may also need time to project. Verify the active gateway configuration before retrying the same credential repeatedly.

Deleting a Referenced Key

Removing a provider-key resource while models still reference it breaks those models. Query or inventory references first, then remove the key last.

Testing Only One Endpoint

Chat success does not validate embeddings, audio, images, regional endpoints, or streaming. Test the production request shapes that depend on the key.

Confusing Caller and Provider Keys

Rotating an upstream provider credential should not require reissuing caller API keys. If applications receive the provider key directly, the architecture has lost the separation AISIX is intended to provide.

Treating AISIX as the Only Secret System

AISIX controls how provider credentials enter the AI traffic path. Open-source deployments still need an operator-owned secret and configuration workflow. Cloud deployments still need provider-side issuance, revocation, ownership, and incident response. An AI gateway complements those systems; it does not replace them.

Conclusion

Provider key rotation is successful when callers keep the same gateway contract and operators can prove which upstream credential served each affected model. AISIX enables that separation through caller API keys, model aliases, and provider keys, but the activation and evidence path depends on the deployment model.

Use in-place rotation for a small, well-understood dependency set. Use a replacement provider key when models need staged migration and rollback. In both cases, inventory first, verify through real aliases and endpoint types, inspect fallback, and revoke the old credential only after every reference has moved.

Start with the product-specific AISIX provider key documentation, then make the runbook match the exact Cloud or open-source gateway version your applications depend on.