AI Gateway Configuration Changes: Validate, Reload, Verify, and Roll Back

An AI gateway configuration change is complete only when four separate gates pass: the candidate file is valid, the gateway accepts it, the effective configuration changes as intended, and caller behavior still meets its contract. A healthy process alone is not proof that the new configuration was applied.

This runbook uses AISIX OSS 1.8.0 in declarative file mode. It covers resources.yaml, offline validation, SIGHUP reload, /status/config, negative caller tests, and rollback to a known-good file. AISIX Enterprise and AISIX Cloud use a control plane instead of this file-mode procedure; apply the same four gates through their documented change and status surfaces.

The commands are documentation- and source-backed examples. They are not presented as a live production deployment test. Use an isolated gateway and non-production provider credentials before adapting them to your environment.

The Four Gates

Do not merge these gates into one “deployment succeeded” signal. A YAML parser cannot prove references resolve. A successful signal delivery cannot prove the reload was accepted. Readiness can remain healthy while the gateway serves its last valid configuration after a rejected reload.

Prerequisites and Starting State

Prepare:

• an AISIX OSS 1.8.0 container running in file mode;
• the exact resources.yaml mounted into the container;
• every environment variable referenced by that file;
• a known-good copy stored outside the container's writable layer;
• one allowed caller credential and one intentionally denied credential or model;
• a low-risk route and provider suitable for a smoke request; and
• a change record with owner, intent, affected resources, validation window, and rollback trigger.

The quickstart-style examples below assume a working directory containing both config.yaml and resources.yaml. The directory mount keeps the same file path visible across edits; a single-file bind mount can retain the old inode after an atomic host-side replacement. For a clean, isolated exercise, use these complete starting files:

# config.yaml
resources_file: /etc/aisix/resources.yaml
proxy:
  addr: "0.0.0.0:3000"
admin:
  enabled: false
observability:
  metrics:
    prometheus:
      enabled: true
      addr: "0.0.0.0:9090"

# resources.yaml
_format_version: "1"
provider_keys:
  - display_name: openai-main
    provider: openai
    api_key: ${OPENAI_API_KEY}
models:
  - display_name: my-model
    provider: openai
    model_name: gpt-4o-mini
    provider_key: openai-main
api_keys:
  - display_name: local-dev
    key_env: CALLER_API_KEY
    allowed_models: ["my-model"]

Have your secret mechanism export a valid, non-production OPENAI_API_KEY and a separate non-production CALLER_API_KEY into the shell environment. An unexported shell variable can pass a local test but will not reach docker run -e. Confirm both are nonempty without printing them. This example makes a provider request and may incur provider charges. Start the isolated gateway before making the candidate edit:

test -n "$OPENAI_API_KEY" && test -n "$CALLER_API_KEY"
docker run -d --name aisix-quickstart \
  -v "$PWD:/etc/aisix:ro" \
  -e OPENAI_API_KEY -e CALLER_API_KEY \
  -p 127.0.0.1:3000:3000 -p 127.0.0.1:9090:9090 \
  ghcr.io/api7/aisix:1.8.0
curl -fsS http://127.0.0.1:9090/status/config

The expected starting status is synced, with one model and one caller key in applied.resource_counts. If startup fails, inspect docker logs aisix-quickstart; the image entrypoint requires /etc/aisix/config.yaml. The example below uses these names and addresses:

container: aisix-quickstart
resources: $PWD/resources.yaml -> /etc/aisix/resources.yaml
proxy:     http://127.0.0.1:3000
status:    http://127.0.0.1:9090/status/config

Do not put provider or caller secrets into the file, shell history, screenshots, or change record. Reference environment variables and provide them through the deployment's secret mechanism.

Gate 1: Validate the Exact Candidate

Make a recoverable backup before editing:

cp resources.yaml resources.known-good.yaml

For this exercise, replace resources.yaml with the complete candidate below. It adds my-model-next using the same provider key and model_name: gpt-4o-mini, then permits the original caller to use it. This is an alias and access-list change; it does not test a new provider model.

# resources.yaml (complete candidate)
_format_version: "1"
provider_keys:
  - display_name: openai-main
    provider: openai
    api_key: ${OPENAI_API_KEY}
models:
  - display_name: my-model
    provider: openai
    model_name: gpt-4o-mini
    provider_key: openai-main
  - display_name: my-model-next
    provider: openai
    model_name: gpt-4o-mini
    provider_key: openai-main
api_keys:
  - display_name: local-dev
    key_env: CALLER_API_KEY
    allowed_models: ["my-model", "my-model-next"]

If the edit uses only environment variables already present in the running container, validate through that container:

docker exec aisix-quickstart \
  /usr/local/bin/aisix validate --resources /etc/aisix/resources.yaml

A successful validation reports that the file loaded and includes its resource count. Record the exit status and sanitized output.

If the candidate introduces a new ${VARIABLE}, the running container cannot inherit a variable exported later on the host. Validate in a short-lived container with the complete environment instead:

docker run --rm \
  -v "$PWD:/etc/aisix:ro" \
  -e OPENAI_API_KEY \
  -e CALLER_API_KEY \
  -e NEW_PROVIDER_API_KEY \
  --entrypoint /usr/local/bin/aisix \
  ghcr.io/api7/aisix:1.8.0 \
  validate --resources /etc/aisix/resources.yaml

AISIX 1.8.0 runs the same file-loading pipeline for validation, startup, and SIGHUP: environment interpolation, schema decoding, name-reference resolution, and enabled guardrail build checks. Review every warning as well as the exit status: some conditions warn rather than fail, including unattached guardrails and sanitized MCP tool-name collisions. Validation is stronger than YAML syntax checking, but it still does not call the provider or prove application behavior.

Negative Validation Is Part of the Evidence

Keep one sanitized negative fixture for the class of change you are making. Examples include:

• a model referencing a missing provider credential;
• a caller key allowing a model alias that does not exist;
• an invalid resource field or schema shape;
• a missing environment variable; or
• a duplicate or invalid resource identifier.

The negative fixture should fail validation. If both the good and deliberately broken files pass, the gate is not testing the dependency you think it is.

Gate 2: Apply the Candidate

Capture the current status before reloading:

curl -fsS http://127.0.0.1:9090/status/config > status.before.json

For a candidate that introduces no new environment variables, request a reload:

docker kill --signal=HUP aisix-quickstart

If new environment variables are required, recreate the container with the same mounted file and the expanded environment. A signal cannot add variables to a running process:

docker rm -f aisix-quickstart
docker run -d --name aisix-quickstart \
  -v "$PWD:/etc/aisix:ro" \
  -e OPENAI_API_KEY \
  -e CALLER_API_KEY \
  -e NEW_PROVIDER_API_KEY \
  -p 127.0.0.1:3000:3000 -p 127.0.0.1:9090:9090 \
  ghcr.io/api7/aisix:1.8.0

The mounted directory must still contain config.yaml; the container entrypoint exits if that file is absent. Preserve any other original flags, mounts, and environment settings when adapting this recreation command. The status and metrics listener is unauthenticated in this deployment mode, so keep it on host loopback or an equivalently private administrative interface. Recreation changes the failure surface. Verify startup, listener binding, secret availability, and the first configuration apply before sending traffic.

Gate 3: Prove the Effective State

Fetch status again:

curl -fsS http://127.0.0.1:9090/status/config > status.after.json

Check the fields, not only the HTTP status:

1. configuration state is synced;
2. rejected is empty;
3. the applied resource counts match the candidate;
4. apply_seq or the reported configuration hash changed when the effective content changed; and
5. the applied timestamp is later than the change request.

An identical reload may leave apply_seq unchanged. Treat that as “no effective change,” not as a failed signal.

AISIX 1.8.0 retains the last valid file-mode configuration when a later reload is rejected and reports the mismatch through /status/config. This availability behavior is useful, but it creates an important distinction:

ready and serving != candidate accepted and current

Do not use /readyz alone as configuration evidence. Readiness answers whether the instance can serve traffic; configuration status answers which state it accepted and whether the source is in sync.

Verify Caller Behavior

For the concrete alias change above, first check discovery with the allowed caller, then send small smoke requests to both the original and new aliases through the proxy. These commands require a working provider credential, network access, and an upstream model available to your account:

curl -sS http://127.0.0.1:3000/v1/models \
  -H "Authorization: Bearer $CALLER_API_KEY"
curl -sS http://127.0.0.1:3000/v1/chat/completions \
  -H "Authorization: Bearer $CALLER_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"my-model","messages":[{"role":"user","content":"Reply with OK."}]}'
curl -sS http://127.0.0.1:3000/v1/chat/completions \
  -H "Authorization: Bearer $CALLER_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"my-model-next","messages":[{"role":"user","content":"Reply with OK."}]}'

Expect my-model-next in the allowed model list and a successful chat response. A provider error is not proof of an accepted caller contract; check the provider key, model availability, and gateway logs. Send negative requests and record their response bodies and status codes:

curl -sS -w '\nHTTP %{http_code}\n' http://127.0.0.1:3000/v1/models \
  -H 'Authorization: Bearer deliberately-invalid-key'
curl -sS -w '\nHTTP %{http_code}\n' http://127.0.0.1:3000/v1/chat/completions \
  -H "Authorization: Bearer $CALLER_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"unknown-alias","messages":[{"role":"user","content":"Reply with OK."}]}'

The exact error code can depend on the failed layer, so compare it with your version's documented behavior rather than asserting a universal code. Run at least these four request cases:

When the change affects streaming, tools, structured output, guardrails, caching, or fallback, add a contract test for that behavior. A non-streaming 200 does not prove streaming compatibility, and a provider response does not prove a denial policy was evaluated.

Record request IDs, requested aliases, resolved upstream targets when exposed, status or error class, and the gateway configuration hash. Do not record prompt bodies or secrets unless a separate privacy and retention policy permits it. The four request cases are an acceptance checklist, not a claim that this article's author executed them against a live gateway.

Gate 4: Roll Back Deliberately

Define rollback triggers before applying the candidate. Examples:

• configuration state becomes out_of_sync;
• any required resource is rejected;
• an allowed caller loses access;
• a denied caller gains access;
• streaming or tool behavior violates the contract;
• error rate or latency crosses the preapproved window; or
• the new provider or dependency is unavailable.

Restore the known-good file through your deployment mechanism, validate it, and then reload or recreate the container according to its environment-variable needs. For production, use the deployment's atomic file replacement method and keep the directory mounted so the gateway sees the new inode. Do not edit the live file repeatedly until the status turns green; that destroys the exact candidate and makes the incident harder to reconstruct.

For a local exercise:

cp resources.known-good.yaml resources.yaml
docker exec aisix-quickstart \
  /usr/local/bin/aisix validate --resources /etc/aisix/resources.yaml
docker kill --signal=HUP aisix-quickstart
curl -fsS http://127.0.0.1:9090/status/config

Then repeat the positive and negative caller checks. In this exercise, my-model should remain available and my-model-next should disappear from discovery and fail as an unknown alias. Rollback is complete only when the previous behavior is restored, not when the old file is copied back. If the candidate introduced a new environment variable and required recreation, restore the original environment and other container settings as part of the rollback; SIGHUP cannot remove or add process environment variables.

Troubleshoot by Failure Stage

File Mode, Enterprise, and Cloud Are Different Paths

This procedure is for AISIX OSS file mode. A directly operated OSS gateway can also use etcd; that path has watch, revision, and connectivity semantics that are not equivalent to SIGHUP.

AISIX Enterprise and AISIX Cloud distribute configuration through their control planes. Do not copy file-mode commands into those deployments. Use their version-specific status, rollout, and audit surfaces, and validate mixed-version compatibility before applying a field that an older gateway cannot load.

Across all modes, preserve the same evidence chain:

candidate -> validation -> accepted revision -> caller contract -> rollback proof

Definition of Done

A configuration change is complete when:

• the exact candidate and known-good files are retained with hashes;
• offline validation passed with the same environment-variable set used at apply time;
• a deliberately broken fixture failed for the expected reason;
• the gateway reported the intended effective state, not only process readiness;
• positive and negative caller tests passed through the real gateway path;
• version, request, configuration, and operator evidence can be correlated without secrets;
• rollback triggers were declared before rollout; and
• rollback was either exercised or proven through a recent equivalent drill.

The goal is not zero configuration failures. It is a change process in which a rejected candidate is visible, the last valid state is understood, and operators can restore the caller contract without guessing.

Start with the AISIX configuration propagation guide, then adapt this runbook to the deployment mode and release you actually operate.