AI Gateway Security Checklist: Identity, Guardrails, and Failure Modes

Evaluating AI gateway security requires more than a feature list. For each control, ask which actor it identifies, which traffic it covers, when it runs, how it fails, and what evidence shows that the configured policy executed.

This checklist uses AISIX AI Gateway 1.6.0, released on October 12, 2026, and the corresponding documentation reviewed on October 20. The implementation tag points to commit 1a20831d7155ef93140d19dd6dc5345f219291bf. Verify the same behaviors against the version you deploy; a live documentation page may later describe a newer release.

The scope is AI traffic that actually crosses AISIX. Local shell commands, filesystem access, direct provider calls, retrieval authorization, and tool traffic that bypasses the gateway remain outside this boundary.

Key Takeaways

• Separate the caller's gateway identity from the credential AISIX uses with a model provider.
• Treat a shared caller key as attribution to that key, not proof of an individual human identity.
• Verify guardrail attachments; a defined but unattached guardrail inspects no traffic.
• Review hook, enforcement mode, failure policy, and streaming behavior together.
• An input block can stop provider dispatch; an output block happens after the provider has responded and cannot undo upstream cost or side effects.
• Preserve bounded operational evidence without turning prompts and responses into an uncontrolled sensitive-data archive.
• Run negative tests on one non-production route before expanding policy.

Start with the AI Traffic Trust Boundary

Draw the actors before selecting controls:

flowchart LR
    Caller[Application or agent] -->|caller credential| Gateway[AISIX AI Gateway 1.6.0]
    Gateway -->|provider credential| Provider[Model provider]
    Gateway --> Evidence[policy and usage evidence]
    Caller -. local tools or direct calls .-> Outside[paths outside this gateway]

AISIX authenticates traffic on its proxy surface; it does not establish the identity of activity that never reaches it. A model-request policy also does not automatically govern local tools, MCP servers, A2A agents, or passthrough routes. Those surfaces need explicit controls.

Use the AI Gateway Guide for the broader operating model. Use this page to review the security behavior of one deployed route.

Separate Caller Identity from Provider Credentials

AISIX models caller and provider credentials as different resources. The AISIX OSS quickstart demonstrates the application presenting a caller API key while the gateway uses a separate provider key to call the upstream model.

Review caller identity with these questions:

• Who owns the credential, and which environment uses it?
• Which model aliases may it request?
• Is it shared by several applications or users?
• How is it issued, stored, rotated, disabled, and expired?
• Which logs and usage records identify the resolved key without exposing its plaintext value?

The caller API key documentation says AISIX stores the SHA-256 hash in the key resource. A shared key still identifies only the shared application credential. If a policy needs a short-lived external identity, AISIX can map a verified OIDC-issued JWT to a caller key; the identity provider's issuance and claim policy remain part of the trust chain.

Review provider credentials separately:

• Which organization, environment, and model may use the key?
• Can an application read or override it?
• How is rotation tested without distributing a new secret to callers?
• What happens when the provider refuses or rate-limits it?

Centralizing a provider key reduces secret copies. Operator access, control-plane permissions, backups, environment variables, and incident response still need least privilege.

Authorize Models, Tools, Routes, and Environments

Authentication answers which configured caller key a request resolves to. Authorization answers whether that key may access the requested resource.

AISIX's resource model ties a caller key to allowed model aliases. MCP, agent, and passthrough traffic have separate resource and route semantics. Do not infer that permission to call one model grants permission to invoke tools or agents.

For each traffic surface, record:

Review item	Evidence to capture
Requested resource	Model alias, MCP server/tool, agent, or passthrough route
Resolved caller	Key ID or mapped identity, without the secret
Grant or denial	Matching allowlist/route condition and observed error envelope
Scope	Environment, team, caller, destination, and source constraints

Do not copy an expected status code from another endpoint. AISIX documents endpoint-specific errors in Headers and Error Codes, and streaming methods may report a later failure inside a terminal event after response headers have already been sent.

Place Guardrails at the Correct Hook

A guardrail definition is not proof of enforcement. In AISIX 1.6.0, attachments define where the guardrail applies. For open-source resources, guardrail_attachments connects a guardrail to its scope. The official Guardrail Behavior reference is explicit: a guardrail with no attachment inspects no traffic.

Coverage also depends on the API surface. Guardrails do not run on Files API operations in 1.6.0, so empty guardrail evidence there does not prove that content was checked.

Review four fields together:

1. Attachment: which environment, model, caller key, team, or supported server scope can match?
2. Hook: input, output, or both?
3. Enforcement: block or monitor?
4. Failure policy: what happens when a remote detector or output scanner cannot complete?

An input block occurs before provider dispatch. A matching block can therefore prevent the prompt from reaching the provider. An output block occurs after the upstream has produced a response; it can withhold content from the caller, but it cannot reverse provider cost or an upstream side effect.

Monitor mode does not block or mask caller-visible content. It records what the guardrail would have done. Use it to calibrate a detector, but do not report monitor hits as prevented events.

Design Failure Behavior Before Enforcement

Remote guardrails add another request-path dependency. AISIX 1.6.0 has separate input and output failure settings. The documented defaults fail closed; fail_open or output_fail_open can release traffic the remote check did not complete. Set and test the intended values explicitly rather than relying on memory of a default.

Built-in keyword and PII checks run inside the gateway, so a remote-provider timeout does not apply to them. They still have content-shape and write-back boundaries. For example, the AISIX behavior reference distinguishes bodies it can scan from provider-native shapes it cannot rewrite for a mask action.

Create a failure table for each guardrail:

Condition	Expected caller result	Provider contacted?	Evidence
Input match	Block or observe, according to mode	No for an enforcing block	Guardrail name, input hook, policy result
Output match	Withhold, mask, or observe, according to kind and mode	Yes	Output hook and terminal result
Remote timeout	Fail open or closed, as configured	Depends on hook	Bypass or unavailable reason
Output buffer exceeded	Release or refuse per on_buffer_exceeded	Yes	Bounded bypass/refusal tag
Unreadable output	Release or refuse per the applicable failure policy	Yes	Bounded bypass/refusal tag
Held stream with nothing scannable	Refuse regardless of fail_open	Yes	Terminal refusal evidence

Treat Streaming as a Terminal Outcome

Streaming complicates security language. A caller may receive HTTP 200 before the stream completes. That status means the stream opened; it does not prove that every later frame was delivered successfully.

When an enforcing output guardrail applies, AISIX can hold streamed content until the relevant check runs. Depending on the guardrail kind, it buffers windows or the full response. A later block ends the stream with an error event and drops held content. A monitor-only output chain does not hold the stream; it observes after completion and leaves caller-visible content unchanged.

The documented default buffer cap for guardrail kinds that expose it is 262,144 bytes with fail-closed overflow behavior. Some kinds use a built-in cap instead. Record the guardrail kind, route shape, buffer settings, readable frame format, and terminal event in the test result. Do not generalize one /v1/responses test to every provider adapter or passthrough route.

Protect Sensitive Data Without Creating a Logging Risk

Prompt and response capture can help debugging, but it can also duplicate personal data, source code, credentials, or customer content into an analytics system.

For routine operations, prefer bounded metadata: resolved caller and route, alias and provider target, status and latency, endpoint-reported usage, guardrail result, and a documented request ID.

A caller-supplied request ID supports correlation but is not a trusted identity. If full content capture is required for an investigation, define access, retention, encryption, regional storage, deletion, and incident handling before enabling it.

For the compliance-specific workflow, see AI Gateway Compliance Controls. Guardrail execution evidence can support an audit, but it does not by itself prove legal compliance, detector completeness, output quality, or equality with a provider invoice.

Verify Controls with Negative Tests

Run the matrix on one non-production model route:

1. invalid, disabled, and out-of-scope caller credentials;
2. an allowed model and a forbidden alias;
3. input and output matches in monitor and block modes;
4. remote timeout with each supported failure policy;
5. a response above the configured output buffer cap;
6. a stream that fails after headers open;
7. a direct or local-tool path that bypasses AISIX;
8. rollback by detaching or disabling the policy, with the resulting scope confirmed.

For each case, capture the route, resolved resources, caller-visible status or terminal event, whether the provider was contacted, and the evidence field operators can alert on. Repeat after upgrades or resource-model changes.

What an AI Gateway Does Not Secure by Itself

An AI gateway is one runtime control point. It does not independently:

• eliminate prompt injection or prove that a detector finds every unsafe instruction;
• authorize RAG documents or database rows for the end user;
• sandbox local agent filesystem, shell, browser, or network access;
• approve high-impact tool actions on behalf of a human;
• govern direct calls that bypass its listeners;
• guarantee that model output is correct;
• turn usage telemetry into an exact provider bill;
• establish compliance without organizational, legal, and data controls.

Keep these non-claims in the architecture and incident runbook. They prevent a strong gateway control from becoming an excuse to remove another necessary layer.

Conclusion

AI gateway security becomes testable when each control has an actor, scope, phase, failure path, and evidence record. Start by separating caller identity from provider credentials. Then verify resource grants, guardrail attachments, input and output hooks, monitor versus block behavior, remote failure policy, streaming terminal outcomes, and logging boundaries.

Run the AISIX 1.6.0 security test matrix on one non-production route. Use the Caller API Keys and Guardrail Behavior references to record expected results, then compare them with what the caller, gateway, and provider actually observe. Expand the policy only after the negative paths are as clear as the successful one.