AI Gateway for AI Agents: An Onboarding Acceptance Checklist

An AI agent is ready to use an AI gateway when it can authenticate with its own caller identity, reach only approved model aliases, keep provider credentials out of application code, behave predictably when policy or providers fail, and leave enough evidence to explain each request. A successful chat response is not an onboarding test.

This checklist uses AISIX OSS 1.8.0 as the gateway implementation scope. AISIX OSS is API7.ai's open-source, Rust-native gateway for model and agent traffic. AISIX Enterprise and AISIX Cloud use the same gateway binary and add control-plane workflows such as centralized usage views and budgets. In 1.8.0, their control plane pushes budget state to the gateway, which makes hard-stop decisions locally; do not imply those managed workflows for AISIX OSS.

Before starting, deploy a non-production AISIX gateway, configure one provider key and one gateway-facing model alias, and prepare a disposable caller key. Export AISIX_PROXY and AGENT_API_KEY only in the test shell. Keep the provider credential in the gateway's configured secret source and use a low-cost or fixture upstream for repeated negative tests.

Define the Agent Contract First

Write one page before creating keys or routes:

Agent: support-triage-prod
Owner: Customer Platform
Environment: production
Allowed model aliases: support-fast, support-reasoning
Forbidden model aliases: every unlisted alias
Expected request path: POST /v1/chat/completions
Monthly spend owner: Customer Platform cost center
Failover: only to approved models in the same data region
Prompt and response retention: metadata only; raw content disabled
Tool boundary: ticket creation is handled by the MCP/API gateway path, not the model route
Rollback: disable the caller key and restore the previous application base URL

This separates application intent from gateway implementation. It also exposes a critical boundary: an AI gateway can govern requests that pass through it. It cannot authorize a local shell command, a direct database connection, or a tool call that bypasses its API or MCP path.

Map Ownership Across Four Boundaries

Do not collapse these identities. A caller API key proves which gateway principal made a model request; it is not the provider key and does not become the end user's identity unless your application binds and preserves that context correctly.

1. Prove Caller Identity and Revocation

Give each production agent or application workload its own caller key. Do not share a human's key across several agents when attribution and revocation matter.

AISIX caller keys control which model aliases a request may use. The plaintext value is sensitive and should be provisioned to the workload through your secret delivery path, not committed to code or pasted into prompts.

For ordinary model requests, the caller can send the key as Authorization: Bearer <key> or use x-api-key as a fallback. AISIX 1.8.0 also accepts x-aisix-api-key when Authorization must carry the caller's own upstream token, such as delegated MCP authentication. Credential precedence is x-aisix-api-key, then Authorization, then x-api-key. The first present header is the only one AISIX reads for gateway authentication; an empty or malformed value is a 401 rather than a reason to fall through. AISIX never forwards x-aisix-api-key upstream.

Run these checks against the exact production route:

Record the gateway status, response body class, request correlation value, and whether an upstream request occurred. Do not publish the key or raw authorization header in evidence.

2. Prove Model Access, Not Just Model Discovery

An agent should call stable gateway-facing aliases such as support-fast, not provider model IDs scattered through code. The alias can hide provider credentials and routing details while keeping the application's request contract stable.

Discovery is useful, but enforcement matters more. Use the caller key to list available models, then test both an allowed and a forbidden alias:

curl -fsS "$AISIX_PROXY/v1/models" \
  -H "Authorization: Bearer $AGENT_API_KEY"

curl -sS -o allowed.json -w '%{http_code}\n' \
  "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer $AGENT_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "support-fast",
    "messages": [{"role": "user", "content": "Reply with OK"}]
  }'

curl -sS -o denied.json -w '%{http_code}\n' \
  "$AISIX_PROXY/v1/chat/completions" \
  -H "Authorization: Bearer $AGENT_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "unapproved-model",
    "messages": [{"role": "user", "content": "Reply with OK"}]
  }'

The second request must fail before an unapproved provider is called. Keep the exact status and error schema from your deployed version in the application contract; do not hard-code a status copied from a different gateway or release.

3. Remove Provider Credentials from the Agent

The application should send its caller credential to the gateway. The gateway should obtain the provider credential from its configured provider-key resource and use it only for the upstream provider call.

Verify this with a deployment review and a negative test:

1. Search the agent's source, environment templates, CI variables, and deployment secret references for provider keys.
2. Confirm that only the gateway's secret source can resolve the provider credential.
3. Rotate the provider credential in a non-production environment.
4. Prove that the agent configuration does not change and that the old provider key no longer authenticates directly.

This test proves separation and rotation behavior. It does not prove that every host in the gateway environment is free of unrelated credentials.

4. Test Limits and Budgets at Their Real Scope

Rate limits and budgets answer different questions. A rate limit controls traffic over a short window; a budget controls recorded spend over a configured period. An agent loop can make several model calls for one user task, so limits designed per chat message can interrupt a tool loop mid-task.

For AISIX OSS, test the configured rate_limit and rate_limit_policy resources; managed budgets are not part of its resource model. For AISIX Enterprise or AISIX Cloud, the control plane defines budgets and pushes their state to each 1.8.0 gateway. The gateway then enforces blocking decisions before provider dispatch. The default sticky mode, labeled Keep last known state in the control plane, keeps the last pushed exhausted state through an outage, but that state expires at the next budget-period boundary even if the control plane remains unavailable. In closed mode, labeled Strict, a prolonged usage-delivery failure also rejects covered traffic that is not already known to be exhausted with 503 budget_unverified after the configured grace period. A known exhausted budget still returns 429 budget_exceeded, and traffic outside a blocking budget remains unaffected.

The acceptance matrix should include:

• one request below the limit;
• repeated requests until the limit is crossed;
• a second caller to prove whether the counter is shared or isolated;
• a request after the reset window;
• a request whose completed usage crosses a blocking budget, followed by a later matching request;
• an exhausted or unverified provider-key budget when another approved routing target remains: the gateway must skip that target and use the eligible one;
• a model group with no eligible target: expect 429 budget_exceeded if any target is known exhausted, otherwise 503 budget_unverified when all applicable targets are unverified;
• a usage-delivery or control-plane outage under the selected sticky or closed mode;
• a period rollover while the control plane is unavailable, proving that an expired exhausted mark stops blocking;
• an unpriced model or missing usage case when budgets are in scope;
• a request already in flight when a spend threshold is crossed.

Do not promise that every over-budget request is stopped at the exact monetary boundary. The request that crosses a limit can complete because provider-reported usage arrives after the response, and concurrent requests may already be in flight. Pricing or attribution gaps can also make recorded spend differ from an estimate. Record the budget scope, enforcement mode, grace period, and observed 429 budget_exceeded or 503 budget_unverified response for the product and version you deploy.

5. Define Failover Eligibility and Failure Timing

Failover is safe only when every fallback is approved for the workload. Define eligible providers, regions, models, data handling, and error classes before enabling it.

Test at least:

• primary provider connection failure;
• provider timeout before response headers;
• error after a streaming response has begun;
• authentication failure caused by an invalid provider key;
• fallback target unavailable;
• caller-forbidden model presented as a requested alias.

A gateway may retry or select a fallback before response delivery. It cannot safely replay a non-idempotent downstream action merely because a connection failed, and it cannot retract tokens already streamed to the caller. Keep retry and fallback claims next to the phase in which they apply.

6. Keep Model, Tool, and Local-Action Governance Separate

Model traffic and tool traffic have different authorization surfaces.

• Model requests use caller identity, model allowlists, routing, and model-scoped policy.
• MCP calls use tool discovery and call-time authorization for the selected server and tool.
• Traditional APIs need their own route authentication and authorization.
• Local file, shell, or browser actions remain inside the agent runtime unless they are deliberately mediated through a governed service.

If the agent can call an MCP server, test discovery and execution separately. A hidden tool should not appear in tools/list; a forbidden tool must also be rejected at tools/call, because discovery filtering alone is not enforcement.

For the protocol-specific trust boundaries, use the MCP security best practices and least-privilege MCP access guides rather than duplicating them in the model onboarding checklist.

7. Prove the Audit Trail Without Logging Secrets

For one allowed request and each denial, operators should be able to answer:

• which caller key identity sent it;
• which gateway-facing model alias was requested;
• which provider and upstream model handled an attempt, if dispatch occurred;
• which policy allowed, limited, or denied it;
• when the decision happened relative to provider dispatch or response delivery;
• what correlation value connects application and gateway records.

Log identifiers and decisions, not raw credentials. Decide separately whether prompts, tool arguments, retrieved documents, or model responses may be retained. A correlation header supplied by a caller helps tracing but is not a trusted identity by itself.

Use the AI gateway audit logging guide to separate runtime request evidence from control-plane change evidence.

Failure Triage

Onboarding Sign-Off

Approve the agent only when the evidence package contains:

• a named owner and deployment environment;
• one workload-specific caller identity with a tested revocation path;
• positive and negative model-allowlist results;
• provider credentials absent from the agent deployment;
• limit and, when applicable, budget boundary tests;
• eligible failover targets and tested failure cases;
• separate model, API, MCP, and local-action boundaries;
• correlated allowed and denied request records with secrets redacted;
• a rollback procedure that disables the caller and restores the prior base URL.

The value of an AI gateway for AI agents is not that it makes an agent safe by default. It gives platform teams a place to state and test a runtime contract for the traffic that actually crosses the gateway.

Start with the AISIX OSS 1.8.0 quickstart, then turn the working request into this negative-path acceptance suite before onboarding production traffic.