AI Gateway Cost Control: Attribute Spend and Test Budgets

AI Gateway cost control works when you can answer three questions for a request: who owns its spend, how that spend is calculated, and what happens when its budget is exhausted. Start with reliable attribution and prices, enable budgets in a test environment, and verify the rejection path before enforcing them in production.

A dashboard total is insufficient. An unpriced model can contribute tokens but zero recorded spend. A key without a team binding can miss the team budget you expected. A request already in flight can finish after the budget threshold is crossed.

This guide turns those cases into a configuration and acceptance workflow for AISIX Enterprise or AISIX Cloud 1.7.0. Managed budgets and organization-level Cost Analysis are not AISIX OSS resources. The 1.7.0 release article covers what changed; this guide covers how to put the operating controls into use.

The tag and budget API examples are adapted from the official cost attribution guide and budget guide. The Python reconciliation example was run against the synthetic CSV shown below. The budget matrix states documented acceptance expectations; it does not report measurements from an Enterprise or Cloud deployment.

Choose the Owner Before Choosing the Limit

Use the narrowest stable ownership boundary that answers the business question:

Directory membership is not a substitute for the key's team or member binding. A person in two teams can use keys whose spend belongs to different teams, or to neither.

Budgets are monetary controls. Rate limits control traffic and token consumption over a window. Use both when a workload can be expensive even at a low request rate.

Prepare an organization with administrative access, a non-production environment, a caller key, and one working model alias. Confirm that the gateways processing these requests run 1.7.0 before using tag-based controls. Keep provider credentials at the provider-key boundary; use a separate caller key in the application.

Configure a Small Attribution Vocabulary

Suppose the finance question is: “How much did the R&D department spend across its applications?” Declare department; avoid three competing spellings such as dept, Department, and department.

AISIX resolves each tag independently: caller-key value first, otherwise its bound team's value, otherwise its owning member's value. A request cannot set these attribution tags through its headers or body.

A useful acceptance fixture is:

K1 tests the override. K2 tests inheritance. K3 makes the gap visible.

In the dashboard, declare the key under Tag Keys, then set values on the relevant team, member, or caller key. For API-driven setup, set these variables in your shell using a scoped admin token. Do not put real credentials in article examples, shell tracing, or source control:

# The control-plane API URL includes /api and has no trailing slash.
export AISIX_CP="https://your-control-plane.example/api"
export TEAM_ID="your-existing-team-id"
# Set AISIX_TOKEN through your approved secret-injection mechanism.
cp_api() {
  printf 'header = "Authorization: Bearer %s"\n' "${AISIX_TOKEN:?}" |
    curl --fail-with-body -sS --config - "$@"
}

cp_api -X POST "${AISIX_CP}/tag_keys" \
  -H "Content-Type: application/json" \
  -d '{"key":"department","display_name":"Department"}'

cp_api -X PATCH "${AISIX_CP}/teams/${TEAM_ID}" \
  -H "Content-Type: application/json" \
  -d '{"tags":{"department":"rnd"}}'

These commands require curl 7.76 or later for --fail-with-body. Skip the declaration if the key already exists. A PATCH replaces the entire tags object: include every existing tag you intend to retain, rather than copying the one-tag fixture onto a populated team.

In the environment's API key settings, bind K2 to this team. Use a separate test key for K1's override and a key with no applicable tags for K3. The cost attribution documentation documents the resource fields and full update semantics.

Tags are recorded with usage when the request runs. Retagging a workload does not reclassify old usage. Before changing ownership, record the effective date and compare the before/after intervals separately.

Make Recorded Spend Explainable

Before creating a hard-stop budget, send a small known request and inspect both its usage and pricing.

In Model Pricing, check the exact provider and upstream model name that served the request, not just the alias the client requested. AISIX resolves an organization override before a matching catalog price. USD organizations can use catalog defaults; CNY organizations need their own prices. Missing matches record zero spend, which must not be interpreted as free provider usage.

For the first rollout, use one known, directly addressed model. Then expand the reconciliation to routing, retries, and ensembles. They can create several usage events for one logical request; an event is not necessarily a separate user task.

Check these pricing boundaries:

• A response-cache hit is priced at zero in AISIX 1.7.0. Provider-side prompt caching is a different mechanism with its own token classes and prices.
• Model-group pricing, input-length tiers, service-tier rates, and off-peak discounts can change recorded cost. Do not apply a simple input/output formula to traffic using those options.
• Organization currency is a reporting and policy setting, not an exchange-rate conversion. Even API fields retaining a _usd suffix carry the organization's currency.
• Gateway-recorded spend does not replace provider invoices, negotiated contracts, tax, application compute, or other tool costs.

Use the model pricing workflow to resolve a mismatch. Keep pricing changes separate from attribution changes so you can explain which one altered the report.

Create the Budget and Test Its Boundary

Start with a warn-only budget while you reconcile ownership. Once the fixture is correct, enable Hard stop at limit for the isolated test scope.

For example, this creates a monthly tag budget with a limit of 500.00 in the organization's currency:

cp_api -X POST "${AISIX_CP}/budgets" \
  -H "Content-Type: application/json" \
  -d '{
    "display_name": "R&D monthly",
    "scope": "tag",
    "scope_ref": "department=rnd",
    "limit_cents": 50000,
    "period": "month",
    "hard_stop": true
  }'

Choose a smaller limit for a controlled acceptance test and set an explicit test-spend allowance. A new budget can start with spend already recorded for its current period; it does not necessarily start at zero.

The control plane counts reported usage, and the gateway checks budget decisions before provider dispatch. The request that crosses the limit can complete. The gateway caches decisions for five seconds, and usage reporting also takes time. This is not a prepaid reservation that guarantees the provider bill cannot exceed the configured amount.

Run this matrix against your deployment and save the actual status, error code, relevant usage record, and observation time:

A tag budget is not a fallback cap for untagged traffic. Use an appropriate organization, environment, or caller-key budget for that coverage.

Tag budgets cannot be saved while registered gateways lack tag support, but that compatibility check is not permanent admission control. A later rollback or newly joined 1.6.x gateway creates the documented counting/enforcement gap. Include gateway version inventory in the operating checklist.

Handle Rejections Without Creating a Retry Loop

For an OpenAI-compatible request, inspect the JSON error code, not HTTP 429 alone. A budget rejection has error.code: "budget_exceeded" and identifies the scope and target. Treat it as an exhausted allowance: stop automatic retries, surface the owner and reset information where available, or use a separately approved fallback.

A transient rate limit needs a different retry policy. Do not silently route around an exhausted budget with a second key.

Anthropic-compatible Messages responses use their own error format and do not include the extra budget fields. Adapt the client handler to its protocol rather than assuming the OpenAI error envelope.

Also test control-plane unavailability in an isolated environment. With no cached budget decision the gateway denies. With a cached decision, stale reuse is bounded by AISIX_DP_BUDGET_STALE_MAX_SECONDS, defaulting to 600 seconds; after that ceiling the configured sticky, fail-open, or fail-closed behavior applies. Record the actual mode and outage result before relying on a hard-stop policy. The budget documentation explains these paths.

Reconcile Attempts Without Hiding Missing Prices

Use a worksheet that separates known cost, unknown prices, and unclassified ownership. The following CSV is synthetic teaching data, not an AISIX export schema, price quote, or measured deployment result. Normalize your approved usage extract into these columns, using a unique event identifier and the recorded attempt cost:

event_id,request_id,department,currency,cost,price_known
e1,r1,rnd,USD,0.018,yes
e2,r2,rnd,USD,0.012,yes
e3,r2,rnd,USD,0.020,yes
e4,r3,support,USD,0,yes
e5,r4,,USD,0.030,yes
e6,r5,rnd,USD,,no

Here r2 has two priced attempts; r3 represents a zero-cost response-cache event; r4 has no department; r5 has no verified price. The known total is USD 0.080, with one unpriced event. It is a partial total, not a complete bill.

Save the CSV as usage.csv and this Python 3 script as reconcile.py:

import csv
import sys
from collections import defaultdict
from decimal import Decimal

groups = defaultdict(lambda: [Decimal("0"), 0, 0])
seen = set()
currency = None
with open(sys.argv[1], newline="", encoding="utf-8") as source:
    for row in csv.DictReader(source):
        event_id = row["event_id"]
        if not event_id or event_id in seen:
            raise ValueError("Missing or duplicate event_id")
        seen.add(event_id)
        if currency is None:
            currency = row["currency"]
        if not currency or row["currency"] != currency:
            raise ValueError("Use one nonempty currency per worksheet")
        group = groups[row["department"] or "NOT_SET"]
        group[1] += 1
        if row["price_known"] == "no":
            if row["cost"]:
                raise ValueError("Unknown price must have an empty cost")
            group[2] += 1
        elif row["price_known"] == "yes":
            amount = Decimal(row["cost"])
            if not amount.is_finite() or amount < 0:
                raise ValueError("Cost must be finite and nonnegative")
            group[0] += amount
        else:
            raise ValueError("price_known must be yes or no")

print("department,currency,known_cost,events,unpriced")
for department, (cost, events, unpriced) in sorted(groups.items()):
    print(f"{department},{currency},{cost:.6f},{events},{unpriced}")
print(f"KNOWN_TOTAL,{currency},"
      f"{sum(v[0] for v in groups.values()):.6f},"
      f"{len(seen)},{sum(v[2] for v in groups.values())}")

Run python3 reconcile.py usage.csv. Expected output:

department,currency,known_cost,events,unpriced
NOT_SET,USD,0.030000,1,0
rnd,USD,0.050000,4,1
support,USD,0.000000,1,0
KNOWN_TOTAL,USD,0.080000,6,1

Do not sum both request-level totals and their underlying attempts. Resolve duplicate events and missing prices before comparing this worksheet with a budget period.

Turn the Report Into an Operating Decision

In Cost Analysis, choose a matching time range, filter the environment or business group, then group by department. Investigate Not set, unknown model prices, and a change in resolved models before blaming growth on one team. Whole-hour report boundaries and usage arrival timing matter when comparing with a freshly created budget.

Keep this weekly review record:

Usage Reporting describes reporting lag and estimation. The FinOps allocation framework provides the broader ownership process.

Finish the non-production acceptance matrix, assign an owner to the exceptions, and only then promote the tested budget policy. The result should be an explainable operating limit with a known failure path—not just a smaller number on a dashboard.