---
title: "Decision Models Through an AI Gateway: Test the API Contract - API7.ai"
description: "Validate AISIX decision-model requests, answer ordering, refusals, token usage, and failure paths with a runnable application-side contract checker."
url: "https://api7.ai/blog/decision-model-api-contract-testing"
---

[Blog](https://api7.ai/blog)

[Products](https://api7.ai/category/products)[Technology](https://api7.ai/category/technology)[Ecosystem](https://api7.ai/category/ecosystem)[Case Study](https://api7.ai/category/usercase)[Team & Community](https://api7.ai/category/community)

Search

⌘ + K

[Blog](https://api7.ai/blog)

Technology

# Decision Models Through an AI Gateway: Test the API Contract

[![Yilia Lin](https://github.com/Yilialinn.png)](https://github.com/Yilialinn)

[Yilia Lin](https://github.com/Yilialinn)

October 27, 2026

[Technology](https://api7.ai/category/technology)

Before routing application decisions through an AI gateway, test the contract you will actually consume: the request shape, answer order and types, refusal handling, usage, and transport failures. A `200` response means the HTTP request succeeded; it does not prove the answers are usable or correct.

This tutorial targets **AISIX 1.7.0**, a direct TypeSafe decision model, and the client-facing `POST /v1/decisions` format. The gateway examples follow the official [decision-model guide](https://docs.api7.ai/ai-gateway/endpoints/decision-models): AISIX converts that request to the upstream's System One format. The application-side response checker was run against synthetic valid and invalid responses. It checks the application profile and refusal handling; it does not measure a live model, gateway throughput, or provider accuracy.

**AISIX 1.7.0 does not run model or caller-key guardrails on either decision endpoint.** Put required input validation, sensitive-data handling, and application authorization outside this call. Do not assume an existing chat guardrail also screens decision requests.

## Start With a Concrete Decision Task

Suppose a support application needs two judgments:

1. Which queue should receive a ticket: billing or engineering?
2. How likely is the ticket to describe a service outage?

The application can use the first answer to suggest a queue and the second to request human escalation. Neither answer grants access, issues a refund, or overrides an authorization rule.

Our existing [Decisions API architecture article](https://api7.ai/blog/openai-decisions-api-gateway) explains that policy boundary. This tutorial adds the implementation acceptance path.

Recent [LiteLLM decision-model reranking coverage](https://docs.litellm.ai/blog/jevrank) shows interest in using structured judgments inside applications. That is a discovery signal, not evidence that AISIX implements JevRank or converts its rerank interface. The integration procedure below follows AISIX's documented decision endpoint.

## Pin the Endpoint, Provider, and Model

Record the gateway image version/digest, deployment mode, provider, upstream model revision where available, model alias, and question-set version. A moving alias such as `jev-latest` is not a fixed model snapshot; record that limitation if the provider does not expose a revision.

AISIX exposes two distinct client contracts:

| Endpoint        | Input and questions                   | Answer shape                                       |
| --------------- | ------------------------------------- | -------------------------------------------------- |
| `/v1/decisions` | `input`, plus a questions array       | Answers array in question order; includes refusals |
| `/v1/systemone` | `state`, plus questions keyed by name | Answers object keyed by name; no refusal type      |

The gateway chooses the upstream format from the model's provider key. In 1.7.0 the decision providers are TypeSafe, Microsoft Foundry through `azure`, OpenRouter, and OpenAI. Both endpoints are non-streaming. A chat model on an arbitrary provider is not automatically eligible.

For this tutorial, prepare an existing AISIX Enterprise/Cloud 1.7.0 environment:

1. Create a **Provider Key** with provider `typesafe`, a valid provider credential, and access to the test environment.
2. Create a direct model with alias `decision-check`, that provider key, and an upstream model ID your TypeSafe account supports.
3. Grant `decision-check` to a dedicated caller key. Do not give the application the provider key or an admin token.
4. Configure the matching provider/model price if you need recorded spend or managed budget tests. Token usage alone does not establish priced spend.
5. Keep chat-based background health checks off for this decision-only model. Use the endpoint-specific probes below.

Use the [AISIX decision-model setup guide](https://docs.api7.ai/ai-gateway/endpoints/decision-models) for the deployment-specific API or OSS resources equivalent. OSS uses the same proxy endpoints, but managed budgets and organization-level cost reporting require Enterprise or Cloud.

Do not use the chat-only dashboard playground as proof that a decision model works.

## Send a Small, Versioned Request

Save this as `decision-request.json`. It adapts the official guide’s Decisions-format choice example, adds a documented predicate question, and uses the application alias configured above. The text is synthetic and contains no customer data.

```
{
  "model": "decision-check",
  "input": "The API returns 500 on every call; engineering should look.",
  "questions": [
    {
      "type": "choice",
      "name": "team",
      "instructions": "Which team should handle this ticket?",
      "choices": [
        {"value": "billing", "description": "Charges and invoices"},
        {"value": "engineering", "description": "Bugs and outages"}
      ]
    },
    {
      "type": "predicate",
      "name": "is_outage",
      "instructions": "Does this describe a service outage?"
    }
  ]
}
```

Use unique, stable question names and keep their order in the application's versioned profile. AISIX has rules for absent or repeated names during conversion, but the application need not depend on that ambiguity.

Set `AISIX_PROXY` to the gateway origin without `/v1` or a trailing slash, and inject `AISIX_API_KEY` through your normal credential mechanism. This example passes the bearer header through curl's standard input rather than writing its value in the command arguments:

```
umask 077
status=$(
  printf 'header = "Authorization: Bearer %s"\n' "${AISIX_API_KEY}" |
    curl -sS --config - \
      -o decision-response.json -w '%{http_code}' \
      -H "Content-Type: application/json" \
      --data-binary @decision-request.json \
      "${AISIX_PROXY}/v1/decisions"
)
test "$status" = "200" || {
  echo "Decision call failed with HTTP $status; inspect a sanitized error."
  exit 1
}
```

Keep TLS verification enabled. Store response captures under restricted permissions and sanitize them before sharing; even a decision response can reveal sensitive input-derived information.

Do not compare model probabilities for exact equality between direct and gateway calls. Start by comparing schema, identity, question mapping, refusal semantics, and usage presence. Use a separate representative labeled dataset to evaluate answer quality.

## Check the Application Contract

The following **synthetic response fixture** illustrates a valid answer. Its probabilities and token counts are not measured results:

```
{
  "model": "decision-check",
  "answers": [
    {
      "type": "choice",
      "name": "team",
      "choice": "engineering",
      "confidence": 0.97,
      "probabilities": [
        {"value": "billing", "probability": 0.03},
        {"value": "engineering", "probability": 0.97}
      ]
    },
    {"type": "predicate", "name": "is_outage", "probability": 0.92}
  ],
  "usage": {"input_tokens": 381, "output_tokens": 62, "total_tokens": 443}
}
```

Save it as `decision-fixture.json`. Save this checker as `decision_check.py`:

```
import json
import math
import sys

def number(value):
    return (isinstance(value, (int, float))
            and not isinstance(value, bool) and math.isfinite(value))

def probability(value):
    return number(value) and 0 <= value <= 1

def check(body):
    if not isinstance(body, dict) or body.get("model") != "decision-check":
        raise ValueError("Unexpected response model or envelope")
    answers = body.get("answers")
    if not isinstance(answers, list) or len(answers) != 2:
        raise ValueError("Expected two ordered answers")
    review = False
    for answer, name, kind in zip(
        answers, ("team", "is_outage"), ("choice", "predicate")
    ):
        if not isinstance(answer, dict) or answer.get("name") != name:
            raise ValueError("Question identity or order changed")
        if answer.get("type") == "refusal":
            review = True
            continue
        if answer.get("type") != kind:
            raise ValueError("Unexpected answer type")
        if kind == "predicate":
            if not probability(answer.get("probability")):
                raise ValueError("Invalid outage probability")
        else:
            allowed = {"billing", "engineering"}
            if (not isinstance(answer.get("choice"), str)
                    or answer["choice"] not in allowed):
                raise ValueError("Unknown team")
            if not probability(answer.get("confidence")):
                raise ValueError("Invalid confidence")
            values = answer.get("probabilities")
            if not isinstance(values, list) or len(values) != 2:
                raise ValueError("Expected both team probabilities")
            seen = set()
            total = 0.0
            for item in values:
                if not isinstance(item, dict):
                    raise ValueError("Invalid probability entry")
                value = item.get("value")
                if not isinstance(value, str) or value not in allowed:
                    raise ValueError("Unknown probability choice")
                if value in seen or not probability(item.get("probability")):
                    raise ValueError("Repeated choice or invalid probability")
                seen.add(value)
                total += item["probability"]
            if abs(total - 1.0) > 0.001:
                raise ValueError("Team probabilities do not sum to one")
    usage = body.get("usage")
    if not isinstance(usage, dict):
        raise ValueError("Missing usage")
    for field in ("input_tokens", "output_tokens", "total_tokens"):
        value = usage.get(field)
        if type(value) is not int or value < 0:
            raise ValueError("Missing or invalid usage counter")
    return "REVIEW" if review else "PASS"

if __name__ == "__main__":
    try:
        with open(sys.argv[1], encoding="utf-8") as source:
            outcome = check(json.load(source))
    except (ValueError, OSError, IndexError):
        print("FAIL: response violates the application contract")
        raise SystemExit(1)
    print(outcome)
    raise SystemExit(2 if outcome == "REVIEW" else 0)
```

Run the fixture first:

```
python3 decision_check.py decision-fixture.json
# Expected: PASS, exit 0
```

Then run the captured gateway response through the same checker. Exit `0` means it satisfies this application's structural profile. Exit `2` means a refusal requires the application's review path. Exit `1` means the response cannot be consumed under this profile.

This is deliberately narrower than the full API. It accepts exactly two uniquely named questions with these types and requires the displayed usage fields. Its team probabilities must sum to one within 0.001. Change the profile and its fixtures together if your application has different needs. It does not certify answer correctness, probability calibration, or completeness of every provider's metadata.

## Exercise Refusals and Failures Deliberately

A refusal is a valid decision outcome, not a probability of zero. For a converted response, AISIX represents an absent answer or an answer of the wrong question type as a refusal on `/v1/decisions`. On `/v1/systemone`, which has no refusal type, the corresponding answer is omitted. Application code must handle both shapes if it uses both endpoints.

Change the fixture's first answer to `{"type":"refusal","name":"team"}` and rerun: expect `REVIEW` and exit `2`. Removing an answer, swapping its position, using an unknown team, or setting a probability above one should produce `FAIL`.

Add these deployment-level tests using a controlled mock upstream or a provider-approved sandbox:

| Case                                       | What to verify                                                               | Application action                   |
| ------------------------------------------ | ---------------------------------------------------------------------------- | ------------------------------------ |
| Missing or unauthorized caller key         | Request cannot access the model under the configured policy                  | Stop; correct identity/access        |
| Supported direct model, valid body         | Alias and ordered answers satisfy the profile                                | Continue to application policy       |
| Refusal or converted missing answer        | Refusal remains visible; no fabricated score                                 | Human review or documented fallback  |
| Unsupported provider/model kind            | Configuration/request is rejected rather than treated as a generic chat call | Fix the model contract               |
| Upstream 4xx                               | Its status and error message are returned                                    | Inspect the specific error           |
| Upstream 5xx                               | AISIX reports a gateway failure; routing may retry/fail over                 | Apply bounded retry policy           |
| Invalid JSON or malformed converted answer | The call does not become a usable answer                                     | Record the failure and attempt count |
| Required guardrail                         | It does not run on these decision endpoints                                  | Enforce the requirement elsewhere    |

The official [malformed-answer documentation](https://docs.api7.ai/ai-gateway/endpoints/decision-models#upstream-errors-and-malformed-answers) explicitly distinguishes versions: **AISIX 1.7.0 or earlier** fails a routing model over to its next target on a malformed converted answer; later behavior does not retry that already-served answer. Do not apply the later no-retry behavior to a 1.7.0 rollout. Start with the direct model described here, then test attempt counts and cost in your deployment before adding routing.

When the upstream and client formats match, AISIX passes through fields it does not know, apart from model-name rewriting and configured body overrides. That path is not a universal schema-validation guarantee. Keep the application checker even if one converted response was validated by the gateway.

## Record Usage Without Calling It a Quality Result

For decision traffic, usage-event `operation` identifies the client endpoint: `decisions` or `systemone`. Check the resolved provider/model, token counts, retries, and recorded cost alongside the application outcome.

Configure a pricing override where the selected provider/model has no catalog price. [AI cost attribution and budgets](https://api7.ai/blog/ai-gateway-cost-control) are useful after the usage is priced, but they cannot tell you whether the chosen support queue was correct.

Keep three separate acceptance results:

1. **Transport and policy:** the intended caller can access the intended model, with known failure handling.
2. **Contract:** the response has the fields and fallback states the application consumes.
3. **Task quality:** representative labeled tickets meet your false-routing and escalation requirements.

The local fixture verifies the checker, not a live provider or gateway deployment. Run the live capture and failure matrix in your environment, record actual versions and observations, and only then approve that integration. No production accuracy, cost reduction, or latency improvement is claimed by this tutorial.

Start with `decision-request.json`, the fixture, and the checker. Link the resulting contract version to your application deployment so a provider or gateway change triggers the same acceptance tests again.

Tags:

[AI Gateway](https://api7.ai/tags/ai-gateway)

[AISIX](https://api7.ai/tags/aisix)

[API Testing](https://api7.ai/tags/api-testing)

Share article link

Share

Related Posts

[![Migrating from LiteLLM to AISIX: Keep Model Profiles, Add Gateway Orchestration](https://static.api7.ai/uploads/2026/11/03/w1ZDPFFD_litellm-migration.png)](https://api7.ai/blog/litellm-migration-aliases-inference-parameters)

## [Migrating from LiteLLM to AISIX: Keep Model Profiles, Add Gateway Orchestration](https://api7.ai/blog/litellm-migration-aliases-inference-parameters)

[![LiteLLM vs AISIX: Choosing an AI Gateway for Production](https://static.api7.ai/uploads/2026/11/05/FKwg8x8B_litellm-vs-aisix-production.png)](https://api7.ai/blog/litellm-vs-aisix-production-ai-gateway)

## [LiteLLM vs AISIX: Choosing an AI Gateway for Production](https://api7.ai/blog/litellm-vs-aisix-production-ai-gateway)

Table of Contents

- [Start With a Concrete Decision Task](#start-with-a-concrete-decision-task)
- [Pin the Endpoint, Provider, and Model](#pin-the-endpoint-provider-and-model)
- [Send a Small, Versioned Request](#send-a-small-versioned-request)
- [Check the Application Contract](#check-the-application-contract)
- [Exercise Refusals and Failures Deliberately](#exercise-refusals-and-failures-deliberately)
- [Record Usage Without Calling It a Quality Result](#record-usage-without-calling-it-a-quality-result)

## Related Posts

[![AI Gateway Configuration Changes: Validate, Reload, Verify, and Roll Back](https://static.api7.ai/uploads/2026/11/09/dGL3lHkE_ai-gateway-configuration-change-runbook-v2.webp)](https://api7.ai/blog/ai-gateway-configuration-change-runbook)

Technology

November 9, 2026

#### [AI Gateway Configuration Changes: Validate, Reload, Verify, and Roll Back](https://api7.ai/blog/ai-gateway-configuration-change-runbook)

Use a four-gate runbook to validate an AI gateway configuration, apply it safely, prove the effective state, and roll back without confusing readiness with freshness.

[AI Gateway](https://api7.ai/tags/ai-gateway)

[AISIX](https://api7.ai/tags/aisix)

[Platform Engineering](https://api7.ai/tags/platform-engineering)

[![Kong vs LiteLLM vs AISIX: Choosing a Self-Hosted AI Gateway](https://static.api7.ai/uploads/2026/11/09/WGLNy0sV_self-hosted-ai-gateway-cover.png)](https://api7.ai/blog/self-hosted-ai-gateway-kong-litellm-aisix)

Technology

November 9, 2026

#### [Kong vs LiteLLM vs AISIX: Choosing a Self-Hosted AI Gateway](https://api7.ai/blog/self-hosted-ai-gateway-kong-litellm-aisix)

Compare Kong, LiteLLM and AISIX for self-hosted AI gateways: deployment dependencies, licensing, shared quotas and the path to centralized management.

[AI Gateway](https://api7.ai/tags/ai-gateway)

[AISIX](https://api7.ai/tags/aisix)

[LiteLLM](https://api7.ai/tags/litellm)

[![Kong AI Gateway vs AISIX: Which Gateway Fits When Teams Share AI Models?](https://static.api7.ai/uploads/2026/11/09/ue5pqd2T_kong-ai-gateway-vs-aisix-v2.png)](https://api7.ai/blog/kong-ai-gateway-vs-aisix)

Technology

November 6, 2026

#### [Kong AI Gateway vs AISIX: Which Gateway Fits When Teams Share AI Models?](https://api7.ai/blog/kong-ai-gateway-vs-aisix)

Compare Kong AI Gateway and AISIX for shared LLM access: open-source options, licensing, self-hosted deployment, failover, rate limits, and cost controls.

[AI Gateway](https://api7.ai/tags/ai-gateway)

[AISIX](https://api7.ai/tags/aisix)

[Kong AI Gateway](https://api7.ai/tags/kong-ai-gateway)
