OpenAI Decisions API: Put Gateway Policy Around AI Judgment

OpenAI's new Decisions API is useful precisely because it does less than a general-purpose generation endpoint. It takes text, images, or both and returns typed answers: a probability for a predicate, a selected choice, a score against ordered levels, or a refusal for a named question. That narrower contract can make classification and prioritization easier to integrate.

It does not turn a model judgment into an authorization fact.

That distinction matters for API and AI platform teams. A probability can help rank a support ticket, route content to a review queue, or identify an image that may need inspection. It should not silently become the sole reason to approve a payment, grant an administrative permission, or bypass a deterministic security rule.

The launch drew substantial attention in this week's Hacker News discussion. The practical architecture question is not whether AI can make a decision. It is where that decision belongs in a system whose identity, policy, limits, and audit requirements remain deterministic.

Key Takeaways

• The Decisions API is a public beta with a dedicated contract; pin that contract and expect change.
• A model probability is evidence for an application policy, not proof of identity or permission.
• Keep authentication, authorization, schema validation, quotas, and audit outside the model call.
• Version the question, allowed choices, threshold, model, and fallback as one decision policy.
• Treat refusals, uncertain answers, missing or malformed responses, and unavailability as explicit application states.
• An AI gateway can govern the HTTP call, but normalized support for /v1/decisions must be verified in the deployed gateway release.

What the Decisions API Actually Returns

As documented on October 8, 2026, the OpenAI Decisions API is in public beta, uses POST /v1/decisions, and currently supports gpt-6-luna. A request supplies a model, shared input, and one or more named questions.

The requested question determines the scored answer shape. For any named question, the response can also contain a refusal:

Answer type	Output	Appropriate use
predicate	Probability from 0 to 1	Estimate whether a stated condition is true
choice	One supplied value plus probabilities	Select among unordered categories
score	A probability-weighted score over ordered levels	Rank severity, quality, or priority
refusal	No scored answer for that named question	Apply the application's refusal fallback

This is not the same as asking the Responses API for arbitrary JSON. OpenAI directs applications that need a custom generated object toward Structured Outputs, while function calling is for asking a model to request a tool with arguments. Decisions is optimized around a small set of judgment shapes.

That narrower shape is operationally helpful. It gives the gateway and application a predictable endpoint, while the application can validate that question names, choice values, and response fields match the policy version it expects. But a typed response can still be wrong, uncertain, stale relative to business policy, or unsuitable for a high-impact action.

Put Deterministic Controls Around Probabilistic Output

The clean design separates the request path into three layers:

flowchart LR
    C[Application] -->|authenticated request| G[API or AI gateway]
    G -->|bounded Decisions call| D[OpenAI Decisions API]
    D -->|typed answer or refusal| P[Application policy]
    P -->|allow, review, deny, or fallback| O[Business action]

The gateway authenticates the caller, authorizes the route, validates the outer request shape, applies traffic and payload limits, protects provider credentials, and records transport-level evidence. The Decisions API produces a typed answer, which may be a refusal. The application combines a scored answer with deterministic business rules or follows its refusal fallback.

Do not collapse those layers. If a predicate returns 0.93, the number says what the model estimated for the supplied question and input. It does not say the caller is entitled to act, the evidence is complete, or the threshold is appropriate for every tenant.

For example, a fraud-review service might use a model score to choose between an automatic low-risk path and manual review. It should still verify the caller's identity, tenant, account state, and transaction limits outside the model. A high score might escalate review; it should not rewrite the authorization policy.

This boundary is consistent with the broader AI gateway architecture: the gateway governs traffic and credentials, while the application owns the meaning of the result.

Version the Entire Decision Policy

A stable endpoint does not make the decision policy stable. The outcome can change when any of these inputs change:

• model snapshot or supported model;
• question text and instructions;
• allowed choices or score levels;
• input preprocessing and image handling;
• threshold that converts probability into an action;
• fallback used when the request fails or is uncertain.

Store a policy version that binds these elements together. Log that version with the decision ID or request correlation ID, the selected outcome, latency, and the final application action. Avoid logging raw prompts or images by default when they may contain personal, regulated, or proprietary data.

Replaying a sample set is useful before changing a threshold or question. The sample needs representative positives, negatives, ambiguous cases, and cases where required evidence is absent. Compare both false positives and false negatives because the safer threshold depends on the consequence of each error.

This is also where an AI gateway audit design helps: transport and policy evidence should explain which request was allowed to reach a provider and which controls ran. It does not prove the model's judgment was correct, so application review outcomes must be correlated separately.

Design the Failure Path Before the Happy Path

The API is in beta, so production integration should make failure behavior explicit. At minimum, define responses for:

1. transport timeout or provider unavailability;
2. rate-limit response;
3. malformed or schema-incompatible response;
4. an explicit type: "refusal" answer for a named question;
5. a missing named answer;
6. probability near the decision threshold;
7. input that exceeds a size or media constraint;
8. a policy version the application no longer recognizes.

A refusal is a valid answer variant, not a malformed response or a zero probability. Match answers by name and check type before reading a probability, choice, or score. Route a refusal through the fallback defined for that decision policy, such as manual review or withholding an action that requires a scored answer.

Fail-open and fail-closed are too coarse as universal advice. A product-recommendation ranker may fall back to a deterministic ordering. A moderation workflow may send uncertain material to human review. A security-sensitive action may deny the operation until required evidence is available. Put the fallback beside the rule it constrains, and test the observable status returned to the caller.

Avoid automatic retries for every failure. Retrying a timeout can amplify a provider incident and may create duplicate application work even if the remote computation completed. Use bounded retry only for documented transient conditions, keep an overall latency budget, and make the surrounding business operation idempotent where duplicate execution would matter.

What the Gateway Can Enforce

A gateway provides a consistent boundary for controls that do not depend on believing the model output. In Apache APISIX 3.19.0, for example, versioned plugins document separate mechanisms for OpenID Connect authentication, request validation, request-count limits, and HTTP logging.

Those mechanisms solve different problems. A valid token does not authorize every decision question. A valid JSON schema does not make an instruction safe. A rate limit does not guarantee an affordable bill when inputs or provider pricing vary. A log proves that an event was recorded according to configuration, not that the result was correct.

For an AI-specific gateway, verify the exact deployed contract. A generic reverse proxy can route an HTTP request to /v1/decisions, but that does not prove the gateway understands its question schema, usage fields, streaming behavior, or error mapping. If the product documentation does not list the endpoint as normalized support, treat it as provider-specific passthrough and test request preservation, authentication, limits, response status, and observability end to end.

This avoids a familiar compatibility trap. “OpenAI-compatible” often refers to selected endpoints such as chat or responses, not every present and future OpenAI API. API7.ai's guide to testing an OpenAI-compatible gateway explains why compatibility must be verified feature by feature.

A Practical Rollout Checklist

Start with a reversible, low-impact classification use case:

• Pin the public-beta API contract and supported model at the time of implementation.
• Give every question a stable unique name and version its instructions and choices.
• Authenticate the caller and authorize which applications may use each decision policy.
• Bound request body size, image types, concurrency, and request rate.
• Keep provider credentials at the gateway or server side, not in untrusted clients.
• Define threshold bands for automatic action, human review, and fallback.
• Test refusals, unavailable or malformed responses, ambiguous answers, and near-threshold probabilities.
• Record the policy version and final application action without exposing sensitive input.
• Re-evaluate the integration when the beta contract or available model changes.

The success metric should not be “the API returned a choice.” Measure whether the complete system makes the intended action with acceptable false-positive, false-negative, latency, cost, and review rates.

Conclusion

The Decisions API makes a common AI workload easier to express: evaluate shared evidence and return a typed answer, including an explicit refusal when it declines a question. Its value comes from the narrow contract, not from replacing policy.

Keep probabilistic reasoning between deterministic boundaries. The gateway should control who can call the endpoint, how much traffic and data can pass, which provider credential is used, and what evidence is recorded. The application should decide how a probability affects a business action, including uncertainty and failure.

Explore API7 Enterprise to centralize authentication, traffic controls, credential handling, and observability around AI and API calls while your application retains ownership of authorization and business decisions.