AISIX 1.3.0: Run Structured AI Agent Workflows Across Providers

An invoice-review agent turns invoice information and purchase-order facts into a structured recommendation for a human reviewer. Changing its model provider tests more than whether a prompt reaches a model: the evidence, requested JSON shape, and safety policy must remain usable. For the shared evaluation below, the application extracts invoice text and retrieves purchase-order facts before submitting them as text in a separate recommendation request. Model-driven tool loops and direct document input require additional, route-specific checks.

AISIX 1.3.0, released on September 18, 2026, expands the gateway's support for that workflow. The strongest reason to evaluate this release is the combination of richer Responses translation, structured output across additional providers, and explicit conversation-level guardrail controls. The invoice agent below is an illustrative evaluation scenario, not a benchmark or a packaged AISIX application.

The deployment boundary matters throughout: AISIX provides the gateway traffic-processing path. AISIX Cloud adds the control plane, dashboard, organizations, and centralized usage views, with On-Premises and Hybrid Cloud deployment options. In both options, you operate the gateway in your own environment.

Key Takeaways

• AISIX 1.3.0 expands structured output and Responses translation. Cross-provider adoption still depends on the selected adapter: richer bridge fields do not guarantee a complete multimodal or tool-calling workflow.
• Teams can choose which conversation turns an input guardrail reads, but must review tool-result coverage and content that remains outside inspection.
• Terminal access logs and abandoned-request usage events make the end of an agent request visible; dashboard baselines and log processing need corresponding updates.
• Upgrade planning must account for configuration-cache persistence becoming opt-in, control-plane-first ordering, and API and deployment compatibility changes.

Choose a Supported Path for the Agent's Evidence

Start with the recommendation request. An application using /v1/responses may target a provider that supports Responses natively or one that needs translation through a chat API. AISIX selects that path from the provider key's API-surface configuration; the selected adapter then determines what reaches the upstream.

In 1.3.0, the Responses bridge maps input_image, input_file, and input_audio into the intermediate chat format. It serializes a JSON function_call_output and translates supported tool-choice controls and freeform custom tools. These additions expand the bridge, but a later adapter can still drop content or lack the state required for a tool loop:

• Gemini through Google AI Studio: do not use /v1/responses or translated /v1/messages for multi-step Gemini 3 tool loops. Those bridges do not replay tool_calls[].extra_content.google.thought_signature, so a subsequent request can fail with 400. Use /v1/chat/completions and preserve the complete assistant tool-call objects when continuing that loop, as the Gemini integration guide describes.
• Gemini through Vertex AI: the current Vertex Gemini adapter is text-only and does not serialize function declarations or tool results as a tool exchange. It does not support the invoice agent's model-driven lookup loop, including through Responses. The AI Studio alternative above does not apply to this adapter; check the Vertex endpoint and content limits.
• Anthropic through chat translation: the adapter drops non-text message parts. If image or document content must reach Claude, use the native Anthropic-style /v1/messages path described in the Anthropic translation guide, rather than assuming the Responses bridge's intermediate representation preserves it end to end.

The text-based recommendation step avoids depending on those multimodal and tool-loop paths. Test the actual adapter and model with that input. The bridge also drops images represented only by a file_id, non-text parts in tool results, hosted tools, and previous_response_id. An allowed_tools selection loses its named subset and keeps only its mode, so it is not a tool-authorization boundary. Review the Responses bridge's full limits before moving from a native endpoint.

Once the supported evidence reaches the model, the next question is what the application can consume from its answer.

Ask for a Structured Result, Then Validate It

Suppose the invoice agent's final recommendation needs three fields: an invoice identifier, a review decision, and an explanation. In 1.3.0, a Responses request's text.format is translated into response_format, then into the structured-output mechanism used by the selected provider.

Choose the structured-output path by adapter as well as model:

Provider path	How AISIX carries the requested schema
Anthropic	Claude 4.5 and later use output_config.format; older families and compatible third parties use the synthetic-tool path.
Vertex AI, Gemini 2.x and later	Uses generationConfig.responseJsonSchema and forwards the caller's JSON Schema unchanged.
Vertex AI, Gemini 1.x	Uses responseSchema with older-dialect conversions: additionalProperties is removed, while supported bounds such as minimum, maxLength, and pattern remain.
Bedrock	Claude 4.5 and later use output_config.format on Messages or outputConfig.textFormat on Converse. AISIX uses a synthetic tool only for recognized tool-capable publishers: Anthropic, Amazon Nova, Meta, Mistral, and Cohere. Models without tool support leave the format unhonored because attaching toolConfig would fail the request even when the caller supplied no tools. AISIX also leaves the format unhonored for unclassified publishers to avoid a possible toolConfig failure.

The Gemini mappings in this table belong to the Vertex adapter. Google AI Studio uses the OpenAI-compatible endpoint through AISIX's openai adapter, so those generationConfig conversions do not describe that path.

Anthropic and Bedrock narrow schemas to their supported subset, move removed constraints into property descriptions, and set additionalProperties: false. Vertex Gemini 2.x and later do not receive that rewrite; the 1.x conversion removes unsupported keywords and restates selected constraints in descriptions while keeping supported bounds. These paths preserve the caller's required list. A constraint stated in prose is not decoder enforcement. Validate every returned JSON recommendation against the application's own contract.

On the Anthropic synthetic-tool path, AISIX appends json_tool_call alongside the caller's tools. It forces that tool unless the caller supplies a non-null tool_choice (including auto) or enables extended thinking. Merely supplying a business tool does not give it precedence. Bedrock's synthetic-tool forcing also depends on the publisher: Claude and Amazon Nova honor explicit selection; other supported families are offered the tool without that forcing. See the Bedrock structured-output behavior.

Keep the recommendation request separate from the purchase-order lookup, and verify any explicit tool choice before combining them. A streaming request on a synthetic-tool path runs its upstream leg non-streaming, then streams the completed result to the client, increasing time to first byte. It is not native upstream streaming.

This gives the team a concrete acceptance test: the same supported input reaches each chosen model, and its result passes application validation. It does not establish that every provider will follow every schema constraint or reach the same decision.

Decide Which Parts of the Conversation a Guardrail Reads

After a recommendation, the reviewer may ask a follow-up question and the application may replay conversation history. A policy matching an old message can keep blocking later requests even when their new input is acceptable.

AISIX 1.3.0 adds input_messages to every guardrail kind. Its default, all, keeps the whole-request input window. latest_turn narrows the window to messages after the last assistant message, excluding system messages; a trailing assistant prefill remains part of the current turn. The setting affects input checks only. In AISIX Cloud, configuring input_messages: latest_turn with hook_point: output is rejected with 400 INVALID_REQUEST.

For the invoice agent, the choice depends on the rule's purpose. A rule concerned with a new instruction can evaluate the latest turn. A rule responsible for masking sensitive information throughout the conversation should retain all: with latest_turn, earlier history is still forwarded unchanged. A blocked message replayed without an intervening assistant reply remains in the current turn and is still checked. Narrowing the window does not automatically clear a blocked conversation.

There is also an upgrade-sensitive change in tool handling. On /v1/responses, AISIX now classifies replayed function_call names and arguments as assistant items and function_call_output as tool results, making them available to guardrails that inspect the corresponding roles. Previously, the tool-call names and arguments were skipped, while function_call_output was misclassified as user text and was therefore scanned by the default semantic and Azure moderation settings. Their default text_source now reads user-role messages, so it excludes both assistant tool-call names and arguments and tool-role results; select their all-messages setting when they must inspect those items. Other guardrail kinds that read all roles can inspect them directly. The input window and the guardrail's text-source selection both constrain coverage.

For an invoice workflow, inspect these boundaries before enforcing policy:

• Uploaded files: /v1/files uploads and downloads bypass input and output guardrail inspection. Screen required file content before it reaches that route; forwarding a file reference is not evidence that its contents were screened.
• Generated reasoning: model-generated reasoning remains outside output scanning and masking, even when the bridge returns it as a Responses reasoning item. Replayed readable reasoning has different input coverage; encrypted reasoning payloads remain unscanned and unmodified.
• Streaming output: enabling output guardrails buffers the stream for inspection, affecting when the user receives content. A buffered frame the gateway cannot parse is dropped; if nothing remains, the response is refused with 422.

Use the guardrail behavior reference to test the exact attachments, input window, text source, and enforcement action. A valid structured answer and a completed guardrail check answer different questions.

Observe How the Request Actually Ends

The human reviewer may close the page while the agent is still working. Previously, a caller disconnect could leave an access log without a usage row, while a streaming access log written at response-head time could report 200 before the stream was abandoned.

In 1.3.0, a caller that disconnects before AISIX sends response headers produces an access-log entry with 499 and error_kind = "client_disconnected", without token fields. Its usage event uses error_class = "client_disconnected", with zero tokens and zero cost. This pre-response cancellation must not be confused with a stream that has already begun: a mid-stream disconnect still produces a 499 usage event, and its token counts cover the content delivered before the disconnect. A streamed request writes its completion evidence at the end, so operators can distinguish an incomplete stream from a delivered response without treating the initial response header as completion.

The pre-response record does not prove that a provider incurred no work or will make no charge. For a mid-stream disconnect, use the recorded or estimated usage when interpreting spend, budgets, and token rate limits; it remains part of that accounting even though the caller abandoned the response. In AISIX Cloud, include these 499 events deliberately when calculating request totals and success rates; latency percentiles already filter to successful rows.

Access logs now separate duration_ms, the complete gateway occupancy time, from latency_ms, which remains time to first token for a stream. They also identify the selected upstream model and provider key. Model-group cache hits no longer attribute traffic to a provider that was never called. Follow the access-log and correlation guide when adjusting dashboards and parsers.

For the invoice agent's rollout, evaluate one completed response, one policy refusal, one cache hit, and one disconnected stream. Confirm both the caller outcome and its recorded evidence.

Plan the Upgrade Around the Deployment You Operate

The release also adds deployment and configuration capabilities outside this agent workflow: stable resource-ID references, shared pricing documents, organization export/import, private endpoint addresses, multiple proxy listeners, and ARM64 images. They address operational choices rather than extending structured-output guarantees.

Use the upgrade procedure and every intervening release's notes. The supported upgrade floor is 0.12.0; older installations need an intermediate upgrade. Back up the database before upgrading an AISIX Cloud On-Premises control plane, then upgrade the control plane before gateways. Rollback reverses that order and depends on the database backup. Hybrid Cloud users coordinate the API7-hosted control-plane step rather than operating it themselves.

Review the following before rollout:

Area	Action or boundary
Restart resilience	Disk snapshots are now opt-in in managed and self-hosted etcd modes. Set managed.snapshot_cache_enabled: true or AISIX_MANAGED__SNAPSHOT_CACHE_ENABLED=true if restart recovery depends on them. A path alone no longer enables persistence. Without it, a restart waits for configuration before opening the proxy listener when the control plane is unreachable. In-memory last-known-good serving is unchanged.
Mixed-version pricing	Pre-1.3.0 gateways can report a rejected pricing row and aisix_config_last_reload_successful 0 after the control-plane upgrade, while traffic, readiness, and later updates remain unaffected. Upgrade gateways promptly and account for this documented signal in alerts.
Model references	Renames preserve ID-bound references, but name-based rate-limit conditions do not follow them. Deleting a model used by a model-scoped rate limit now returns 409 MODEL_IN_USE. Re-save cache selectors after deleting and recreating a selected model; the new resource has a different ID.
Pricing and reasoning automation	Re-save organization price overrides whose audio duration rate changed after their first save. Update effort_mapping clients for nullable values; {"": null} and mappings to "" are now rejected. Reserved entries also receive 422 DP_INCOMPATIBLE while affected gateways are too old.
Private endpoints	resolve_addresses preserves the endpoint hostname for HTTP and TLS while changing dial addresses. It excludes Bedrock and /v1/realtime, and has no effect through a forward proxy. All target gateways must support it; older gateways would ignore it and use DNS, so saving it is refused with 422 DP_INCOMPATIBLE.
Listeners and routing	A nonempty proxy.listeners replaces proxy.addr; combining it with proxy.tls is an error. Scope URL rewrite rules with hosts when they should not also affect host-matched passthrough routes.
Network settings	Review newly honored upstream.pool_idle_timeout_secs and upstream.connect_timeout_ms for Bedrock, object-store exporters, and Realtime. Validate control-plane addresses and update allowlists expecting the old aisix/0.1 User-Agent. Invalid AISIX_TRUSTED_ORIGINS entries are dropped and sign-in from them is refused.
Deployment packages	Choose the matching linux/amd64 or linux/arm64 offline package. The control-plane chart no longer pins runAsUser, runAsGroup, or fsGroup; restore explicit values where mounts or cluster policies require them. OpenShift deployments must separately adjust the bundled PostgreSQL security contexts.
Export and import	Organization bundles require the same control-plane version. Unredacted backups require owner privileges and retain credentials encrypted under the deployment master key; a redacted support bundle substitutes known credentials and a published fixed password, so it is not a production credential backup. Import has authentication, master-key, organization, and certificate-authority checks; follow the documented restore procedure and restart cp-api and dp-manager if the authority is replaced.

The complete 1.3.0 release notes include the exact API changes and deployment conditions. In particular, read the export/import authorization rules before moving an organization's configuration; those are control-plane operations, not standalone gateway features.

Evaluate One Complete Agent Exchange

Begin with the shared recommendation step: invoice text and purchase-order facts in, a validated recommendation out, and an observable final outcome. Repeat it on each intended adapter and model, including a refused input and a cancelled stream. Evaluate model-driven tool loops and multimodal input separately on the routes that support them; keep application authorization and result validation explicit.

AISIX 1.3.0 makes more of that exchange portable across supported provider paths. Adoption should follow demonstrated compatibility and policy coverage for your workflow. Start with the Responses API guide, then apply the upgrade checks to the deployment you operate.