AISIX 1.10.0: Keep AI Coding Sessions on a Matched Semantic Route
November 12, 2026
AI coding and agent clients do not send one isolated prompt. A working session can include a task request, tool results, title-generation requests, token counts, and follow-up turns. If the gateway makes a new semantic-routing decision for every request, one conversation can move between policies or model classes even when the original task has not changed.
AISIX 1.10.0, released November 12, 2026, extends semantic routing to /v1/messages, /v1/messages/count_tokens, and /v1/responses. It also lets a semantic route target a model group and adds optional session affinity. Together, these changes let a supported coding or agent session reuse the route in its first established pin while that route can still use a model group's own retries, health checks, fallbacks, and availability policy.
This is not a promise that one user or one conversation always receives one model. Affinity is off by default, needs a usable session identifier, and is scoped to a semantic router, an API key, and a session. The pinned route can still fall back when its target cannot serve the caller.
Why a Session Needs One Routing Decision
Consider a team that uses one AI gateway alias for coding assistance. Its semantic router sends security-sensitive implementation work to a review-oriented route and ordinary developer questions to a general route. A typical client session can include the original user request, an auxiliary title request, tool outputs, a token-count request, and follow-up turns with little or no new natural-language text.
Semantic matching can be influenced by adversarial prompts, so do not rely on route selection alone as a security or compliance boundary. When the destination has security or compliance consequences, attach input guardrails to the semantic router the caller addresses so they run before dispatch; guardrails attached only to a destination model do not run for that request.
In a local demonstration with the AISIX 1.10.0 gateway, a simulated embedding service, and simulated model endpoints, a contract prompt matched a route targeting a model group. A subsequent Python prompt with the same API key and session ID stayed on that route; the access log reported semantic_pinned=true. The same Python prompt without a session ID selected the default instead. These simulated responses demonstrate the tested routing path, not model quality or provider billing.

The AISIX 1.10.0 dashboard's route test selects the contract route and demo-legal-group using a simulated embedding service. It shows the route test, not the subsequent session pin or an actual provider's model behavior. View the full-resolution screenshot.
Before 1.10.0, semantic routers worked on /v1/chat/completions; the Messages and Responses surfaces could not use a semantic model. In 1.10.0, all four supported surfaces use the same routing resolution path. For /v1/messages/count_tokens, the selected upstream must support native Anthropic Messages and token counting, such as an Anthropic adapter or a provider key declaring apis.messages; a chat-only target can return 400. A model group attempts compatible members, so verify the capability of group members and fallback targets. The router classifies the newest user turn that has human text. Tool results, images, complete <system-reminder> blocks, the top-level system prompt, and Responses instructions are not classifier input.
For Codex callers, AISIX also removes Codex-injected context blocks, such as <environment_context> and # AGENTS.md instructions for …, before classification. That distinction matters when the newest user turn has no text: routing should be based on the user's usable text, not client-injected operating context. An unclosed block is not silently removed. Without a valid pin, if no usable user text remains, the router attempts default with semantic_fallback=empty_prompt; the request can fail if the default is ineligible or unavailable.
Add Session Affinity Deliberately
The new setting is optional. An absent session_affinity block is off, and enabled defaults to false.

In the AISIX 1.10.0 local demonstration, session affinity is enabled with a 3600-second TTL. This configuration view does not itself demonstrate a pinned request; the access-log observations above do. View the full-resolution screenshot.
semantic: session_affinity: enabled: true ttl_seconds: 3600
ttl_seconds defaults to 3600 and must be between 60 and 604800. A matching request with a session identifier can create a pin; the TTL slides when the pin is used. AISIX can obtain that identifier from x-aisix-session-id, recognized vendor session headers, supported Codex session headers, x-session-id, selected request metadata, or the session.id entry in W3C baggage. A request without a usable identifier is classified normally and creates no pin.
The pin key includes the semantic-router model ID, the caller's API-key ID, and the session ID. Two API keys therefore do not share an affinity record merely because they present the same session identifier. In a managed deployment, pins can be shared by gateways through the data-plane manager. If that store is unavailable, times out, or refuses a key, the gateway falls back to a process-local pin without failing the request; replicas can then differ. Without a shared store, pins are not shared between replicas and are lost on restart.
For a new session, AISIX creates a pin only when a route matches and its eligible target is selected for dispatch. Concurrent first requests follow the pin that is successfully established; the earliest arrival is not guaranteed to win. The pin does not prove that an upstream request succeeded. It does not pin no_match, low_confidence, empty_prompt, decision-failure, or target-unavailable outcomes. /v1/messages/count_tokens and recognized client-side requests can follow an existing pin, but do not create one. AISIX recognizes auxiliary requests using specific headers or known client prompts; an unrecognized title-generation or auxiliary request can create a pin, including after a client changes its prompts. Test the client version you deploy. If a pinned target is no longer a route target, AISIX drops the pin and classifies again. If it is temporarily unavailable, AISIX attempts default and retains the pin; an ineligible or unavailable default can still make the request fail.
Use the access log to verify the behavior in your own deployment. A request using a pin has semantic_pinned=true and does not report a new semantic score. When the pinned target is available, the log reports its route; a target_unavailable fallback can omit semantic_route. Do not treat semantic_pinned as proof that a particular upstream model served the request: the selected route can point to a model group, whose member selection remains a separate decision.
Let a Semantic Route Use a Model Group
In 1.10.0, a semantic route, its default, or its embedding/classifier failure target can name a model group. The group must include at least one member whose provider supports Chat Completions. This is useful when the policy decision is “use the coding route,” while the operational decision inside that route is “choose an eligible member according to the group's configured strategy.”
The group retains the behavior it has when addressed directly: its strategy, retries, fallbacks, cooldown, health checks, and when_all_unavailable setting apply. If a route's group cannot serve this caller, AISIX treats that target as unavailable and attempts the router's default with semantic_fallback=target_unavailable. The request can fail if the default cannot serve the caller either.
That fallback is an availability behavior, not a guarantee that the default has the same capability, cost, or policy as the intended route. Design the default so it is safe for the traffic that may reach it, and test the unavailable-group path. Guardrails, model access, the router's allowed_cidrs, the response cache, and the requested-model identity remain associated with the router rather than becoming group-wide claims.
Upgrade Without Surprising Existing Clients
Upgrade the gateways before saving configurations that depend on the new capabilities. While an environment has a registered gateway older than 1.10.0, the control plane refuses a semantic-router target that is a model group with 422 DP_INCOMPATIBLE. It saves session_affinity with a field_ignored warning instead; older gateways route without pinning. An older gateway also ignores window_type and enforces a fixed rate-limit window. The control plane rejects saving a sliding-window limit with 422 DP_INCOMPATIBLE, including unrelated edits that rewrite an existing sliding configuration, such as disabling or rotating a key. Upgrade all registered gateways before making those changes.
Existing chat routers can classify differently after this upgrade. An image-only or reminder-only newest user message can now cause AISIX to look back to the newest earlier user turn with text instead of immediately selecting default. Multiple text parts are joined with a space, and Codex-injected context is excluded for Codex callers. Validate representative coding and agent requests before rollout, especially if route choice affects cost, tool availability, or a policy boundary.
The semantic-routing story is not the whole upgrade. Review these independent changes before rollout:
- In OpenAI-format budget
429responses,error.limit_usdanderror.spent_usdbecomeerror.limitanderror.spent. Rename dashboards and alerts usingaisix_budget_limit_usd,aisix_budget_spent_usd, oraisix_budget_remaining_usdto use their names without_usd. - Exported usage events no longer contain
cost_usd; update exporter consumers. In usage-events CSV exports, the column is renamed fromcost_usdtocost. In managed deployments, the control plane's usage data is authoritative for cost; a standalone gateway now exports no cost for any request. - OpenAI-style cache writes now use the configured cache-write price for new events, including events from older gateways reporting
cache_write_tokens; budgets count the new cost. Existing history is not repriced, and models without a cache-write price are unchanged. Review the 12-hour pricing rules in the release notes. - The default gateway log level changes from
infotowarn; setlog_level: infoif existing tooling needs INFO lines, or use/readyzfor readiness. An enabled access log is now written independently of log level, andaccess_log: falseturns it off. - Model-group member limits count each upstream attempt, including same-target retries, while certain group-matching policies count once per request. Recheck limits and alerts against your actual policies and retry behavior.
- A
/v1/responsesrequest withadditional_toolsinput items can now take the Responses-to-Chat bridge even for a key that otherwise forwards Responses natively. The new reroute excludes OpenAI, bring-your-own, a provider omitted in an AISIX OSS resource, and literalazurewith an adapter other thanbedrock,vertex, orazure-openai. Azure OpenAI keys already bridge all Responses requests. The decision is per upstream attempt, so a group can use different paths on fallback. Rerouted requests do not resolveprevious_response_id; test clients that rely on stored response state. Requests withoutadditional_toolsdo not undergo this specific rerouting. - Built-in PII detectors now match values next to CJK text, and
bank_cardmatches supported grouped numbers that pass its prefix and Luhn checks. Conversely,us_ssnandip_addressno longer match values written entirely in full-width or other non-ASCII digits. Review and test affected masking and blocking rules.
Also review telemetry for MCP or A2A failures delivered inside HTTP 200 responses in the full 1.10.0 release notes. These changes are not enabled by session affinity.
Start with a Bounded Rollout
Use session affinity when an initial meaningful user request is a sound basis for a continuing route decision, and when your default route is safe if the pinned target becomes unavailable. Keep it off when each turn should be reclassified independently or when a caller-provided session identifier is not an appropriate continuity key for your policy.
Start in a non-production environment with a small set of distinct routes. Verify the initial match, a follow-up that would otherwise classify differently, an auxiliary or title request, a count-token request, a request without a session identifier, and an unavailable-target fallback. Then inspect semantic_route, semantic_fallback, and semantic_pinned in the access log alongside the served member and the application's own outcomes.
See the 1.10.0 release notes for the routing changes and all upgrade considerations.


