API7 Gateway 3.10.4: Reliability Through Runtime Change

Production reliability often fails during ordinary change rather than an extraordinary outage. A client uploads a larger payload. An upstream scales from four nodes to six. A Redis-backed rate limiter sees a traffic spike. A metrics dictionary reaches capacity. A database primary changes roles. None of these events is unusual, but each can expose hidden assumptions about how much work the API gateway performs and which runtime state survives a transition.

That creates two practical requirements for an enterprise API gateway. First, expensive work needs explicit limits so one request or one dependency cannot consume an unbounded share of resources. Second, useful runtime state should survive routine topology and infrastructure changes instead of resetting at the moment operators need continuity most.

API7 Gateway 3.10.4, released on July 27, 2026, addresses both requirements. The release adds body-buffer limits across plugins, makes Redis connection reuse configurable and effective, preserves health-check and least_conn state during upstream changes, prevents a full Prometheus dictionary from trapping a worker in a CPU loop, and bounds Control Plane database connection lifetime. It also improves restart recovery and protects concurrent configuration updates.

These are not disconnected fixes. Together, they support a clearer reliability model: bound the work a request can create, preserve the state that makes routing decisions accurate, and make dependency recovery part of normal operation.

flowchart LR
    change[Runtime Change] --> work{Is work bounded?}
    change --> state{Is useful state preserved?}
    work -- No --> pressure[Memory, CPU, or Connection Pressure]
    work -- Yes --> contain[Contained Resource Use]
    state -- No --> reset[Cold or Incorrect Decisions]
    state -- Yes --> continue[Continuous Health and Load Context]
    contain --> reliable[Predictable Gateway Operation]
    continue --> reliable
    pressure --> incident[Operational Incident]
    reset --> incident

Reliability Starts by Putting a Ceiling on Request Work

Several gateway plugins need to read a request or response body before they can transform, validate, inspect, or cache it. Without a clear ceiling, a valid but unusually large body can force a worker to retain far more data than the route's ordinary traffic requires. Under concurrency, that amplification can turn one payload pattern into fleet-wide memory pressure.

API7 Gateway 3.10.4 adds max_req_body_size and max_resp_body_size to body-buffering plugins, with a default of 67,108,864 bytes (64 MiB). The controls apply to AI Proxy, AI Proxy Multi, AI Request Rewrite, AI Prompt Decorator, AI Prompt Template, AI Prompt Guard, Request Validation, OAS Validator, Body Transformer, Response Rewrite, Proxy Cache, gRPC Transcode, SOAP, and other plugins that buffer bodies.

The outcome depends on which side crosses the boundary. A request body larger than the configured limit is rejected. A response body is truncated at the limit. Proxy Cache is deliberately different: when an upstream response is too large to cache, the gateway streams it through without caching it. That exception protects the worker while allowing the response to reach the client.

The default is a safety boundary, not a universal sizing recommendation. A route accepting video, model input, large SOAP envelopes, or bulk data may need a different limit from a JSON API. Operators should set the limit from the route's contract and expected concurrency, then test the exact over-limit behavior expected by clients. Raising the value should be paired with enough memory headroom for concurrent buffered requests; lowering it should be paired with clear client errors and documented payload limits.

Logger configuration now has a related upgrade check. The ClickHouse Logger, Elasticsearch Logger, File Logger, Loggly, Loki Logger, SkyWalking Logger, Alibaba Cloud Logging (SLS), and Syslog schemas validate max_req_body_bytes and max_resp_body_bytes as positive integers. Values such as 0, negative numbers, or quoted numbers like "1024" are rejected with HTTP 400. Existing routes with those values appear as errors in the gateway group's compatibility report and are not published to the Data Plane, while other routes remain unaffected. Before upgrading, replace invalid values with positive integers or remove the fields to use the default of 524,288 bytes.

This is an important form of reliability: reject an invalid resource limit before it reaches traffic, and isolate the failure to the affected configuration rather than destabilizing unrelated routes.

Connection Reuse Turns Dependency Capacity into a Policy

Rate limits and AI caching may make a decision for every request, which means their Redis connection behavior becomes part of request-path capacity. If each request opens a new connection, connection setup adds latency and can overwhelm Redis, network address translation tables, or local sockets long before application traffic reaches its intended throughput.

Version 3.10.4 adds redis_keepalive_timeout and redis_keepalive_pool to the Redis and Redis Cluster policies in Limit Conn, Limit Req, and AI Cache. It also fixes a defect in Limit Conn and Limit Req that prevented Redis connections from returning to the keepalive pool. With the fix, connections can actually be reused instead of being opened again for every request.

The settings make connection reuse an explicit capacity decision. The pool should be large enough for the concurrency of each worker and policy, but not so large that idle connections consume the dependency's connection budget. The timeout should retain connections long enough to avoid repeated handshakes without holding stale connections indefinitely. Teams should observe Redis connection counts, connection-establishment latency, command latency, gateway errors, and pool saturation while tuning them.

Connection reuse does not remove dependency failure. Redis can still become unavailable, slow, or partitioned. The operational advantage is that normal traffic no longer manufactures avoidable connection churn, leaving more of the dependency's capacity available for real policy work and making abnormal behavior easier to distinguish in telemetry.

Scaling Should Not Erase the State Behind Routing Decisions

An upstream topology change is common: an autoscaler adds nodes, a deployment removes old pods, or an operator changes weights. The risky part is not the new list itself. It is what happens to the state already learned about nodes that did not change.

API7 Gateway 3.10.4 upgrades the health-check engine so targets are reconciled incrementally rather than destroyed and rebuilt. When an upstream scales, unchanged targets retain their accumulated health state and failure counters. The gateway also avoids the earlier window in which rebuilding the target set could leave no node under active health checking.

The least_conn load balancer receives the same continuity principle. Previously, adding or removing an upstream node discarded tracked connection counts. The algorithm temporarily degraded toward round robin and could send new requests to nodes already carrying long-lived connections. Version 3.10.4 preserves the load state across upstream scaling, so the next routing decision still reflects the connections already in progress.

This matters especially for streaming APIs, WebSocket-style long-lived traffic, large downloads, and LLM responses. A node with only a few active requests may still be much busier than a peer if those requests remain open for a long time. Preserving connection counts prevents an ordinary scale event from making that workload invisible.

Operators should validate this behavior with a representative change, not only a static health check. Keep long-lived requests open, add and remove an upstream node, and confirm that unchanged nodes retain health status and that new requests do not suddenly concentrate on an already busy node. Reliability is demonstrated across the transition.

sequenceDiagram
    participant O as Operator or Autoscaler
    participant G as API7 Gateway
    participant A as Existing Node A
    participant B as Existing Node B
    participant C as New Node C

    G->>A: Track health and active connections
    G->>B: Track health and active connections
    O->>G: Add Node C to upstream
    G->>G: Reconcile target set incrementally
    Note over G,A: Preserve A's health counters and load state
    Note over G,B: Preserve B's health counters and load state
    G->>C: Start health checks for new target
    G->>G: Route using preserved and newly learned state

Observability Must Degrade Without Becoming the Outage

Monitoring protects reliability only while its own resource use remains bounded. In earlier behavior, a full shared dictionary used by the Prometheus plugin could trigger a loop that held a gateway worker at 100% CPU and did not recover after traffic stopped. A capacity limit in the telemetry path could therefore become a traffic-processing incident.

API7 Gateway 3.10.4 changes that failure mode. When the shared dictionary is full, the gateway degrades gracefully and logs that reported metric data may be incomplete. Losing some new metric samples is still an operational warning, but it is safer than allowing metrics collection to monopolize a worker.

The correct response is not to ignore the warning. Teams should alert on the log signal, inspect metric cardinality, review which route, service, consumer, and other labels are enabled, and size the shared dictionary for the intended series count. The release also rejects disabled_labels settings that remove structural labels such as type from latency metrics or code from status metrics, because removing those dimensions collapses distinct measurements into one misleading series. Non-structural labels such as route, service, and consumer can still be disabled.

Together, these behaviors establish two boundaries: telemetry must not consume the worker indefinitely, and cardinality reduction must not destroy the meaning of the metric. Reliable observability needs both.

Dependency Recovery Should Happen Without Process Luck

Long-lived connections are useful until the endpoint behind them changes roles. Before 3.10.4, pooled Control Plane database connections could be reused indefinitely. After a database failover, a connection could remain pinned to a demoted, read-only primary, leaving the Control Plane dependent on whether and when that connection happened to be replaced.

Version 3.10.4 bounds database connection lifetime at one hour by default and exposes database.max_lifetime for tuning. This does not make failover instantaneous, and the right value depends on database topology and connection costs. It does guarantee that pooled connections age out instead of persisting without a limit. Failover tests should measure how quickly the Control Plane resumes writes and whether the configured lifetime matches the recovery objective.

The release applies the same recovery mindset to gateway processes. The CLI now waits for a previous gateway instance to finish exiting before starting a replacement. If a container was killed without a clean shutdown, leftover worker event sockets are removed during startup so they do not prevent the new process from binding. These changes turn common stop-start races and unclean container exits into conditions the software handles directly.

Configuration updates also gain stronger state protection. Concurrent PATCH requests against the same resource are serialized so both no longer report success while the last write silently erases the other. The merged route produced by PATCH /apisix/admin/routes is validated against the route schema before storage. Operational automation still needs coordination and error handling, but the Control Plane now protects the resource from two subtle forms of invalid state.

A Reliability Review for API7 Gateway 3.10.4

Use the version-specific release notes with the rolling upgrade guide. A focused review should cover:

1. Inventory body-buffering routes. Record normal and maximum payload sizes, concurrency, client behavior on rejection or truncation, and the special Proxy Cache pass-through behavior.
2. Validate logger limits. Find max_req_body_bytes and max_resp_body_bytes values that are zero, negative, quoted, or otherwise invalid before checking the compatibility report.
3. Size Redis reuse. Set keepalive timeout and pool values from observed concurrency and Redis connection capacity; monitor both the gateway and Redis during a canary.
4. Exercise topology changes. Add and remove upstream nodes while health checks and long-lived requests are active; verify continuity of health counters and least_conn decisions.
5. Test telemetry saturation. Alert on incomplete-metrics warnings, measure cardinality, and confirm that structural Prometheus labels remain enabled.
6. Test database failover. Verify Control Plane write recovery and tune database.max_lifetime only with an explicit recovery and connection-cost target.
7. Test restart recovery. Include a normal restart and an unclean container termination, then confirm the replacement starts and serves traffic.
8. Protect configuration writers. Retry failed updates safely, inspect HTTP 400 validation responses, and ensure automation does not assume that every patch is valid.

The canary should combine these checks. A gateway that handles ordinary requests but has not experienced a payload boundary, upstream scale event, metrics-pressure condition, dependency transition, or restart has not yet demonstrated the reliability properties introduced in this release.

Make Change a Normal Operating Condition

API gateway reliability is not only the ability to remain available when nothing changes. Production systems continuously change payload sizes, traffic rates, upstream membership, connection state, metrics cardinality, database roles, and configuration revisions.

API7 Gateway 3.10.4 makes several of those transitions safer by bounding memory and connection work, preserving health and load state, degrading telemetry without consuming a worker, expiring database connections, recovering from stale process artifacts, and preventing ambiguous concurrent updates.

The common outcome is predictable behavior under motion. Operators get limits they can size, state they can trust across topology changes, and recovery paths they can test before an incident.

Read the full API7 Gateway 3.10.4 release notes, follow the rolling upgrade procedure, and validate each transition with representative traffic before expanding the rollout.