Distributed API Gateway Rate Limiting: Local vs Redis Counters and Accuracy Trade-offs

Use node-local counters when the policy is a fast protective guardrail for each gateway instance and some fleet-level drift is acceptable. Use a shared Redis-backed counter when several gateway instances must enforce one tenant, credential, or route budget. The shared design improves coordination, but it adds a network dependency and forces an explicit decision about what happens when Redis is slow or unavailable.

There is no universally exact distributed limiter. Load-balancer skew, retries, counter windows, clock behavior, autoscaling, and dependency failures all affect what clients observe. Define the business invariant first, then select the counter and failure policy that protect it.

Key Takeaways

• Local counters are low-latency and failure-isolated, but every gateway instance has independent state.
• Redis counters coordinate a fleet, but Redis latency and availability become part of the request path.
• A stable authenticated identity is usually a fairer key than a source IP.
• Decide fail-open versus fail-closed per route; allow_degradation is an availability and security decision, not a tuning shortcut.
• Test scaling, load-balancer skew, counter-store failure, and hot tenants before production.

Define the Limit You Actually Need

“100 requests per second” is incomplete. Specify all of these:

A login endpoint may require a conservative, identity-aware abuse limit. A public catalog read may use a local guardrail to keep each gateway healthy. A paid inference endpoint may need a shared tenant quota because accepting requests on every node multiplies spend.

Local and Redis Counters Solve Different Problems

Apache APISIX's limit-req plugin implements a leaky-bucket limiter and documents local, redis, and redis-cluster policies.

Local policy

With policy: local, each APISIX instance maintains its own counters. This avoids a remote lookup and prevents a Redis outage from disabling the limiter. It is a good fit for instance protection and deployments where traffic is predictably partitioned.

The trade-off is scope. If a client is spread evenly across four instances configured for 100 requests per second, the fleet can admit roughly four times the per-instance setting. The exact result depends on distribution and bursts; it is not a guaranteed multiplier. Scaling the fleet also changes aggregate admission unless you recalculate the per-instance rate.

Redis policy

With policy: redis, APISIX instances use shared counter state. This supports a fleet-wide policy for the same key and avoids multiplying a tenant allowance merely because more gateway nodes were added.

Redis is now on the policy path. Network latency, connection limits, authentication, TLS, cluster failover, and hot-key load must be included in the design. A shared counter can improve consistency without becoming mathematically exact under every race and failure condition.

Choose a Stable Key

The key determines who shares a bucket. Prefer an authenticated API consumer or tenant identifier when the policy represents entitlement. Use source IP only when the trusted proxy chain has already restored the real address and when shared NATs, IPv6 privacy addresses, and mobile networks are acceptable sources of error.

Avoid unbounded attacker-selected keys. A raw query parameter can create high-cardinality counter state and let a caller obtain a new bucket at will. Normalize composite keys, namespace them by environment and route, and define retention or expiration behavior.

Configure a Shared APISIX Limit

The following illustrative Apache APISIX 3.18 route applies a shared request rate to an authenticated consumer name. Replace the endpoint, credentials, and limits for your environment. Store secrets through your deployment's secret workflow; never commit them.

curl "http://127.0.0.1:9180/apisix/admin/routes/report-export" \
  -X PUT \
  -H "X-API-KEY: ${admin_key}" \
  -d '{
    "uri": "/v1/reports/export",
    "plugins": {
      "key-auth": {},
      "limit-req": {
        "rate": 5,
        "burst": 2,
        "key_type": "var",
        "key": "consumer_name",
        "policy": "redis",
        "redis_host": "redis.internal",
        "redis_port": 6379,
        "redis_password": "<managed-secret>",
        "redis_database": 2,
        "redis_timeout": 1000,
        "allow_degradation": false,
        "rejected_code": 429,
        "rejected_msg": "Rate limit exceeded"
      }
    },
    "upstream": {
      "type": "roundrobin",
      "nodes": {"reports.internal:8080": 1}
    }
  }'

The example is reviewable configuration, not a production recommendation for the numeric values. The limit-req reference is the source of truth for fields supported by your APISIX release. If Redis crosses an untrusted network, configure encryption and certificate verification according to the documented options and your distribution rather than relying on defaults.

Decide Failure Behavior Deliberately

APISIX documents allow_degradation: false as the default. When it is true, requests can bypass the plugin if the limiting dependency fails. That can preserve availability, but it removes the intended protection precisely when the policy system is impaired.

Use fail-closed behavior for a hard financial, abuse, or contractual boundary when rejecting is safer than uncontrolled execution. Consider fail-open behavior for a low-risk read path when the limiter is advisory and rejecting all legitimate traffic would cause greater harm. A local emergency fallback may be desirable, but do not assume the plugin automatically provides the fallback semantics you designed; verify the implementation for your release.

Alert on limiter errors separately from ordinary 429 responses. A counter-store outage is an infrastructure incident, while a valid rejection is a policy outcome.

Operate the Shared Counter Safely

• Isolate counter traffic from unrelated Redis workloads or define explicit capacity and eviction policy.
• Namespace keys by environment, policy, route, and subject to avoid accidental collisions.
• Protect credentials, enable transport security where required, and restrict network access.
• Watch command latency, timeouts, connection saturation, errors, memory, and hot keys.
• Plan Redis maintenance and failover; exercise the configured request behavior during both.
• Version policy changes and roll them out gradually, especially when changing keys or scope.
• Keep response semantics stable so clients know when to slow down.

Do not add retries around every failed counter operation without a budget. Retrying a degraded Redis service can amplify load and increase gateway latency.

Validate Before Production

The APISIX rate-limiting tutorial demonstrates basic plugin behavior. Extend testing to the topology you will run:

1. Send one identity through one instance and confirm the steady rate and burst behavior.
2. Spread the same identity across all gateway instances and compare local and Redis policies.
3. Add and remove instances while traffic continues.
4. Concentrate traffic on one node to expose load-balancer skew.
5. Stop or delay Redis and verify the chosen fail-open or fail-closed result.
6. Drive many identities plus one hot tenant and observe latency and counter-store saturation.
7. Confirm that rejected requests do not reach the protected upstream.
8. Verify dashboards distinguish 429 policy outcomes from limiter failures.

Decision Checklist

• Is the limit a per-node safety rail or a fleet-wide business invariant?
• Which authenticated or trusted attribute owns the bucket?
• What burst behavior do legitimate clients require?
• How does scaling change the effective local rate?
• Can the shared store meet the latency and availability budget?
• Should this route fail open or fail closed when counters are unavailable?
• Are secrets, TLS, key namespaces, and store capacity managed?
• Have multi-node, skew, failover, and hot-key tests passed?

Summary

Local rate limiting is usually the simplest way to protect each API gateway instance. Redis-backed limiting is appropriate when one budget must follow a subject across the fleet. The price of coordination is a new dependency and a more consequential failure policy. Define subject, scope, burst, response, and failure behavior explicitly, then test the whole topology instead of validating one gateway in isolation.

FAQ

Does Redis make a distributed rate limit exact?

It provides shared state and improves fleet-wide coordination, but observed behavior can still be affected by algorithm semantics, network failures, races, retries, and topology. Treat it as a defined consistency design, not a universal exactness guarantee.

Can I divide a global limit by the number of gateway nodes?

Only as an approximation when traffic distribution is predictable. Autoscaling and load-balancer skew change the effective aggregate and per-client experience.

Should rate limiting fail open?

It depends on the protected operation. Fail closed for hard abuse or cost boundaries when uncontrolled execution is worse; fail open only when availability is the explicit priority and the residual risk is acceptable.

Next Steps

Use API gateway concurrency control to bound in-flight work as well as arrival rate. For a managed control plane and enterprise policy workflow, explore API7 Enterprise.