Traefik vs Kong vs APISIX: A Reproducible API Gateway Benchmark Method

A fair Traefik, Kong, and Apache APISIX benchmark does not start with a requests-per-second chart. It starts with an experimental contract: exact versions, equivalent traffic paths, identical resource limits, a controlled upstream, defined policy profiles, an offered-load model, invalid-run criteria, and raw results that another team can reproduce.

This guide intentionally publishes a method, not a winner. A proxy-only HTTP/1.1 test can measure one narrow data-plane path; it cannot answer which gateway best fits a Kubernetes platform, a plugin-heavy security policy, or a multi-region operating model.

Key Takeaways

Benchmark a decision-relevant workload, not an unnamed “default” configuration.
Keep hardware, placement, protocol, TLS, upstream, connection reuse, and policies equivalent.
Report latency percentiles, errors, offered and achieved load, and resource saturation together.
Use both a capacity search and a fixed-rate test to expose overload behavior.
Publish manifests, commands, raw output, exclusions, and environment data with conclusions.

Write the Benchmark Contract First

Record these fields before installing a gateway:

Dimension	Required disclosure
Products	Image digest, gateway version, edition, build flags, plugin versions
Platform	CPU model, cores, memory, OS, kernel, container runtime, CPU pinning and limits
Topology	Load generator, gateway, upstream, control plane, and network hops
Protocol	HTTP version, TLS version, cipher, keep-alive, connection count, payload sizes
Route	Match rules, rewrite behavior, upstream algorithm, health checks
Policy	Authentication, rate limit, logging, tracing, or explicitly none
Load	Open- or closed-loop model, warm-up, duration, rates, concurrency, repetitions
Validity	Maximum generator CPU, allowed errors, upstream baseline, clock and telemetry checks

RFC 9411 requires benchmark reports to identify the measured layer and defines HTTP/HTTPS transaction measurements including time to first and last byte. Although an API gateway is broader than the network-security devices in that RFC, its disclosure discipline is useful: a throughput number without test topology and latency is not comparable evidence.

Build an Equivalent Testbed

Use separate, stable resources for the load generator and upstream so the gateway is the intended system under test. First measure the direct upstream baseline through the same network. If the upstream saturates before the gateway or the generator cannot sustain the target rate, mark the gateway run invalid.

Give each gateway the same CPU quota, memory, replicas, listener protocol, TLS termination point, upstream, route count, and logging policy. Disable access logs and telemetry only if you disable them for every candidate and label the result as a stripped proxy profile.

Topology choices must be named. APISIX supports traditional, decoupled, and standalone modes. Kong documents traditional, hybrid, DB-less, and hosted topologies. Traefik installation options are described in its official installation guide. Control-plane components may be outside the request path, but they still affect change propagation, memory, recovery, and operational cost; measure those separately rather than hiding them.

Define Three Workload Profiles

1. Proxy-only baseline

one exact path and one upstream;
small fixed response;
keep-alive enabled;
no authentication, rate limit, tracing, or body inspection;
both cleartext and TLS variants if both matter.

This isolates forwarding overhead. It is not a production ranking.

2. Common policy profile

Choose behavior that all candidates can implement equivalently, such as one JWT verification step, one local rate policy, bounded access logging, and one trace sampling rate. Verify policy correctness before measuring it. If implementations use different algorithms or shared stores, report them as different experiments rather than pretending they are equivalent.

3. Production-shaped profile

Use sanitized distributions for route count, methods, payload sizes, response sizes, upstream latency, keep-alive, TLS, and policy mix. Include a small fraction of invalid credentials, rejected requests, upstream errors, and slow responses. Record every simplification from production.

Use Load Models Deliberately

The official wrk README documents threads, connections, duration, timeouts, headers, scripts, and latency output. This command is a smoke test, not a complete benchmark:

wrk --threads 4 \
  --connections 128 \
  --duration 60s \
  --timeout 2s \
  --latency \
  http://gateway.example.test/benchmark

It is a closed-loop workload: each connection generally waits for a response before sending more work. When latency rises, offered load can fall, hiding part of the overload. wrk2 adds constant-throughput operation and discusses coordinated omission. Use an open-loop or constant-rate tool for fixed offered-load steps, and verify that the generator itself is not saturated.

Run both:

Capacity search: increase offered load in steps until the service-level objective or error threshold fails.
Fixed-rate stability: hold several rates below and around the boundary long enough to observe queues, memory, garbage collection, and recovery.

Randomize candidate order when practical, warm every candidate consistently, and repeat runs. Report median results plus run-to-run spread; do not select the best single run.

Measure a Complete Outcome

At minimum, capture:

offered requests per second and successful responses per second;
p50, p90, p95, p99, and maximum latency with the measurement point named;
HTTP status distribution, connection errors, timeouts, and invalid responses;
gateway CPU, resident memory, network bytes, file descriptors, and restarts;
upstream CPU, latency, and errors;
load-generator CPU, ports, sockets, and achieved send rate;
configuration convergence time and behavior during a control-plane outage, as a separate test.

Define the capacity boundary before seeing results. For example: the highest offered rate at which p99 stays below the target, non-policy 5xx and timeouts remain below the error budget, and no component exceeds its sustained resource ceiling. This prevents a gateway from “winning” by returning fast errors.

Add Overload and Recovery Tests

Steady state is only half the decision. Test:

a step from normal load to overload and back;
a slow or unavailable upstream;
connection churn and TLS handshakes;
large headers and bounded request bodies;
a configuration update during traffic;
one gateway replica terminating;
a policy dependency timing out;
recovery time and whether queued work causes a second spike.

Track rejected work separately from failed work. Intentional 429 or load-shed responses may protect the upstream, but they still represent requests not served and must match the configured policy.

Avoid Common Benchmark Errors

Different routes: regex versus prefix matching or different route counts changes the question.
Different TLS paths: terminating TLS in only one gateway invalidates the comparison.
Unpinned images: latest makes later reproduction impossible.
Hidden defaults: worker count, connection pool, access log, dashboard, and telemetry settings matter.
Generator saturation: a flat graph can describe the client, not the gateway.
No correctness check: a fast route that bypasses a plugin is not a valid result.
Average-only reporting: averages hide tail latency and overload collapse.
Marketing-result reuse: a result from other hardware and workload is context, not evidence for yours.

Reproducibility Package

Publish:

benchmark/
  README.md                 # decision, scope, run order, invalid-run rules
  environment.txt           # hardware, OS, kernel, runtime
  manifests/                # pinned gateway and upstream definitions
  config/                   # equivalent routes and policies
  load/                     # scripts and payload fixtures
  raw/                      # unedited tool and telemetry output
  analysis/                 # parsing code and generated tables
  checksums.txt              # artifact integrity

The README should state who ran the test, when, on which commit, and which results were excluded with reasons. Keep credentials out of the package and replace production data with synthetic fixtures.

Decision Checklist

Does the workload represent the intended gateway job?
Are versions, editions, resources, topology, and configuration pinned?
Is policy behavior proven equivalent before load begins?
Is the upstream baseline safely above the tested range?
Are offered load, tail latency, errors, and utilization reported together?
Do repeated runs and overload recovery tell the same story?
Can another engineer reproduce the result from published artifacts?

Summary

A decision-grade gateway benchmark is a controlled experiment, not a race to the highest requests-per-second number. Equalize the path, separate proxy and policy profiles, measure correctness and tails, expose saturation, and publish everything needed to repeat the work. Only then can Traefik, Kong, and APISIX results inform a specific team's choice.

FAQ

Which gateway is fastest?

That question is incomplete without versions, topology, protocol, policies, hardware, and a success criterion. This method is designed to answer the narrower question that matters for your workload.

Is `wrk` enough?

It is useful for HTTP/1.1 smoke and capacity tests, but a complete study may need a fixed-rate generator, protocol-specific tooling, system telemetry, correctness checks, and failure injection.