wrk vs wrk2: Coordinated Omission in Latency Tests

wrk and wrk2 can report very different latency percentiles for the same server. The difference is usually not a calculation bug. It comes from how the tools generate requests and account for coordinated omission.

Standard wrk uses a closed-loop model: each connection sends another request after the previous response completes. When the server slows down, the load generator also sends fewer requests. That behavior can omit latency that users would have experienced if requests had continued to arrive at the intended rate.

wrk2 adds a target request rate and records latency relative to when each request was scheduled to start. This makes it better suited to fixed-rate tests where you need to understand queueing and tail latency under a defined offered load.

wrk vs wrk2

Neither tool makes a benchmark valid by itself. The result still depends on the workload, request mix, test duration, connection count, generator capacity, network path, and whether the offered rate matches a realistic production scenario.

What Is Coordinated Omission?

Coordinated omission occurs when a measurement system waits for a slow operation to finish before scheduling more work. The measurement process then coordinates with the system under test and skips samples during the slowdown.

Consider a service expected to receive one request every 10 milliseconds. If one request takes one second and the load generator waits for it to finish, many requests that should have arrived during that second are never sent. The recorded sample includes the one-second response, but it omits the waiting time those missing requests would have experienced.

This distinction is especially important for p99 and higher percentiles. Average latency can look acceptable while a short stall creates a large backlog for real users.

Why wrk Adjusts Its Latency Distribution

Current wrk source applies stats_correct before printing latency statistics. It derives an expected interval from the test runtime, completed requests, and connection count, then adds corrected samples for long observations.

That correction acknowledges coordinated omission, but it is not the same as running a workload with an explicit arrival-rate plan. The expected interval is inferred after the run, while the test itself remains connection-driven.

Removing the correction code is therefore not a general solution. It changes what the histogram represents and can make tail latency look better without fixing the workload model.

How wrk2 Changes the Measurement Model

wrk2 asks you to define the total request rate. It then measures latency from the time a request was supposed to be sent to the time the response completes. If the server falls behind, the delay remains visible in the histogram instead of disappearing because the generator slowed down with the server.

The official wrk2 documentation also notes two practical constraints:

• allow enough time for its calibration period; very short runs are not useful;
• recorded latency has roughly one-millisecond granularity because of operating-system sleep behavior.

These limits should be included in any interpretation of the result.

Compare the Two Tools Safely

First run a conventional wrk test:

wrk -t4 -c128 -d30s --latency http://127.0.0.1:9080/

Then run wrk2 at an explicit rate that the load generator can sustain:

wrk -t4 -c128 -d30s -R2000 --latency http://127.0.0.1:9080/

The wrk2 executable is also named wrk, so verify which binary is installed before interpreting -R. Standard wrk does not provide that option.

Do not compare the two outputs as though the tests offered identical traffic. The first test sends as quickly as its connections and the server allow. The second asks for 2,000 requests per second. A useful comparison requires you to record:

1. the offered and achieved request rates;
2. connection and socket errors;
3. non-success response counts;
4. p50, p90, p99, p99.9, and maximum latency;
5. CPU, memory, network, and connection pressure on both the server and load generator.

A Practical HTTP Benchmark Checklist

Define the Question

Use a maximum-throughput test when the question is, "How much traffic can this setup complete?" Use a fixed-rate test when the question is, "What latency should users expect at this arrival rate?"

Do not use one result to answer both questions.

Keep the Generator Out of the Bottleneck

Run the load generator on a separate host when possible. Confirm that it has enough CPU, available ports, file descriptors, and network capacity. A saturated generator can create misleading latency and throughput results.

Model Real Requests

Match production request sizes, methods, headers, upstream behavior, keep-alive settings, and authentication work. A single cached GET / request does not represent a mixed API workload.

Use a Warm-Up and a Long Enough Measurement Window

Allow connections, caches, JIT compilation, and autoscaling behavior to stabilize before collecting the result. Run long enough to observe infrequent stalls and meaningful high-percentile samples.

Treat Errors as Part of the Result

Low latency is not a success if requests are timing out, being rejected, or returning error responses. Report latency together with achieved throughput and the complete error rate.

Repeat the Test

Run the same scenario multiple times and explain material variance. A single result can be affected by background work, noisy neighbors, connection establishment, or temporary resource pressure.

Which Tool Should You Use?

Use wrk for a quick closed-loop benchmark or when you intentionally want to measure the maximum rate reached by a fixed number of connections. Use wrk2 when the offered request rate is part of the test definition and tail latency during overload matters.

For API gateway tests, record the exact gateway configuration, plugin set, upstream behavior, TLS mode, logging settings, and hardware alongside the load-generator parameters. That context matters more than publishing a single requests-per-second number.

Further Reading

• wrk source and usage
• wrk2 and its coordinated-omission measurement model
• Understanding and managing API gateway latency
• API7 API Gateway performance benchmark methodology