OpenResty FAQ | Network Structure for Testing, SSL-related Features, DSL, `ab` Tool

The OpenResty article series has been updated so far, and we're done learning the part about testing. Congratulations on not falling behind, actively learning and practicing operations, and enthusiastically leaving your thoughts behind.

Many of the questions raised in the comments are valuable, and some of the more typical and interesting ones have been specially extracted as today's Q&A.

Let's take a look at these five questions today.

Question 1: How to build the network structure for testing?

Q: Should the client running wrk be placed on a machine on the extra net or a machine on the same LAN as the server? Which of these two is more meaningful for performance testing?

A: For testing web-related services, choosing the right testing tool is only a good start, and how to build the network structure for testing is also an essential part of the follow-up.

Generally speaking, we want to test the performance limits of the service alone, excluding all network interferences. For this purpose, we can have two ways to build the network to do the stress test.

The first method is to deploy the wrk and server applications on the same machine with good performance. For example, we can turn on eight workers in NGINX and divide the remaining CPU resources among wrk, so there is only local network communication, minimizing the impact of the network.

The second method is to build a LAN with a dedicated router and connect the machine where wrk is located to the machine where the server is located.

You are not recommended to test directly in an existing network because most networks have switches and firewalls, which may limit the stress testing of large traffic volumes and cause inaccurate test results.

In addition, I would like to mention a few more words about performance testing tools. First, performance testing tools may have Coordinated Omission issues, which you must pay special attention to when analyzing the tool's latency data.

Coordinated Omission means that when doing performance tests, it is not enough to count the time between sending and receiving a response, which is only the service time, so the statistics will miss a lot of potential problems. Therefore, we also need to include the test request's waiting time, which overall is the response time that users care about. Of course, if your server-side program may be blocking, you just need to consider this issue. Otherwise, you can ignore it.

Question 2: Can test::nginx test SSL-related features?

Q: Is it impossible to test SSL-related features with test::nginx?

A: This is not the case. test::nginx can test SSL-related features; you can refer to ssl.t. This test case file tests the whole process of the SSL certificate. First, as you can see, the test case uses Lua code to read the public and private keys of the local certificate; then, it sets up the certificate through the HTTP API; finally, it uses cosocket to do SSL handshake and access to verify whether the certificate is valid.

In fact, not only the SSL but also the features included in OpenResty can be overridden by using test::nginx.

When you are not sure if a particular feature can be implemented with test::nginx, you can search the test cases of lua-nginx-module and other OpenResty open-source projects first, and you can usually find corresponding examples. After all, test::nginx is very playable and variable, and some unexpected combinations and tricks are waiting to be discovered.

Question 3: What exactly is DSL?

Q: What exactly are DSL and test::nginx?

A: DSL stands for Domain Specific Language. The purpose of DSL is different from the common development languages. It is not to solve the needs of a general domain but that of some domain. The most famous DSL is SQL, the Structured Query Language used in the database domain.

As for test::nginx, it is a DSL created to address the testing needs of NGINX and OpenResty. In fact, OpenResty authors have invented many DSL ideas and brought many new attempts and solutions to the OpenResty community. However, as mentioned in the previous article, DSL is a double-edged sword, and the main criterion to measure the value of DSL is whether it can bring productivity improvement to the end users.

Question 4: test::nginx installation problem

Q: After executing the git clone, do I need to run the following command to install test::nginx?

cd test-nginx
perl Makefile.PL
make
sudo make install

A: In fact, this is not the case. Here you can refer to some open-source projects.

Step 1: install it via the package manager

sudo cpanm --notest Test::Nginx >build.log 2>&1 || (cat build.log && exit 1)

Step 2: git clone the latest test::nginx

git clone https://github.com/openresty/test-nginx.git test-nginx

Step 3: include the test-nginx directory when you use the prove command

prove -Itest-nginx/lib -r t

As I mentioned earlier, the best guides for installing OpenResty and the surrounding projects are in CI, not in the documentation. This may be different from other projects, mainly because OpenResty maintains its own fork or specific versions of some of the surrounding projects and because OpenResty is also strongly dependent on CI. Therefore, you should use and test OpenResty in the same way as it is built in CI to ensure consistency with the official one.

Question 5: Is the ab test tool good or not?

Q: How do I remember Yichun Zhang frequently mentioned ab as the best testing tool in Google Groups?

A: As I mentioned in the article, ab is not a good performance testing tool regarding tool features alone because it cannot generate enough request pressure. However, the server-side program performance is powerful now. We use ab instead of wrk in test::nginx because in TEST_NGINX_BENCHMARK mode, test::nginx uses either ab or weighttp, depending on the HTTP protocol version, as the tool for performance testing.

In addition, I hope you note that Internet technology is changing very fast, and each of us needs to update our knowledge and skills in time. For example, in my opinion, this choice of test::nginx now needs to be updated, and Yichun Zhang may not know the existence of wrk at that time. Of course, perhaps there will be better performance testing tools in time than wrk, and we should naturally have a positive and open mind to learn and choose.

Today, I mainly answer the questions mentioned above. I hope that through communication and Q&A, I can help you turn what you learn into what you get. You are also welcome to forward this article so that we can communicate and progress together.