Non-blocking I/O - The key to improving OpenResty performance

In the Performance Optimization chapter, I'll take you through all aspects of performance optimization in OpenResty and summarize the bits and pieces mentioned in the previous chapters into a comprehensive OpenResty coding guide so you can write better quality OpenResty code.

Improving performance is not easy. We must consider system architecture optimization, database optimization, code optimization, performance testing, flame graph analysis, and other steps. But it is easy to reduce performance, and as the title of today's article suggests, you can reduce performance by a factor of 10 or more by adding just a few lines of code. If you're using OpenResty to write your code, but the performance has not been improved, then it is probably because of blocking I/O.

So, before we get into the specifics of performance optimization, let's look at an important principle in OpenResty programming: Non-blocking I/O first.

We have been taught by our parents and teachers since childhood not to play with fire and not to touch the plug, which are dangerous behaviors. The same kind of dangerous behavior exists in OpenResty. If you have to block I/O operations in your code, it will cause a dramatic drop in performance, and the original purpose of using OpenResty to build a high-performance server will be defeated.

Why can't we use blocking I/O operations?

Understanding which behaviors are dangerous and avoiding them is the first step in performance optimization. Let's start by reviewing why blocking I/O operations can affect OpenResty's performance.

OpenResty can maintain high performance simply because it borrows the event handling of NGINX and the coroutine of Lua, so:

• When you encounter an operation such as network I/O that requires you to wait for a return before continuing, you call the Lua coroutine yield to hang yourself and then register a callback in NGINX.
• After the I/O operation completes (or a timeout or error occurs), NGINX calls resume to wake up the Lua coroutine.

Such a process ensures that OpenResty can always use CPU resources efficiently to process all requests.

In this processing flow, LuaJIT does not give control to NGINX's event loop if it does not use a non-blocking I/O method like cosocket, but instead uses a blocking I/O function to handle I/O. This results in other requests waiting in line for the blocking I/O event to finish processing before they get a response.

To sum up, in OpenResty programming, we should be especially careful about function calls that may block I/O; otherwise, a single line of blocking I/O code can bring down the performance of the whole service.

Below, I will introduce a few common problems, some often misused blocking I/O functions; let's also experience how to use the easiest way to "mess up" and quickly make your service performance drop 10 times.

Execute external commands

In many scenarios, developers do not just use OpenResty as a web server but endow it with more business logic. In this case, calling external commands and tools may be necessary to help complete some operations.

For example, to kill a process.

os.execute("kill -HUP " .. pid)

Or for more time-consuming operations such as copying files, using OpenSSL to generate keys, etc.

os.execute(" cp test.exe /tmp ")

os.execute(" openssl genrsa -des3 -out private.pem 2048 ")

On the surface, os.execute is a built-in function in Lua, and in the Lua world, it is indeed the way to call external commands. However, it is important to remember that Lua is an embedded programming language and will have different recommended usage in other contexts.

In OpenResty's environment, os.execute blocks the current request. So, if the execution time of this command is particularly short, then the impact is not very big. Still, if the command, takes hundreds of milliseconds or even seconds to execute, then there will be a sharp drop in performance.

We understand the problem, so how should we solve it? Generally speaking, there are two solutions.

1. If there is an FFI library available, then we give preference to the FFI way to call it

For example, if we used the OpenSSL command line to generate the key above, we can change it to use FFI to call the OpenSSL C function to bypass it.

For killing a process, you can use lua-resty-signal, a library that comes with OpenResty, to solve it non-blocking. The code implementation is as follows. Of course, here, lua-resty-signal is also solved using FFI to call system functions.

local resty_signal = require "resty.signal"
local pid = 12345
local ok, err = resty_signal.kill(pid, "KILL")

In addition, LuaJIT's official website has a particular page that introduces various FFI binding libraries in different categories. For example, when dealing with images, encryption, and decryption of CPU-intensive operations, you can go there first to see if there are libraries that have been encapsulated and can be used directly.

2. Use the lua-resty-shell library based on ngx.pipe

As previously described, you can run your commands in shell.run, a non-blocking I/O operation.

$ resty -e 'local shell = require "resty.shell"
local ok, stdout, stderr, reason, status =
    shell.run([[echo "hello, world"]])
    ngx.say(stdout) '

Disk I/O

Let's look at the scenario of handling disk I/O. In a server-side application, it is a common operation to read a local configuration file, such as the following code.

local path = "/conf/apisix.conf"
local file = io.open(path, "rb")
local content = file:read("*a")
file:close()

This code uses io.open to get a certain file's contents. However, although it is a blocking I/O operation, don't forget that things must be considered in a real-world scenario. So if you call it init and init worker, it's a one-time action that doesn't affect any client requests and is perfectly acceptable.

Of course, it becomes unacceptable if every user request triggers a read or write to the disk. At that point, you need to seriously consider the solution.

Firstly, we can use the lua-io-nginx-module, a third-party C module. It provides a non-blocking I/O Lua API for OpenResty, but you can't just use it as you like with cosocket. Because disk I/O consumption doesn't just disappear for no reason, it's just a different way of doing things.

This approach works because the lua-io-nginx-module takes advantage of NGINX thread pooling to move disk I/O operations from the main thread to another thread to process them so that the main thread does not get blocked by disk I/O operations.

You need to recompile NGINX when using this library since it is a C module. It is used in the same way as Lua's I/O library.

local ngx_io = require "ngx.io"
local path = "/conf/apisix.conf"
local file, err = ngx_io.open(path, "rb")
local data, err = file: read("*a")
file:close()

Secondly, try an architectural tweak. Can we change our way for this type of disk I/O and stop reading and writing to local disks?

Let me give you an example so you can learn by analogy. Years ago, I was working on a project that required logging on a local disk for statistical and troubleshooting purposes.

At the time, developers used ngx.log to write these logs, like the following.

ngx.log(ngx.WARN, "info")

This line of code calls the Lua API provided by OpenResty, and it looks like there are no problems. The downside, however, is that you can't call it very often. First, ngx.log itself is a costly function call; second, even with a buffer, large and frequent disk writes can seriously impact performance.

So how do we solve it? Let's go back to the original need - statistics, troubleshooting, and writing logs to the local disk would have been just one of the means to reach the goal.

So you can also send logs to a remote logging server to use cosocket to do non-blocking network communication; that is, throw the blocking disk I/O to the logging service to avoid blocking the external service. You can use lua-resty-logger-socket to do this.

local logger = require "resty.logger.socket"
if not logger.initted() then
    local ok, err = logger.init{
        host = 'xxx',
        port = 1234,
        flush_limit = 1234,
        drop_limit = 5678,
    }
local msg = "foo"
local bytes, err = logger.log(msg)

As you should have noticed, both methods above are the same: if blocking I/O is unavoidable, don't block the main worker thread; throw it to other threads or services outside.

luasocket

Finally, let's talk about luasocket, a Lua built-in library easily used by developers and often confused between luasocket and the cosocket provided by OpenResty. luasocket can also perform network communication functions. Still, it does not have the advantage of non-blocking. As a result, if you use luasocket, performance drops dramatically.

However, luasocket also has its unique usage scenarios. For example, I don't know if you remember cosocket is not available at several phases, and we can usually bypass it by using ngx.timer. Also, you can use luasocket for cosocket functions in one-off phases like init_by_lua* and init_worker_by_lua*. The more familiar you are with the similarities and differences between OpenResty and Lua, the more interesting solutions like these you will find.

In addition, lua-resty-socket is a secondary wrapper for an open-source library that makes luasocket and cosocket` compatible. This content is also worthy of further study. If you are still interested, I have prepared materials for you to continue learning.

Summary

In general, in OpenResty, recognizing the types of blocking I/O operations and their solutions is the foundation of good performance optimization. So, have you ever encountered similar blocking I/O operations in actual development? How do you find and solve them? Feel free to share your experience with me in the comments, and feel free to share this article.