--- title: "Non-blocking I/O - The key to improving OpenResty performance - API7.ai" description: "Avoid using blocking I/O operations in OpenResty as they affect performance. Some often misused operations are: executing external commands, handling disk I/O and using lusockets" url: "https://api7.ai/learning-center/openresty/blocking-io" --- # Non-blocking I/O - The key to improving OpenResty performance API7.ai December 2, 2022 [OpenResty (NGINX + Lua)](https://api7.ai/learning-center/openresty) In the Performance Optimization chapter, I'll take you through all aspects of performance optimization in OpenResty and summarize the bits and pieces mentioned in the previous chapters into a comprehensive OpenResty coding guide so you can write better quality OpenResty code. Improving performance is not easy. We must consider system architecture optimization, database optimization, code optimization, performance testing, flame graph analysis, and other steps. But it is easy to reduce performance, and as the title of today's article suggests, you can reduce performance by a factor of 10 or more by adding just a few lines of code. If you're using OpenResty to write your code, but the performance has not been improved, then it is probably because of blocking I/O. So, before we get into the specifics of performance optimization, let's look at an important principle in OpenResty programming: **Non-blocking I/O first.** We have been taught by our parents and teachers since childhood not to play with fire and not to touch the plug, which are dangerous behaviors. The same kind of dangerous behavior exists in OpenResty. If you have to block I/O operations in your code, it will cause a dramatic drop in performance, and the original purpose of using OpenResty to build a high-performance server will be defeated. ## Why can't we use blocking I/O operations? Understanding which behaviors are dangerous and avoiding them is the first step in performance optimization. Let's start by reviewing why blocking I/O operations can affect OpenResty's performance. OpenResty can maintain high performance simply because it borrows the event handling of NGINX and the coroutine of Lua, so: - When you encounter an operation such as network I/O that requires you to wait for a return before continuing, you call the Lua coroutine `yield` to hang yourself and then register a callback in NGINX. - After the I/O operation completes (or a timeout or error occurs), NGINX calls `resume` to wake up the Lua coroutine. Such a process ensures that OpenResty can always use CPU resources efficiently to process all requests. In this processing flow, LuaJIT does not give control to NGINX's event loop if it does not use a non-blocking I/O method like `cosocket`, but instead uses a blocking I/O function to handle I/O. This results in other requests waiting in line for the blocking I/O event to finish processing before they get a response. To sum up, in OpenResty programming, we should be especially careful about function calls that may block I/O; otherwise, a single line of blocking I/O code can bring down the performance of the whole service. Below, I will introduce a few common problems, some often misused blocking I/O functions; let's also experience how to use the easiest way to "mess up" and quickly make your service performance drop 10 times. ## Execute external commands In many scenarios, developers do not just use OpenResty as a web server but endow it with more business logic. In this case, calling external commands and tools may be necessary to help complete some operations. For example, to kill a process. ``` os.execute("kill -HUP " .. pid) ``` Or for more time-consuming operations such as copying files, using OpenSSL to generate keys, etc. ``` os.execute(" cp test.exe /tmp ") os.execute(" openssl genrsa -des3 -out private.pem 2048 ") ``` On the surface, `os.execute` is a built-in function in Lua, and in the Lua world, it is indeed the way to call external commands. However, it is important to remember that Lua is an embedded programming language and will have different recommended usage in other contexts. In OpenResty's environment, `os.execute` blocks the current request. So, if the execution time of this command is particularly short, then the impact is not very big. Still, if the command, takes hundreds of milliseconds or even seconds to execute, then there will be a sharp drop in performance. We understand the problem, so how should we solve it? Generally speaking, there are two solutions. ### 1. If there is an `FFI` library available, then we give preference to the FFI way to call it For example, if we used the OpenSSL command line to generate the key above, we can change it to use `FFI` to call the OpenSSL C function to bypass it. For killing a process, you can use `lua-resty-signal`, a library that comes with OpenResty, to solve it non-blocking. The code implementation is as follows. Of course, here, `lua-resty-signal` is also solved using `FFI` to call system functions. ``` local resty_signal = require "resty.signal" local pid = 12345 local ok, err = resty_signal.kill(pid, "KILL") ``` In addition, LuaJIT's official website has a [particular page](https://luajit.org/) that introduces various FFI binding libraries in different categories. For example, when dealing with images, encryption, and decryption of CPU-intensive operations, you can go there first to see if there are libraries that have been encapsulated and can be used directly. ### 2. Use the `lua-resty-shell` library based on `ngx.pipe` As previously described, you can run your commands in `shell.run`, a non-blocking I/O operation. ``` $ resty -e 'local shell = require "resty.shell" local ok, stdout, stderr, reason, status = shell.run([[echo "hello, world"]]) ngx.say(stdout) ' ``` ## Disk I/O Let's look at the scenario of handling disk I/O. In a server-side application, it is a common operation to read a local configuration file, such as the following code. ``` local path = "/conf/apisix.conf" local file = io.open(path, "rb") local content = file:read("*a") file:close() ``` This code uses `io.open` to get a certain file's contents. However, although it is a blocking I/O operation, don't forget that things must be considered in a real-world scenario. So if you call it `init` and `init worker`, it's a one-time action that doesn't affect any client requests and is perfectly acceptable. Of course, it becomes unacceptable if every user request triggers a read or write to the disk. At that point, you need to seriously consider the solution. Firstly, we can use the `lua-io-nginx-module`, a third-party C module. It provides a non-blocking I/O Lua API for OpenResty, but you can't just use it as you like with `cosocket`. Because disk I/O consumption doesn't just disappear for no reason, it's just a different way of doing things. This approach works because the `lua-io-nginx-module` takes advantage of NGINX thread pooling to move disk I/O operations from the main thread to another thread to process them so that the main thread does not get blocked by disk I/O operations. You need to recompile NGINX when using this library since it is a C module. It is used in the same way as Lua's I/O library. ``` local ngx_io = require "ngx.io" local path = "/conf/apisix.conf" local file, err = ngx_io.open(path, "rb") local data, err = file: read("*a") file:close() ``` Secondly, try an architectural tweak. Can we change our way for this type of disk I/O and stop reading and writing to local disks? Let me give you an example so you can learn by analogy. Years ago, I was working on a project that required logging on a local disk for statistical and troubleshooting purposes. At the time, developers used `ngx.log` to write these logs, like the following. ``` ngx.log(ngx.WARN, "info") ``` This line of code calls the Lua API provided by OpenResty, and it looks like there are no problems. The downside, however, is that you can't call it very often. First, `ngx.log` itself is a costly function call; second, even with a buffer, large and frequent disk writes can seriously impact performance. So how do we solve it? Let's go back to the original need - statistics, troubleshooting, and writing logs to the local disk would have been just one of the means to reach the goal. So you can also send logs to a remote logging server to use `cosocket` to do non-blocking network communication; that is, throw the blocking disk I/O to the logging service to avoid blocking the external service. You can use `lua-resty-logger-socket` to do this. ``` local logger = require "resty.logger.socket" if not logger.initted() then local ok, err = logger.init{ host = 'xxx', port = 1234, flush_limit = 1234, drop_limit = 5678, } local msg = "foo" local bytes, err = logger.log(msg) ``` As you should have noticed, both methods above are the same: if blocking I/O is unavoidable, don't block the main worker thread; throw it to other threads or services outside. ## `luasocket` Finally, let's talk about `luasocket`, a Lua built-in library easily used by developers and often confused between `luasocket` and the `cosocket` provided by OpenResty. `luasocket` can also perform network communication functions. Still, it does not have the advantage of non-blocking. As a result, if you use `luasocket`, performance drops dramatically. However, luasocket also has its unique usage scenarios. For example, I don't know if you remember `cosocket` is not available at several phases, and we can usually bypass it by using `ngx.timer`. Also, you can use `luasocket` for `cosocket` functions in one-off phases like `init_by_lua*` and `init_worker_by_lua*`. The more familiar you are with the similarities and differences between OpenResty and Lua, the more interesting solutions like these you will find. In addition, `lua-resty-socket` is a secondary wrapper for an open-source library that makes luasocket and cosocket\` compatible. This content is also worthy of further study. If you are still interested, I have prepared [materials](https://github.com/thibaultcha/lua-resty-socket/) for you to continue learning. ## Summary In general, in OpenResty, recognizing the types of blocking I/O operations and their solutions is the foundation of good performance optimization. So, have you ever encountered similar blocking I/O operations in actual development? How do you find and solve them? Feel free to share your experience with me in the comments, and feel free to share this article. Share article link Share Related Posts [![Advantages and Disadvantages of \`string\` in OpenResty](https://static.api7.ai/2022/12/09/6392eeb028c7f.png)](https://api7.ai/learning-center/openresty/advantages-and-disadvantages-of-string) ## [Advantages and Disadvantages of \`string\` in OpenResty](https://api7.ai/learning-center/openresty/advantages-and-disadvantages-of-string) [![OpenResty FAQ | How OpenResty Is Used in Practice](https://static.api7.ai/uploads/2023/02/12/jbv9M0zi_Hello%20World%20in%20OpenResty%20%283%29.png)](https://api7.ai/learning-center/openresty/faq-5) ## [OpenResty FAQ | How OpenResty Is Used in Practice](https://api7.ai/learning-center/openresty/faq-5) Table of Contents - [Why can't we use blocking I/O operations?](#why-cant-we-use-blocking-io-operations) - [Execute external commands](#execute-external-commands) - [1. If there is an `FFI` library available, then we give preference to the FFI way to call it](#1-if-there-is-an-ffi-library-available-then-we-give-preference-to-the-ffi-way-to-call-it) - [2. Use the `lua-resty-shell` library based on `ngx.pipe`](#2-use-the-lua-resty-shell-library-based-on-ngxpipe) - [Disk I/O](#disk-io) - [`luasocket`](#luasocket) - [Summary](#summary)