The core of OpenResty: cosocket
Today we will learn about the core technology in OpenResty: cosocket.
We've mentioned it many times in the previous articles, cosocket is the basis of various
lua-resty-* non-blocking libraries. Without cosocket, developers can't use Lua to connect to external web services quickly.
In earlier versions of OpenResty, if you wanted to interact with services like Redis and memcached, you needed to use
memc-nginx-module C-modules. These modules are still available in the OpenResty distribution.
However, with the addition of the cosocket feature, the C modules have have been replaced by
lua-resty-memcached. No one uses C modules to connect to external services anymore.
What is cosocket?
So what exactly is the cosocket? cosocket is a proper noun in OpenResty. The name cosocket consists of
coroutine + socket.
cosocket requires Lua concurrency feature support and the fundamental event mechanism in NGINX, which combines to enable non-blocking network I/O. cosocket also supports TCP, UDP, and Unix Domain Socket.
The internal implementation looks like the following diagram if we call a cosocket-related function in OpenResty.
I also used this diagram in the previous article on OpenResty principles and basic concepts. As you can see from the diagram, for every network operation triggered by the user's Lua script, Both will have the
resume of the coroutine.
When encountering network I/O, it registers the network event to the NGINX Listener list and transfers control (yield) to NGINX. When an NGINX event reaches the trigger condition, it wakes the coroutine to continue processing (resume).
The above process is the blueprint OpenResty uses to encapsulate the connect, send, receive, etc. operations that make up the cosocket APIs we see today. I'll use the API for handling TCP as an example. The interface for controlling UDP and Unix Domain sockets is the same as that of TCP.
Introduction to the cosocket APIs and commands
The TCP-related cosocket APIs can be divided into the following categories.
- Create objects:
- Set timeout:
- Establish connection:
- Send data:
- Receive data:
- Connection pooling:
- Close the connection:
We should also pay special attention to the contexts in which these APIs can be used.
rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*_
Another point I also want to emphasize is that many unavailable environments exist due to various limitations in the NGINX kernel. For example, the cosocket API is unavailable in
body_filter_by_lua*. It is unavailable in
init_worker_by_lua* for now, but the NGINX kernel does not restrict these two phases, the support for which can be added later.
There are eight NGINX commands starting with
lua_socket_ related to these APIs. Let's have a look briefly.
lua_socket_connect_timeout: connection timeout, default 60 seconds.
lua_socket_send_timeout: send timeout, default 60 seconds.
lua_socket_send_lowat: send threshold (low water), default is 0.
lua_socket_read_timeout: read timeout, default 60 seconds.
lua_socket_buffer_size: buffer size for reading data, default 4k/8k.
lua_socket_pool_size: connection pool size, default 30.
lua_socket_keepalive_timeout: idle time of connection pool cosocket object, default 60 seconds.
lua_socket_log_errors: whether to log cosocket errors when they occur, default is
Here you can also see that some commands have the same functionality as the API, such as setting the timeout and connection pool size. However, if there is a conflict between the two, the API has higher priority than the commands and will override the value set by the order. So, generally speaking, we recommend using the APIs to do the settings, which is also more flexible.
Next, let's look at a concrete example to understand how to use these cosocket APIs. The function of the following code is simple, which sends a TCP request to a website and prints out the returned content:
$ resty -e 'local sock = ngx.socket.tcp() sock:settimeout(1000) -- one second timeout local ok, err = sock:connect("api7.ai", 80) if not ok then ngx.say("failed to connect: ", err) return end local req_data = "GET / HTTP/1.1\r\nHost: api7.ai\r\n\r\n" local bytes, err = sock:send(req_data) if err then ngx.say("failed to send: ", err) return end local data, err, partial = sock:receive() if err then ngx.say("failed to receive: ", err) return end sock:close() ngx.say("response is: ", data)'
Let's analyze this code in detail.
- First, create a TCP cosocket object with the name
- Then, use
settimeout()to set the timeout to 1 second. Note that the timeout here does not differentiate between connecting and receiving; it is a uniform setting.
- Next, use
connect()API to connect to port 80 of the specified website and exit if it fails.
- If the connection is successful, use
send()to send the constructed data and exit if it fails.
- If the data send successfully, use
receive()to receive the data from the website. Here, the default parameter of
*l, which means only the first line of data returns. If the parameter is set to
*a, it receives data until the connection is closed.
- Finally, call
close()to close the socket connection actively.
As you can see, using the cosocket APIs to do network communication is simple in just a few steps. Let's make some adjustments to explore the example deeply.
1. Set the timeout time for each of the three actions: socket connect, send and read.
settimeout() we used to set the timeout time to a single value. To put timeout time separately, you need to use the
settimeouts() function, such as the following.
sock:settimeouts(1000, 2000, 3000)
The parameters of
settimeouts are in milliseconds. This line of code indicates a connection timeout of
1 second, a send timeout of
2 seconds, and a read timeout of
In the OpenResty and lua-resty libraries, most of the parameters of the time-related APIs are in milliseconds. But there are exceptions that you need to pay special attention to when calling them.
2. Receives the contents of the specified size.
As I just said, the
receive() API can receive one line of data or receive data continuously. However, if you only want to receive data of 10K in size, how should you set it up?
receiveany() comes in. It designs to meet this need, so look at the following line of code.
local data, err, partial = sock:receiveany(10240)
This code means that only up to 10K of data will be received.
Of course, another general user requirement for
receive() is to keep fetching data until it encounters the specified string.
receiveuntil() is designed to solve this kind of problem. Instead of returning a string like
receiveany(), it will return an iterator. This way, you can call it in a loop to read the matched data in segments and return nil when the reading is done. Here is an example.
local reader = sock:receiveuntil("\r\n") while true do local data, err, partial = reader(4) if not data then if err then ngx.say("failed to read the data stream: ", err) break end ngx.say("read done") break end ngx.say("read chunk: [", data, "]") end
receiveuntil returns the data before
\r\n and reads four bytes of it at a time through the iterator.
3. Instead of closing the socket directly, put it in the connection pool.
As we know, without connection pooling, a new connection has to be created, causing cosocket objects to create each time a request comes in and frequently destroyed, resulting in unnecessary performance loss.
To avoid this problem, after you finish using a cosocket, you can call
setkeepalive() to put it in the connection pool, like the following.
local ok, err = sock:setkeepalive(2 * 1000, 100) if not ok then ngx.say("failed to set reusable: ", err) end
This code sets the connection idle time to
2 seconds and the connection pool size to
100 so that when the
connect() function is called, the cosocket object will be fetched from the connection pool first.
However, there are two things we need to be aware of when using connection pooling.
- First, you can't put an error connection into the connection pool. Otherwise, the next time you use it, it will fail to send and receive data. It is one of the reasons why we need to determine whether each API call is successful or not.
- Second, we need to figure out the number of connections. Connection pooling is
Worker-level, and each Worker has its connection pool. If you have 10
Workers and the connection pool size is set to
30, that's 300 connections for the backend service.
To summarize, we learned the basic concepts, the related commands, and the APIs of cosocket. A practical example made us familiar with how to use TCP-related APIs. UDP and Unix Domain Socket use is similar to that of TCP. You can easily handle all these questions after understanding what we learned today.
We know the cosocket is relatively easy to use, and we can connect to various external services by using it well.
Finally, we can think about two questions.
The first question, in today's example,
tcpsock:send sends a string; what if we need to send a table composed of string?
The second question, as you can see, cosocket can't be used in many stages, so can you think of some ways to bypass it?
Please feel free to leave a comment and share it with me. Welcome to share this article with your colleagues and friends so we can communicate and progress together.