The core of OpenResty: cosocket

API7.ai

October 28, 2022

OpenResty (NGINX + Lua)

Today we will learn about the core technology in OpenResty: cosocket.

We've mentioned it many times in the previous articles, cosocket is the basis of various lua-resty-* non-blocking libraries. Without cosocket, developers can't use Lua to connect to external web services quickly.

In earlier versions of OpenResty, if you wanted to interact with services like Redis and memcached, you needed to use redis2-nginx-module, redis-nginx-module and memc-nginx-module C-modules. These modules are still available in the OpenResty distribution.

However, with the addition of the cosocket feature, the C modules have have been replaced by lua-resty-redis and lua-resty-memcached. No one uses C modules to connect to external services anymore.

What is cosocket?

So what exactly is the cosocket? cosocket is a proper noun in OpenResty. The name cosocket consists of coroutine + socket.

cosocket requires Lua concurrency feature support and the fundamental event mechanism in NGINX, which combines to enable non-blocking network I/O. cosocket also supports TCP, UDP, and Unix Domain Socket.

The internal implementation looks like the following diagram if we call a cosocket-related function in OpenResty.

call cosocket-related function

I also used this diagram in the previous article on OpenResty principles and basic concepts. As you can see from the diagram, for every network operation triggered by the user's Lua script, Both will have the yield and resume of the coroutine.

When encountering network I/O, it registers the network event to the NGINX Listener list and transfers control (yield) to NGINX. When an NGINX event reaches the trigger condition, it wakes the coroutine to continue processing (resume).

The above process is the blueprint OpenResty uses to encapsulate the connect, send, receive, etc. operations that make up the cosocket APIs we see today. I'll use the API for handling TCP as an example. The interface for controlling UDP and Unix Domain sockets is the same as that of TCP.

Introduction to the cosocket APIs and commands

The TCP-related cosocket APIs can be divided into the following categories.

  • Create objects: ngx.socket.tcp.
  • Set timeout: tcpsock:settimeout and tcpsock:settimeouts.
  • Establish connection: tcpsock:connect.
  • Send data: tcpsock:send.
  • Receive data: tcpsock:receive, tcpsock:receiveany, and tcpsock:receiveuntil.
  • Connection pooling: tcpsock:setkeepalive.
  • Close the connection: tcpsock:close.

We should also pay special attention to the contexts in which these APIs can be used.

rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*_

Another point I also want to emphasize is that many unavailable environments exist due to various limitations in the NGINX kernel. For example, the cosocket API is unavailable in set_by_lua*, log_by_lua*, header_filter_by_lua*, and body_filter_by_lua*. It is unavailable in init_by_lua* and init_worker_by_lua* for now, but the NGINX kernel does not restrict these two phases, the support for which can be added later.

There are eight NGINX commands starting with lua_socket_ related to these APIs. Let's have a look briefly.

  • lua_socket_connect_timeout: connection timeout, default 60 seconds.
  • lua_socket_send_timeout: send timeout, default 60 seconds.
  • lua_socket_send_lowat: send threshold (low water), default is 0.
  • lua_socket_read_timeout: read timeout, default 60 seconds.
  • lua_socket_buffer_size: buffer size for reading data, default 4k/8k.
  • lua_socket_pool_size: connection pool size, default 30.
  • lua_socket_keepalive_timeout: idle time of connection pool cosocket object, default 60 seconds.
  • lua_socket_log_errors: whether to log cosocket errors when they occur, default is on.

Here you can also see that some commands have the same functionality as the API, such as setting the timeout and connection pool size. However, if there is a conflict between the two, the API has higher priority than the commands and will override the value set by the order. So, generally speaking, we recommend using the APIs to do the settings, which is also more flexible.

Next, let's look at a concrete example to understand how to use these cosocket APIs. The function of the following code is simple, which sends a TCP request to a website and prints out the returned content:

$ resty -e 'local sock = ngx.socket.tcp()
sock:settimeout(1000) -- one second timeout
local ok, err = sock:connect("api7.ai", 80)
if not ok then
    ngx.say("failed to connect: ", err)
    return
end
local req_data = "GET / HTTP/1.1\r\nHost: api7.ai\r\n\r\n"
local bytes, err = sock:send(req_data)
if err then
    ngx.say("failed to send: ", err)
    return
end
local data, err, partial = sock:receive()
if err then
    ngx.say("failed to receive: ", err)
    return
end
sock:close()
ngx.say("response is: ", data)'

Let's analyze this code in detail.

  • First, create a TCP cosocket object with the name sock using ngx.socket.tcp().
  • Then, use settimeout() to set the timeout to 1 second. Note that the timeout here does not differentiate between connecting and receiving; it is a uniform setting.
  • Next, use connect() API to connect to port 80 of the specified website and exit if it fails.
  • If the connection is successful, use send() to send the constructed data and exit if it fails.
  • If the data send successfully, use receive() to receive the data from the website. Here, the default parameter of receive() is *l, which means only the first line of data returns. If the parameter is set to *a, it receives data until the connection is closed.
  • Finally, call close() to close the socket connection actively.

As you can see, using the cosocket APIs to do network communication is simple in just a few steps. Let's make some adjustments to explore the example deeply.

1. Set the timeout time for each of the three actions: socket connect, send and read.

The settimeout() we used to set the timeout time to a single value. To put timeout time separately, you need to use the settimeouts() function, such as the following.

sock:settimeouts(1000, 2000, 3000)

The parameters of settimeouts are in milliseconds. This line of code indicates a connection timeout of 1 second, a send timeout of 2 seconds, and a read timeout of 3 seconds.

In the OpenResty and lua-resty libraries, most of the parameters of the time-related APIs are in milliseconds. But there are exceptions that you need to pay special attention to when calling them.

2. Receives the contents of the specified size.

As I just said, the receive() API can receive one line of data or receive data continuously. However, if you only want to receive data of 10K in size, how should you set it up?

That's where receiveany() comes in. It designs to meet this need, so look at the following line of code.

local data, err, partial = sock:receiveany(10240)

This code means that only up to 10K of data will be received.

Of course, another general user requirement for receive() is to keep fetching data until it encounters the specified string.

The receiveuntil() is designed to solve this kind of problem. Instead of returning a string like receive() and receiveany(), it will return an iterator. This way, you can call it in a loop to read the matched data in segments and return nil when the reading is done. Here is an example.

local reader = sock:receiveuntil("\r\n")
     while true do
         local data, err, partial = reader(4)
         if not data then
             if err then
                 ngx.say("failed to read the data stream: ", err)
                 break
             end
             ngx.say("read done")
             break
         end
         ngx.say("read chunk: [", data, "]")
     end

The receiveuntil returns the data before \r\n and reads four bytes of it at a time through the iterator.

3. Instead of closing the socket directly, put it in the connection pool.

As we know, without connection pooling, a new connection has to be created, causing cosocket objects to create each time a request comes in and frequently destroyed, resulting in unnecessary performance loss.

To avoid this problem, after you finish using a cosocket, you can call setkeepalive() to put it in the connection pool, like the following.

local ok, err = sock:setkeepalive(2 * 1000, 100)
if not ok then
    ngx.say("failed to set reusable: ", err)
end

This code sets the connection idle time to 2 seconds and the connection pool size to 100 so that when the connect() function is called, the cosocket object will be fetched from the connection pool first.

However, there are two things we need to be aware of when using connection pooling.

  • First, you can't put an error connection into the connection pool. Otherwise, the next time you use it, it will fail to send and receive data. It is one of the reasons why we need to determine whether each API call is successful or not.
  • Second, we need to figure out the number of connections. Connection pooling is Worker-level, and each Worker has its connection pool. If you have 10 Workers and the connection pool size is set to 30, that's 300 connections for the backend service.

Summary

To summarize, we learned the basic concepts, the related commands, and the APIs of cosocket. A practical example made us familiar with how to use TCP-related APIs. UDP and Unix Domain Socket use is similar to that of TCP. You can easily handle all these questions after understanding what we learned today.

We know the cosocket is relatively easy to use, and we can connect to various external services by using it well.

Finally, we can think about two questions.

The first question, in today's example, tcpsock:send sends a string; what if we need to send a table composed of string?

The second question, as you can see, cosocket can't be used in many stages, so can you think of some ways to bypass it?

Please feel free to leave a comment and share it with me. Welcome to share this article with your colleagues and friends so we can communicate and progress together.