---
title: "Beyond the Web Server: Privileged Process and Timer Tasks - API7.ai"
description: "Explore OpenResty's advanced features like timer tasks, privileged processes, and non-blocking ngx.pipe for optimized NGINX performance and maintenance."
url: "https://api7.ai/learning-center/openresty/privileged-process-and-timer-tasks"
---

# Beyond the Web Server: Privileged Process and Timer Tasks

API7.ai

November 3, 2022

[OpenResty (NGINX + Lua)](https://api7.ai/learning-center/openresty)

In the previous article, we introduced the OpenResty APIs, `shared dict` , and `cosocket`, all of which implement functionality within the realm of NGINX and web servers, providing a programmable Web server that is lower-cost, easier-to-maintain implementation.

However, Openresty can do more than that. Let's pick a few features in OpenResty that go beyond the Web server and introduce them today. They are timer tasks, privileged process, and non-blocking `ngx.pipe`.

## Timer Tasks

In OpenResty, we sometimes need to regularly perform specific tasks in the background, such as synchronizing data, cleaning up logs, etc. If you were to design it, how would you do it? The easiest way to think of is to provide an API interface to the outside world to perform these tasks, then use the system's `crontab` to call `curl` at regular intervals to access this interface, and then implement this requirement in an indirect way.

However, this would not only be fragmented but also bring higher complexity to the operation and maintenance. So, OpenResty provides `ngx.timer` to solve this kind of requirement. You can take `ngx.timer` as a client request simulated by OpenResty to trigger the corresponding callback function.

OpenResty's timer tasks can be divided into the following two types.

- `ngx.timer.at` is used to execute one-time timer tasks.
- `ngx.timer.every` is used to execute fixed-period timer tasks.

Remember the thought-provoking question I left at the end of the last article? The question was how to break the restriction that `cosocket` cannot be used in `init_worker_by_lua`, and the answer is `ngx.timer`.

The following code starts a timer task with a delay of `0`. It starts the callback `handler` function, and in this function, it uses `cosocket` to access a website.

```
init_worker_by_lua_block {
    local function handler()
        local sock = ngx.socket.tcp()
        local ok, err = sock:connect(“api7.ai", 80)
    end

    local ok, err = ngx.timer.at(0, handler)
}
```

This way, we get around the restriction that `cosocket` cannot be used at this stage.

Going back to the user requirement we mentioned at the beginning of this section, `ngx.timer.at` does not address the need to run periodically; in the code example above, it is a one-time task.

So, how do we do this periodically? You seem to have two options based on the `ngx.timer.at` API.

- You can implement the periodic task yourself by using a `while true` infinite loop in the callback function that `sleep`s for a while after executing the task.
- You can also create another new timer at the end of the callback function.

However, before making a choice, there is one thing we need to clarify: the timer is essentially a request, although the request was not initiated by the client. For the request, it has to exit after completing its task and can not always be resident. Otherwise, it is easy to cause a variety of resource leakage.

Therefore, the first solution of using `while true` to implement periodic tasks is unreliable. The second solution is feasible but recursively creates timers, which is not easy to understand.

So, is there a better solution? The new `ngx.timer.every` API behind OpenResty is specifically designed to solve this problem, and it is a solution closer to `crontab`.

The downside is that you never have a chance to cancel a timer task after starting it. After all, `ngx.timer.cancel` is still a to-do function.

At this point, you will face a problem: the timer is running in the background and cannot be canceled; if there are many timers, it is easy to run out of system resources.

Therefore, OpenResty provides two directives, `lua_max_pending_timers` and `lua_max_running_timers` to limit them. The former represents the maximum number of timers waiting to be executed, and the latter represents the maximum number of currently running timers.

You can also use the Lua API to get the values of the currently waiting and running timer tasks, as shown in the following two examples.

```
content_by_lua_block {
    ngx.timer.at(3, function() end)
    ngx.say(ngx.timer.pending_count())
}
```

This code will print a `1`, indicating that there is one scheduled task waiting to be executed.

```
content_by_lua_block {
    ngx.timer.at(0.1, function() ngx.sleep(0.3) end)
    ngx.sleep(0.2)
    ngx.say(ngx.timer.running_count())
}
```

This code will print a `1`, indicating that there is one scheduled task running.

## Privileged Process

Next, let's look at the privileged process. As we all know, NGINX is divided into `Master` process and `Worker` processes, where the worker processes handle user requests. We can get the type of the process through the `process.type` API provided in `lua-resty-core`. For example, you can use `resty` to run the following function.

```
$ resty -e 'local process = require "ngx.process"
ngx.say("process type:", process.type())'
```

You will see that it returns a `single` result instead of `worker`, which means that `resty` starts NGINX with a `Worker` process, not a `Master` process. This is true. In the `resty` implementation, you can see that the `Master` process is turned off with a line like this.

```
master_process off;
```

OpenResty extends NGINX by adding a `privileged agent`, The privileged process has the following special features.

- It does not monitor any ports, which means it does not provide services to the outside world.
- It has the same privileges as the `Master` process, which is generally the `root` user's privilege, allowing it to do many tasks that are impossible for the `Worker` process.
- The privileged process can only be opened in the `init_by_lua` context.
- Also, the privileged process only makes sense if they run in the `init_worker_by_lua` context because no requests are triggered, and they don't go to the `content`, `access`, etc. contexts.

Let's look at an example of a privileged process that is turned on.

```
init_by_lua_block {
    local process = require "ngx.process"

    local ok, err = process.enable_privileged_agent()
    if not ok then
        ngx.log(ngx.ERR, "enables privileged agent failed error:", err)
    end
}
```

After opening the privileged process with this code and starting the OpenResty service, we can see that the privileged process is now part of the NGINX process.

```
nginx: master process
nginx: worker process
nginx: privileged agent process
```

However, if privileges are only run once during the `init_worker_by_lua` phase, which is not a good idea, how should we trigger the privileged process?

Yes, the answer is hidden in the knowledge just taught. Since it doesn't listen to ports, i.e., it can't be triggered by terminal requests, the only way to trigger it periodically is to use the `ngx.timer` we just introduced:

```
init_worker_by_lua_block {
    local process = require "ngx.process"

    local function reload(premature)
        local f, err = io.open(ngx.config.prefix() .. "/logs/nginx.pid", "r")
        if not f then
            return
        end
        local pid = f:read()
        f:close()
        os.execute("kill -HUP " .. pid)
    end

    if process.type() == "privileged agent" then
         local ok, err = ngx.timer.every(5, reload)
        if not ok then
            ngx.log(ngx.ERR, err)
        end
    end
}
```

The code above implements the ability to send `HUP` semaphores to the master process every 5 seconds. Naturally, you can build on this to do more exciting things, such as polling the database to see if there are tasks for the privileged process and executing them. Since the privileged process has `root` privileges, this is obviously a bit of a "backdoor" program.

## Non-blocking `ngx.pipe`

Finally, look at the non-blocking `ngx.pipe`, which uses Lua's standard library to execute an external command line that sends a signal to the `Master` process in the code example we just described.

```
os.execute("kill -HUP " .. pid)
```

Naturally, this operation will block. So, is there a non-blocking way to call external programs in OpenResty? After all, you know that if you are using OpenResty as a complete development platform and not as a web server, this is what you need. For this reason, the `lua-resty-shell` library was created, and using it to invoke the command line is non-blocking:

```
$ resty -e 'local shell = require "resty.shell"
local ok, stdout, stderr, reason, status =
    shell.run([[echo "hello, world"]])
    ngx.say(stdout)
```

This code is a different way of writing `hello world`, calling the system's `echo` command to complete the output. Similarly, you can use `resty.shell` as an alternative to the `os.execute` call in Lua.

We know that the underlying implementation of `lua-resty-shell` relies on the `ngx.pipe` API in `lua-resty-core`, so this example uses `lua-resty-shell` to print out `hello world`, using `ngx.pipe` instead, would look like this.

```
$ resty -e 'local ngx_pipe = require "ngx.pipe"
local proc = ngx_pipe.spawn({"echo", "hello world"})
local data, err = proc:stdout_read_line()
ngx.say(data)'
```

The above is the underlying code of the `lua-resty-shell` implementation. You can check out the `ngx.pipe` documentation and test cases for more information on how to use it. Therefore, I won't go into it here.

## Summary

That's it. We've done with the main content for today. From the above features, we can see that OpenResty is also trying to move closer to the direction of a universal platform while making a better NGINX, hoping that developers can try to unify the technology stack and use OpenResty to solve their development needs. This is quite friendly to operations and maintenance because maintenance costs are lower as long as you deploy an OpenResty on it.

Finally, I'll leave you with a thought-provoking question. Since there may be multiple NGINX `Worker`s, the `timer` will run once for each `Worker`, which is unacceptable in most scenarios. How can we ensure that the `timer` runs only once?

Feel free to leave a comment with your solution, and feel free to share this article with your colleagues and friends so that we can communicate and improve together.

Share article link

Share

Related Posts

[![OpenResty FAQ | How OpenResty Is Used in Practice](https://static.api7.ai/uploads/2023/02/12/jbv9M0zi_Hello%20World%20in%20OpenResty%20%283%29.png)](https://api7.ai/learning-center/openresty/faq-5)

## [OpenResty FAQ | How OpenResty Is Used in Practice](https://api7.ai/learning-center/openresty/faq-5)

[![Part 3: How to Build a Microservices API Gateway Using OpenResty](https://static.api7.ai/uploads/2023/02/07/3XJXY9Jy_45.png)](https://api7.ai/learning-center/openresty/how-to-build-a-microservices-api-gateway-part-3)

## [Part 3: How to Build a Microservices API Gateway Using OpenResty](https://api7.ai/learning-center/openresty/how-to-build-a-microservices-api-gateway-part-3)

Table of Contents

- [Timer Tasks](#timer-tasks)
- [Privileged Process](#privileged-process)
- [Non-blocking `ngx.pipe`](#non-blocking-ngxpipe)
- [Summary](#summary)