---
title: "Roadblock in Contributing Code: `test::nginx` - API7.ai"
description: "test::nginx is a DSL used for TDD in OpenResty, and can be used for NGINX, HTTP testing"
url: "https://api7.ai/learning-center/openresty/introduction-to-test-nginx"
---

# Roadblock in Contributing Code: \`test::nginx\`

API7.ai

November 17, 2022

[OpenResty (NGINX + Lua)](https://api7.ai/learning-center/openresty)

Testing is an essential part of software development. The concept of Test Driven Development (TDD) has become so popular that almost every software company has a QA (Quality assurance) team to take care of the testing work.

Testing is the cornerstone of OpenResty's quality and great reputation, but it is also the most neglected part of OpenResty's open-source projects. Many developers use the `lua-nginx-module` every day and occasionally run a flame graph, but how many people will run the test cases? Even many OpenResty-based open-source projects are without test cases. But an open-source project without test cases and continuous integration is not trustworthy.

However, unlike commercial companies, there are no dedicated software testing engineers in most open-source projects, so how do they ensure the quality of their code? The answer is simple: "test automation" and "continuous integration", with the key points being automative and continuous, both of which OpenResty has achieved to the greatest extent.

OpenResty has 70 open source projects, and their unit testing, integration testing, performance testing, mock testing, fuzz testing, and other workloads are challenging to solve purely manually by the community contributors. Therefore, OpenResty invested more in automation testing at the beginning. This may seem to slow down the project in the short term, but it can be said that the investment in this area is very cost-effective in the long run. So when I talk to other engineers about OpenResty's logic and toolset for testing, they are amazed.

Let's talk about OpenResty's testing philosophy.

## Concept

`test::nginx` is the core of the OpenResty testing architecture, which is used by OpenResty itself and the surrounding `lua-resty` libraries to organize and write test sets. It is a testing framework with a very high threshold. The reason is that, unlike common testing frameworks, `test::nginx` is not based on assertions and does not use the Lua language, which requires developers to learn and use `test::nginx` from scratch and reverse their inherent knowledge of testing frameworks.

I know several OpenResty contributors who can submit C and Lua code to OpenResty but feel it is difficult to write test cases using `test::nginx`. They either didn't know how to write them or how to fix them when encountering test failures. Therefore, I call `test::nginx` a roadblock in contributing code.

`test::nginx` combines Perl, data-driven, and DSL (Domain-specific language). For the same test case set, by controlling the parameters and environment variables, you can achieve different effects such as random execution, multiple repetitions, memory leak detection, stress testing, etc.

## Installation and examples

Before we use `test::nginx`, let's learn how to install it.

As for software installation in the OpenResty system, only the official CI installation method is the most timely and effective; other ways of installation always encounter various problems. That's why I recommend you take official methods as a reference, where you can find the installation and use of `test::nginx` as well. There are four steps.

1. First, install Perl's package manager `cpanminus`.
2. Then, install `test::nginx` via `cpanm`.

```
sudo cpanm --notest Test::Nginx IPC::Run > build.log 2>&1 || (cat build.log && exit 1)
```

3. Next, clone the latest source code.

```
git clone https://github.com/openresty/test-nginx.git
```

4. Finally, load the `test-nginx` library via Perl's `prove` command and run the set of test cases in the `/t` directory.

```
prove -Itest-nginx/lib -r t
```

After the installation, let's look at the simplest test case in `test::nginx`. The following code is adapted from the official documentation, and I have removed all the customized control parameters.

```
use Test::Nginx::Socket 'no_plan';

run_tests();

__DATA__

=== TEST 1: set Server
--- config
    location /foo {
        echo hi;
        more_set_headers 'Server: Foo';
    }
--- request
    GET /foo
--- response_headers
Server: Foo
--- response_body
hi
```

Although `test::nginx` is written in Perl and works as one of the modules, can you see anything in Perl or any other language from the above test? That's right. It's because \`test::nginx is the author's own Perl implementation of DSL, abstracted specifically for testing NGINX and OpenResty.

So, when we first see this kind of test, we most likely don't understand. But don't worry; let's analyze the above test case.

First of all, `use Test::Nginx::Socket;`, which is the way of Perl references libraries, just like the `require` in Lua. This also reminds us that `test::nginx` is a Perl program.

The second line, `run_tests();` is a Perl function in `test::nginx`, the entry function for the testing framework. If you want to call any other Perl functions in `test::nginx`, they must be placed before `run_tests` to be valid.

The `__DATA__` in the third line is a flag indicating that everything below it is test data, and Perl functions should be completed before this flag.

The next `=== TEST 1: set Server`, the title of the test case, indicates the purpose of this test, and it has a tool that automatically arranges the numbering inside.

`--- config` is the NGINX configuration field. In the above case, we used NGINX commands, not Lua, and if you want to add Lua code, you will do so here with a directive like `content_by_lua`.

`--- request` is used to simulate a terminal to send a request, followed by `GET /foo`, which specifies the method and URI of the request.

`--- response_headers`, which is used to detect response headers. The following `Server: Foo` indicates the `header` and `value` that must appear in the response headers. If not, the test will fail.

The last one `--- response_body`, is used to detect the corresponding body. The following `hi` is the string that must appear in the response body; if it does not, the test will fail.

Well, here, the simplest test case is finished analyzing. So, understanding the test case is a prerequisite for completing OpenResty-related development work.

## Write your test cases

Next, it's time to get into the hands-on testing. Remember how we tested Memcached server in the last article? That's right; we used `resty` to send the request manually, which is represented by the following code.

```
resty -e 'local memcached = require "resty.memcached"
    local memc, err = memcached:new()

    memc:set_timeout(1000) -- 1 sec
    local ok, err = memc:connect("127.0.0.1", 11212)
    local ok, err = memc:set("dog", 32)
    if not ok then
        ngx.say("failed to set dog: ", err)
        return
    end

    local res, flags, err = memc:get("dog")
    ngx.say("dog: ", res)'
```

But isn't sending it manually not smart enough? No worries. We can try to turn manual tests into automated ones after learning `test::nginx`. For example:

```
use Test::Nginx::Socket::Lua::Stream;

run_tests();

__DATA__
  
=== TEST 1: basic get and set
--- config
        location /test {
            content_by_lua_block {
                local memcached = require "resty.memcached"
                local memc, err = memcached:new()
                if not memc then
                    ngx.say("failed to instantiate memc: ", err)
                    return
                end

                memc:set_timeout(1000) -- 1 sec
                local ok, err = memc:connect("127.0.0.1", 11212)

                local ok, err = memc:set("dog", 32)
                if not ok then
                    ngx.say("failed to set dog: ", err)
                    return
                end

                local res, flags, err = memc:get("dog")
                ngx.say("dog: ", res)
            }
        }

--- stream_config
    lua_shared_dict memcached 100m;

--- stream_server_config
    listen 11212;
    content_by_lua_block {
        local m = require("memcached-server")
        m.go()
    }

--- request
GET /test
--- response_body
dog: 32
--- no_error_log
[error]
```

In this test case, I have added `--- stream_config`, `--- stream_server_config`, `--- no_error_log` as configuration items, but they are essentially the same, i.e.

**The data and testing of the tests are stripped down to make readability and extensibility better by abstracting the configuration.**

This is where `test::nginx` is fundamentally different from other testing frameworks. This DSL is a double-edged sword in that it makes the test logic clear and easily extensible. However, it raises the learning cost, requiring you to relearn new syntax and configuration before you can start writing test cases.

## Summary

The `test::nginx` is powerful, but many times it may not always be suitable for your scenario. Why break a butterfly on the wheel? In OpenResty, you also have the option to use the assertion-style testing framework `busted`. The `busted` combined with `resty` becomes a command line tool, and can also meet many testing needs.

Finally, I'll leave you with a question. Can you run this test for `Memcached` locally? If you can add a new test case, that would be great.

Share article link

Share

Related Posts

[![OpenResty FAQ | Network Structure for Testing, SSL-related Features, DSL, \`ab\` Tool](https://static.api7.ai/2022/12/02/63896c177fde1.png)](https://api7.ai/learning-center/openresty/faq-3)

## [OpenResty FAQ | Network Structure for Testing, SSL-related Features, DSL, \`ab\` Tool](https://api7.ai/learning-center/openresty/faq-3)

[![Little-Known Usage of \`test::nginx\`](https://static.api7.ai/2022/11/25/63802a7fb4697.png)](https://api7.ai/learning-center/openresty/little-known-usage-of-test-nginx)

## [Little-Known Usage of \`test::nginx\`](https://api7.ai/learning-center/openresty/little-known-usage-of-test-nginx)

Table of Contents

- [Concept](#concept)
- [Installation and examples](#installation-and-examples)
- [Write your test cases](#write-your-test-cases)
- [Summary](#summary)
