What Is the Difference Between OpenResty and NGINX?

The advantages of OpenResty are apparent, before learning in detail, let us briefly review the development process of OpenResty, which will help you better understand the following content.

Development process of OpenResty

OpenResty is not built from scratch like other development languages but is based on mature open-source components - NGINX and LuaJIT. OpenResty was born in 2007, but its first version did not choose Lua, but Perl, which has a lot to do with the technical preference of the author.

But Perl's performance is far from meeting the requirements, so in the second version, Perl was replaced by Lua. However, in the official project of OpenResty, Perl still plays an important role. OpenResty ecosystem projects are built with Perl, such as testing framework, Linter, CLI, etc. We will introduce them step by step later.

Because OpenResty's high performance and dynamic advantages are very suitable for CDN business needs, OpenResty soon became the technical standard of CDN. Moreover, through the rich lua-resty-* libraries, OpenResty began to gradually get rid of the shadow of NGINX and form its ecosystem, which is widely used in API gateway, soft WAF, and other fields.

I often say that OpenResty is a widely used technology, but it is not a hot technology, which sounds contradictory. What does it mean?

It is widely used because OpenResty is now the world's fifth most extensive web server.

It is not popular because the proportion of using OpenResty to build business systems is not high. Most users use OpenResty to process ingress traffic, and they do not go deep into the business. So naturally, the use of OpenResty is only a shallow taste, and it is enough to meet the current needs. This is, of course, also related to OpenResty's lack of mature web frameworks and ecosystems like Java and Python.

Having said so much, next, I will focus on introducing a few places where OpenResty, an open source project, is worthy of praise and learning.

Highlights of OpenResty

Detailed documentation and test cases

Yes, documentation and testing are critical indicators of whether an open source project is reliable, even ahead of code quality and performance.

OpenResty's documentation is very detailed, and the author has written every point that needs attention in the documentation. Most of the time, we only need to look carefully at the documentation to solve the problem we encounter without googling or tracking down the source code. For convenience, OpenResty also comes with a command-line tool, restydoc, which is designed to help you view documentation through the shell and avoid interruptions in the coding process.

However, there are only one or two available code snippets in the documentation, and there are no complete and complex examples. Where can I find such examples?

For OpenResty, it is naturally the /t directory, which contains all the test cases. Each test case includes the complete NGINX configuration and Lua code, as well as the test input and expected output data. However, the test framework OpenResty is entirely different from other assertion-style test frameworks, which I will introduce later in a particular chapter.

Synchronous non-blocking

Coroutine is a new feature that many scripting languages added in recent years to improve performance. But they are not perfectly implemented, some are syntactic sugar, and some require explicit keyword declarations.

OpenResty has supported coroutine since day one and implements a synchronous non-blocking programming model. This is important because programmers are also human, and the code should be more aligned with human thinking habits. Explicit callbacks and async keywords interrupt thinking and make debugging difficult.

So what's synchronous non-blocking? Let's talk about synchronous first. This is very simple: execute sequentially according to the code. For example, the following pseudocode:

local res, err  = query-mysql(sql)
local value, err = query-redis(key)

In the same request, if we have to wait for the query result of MySQL to return before continuing to query Redis, that is synchronous; if we don't need to wait for the return of MySQL and we can continue to query Redis, then it is asynchronous. For OpenResty, most of them are synchronous operations. Only APIs related to background timers, such as ngx.timer are asynchronous operations.

As for non-blocking, which is a concept that is easily confused with asynchronous, when we say blocking here, we mean blocking the operating system thread. Let's continue looking at the above example, assuming it takes 1s to query MySQL. If within this 1s, the operating system's resources (CPU) are idle and foolishly waiting for the return, that is blocking; if the CPU takes the opportunity to process other connections request, that is non-blocking. Non-blocking is also the key to realizing high concurrencies such as C10K and C100K.

The concept of synchronous non-blocking is essential. However, in my opinion, this concept should not be understood by analogy because the inappropriate analogy is likely to confuse you even more.

In OpenResty, the pseudocode above can directly achieve synchronous non-blocking without explicit keywords. It is also reflected here again that making it easier for developers to use is one of the concepts of OpenResty.

Dynamics

One massive advantage of OpenResty, which has not been fully exploited, is its dynamics.

Traditional web servers, such as NGINX, require us to modify the configuration file on disk and reload it to take effect if any changes occur because they do not provide APIs to control runtime behavior. Therefore, in microservices requiring frequent changes, NGINX has tried many times, but nothing has been improved. And the sudden emergence of Envoy, with the dynamic control API of xDS, poses a significant threat of dimensionality reduction attacks to NGINX.

Unlike NGINX and Envoy, OpenResty is controlled by the scripting language Lua, and dynamic is Lua's natural advantage. For example, through the Lua API provided in the lua-nginx-module module in OpenResty, we can dynamically control routes, upstreams, SSL certificates, requests, responses, etc. Even further, we can modify the processing logic of the business without restarting OpenResty, not limited to the Lua API provided by OpenResty.

Here's an excellent analogy to help you understand what's been said about dynamics above. Just think of a web server as a car that is speeding on the highway, NGINX needs to stop to change tires and change the paint color; Envoy can change tires and colors while running; and OpenResty, in addition to the former capabilities, can also turn into an SUV without parking.

After mastering this magic ability, OpenResty's circle of competence and imagination has expanded to other fields, such as Serverless and Edge computing.

What should be learned?

After talking about so many critical features of OpenResty, what should we learn? I prefer focusing on the main line rather than grasping the eyebrows and beards so that we can build a knowledge system with a clear context.

You must know that no matter how comprehensive a course is, it is impossible to cover all problems and cannot directly help you solve every bug and exception online.

Going back to the study of OpenResty, in my opinion, if you want to learn OpenResty well, you should understand the following eight key points:

Synchronous non-blocking programming model
Role of different Request/Response phases
Differences between LuaJIT and Lua
OpenResty API and surrounding libraries
Coroutine and cosocket
Unit testing framework and performance testing tools
Flame graph and surrounding toolchain
Performance optimization

Those points are essential in our study, and I will discuss them separately in each chapter. However, in the process of learning, I hope you can draw inferences from one case and read some chapters in depth according to your interests and background.

If you are a beginner with OpenResty, you can follow the course's progress, install OpenResty in your environment, and run and modify the sample code. Remember, your focus is on building the big picture of OpenResty, not sticking to a single point of knowledge. Of course, if you have any questions, feel free to contact us in live chat.

If you are using OpenResty in your project, that's great! However, I believe that when you read the LuaJIT and performance optimization chapters, you will have more resonance and more practical applications and see the performance improvement before and after optimization in your project.

In addition, if you want to contribute code to OpenResty and surrounding libraries, the most significant threshold does not understand the principles of OpenResty or how to write NGINX C modules but test cases and code specifications. I've seen too many OpenResty contributors (myself included) repeatedly modifying test cases and code styles on a PR, and there are too many unspoken rules. So, the course's code specification and unit testing sections are for you.

And if you are a QA engineer, even if you don't use OpenResty, OpenResty's testing framework and performance analysis toolset will give you a lot of inspiration. After all, OpenResty's investment and accumulation in testing are pretty deep.