OpenResty FAQ | How OpenResty Is Used in Practice
API7.ai
February 10, 2023
By now, we've finished the last section of OpenResty, Microservices API Gateway. Congratulations on not falling behind, actively learning and practicing, and enthusiastically leaving your thoughts behind.
I selected some typical and interesting questions here to share with you. Let's look at these 5 questions for today.
Question 1: How OpenResty is used in practice
Description: The course is almost over, and I can basically understand it, but my own practice is still on the low side (not currently used in my work). because I can't use it in my work. However, this is a very helpful series of articles. Thanks to the author for continuing to share, and I'll introduce it later in my work.
I'd like to talk about introducing OpenResty at work, a topic worth discussing.
OpenResty is based on NGINX and adds the lua-nginx-module C module and numerous lua-resty libraries, so OpenResty is a good alternative for NGINX, which is the cheapest way to start using OpenResty. Of course, there are risks associated with this replacement process, so you need to be aware of the following three points.
First, ensure that the online NGINX version is the same as the main version of OpenResty, such as OpenResty 1.15.8.1, which uses the NGINX 1.15.8. If the current online NGINX version is higher than the latest version of OpenResty, you need to switch to OpenResty cautiously. After all, OpenResty is still slow to upgrade and is six months to a year behind the mainline version of NGINX. If the online NGINX version is the same as or lower than that of OpenResty, then you have the prerequisites to upgrade.
Second, testing. Testing is one of the most important aspects. There is little risk in using OpenResty to replace NGINX, but risks still exist. For example, whether there are custom C modules that need to be compiled, the version of openssl that OpenResty relies on, and whether the patch that OpenResty puts on NGINX will impact the business. You need to replicate some of the business traffic to verify this.
Third, traffic switching. After the basic validation is passed, you still need to verify the canary release of real traffic online. To roll back quickly, we can open a few new servers to deploy OpenResty instead of directly replacing the original NGINX service. If there are no problems, we can either hot upgrade the binary file or gradually remove and replace NGINX from LB to upgrade.
In addition to replacing NGINX, OpenResty has two other easy entry points: WAF and API gateways. Both of these are scenarios with high performance and dynamic requirements and have corresponding open-source projects that can be used out-of-the-box, which I covered in part before.
If we continue to deepen OpenResty at the business level, we need to consider more factors beyond the technology, such as whether it is easy to recruit OpenResty-related engineers, whether OpenResty can be integrated with the company's existing technical systems, etc.
Generally, it's a good idea to start by replacing NGINX and then slowly spread out to use OpenResty.
Question 2: Database encapsulation for OpenResty
Description: According to the previous article, we should use
..(string concatenation operator) as little as possible, especially in the code hot path. But when dealing with database access, I need to build SQL statements dynamically by inserting variables in the statements, which should be a common usage scenario. But for this requirement, I feel that string concatenation is the easiest way, and I can't think of any other simple and high-performance way.
You can first analyze it with SystemTap or other tools we introduced in the previous articles to see if the splicing of SQL statements is the system's bottleneck. If it is not, there is no need to optimize it. After all, premature optimization is the root of all evil.
If the bottleneck is indeed the splicing of SQL statements, then we can use the database prepare statement to do the optimization or an array to do the splicing. But lua-resty-mysql's support for prepare is in TODO status, so we can only use array splicing. This is also a common problem with some lua-resty libraries, which implement most of the functionality and run normally but are not updated in time. In addition to the database prepare statement, lua-resty-redis has no support for cluster.
String splicing and lua-resty libraries are the kind of problems that OpenResty wants to solve completely with DSL - using compiler technology to automatically generate arrays to splice strings, hiding these details from upper-level users; using DSL wirelang to automatically generate various lua-resty network communication libraries, eliminating the need for handwriting.
This sounds wonderful, right? But we must confront a problem: automatically generated code is unfriendly to developers. If you want to learn or modify the generated code, you have to learn compiler technology and a DSL that may not be open-sourced, which makes the barrier to participating in the community higher and higher.
Question 3: OpenResty Web Framework
Description: I want to make a Web project with OpenResty, but it is painful mainly because I can't find a mature framework, and I need to build a lot of wheels. For example, the database operation problem. I didn't find a class library that can dynamically build SQL statements and coherent operations. So I would like to ask the author, could you recommend a good web framework?
In the awesome-resty repository, we can see that there is a special category for a web framework, there are 20 open-source projects, but most of them are stagnant. Among them, Lapis, lor and vanilla are three projects you can try to see which one is more suitable.
Indeed, without a strong web framework to back it up, OpenResty is overwhelmed when dealing with large projects, which is one of the reasons why few people use OpenResty for business systems.
Question 4: How to change the content-length in the response header after modifying the response body?
Description: If I need to modify the content of the response body, I can only make changes in the body filter, but this will cause the body length to be inconsistent with the content-length length. How should I handle it?
In this case, we need to set the content length response header to nil in the header filter phase before the body filter and not return it and stream the output instead.
The following is a sample code:
server {
listen 8080;
location /test {
proxy_pass http://api7.ai;
header_filter_by_lua_block {
ngx.header.content_length = nil
}
body_filter_by_lua_block {
ngx.arg[1] = ngx.arg[1] .. "abc"
}
}
}
As you can see from this code, in the body filter phase, ngx.arg[1] represents the response body. If we add the string abc after it, the response header content length will be inaccurate, so we can disable it in the header filter phase.
Also, this example shows how the various phases of OpenResty work together, which I hope you will notice and think about.
Question 5: Lua package paths in OpenResty
Description: The
lua_package_pathseems to be configured as the search path for Lua dependencies. Forcontent_by_lua_file, I experimented and found that it only searches under prefix based on the relative path to the file provided by the directive, not underlua_package_path. I don't know if my understanding is correct.
lua_package_path is used to load Lua modules, for example, when we call require 'cjson', we will go to the directory specified in lua_package_path and look for the cjson module. In contrast, content_by_lua_file is followed by a file path on disk.
location /test {
content_by_lua_file /path/test.lua;
}
If this is not an absolute path but a relative path.
content_by_lua_file path/test.lua;
Then a splice will be made using the -p directory specified at OpenResty startup to get the absolute path.
Finally, you are welcome to continue to write down your questions in the comment section, and I will continue to answer them. I hope that through communication and Q&A, I can help you turn what you learn into what you get. You are also welcome to forward this article so that we can communicate and improve together.