---
title: "OpenResty FAQ | Dynamic Load, NYI, and Caching of Shared Dict - API7.ai"
description: "This article explains dynamic loading, blocking operations, NYI, dynamic upstream, and caching of shared dictionaries."
url: "https://api7.ai/learning-center/openresty/faq-4"
---

# OpenResty FAQ | Dynamic Load, NYI, and Caching of Shared Dict

API7.ai

January 19, 2023

[OpenResty (NGINX + Lua)](https://api7.ai/learning-center/openresty)

The Openresty article series has been updated so far, and the part about performance optimization is all we have learned. Congratulations to you for not falling behind, still actively learning and practicing, and enthusiastically leaving your thoughts.

We've collected a lot of more typical and interesting questions, and here's a look at five of them.

## Question 1: How do I accomplish dynamic loading of Lua modules?

> **Description:** I have a question about the dynamic loading implemented in OpenResty. How can I use the `loadstring` function to finish loading a new file after it has been replaced? I understand that `loadstring` can only load strings, so if I want to reload a lua file/module, how can I do it in OpenResty?

As we know, `loadstring` is used to load a string, while `loadfile` can load a specified file, for example: `loadfile("foo.lua")`. These two commands achieve the same result. As for how to load Lua modules, here is an example:

```
resty -e 'local s = [[
local ngx = ngx
local _M = {}
function _M.f()
    ngx.say("hello world")
end
return _M
]]
local lua = loadstring(s)
local ret, func = pcall(lua)
func.f()'
```

The content of string `s` is a complete Lua module. So, when you find a change in the code of this module, you can restart the load with `loadstring` or `loadfile`. This way, the functions and variables in it will be updated with it.

To take it a step further, you can also wrap the fetching of changes and reloading with a layer called the `code_loader` function.

```
local func = code_loader(name)
```

This makes code updates much more concise. At the same time, `code_loader` generally uses `lru cache` to cache `s` to avoid calling `loadstring` every time.

## Question 2: Why doesn't OpenResty forbid blocking operations?

> **Description:** Over the years, I've always wondered, since these blocking calls are officially discouraged, why not just disable them? Or add a flag to let the user choose to disable it?

Here's my personal opinion. First, because the ecosystem around OpenResty is not perfect, sometimes we have to call blocking libraries to implement some functions. For example, before version 1.15.8, you had to use the Lua library `os.execute` instead of `lua-resty-shell` to call external commands. For example, in OpenResty, reading and writing files is still only possible with the Lua I/O library, and there is no non-blocking alternative.

Secondly, OpenResty is very cautious about such optimizations. For example, `lua-resty-core` has been developed for a long time, but it has never been turned on by default, requiring you to manually call `require 'resty.core'`. It was turned on until the latest 1.15.8 release.

Finally, the OpenResty maintainers prefer to standardize blocking calls by automatically generating highly optimized Lua code through the compiler and DSL. So, there is no effort to do something like flag options on the OpenResty platform itself. Of course, I am unsure whether this direction can solve the problem.

From an external developer's point, the more practical problem is how to avoid such blocking. We can extend Lua code detection tools, such as `luacheck`, to find and alert on common blocking operations, or we can intrusively disable or rewrite certain functions directly by rewriting `_G`, e.g.:

```
resty -e '_G.ngx.print = function()
ngx.say("hello")
end
ngx.print()'

# hello
```

With this sample code, you can rewrite the `ngx.print` function directly.

## Question 3: Does the operation of LuaJIT's NYI have a significant impact on performance?

> **Description:** `loadstring` shows `never` in LuaJIT's NYI list. Will it have a big impact on performance?

Regarding LuaJIT's NYI, we don't need to be too strict. For operations that can be JIT, the JIT approach is naturally the best; but for operations that cannot be JIT yet, we can continue to use it.

For performance optimization, we need to take a statistically based scientific approach, which is what flame graph sampling is all about. Premature optimization is the root of all evil. We only need to optimize for hot code that makes a lot of calls and consumes a lot of CPU.

Back to `loadstring`, we will only call it to reload when the code changes, not when requested, so it is not a frequent operation. At this point, we don't have to worry about its impact on the overall performance of the system.

In conjunction with the second blocking issue, in OpenResty, we sometimes also invoke blocking file I/O operations during the `init` and `init worker` phases. This operation is more performance-compromising than NYI, but since it is performed only once when the service is started, it is acceptable.

As always, performance optimization should be viewed from a macro perspective, a point you need to pay particular attention to. Otherwise, by obsessing over a particular detail, you will likely optimize for a long time but not have a good effect.

## Question 4: Can I implement dynamic upstream by myself?

> **Description:** For dynamic upstream, my approach is to set up 2 upstreams for a service, select different upstreams according to the routing conditions, and directly modify the IP in the upstream when the machine IP changes. Is there any disadvantage or pitfall in this approach compared to using `balancer_by_lua` directly?

The advantage of `balancer_by_lua` is that it allows the user to choose the load balancing algorithm, for example, whether to use `roundrobin` or `chash`, or any other algorithm that the user implements, which is flexible and high-performance.

If you do it in the way of routing rules, it is the same in terms of the result. But the upstream health check needs to be implemented by you, adding a lot of extra workload.

We can also expand on this question by asking how we should implement this scenario for `abtest`, which requires a different upstream.

You can decide which upstream to use in the `balancer_by_lua` phase based on `uri`, `host`, `parameters`, etc. You can also use API gateways to turn these judgments into routing rules, deciding which route to use in the initial `access` phase, and then finding the specified upstream through the binding relationship between the route and the upstream. This is a common approach to API gateways, and we'll talk more specifically about it later in the hands-on section.

## Question 5: Is caching of `shared dict` mandatory?

> **Description:**
>
> In real production applications, I think the shared dict layer of cache is a must. It seems that everyone only remembers the goodness of `lru cache`, no restrictions on data format, no need to deserialize, no need to calculate memory space based on k/v volume, no contention between workers, no read/write locks and high performance.
>
> However, don't ignore that one of its most fatal weaknesses is that the life cycle of the `lru cache` follows the `Worker`. Whenever NGINX reloads, this part of the cache will be completely lost, and at this point, if there is no `shared dict`, the `L3` data source will be hung in minutes.
>
> Of course, this is the case of higher concurrency, but since caching is used, the business volume is certainly not small, which means that the analysis just mentioned still applies. If I'm right in this view?

In some cases, it is true that, as you said, the `shared dict` is not lost during reload, so it is necessary. But there is a particular case where only the `lru cache` is acceptable if all the data is actively available from `L3`, the data source, in the `init` phase or `init_worker` phase.

For example, if the open source API gateway [APISIX](https://apisix.apache.org/) has its data source in `etcd`, it only fetches data from `etcd`. It caches it in the `lru cache` during the `init_worker` phase, and later cache updates are actively fetched through `etcd`'s `watch` mechanism. This way, even if NGINX reloads, there will be no cache stampede.

So, we can have preferences in choosing technology but don't generalize absolutely because no one silver bullet can fit all caching scenarios. It is an excellent way to build a minimal available solution according to the needs of the actual scenario and then gradually increase it.

Share article link

Share

Related Posts

[![OpenResty FAQ | How OpenResty Is Used in Practice](https://static.api7.ai/uploads/2023/02/12/jbv9M0zi_Hello%20World%20in%20OpenResty%20%283%29.png)](https://api7.ai/learning-center/openresty/faq-5)

## [OpenResty FAQ | How OpenResty Is Used in Practice](https://api7.ai/learning-center/openresty/faq-5)

[![Part 3: How to Build a Microservices API Gateway Using OpenResty](https://static.api7.ai/uploads/2023/02/07/3XJXY9Jy_45.png)](https://api7.ai/learning-center/openresty/how-to-build-a-microservices-api-gateway-part-3)

## [Part 3: How to Build a Microservices API Gateway Using OpenResty](https://api7.ai/learning-center/openresty/how-to-build-a-microservices-api-gateway-part-3)

Table of Contents

- [Question 1: How do I accomplish dynamic loading of Lua modules?](#question-1-how-do-i-accomplish-dynamic-loading-of-lua-modules)
- [Question 2: Why doesn't OpenResty forbid blocking operations?](#question-2-why-doesnt-openresty-forbid-blocking-operations)
- [Question 3: Does the operation of LuaJIT's NYI have a significant impact on performance?](#question-3-does-the-operation-of-luajits-nyi-have-a-significant-impact-on-performance)
- [Question 4: Can I implement dynamic upstream by myself?](#question-4-can-i-implement-dynamic-upstream-by-myself)
- [Question 5: Is caching of `shared dict` mandatory?](#question-5-is-caching-of-shared-dict-mandatory)
