Part 3: How to Build a Microservices API Gateway Using OpenResty

API7.ai

February 3, 2023

OpenResty (NGINX + Lua)

In this article, the construction of the microservices API gateway comes to an end. Let's use a minimal example to put together the previously selected components and run them according to the designed blueprint!

NGINX configuration and initialization

We know that the API gateway is used to handle the traffic entry, so we first need to do a simple configuration in nginx.conf so that all traffic is handled through the Lua code of the gateway.

server {
    listen 9080;

    init_worker_by_lua_block {
        apisix.http_init_worker()
    }

    location / {
        access_by_lua_block {
            apisix.http_access_phase()
        }
        header_filter_by_lua_block {
            apisix.http_header_filter_phase()
        }
        body_filter_by_lua_block {
            apisix.http_body_filter_phase()
        }
        log_by_lua_block {
            apisix.http_log_phase()
        }
    }
}

Here we use the open-source API gateway Apache APISIX as an example, so the above code example has the keyword apisix in it. In this example, we listen to port 9080 and intercept all requests to this port by location /, and process them through access, rewrite, header filter, body filter and log phases, calling the corresponding plugin functions in each phase. The rewrite phase is combined in the apisix.http_access_phase function.

The initialization of the system is handled in the init_worker phase, which includes reading the configuration parameters, presetting the directory in etcd, getting the list of plugins from etcd, and sorting the plugins by priority, etc. I've listed and explained the key parts of the code here, and you can see a more complete initialization function on GitHub.

function _M.http_init_worker()
    -- Initialization Routing, Services, and Plugins - the three most important parts
    router.init_worker()
    require("apisix.http.service").init_worker()
    require("apisix.plugins.ext-plugin.init").init_worker()
end

As you can see from this code, the initialization of the router and plugin parts is a bit more complicated, mainly involving reading configuration parameters and making some choices depending on them. Since this involves reading data from etcd, we use ngx.timer to get around the "can't use cosocket in init_worker phase" restriction. If you are interested in this part, we recommend reading the source code to understand it better.

Matching Routes

At the beginning of the access phase, we first need to match the route based on the request carrying uri, host, args, cookies, etc., to the routing rules that have been set up.

router.router_http.match(api_ctx)

The only code exposed to the public is the line above, where the api_ctx stores the uri, host, args, and cookie information of the request. The specific implementation of the match function uses the lua-resty-radixtree we mentioned earlier. If no route is matched, the request doesn't have an upstream corresponding to it, and it will return 404.

local router = require("resty.radixtree")

local match_opts = {}

function _M.match(api_ctx)
    -- Get the parameters of the request from the ctx and use it as a judgment condition for the route
    match_opts.method = api_ctx.var.method
    match_opts.host = api_ctx.var.host
    match_opts.remote_addr = api_ctx.var.remote_addr
    match_opts.vars = api_ctx.var
    -- Call the judgment function of the route
    local ok = uri_router:dispatch(api_ctx.var.uri, match_opts, api_ctx)
    -- If no route is matched, it returns 404
    if not ok then
        core.log.info("not find any matched route")
        return core.response.exit(404)
    end

    return true
end

Loading plugins

Of course, if the route can be hit, it goes to the step of filtering and loading the plugin, which is the core of the API gateway. Let's start with the following code.

local plugins = core.tablepool.fetch("plugins", 32, 0)
-- The list of plugins in etcd and the list of plugins in the local configuration file are intersected
api_ctx.plugins = plugin.filter(route, plugins)

-- Run the functions mounted by the plugin in the rewrite and access phases in sequence
run_plugin("rewrite", plugins, api_ctx)
run_plugin("access", plugins, api_ctx)

In this code, we first request a table of length 32 through the table pool, which is a performance optimization technique we introduced earlier. Then comes the filter function of the plugin. You may wonder why this step is needed. In the init worker phase of the plugin, don't we already get the list of plugins from etcd and sort them?

The filtering here is done in comparison with the local configuration for the following two reasons:

  1. First, a newly developed plugin needs to be canary released. At this time, the new plugin exists in the etcd list but is only in the open state in some of the gateway nodes. So, we need to do an additional intersection operation.
  2. To support debug mode. Which plugins are processed by the client's request? What is the loading order of these plugins? This information will be useful when debugging, so the filter function will also determine whether it is in debug mode, and record this information in the response header.

So at the end of the access phase, we take these filtered plugins and run them one by one in order of priority, as shown in the following code.

local function run_plugin(phase, plugins, api_ctx)
    for i = 1, #plugins, 2 do
        local phase_fun = plugins[i][phase]
        if phase_fun then
            -- The core calling code
            phase_fun(plugins[i + 1], api_ctx)
        end
    end

    return api_ctx
end

When iterating through the plugins, you can see that we do it in intervals of 2. This is because each plugin will have two components: the plugin object and the plugin's configuration parameters. Now, let's look at the core line of code in the above sample code.

phase_fun(plugins[i + 1], api_ctx)

If this line of code is a bit abstract, let's replace it with a concrete limit_count plugin, which will be much clearer.

limit_count_plugin_rewrite_function(conf_of_plugin, api_ctx)

At this point, we're pretty much done with the overall flow of the API gateway. All of this code is in the same file, which contains more than 400 lines of code, but the core of the code is the few dozen lines we described above.

Writing plugins

Now, there is one thing left to do before a full demo can be run up, and that is to write a plugin. Let's take the limit-count plugin as an example. The full implementation is just over 60 lines of code, which you can see by clicking on the link. Here, I'll explain the key lines of code in detail:

First, we'll introduce lua-resty-limit-traffic as the base library for limiting the number of requests.

local limit_count_new = require("resty.limit.count").new

Then, using the json schema in rapidjson to define what the parameters of this plugin are:

local schema = {
    type = "object",
    properties = {
        count = {type = "integer", minimum = 0},
        time_window = {type = "integer", minimum = 0},
        key = {type = "string",
        enum = {"remote_addr", "server_addr"},
        },
        rejected_code = {type = "integer", minimum = 200, maximum = 600},
    },
    additionalProperties = false,
    required = {"count", "time_window", "key", "rejected_code"},
}

These parameters of the plugin correspond to most of the resty.limit.count parameters, which contain the key of the limit, the size of the time window, and the number of requests to be limited. In addition, the plugin adds a parameter: rejected_code, which returns the specified status code when the request is limited.

In the final step, we mount the plugin's handler function to the rewrite phase:

function _M.rewrite(conf, ctx)
    -- Get the limit count object from the cache, if not, use the `create_limit_obj` function to create a new object and cache it
    local lim, err = core.lrucache.plugin_ctx(plugin_name, ctx,  create_limit_obj, conf)

    -- Get the value of the key from `ctx.var` and compose a new key together with the configuration type and configuration version number
    local key = (ctx.var[conf.key] or "") .. ctx.conf_type .. ctx.conf_version

    --  Function to determine if the limit is entered
    local delay, remaining = lim:incoming(key, true)
    if not delay then
        local err = remaining
        -- If the threshold value is exceeded, the specified status code is returned
        if err == "rejected" then
            return conf.rejected_code
        end

        core.log.error("failed to limit req: ", err)
        return 500
    end

    -- If the threshold is not exceeded, it is released and set the corresponding response header
    core.response.set_header("X-RateLimit-Limit", conf.count,
                             "X-RateLimit-Remaining", remaining)
end

There is only one line of logic in the code above that makes the limit determination, the rest is here to do the preparation work and set the response headers. If the threshold is not exceeded, it will continue to run the next plugin according to the priority.

Summary

Finally, I'll leave you with a thought-provoking question. We know that API gateways can handle not only Layer 7 traffic but also Layer 4 traffic. Based on this, can you think of some usage scenarios for it? Welcome to leave your comments and share this article to learn and communicate with more people.

Previous: Part 1: How to Build a Microservices API gateway using OpenResty Part 2: How to Build a Microservices API gateway using OpenResty