How to Use Secret Manager with API Gateway

What Is Secret?

In an IT environment, a secret is a non-human privileged credential, usually used by systems and applications for authentication or as input to encryption algorithms. For example, passwords needed to connect a program to a database, keys/certificates for encrypted communication, all belong to secret.

Common types of secret include:

• Encryption keys
• Cloud service access credentials
• Application programming interface (API) keys
• Access tokens
• SSH (secure shell) keys

Secret Manager

The Secret Manager stores, retrieves, rotates, and audits secret throughout its lifecycle. HashiCorp Vault is one of the most commonly-used Secret Manager services, and its working principle is as follows (picture from HashiCorp official website):

In addition to HashiCorp Vault, cloud providers such as AWS, Google, and Azure also provide Secret Manager services.

Using a Secret Manager to build applications not only enhances security but also provides the following functions:

• Allow secrets to be easily updated across multiple machines without affecting the application.
• Developers don't need to pay too much attention to the storage of secrets and can focus more on the business logic itself.
• Secret rotation and revocation can be done without redeploying or interrupting active applications.
• Secret Manager provides flexible and detailed audit logs that can track the history of all user access and more easily meet audit and compliance requirements.

Use Secret Manager in API Gateway

As the entrance of business traffic, the API Gateway often contains a huge amount of secret information, such as API keys and tokens for authentication. Therefore it is vital to integrate a Secret Manager into API Gateway.

Scenarios of Using Secret in API Gateway

Key Auth is a simple authentication method in the API Gateway. Users need to preset a key, and when the client has API requests, the preset key is written to the corresponding request header or accessed as a request parameter to pass authentication. However, key leaks are common, and keys are often stored in the configuration files or saved as variables in code. If keys are not properly stored, they could even appear in the public code repositories such as GitHub, posing a significant threat to security. Administrators often update these keys regularly to prevent serious harm caused by key leaks. However, for some static keys, individually updating the configuration file on each machine can also pose a significant threat to the stability of the service.

Integrate Secret Manager into API Gateway

By integrating Secret Manager into API Gateway, the preset key for Key Auth authentication can be stored in the Secret Manager service, and it could effectively resolve a series of problems caused by key leaks. API Gateway is only a key user, and the key's actual value is stored in the external Secret Manager service. Furthermore, the API Gateway only keeps a reference to the key, so even if the API Gateway is hacked, the actual key cannot be obtained from the code or configuration file. In addition, if you need to update the key's value, you only need to update it in the Secret Manager without modifying the code or configuration of the API Gateway, which also effectively avoids restarting the API Gateway.

We will use Apache APISIX as an example below to show how to use the Secret Manager to store the preset key for Key Auth in the API Gateway.

Example of Using Apache APISIX in Secret Manager in Practice

Apache APISIX is a dynamic, real-time, high-performance API Gateway that provides hundreds of functions, such as load balancing, dynamic upstream, canary release, fine-grained routing, rate limiting, service degradation, circuit breaker, authentication, and observability.

In version 3.1.0, Apache APISIX introduced the APISIX Secret to integrate different Secret Manager services. Its work sequence is as follows:

Below, we use Vault as the Secret Manager service. We will use Key Auth authentication to illustrate how to use a Secret Manager to save keys in Apache APISIX.

Create Keys in Vault

Before creating the key, you need to start the Vault service. Since this section shares the best practices for using Vault in the Apache APISIX ecosystem, the configuration of Vault itself is not described in detail and can be found here.

To create the corresponding key in Vault, you could use the following command:

vault secrets enable -version=1 -path=apisix kv

vault kv put apisix/jack auth-key=secret-key

After executing the above command, you will find a key named auth-key under apisix/jack path, with a value of secret-key.

Configuration in APISIX

First of all, add the secret resource through the Admin API and configure the address as well as other connection information for Vault:

1. The APISIX Secret resource id is "gateway", refer to https://apisix.apache.org/docs/apisix/terminology/secret/#usage-1
2. The prefix field is the path of the key in Vault, and the token field is the Vault token.

curl http://127.0.0.1:9180/apisix/admin/secrets/vault/gateway \
-H 'X-API-KEY: Your-API-KEY' -X PUT -d '
{
  "uri": "https://127.0.0.1:8200",
  "prefix": "apisix",
  "token": "hvs.chjDFWvZRcihoq2vpSsVenmR"
}'

Then, use the key-auth plugin to create a consumer and perform Key Auth authentication for the corresponding route or service. The key field refers to the APISIX Secret resource:

curl http://127.0.0.1:9180/apisix/admin/consumers \
-H 'X-API-KEY: Your-API-KEY' -X PUT -d '
{
  "username": "jack",
  "plugins": {
    "key-auth": {
      "key": "$secret://vault/gateway/jack/auth-key"
    }
  }
}'

Via the above two steps, when the user's request hits a key-auth plugin, the Secret component will call the user-configured Secret Manager through the provided interface to obtain the key's actual value in Vault. If the value of the key is not found, the plugin will record the error and fail to perform the Key Auth validation.

Then create a Route to perform Key Auth authentication:

curl -i "http://127.0.0.1:9180/apisix/admin/routes" -H 'X-API-KEY: Your-API-KEY' -X PUT -d '
{
  "id": "getting-started-ip",
  "uri": "/ip",
  "plugins": {
    "key-auth": {}
  },
  "upstream": {
    "type": "roundrobin",
    "nodes": {
      "httpbin.org:80": 1
    }
  }
}'

Finally, send a request without key to the API Gateway to verify the result:

curl -i "http://127.0.0.1:9080/ip"

The result is as follows:

HTTP/1.1 401 Unauthorized
...

{"message":"Missing API key found in request"}

And then, send a request with the key to the API Gateway to verify the result:

curl -i "http://127.0.0.1:9080/ip" -H 'apikey: secret-key'

The result is similar to the following:

HTTP/1.1 200 OK
...

{
  "origin": "127.0.0.1, 59.172.90.243"
}

Besides that, Apache APISIX has expanded environment variables into a simple Secret Manager service, and the APISIX token for connecting to Vault can also be stored in environment variables. So, before starting APISIX, you can set the environment variables with the following command:

export VAULT_TOKEN="root"

Refer to environment variables while adding Secret resource:

curl http://127.0.0.1:9180/apisix/admin/secrets/vault/1
-H 'X-API-KEY: Your-API-KEY' -X PUT -d '
{
  "uri": "https://127.0.0.1:8200"，
  "prefix": "apisix",
  "token": "$ENV://VAULT_TOKEN"
}'

Through the above steps, you can store the key needed for Key Auth authentication in Vault instead of displaying it in plain text while configuring the plugin.

Conclusion

In API Gateway, there is a huge amount of secret information. Using a Secret Manager to manage secrets can ensure that no plain-text secret information in API Gateway, effectively improving the security and stability of the API Gateway. As the world's most active open-source API Gateway, Apache APISIX also supports secret managers greatly. This article also uses Key Auth authentication to illustrate how to use a Secret Manager to manage key information in APISIX.

For more information about API gateway, please visit our blogs or contact us.