API7 Cloud Integrates with Kubernetes Service Discovery

Cost-saving, adaptability, scalability, reliability — the burst of cloud technologies and containerization has brought many advantages over the traditional monolithic architecture. These advantages make it favored by many enterprise users.

What comes along is that the change in architecture also brings new challenges. For example, traditionally, the upstream forwarding IP list for reverse proxying is often unchanged and can be directly written in the static configuration file. In the cloud and containerization era, the IP list of each upstream service is constantly changing. The maintenance of the configuration files is a nightmare if you continue to use static configuration files. Because the timeliness of configuration file updates will directly affect traffic forwarding, users have to introduce additional software to handle the automatic update of configuration files, significantly increasing the architecture's complexity.

To simplify the process, API7 Cloud brings forth a service discovery function, which directly connects to the Kubernetes API Server to obtain real-time Endpoints data, helping users proxy applications deployed in the Kubernetes cluster conveniently.

Service Discovery in API7 Cloud

Preparation

First, we need to deploy an APISIX instance through cloud-cli. Refer to the API7 Cloud documentation: Deploying a gateway instance.

In this example, we will use the Kubernetes deployment guidelines provided in the document to create various resources under the apisix namespace, which we need to create in advance:

kubectl create namespace apisix
cloud-cli deploy kubernetes \
  --name my-apisix \
  --namespace apisix \
  --apisix-image apache/apisix:2.15.0-centos

Create Service Account

Service Account is the core resource for Kubernetes to implement RBAC (Role-based access control). API7 Cloud needs to have the list and watch permissions of the Endpoints resource to obtain the instance list of the upstream Service.

The corresponding yaml configuration is as follows:

# rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: apisix-k8s-sd-watcher
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: apisix-k8s-sd-watcher-binding
subjects:
  - kind: ServiceAccount
    name: apisix-k8s-sd-sa
    namespace: apisix
roleRef:
  kind: ClusterRole
  name: apisix-k8s-sd-watcher
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: apisix-k8s-sd-sa
  namespace: apisix

Create Service Account and obtain a Token:

kubectl apply -f rbac.yaml
SECRET_NAME=`kubectl get sa -n apisix apisix-k8s-sd-sa -ojsonpath='{.secrets[0].name}'`
kubectl get secrets $SECRET_NAME -n apisix -ojsonpath='{.data.token}' | base64 -d

Create Service Registry in API7 Cloud

Go to Control Plane -> Settings -> Service Registry, use the Token obtained in the previous step to create a service registry of type Kubernetes in API7 Cloud (note that the API Server address needs to be filled based on your use case). In this example, APISIX is also deployed in the target Kubernetes cluster, so the API server address we use is kubernetes.default.svc.cluster.local:

Deploy Services in Kubernetes

In order to verify the effectiveness of Kubernetes service discovery, we deployed three replicas of the HTTPBin service and created a corresponding Service.

kubectl create deployment httpbin --image=kennethreitz/httpbin:latest --replicas=3 -n apisix
kubectl create service clusterip httpbin --tcp=80:80 -n apisix

Create Application and API in API7 Cloud

After deploying the test application, go to the API Management -> Applications page of API7 Cloud and click the Create Application button to create a new Application. The basic configuration is as follows:

• sd.httpbin.org as Host
• / as Path prefix

The critical part is to configure the upstream to use Kubernetes and the registry we just created for service discovery instead of a static list of IPs:

Next, we created a /anything API in the Application for testing the proxy:

Testing the Proxy

First map the data plane HTTP port to the local machine through port-forward:

kubectl port-forward -n apisix svc/apisix-gateway 9080:80

Use curl to request the target service:

curl http://127.0.0.1:9080/anything -H 'Host: sd.httpbin.org' -i
HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 373
Connection: keep-alive
Date: Mon, 09 Jan 2023 08:33:44 GMT
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Server: APISIX




{
  "args": {},
  "data": "",
  "files": {},
  "form": {},
  "headers": {
    "Accept": "*/*",
    "Host": "sd.httpbin.org",
    "User-Agent": "curl/7.86.0",
    "X-Api-Key": "edd1c9f034335f136f87ad84b625c8f1",
    "X-Forwarded-Host": "sd.httpbin.org"
  },
  "json": null,
  "method": "GET",
  "origin": "10.0.20.6",
  "url": "http://sd.httpbin.org/anything"
}

You can see that the request has been proxied to the upstream normally. We can check the access.log after multiple requests:

10.0.20.6 - - [09/Jan/2023:08:32:13 +0000] sd.httpbin.org "GET /anything HTTP/1.1" 200 373 0.345 "-" "curl/7.86.0" 172.30.5.172:80 200 0.344 "http://sd.httpbin.org"
10.0.20.6 - - [09/Jan/2023:08:33:09 +0000] sd.httpbin.org "GET /anything HTTP/1.1" 200 373 0.018 "-" "curl/7.86.0" 172.30.82.159:80 200 0.018 "http://sd.httpbin.org"
10.0.20.6 - - [09/Jan/2023:08:33:44 +0000] sd.httpbin.org "GET /anything HTTP/1.1" 200 373 0.016 "-" "curl/7.86.0" 172.30.5.87:80 200 0.016 "http://sd.httpbin.org"

You can see that three httpbin pods are requested in sequence.

It is worth noting that in creating the Service Registry, we can find that API7 Cloud simultaneously supports the configuration of multiple Kubernetes clusters. The configurations take effect dynamically without any operations on the data plane.

Implementation of Kubernetes Service Discovery in API7 Cloud

Below we will briefly explain the principles of how API7 Cloud implements the above feature.

Service and Endpoints

Service and Endpoints are the core concepts of the Kubernetes networking model. Service represents the abstraction of a group of Pods that provide the same service. Service uses the label selector to select a target set of Pods and saves the IP addresses of these Pods in the Endpoints object. So the key to the entire service discovery lies in the Endpoints object, which stores the pods' IP addresses.

During the rolling update process or whenever the health status of the Pod changes, the Pod IP list in Endpoints will change accordingly, and Kubernetes will ensure that the IP list provided in the Endpoints is only constituted of available Pods.

Service Discovery VS Using Service IP Directly

Readers who know Kubernetes may notice that Service in Kubernetes contains a ClusterIP field. This ClusterIP is allocated by Kubernetes and can be accessed within the cluster, so why does API7 Cloud need to perform service discovery based on Endpoints objects instead of directly using the Service ClusterIP as the upstream address?

In fact, using the Service IP as the upstream address is also a valid option to proxy traffic, where Kubernetes, instead of APISIX, will handle the proxying to specific Pods. However, this also introduces new problems:

1. Processing path lengthened: Now the traffic not only needs to be processed by APISIX, but also needs to be processed by the Kubernetes Service network component (kube-proxy). An additional layer means greater performance consumption and more riskiness.
2. Less control: Since APISIX does not directly proxy traffic to specific Pods, it will significantly limit many upstream features of the API Gateway, such as load balancing, health checks, observability, etc.

Best Practices

According to Google SRE statistics, 70% of online failures are triggered by changes. Therefore, ensuring a smooth transition when releasing services is essential to ensure service availability.

In Kubernetes, rolling update is the most commonly used application release strategy. The rolling update means that n or a total of n% of Pods are released each time. After these Pods are ready, the corresponding old version instances will be deleted, and the process is repeated until all instances are updated to the new version.

In this process, how the old instance gracefully exits during the rolling update process is critical. Otherwise, it is prone to cause request timeout and other exceptions. There are multiple Hooks in the Pod life cycle of Kubernetes, among which the PreStop Hook is used to achieve the smooth termination of an instance.

As shown in the figure above, the old instance will first enter the Terminating state, and then there will be two actions at the same time:

1. The Pod will execute the PreStop Hook (if any) defined by the Spec. After the Hook is executed, the SIGTERM signal will be sent to the program.
2. The Kubernetes Controller monitors the change of the Pod status and updates the list of healthy instances in the corresponding Endpoints object. After APISIX observes this event, APISIX will update the forwarding list of the corresponding Upstream.

It can be seen that in action 1, PreStop Hook can delay the time of sending the exit signal to the program so that action 2 can have more “reaction” time, ensuring that the traffic has been transferred before the program exits and the program will also have more time to process the "in-transit" requests.

Summary

In summary, the fully dynamic Kubernetes service discovery of API7 Cloud can directly replace components such as the traditional Ingress Controller. There is no need to deploy additional components and no learning cost of CRD. Users can easily publish applications deployed by Kubernetes in API7 Cloud.

At the same time, because API7 Cloud supports two configuration methods: fixed upstream IP list and service discovery, users can complete API management in multiple deployment modes on one gateway console. It will make their transition to containerized architecture very smooth.