How Does APISIX Ingress Support Custom Plugins
Ingress & Ingress Controller
Ingress is one of the API objects of Kubernetes. It manages external access to the services in a cluster, typically HTTP/HTTPS.
Clients can follow the Ingress rules to route client requests to the Kubernetes cluster services or a specific
The following is an example of an Ingress resource:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: apisix-gateway spec: rules: - host: apisix.apache.org http: paths: - backend: service: name: apisix-gateway port: number: 80 path: / pathType: Exact
The above example contains the following contents:
metadata.name: name of Ingress resource
spec.rules.host: domain used by external access
spec.rules.http.paths.backend: describes related information of services in a cluster
spec.rules.http.paths.path: describes paths used by external access to services in a cluster
spec.rules.http.paths.pathType: describes the path type to manage external access to services in the cluster
Based on the above example, we could notice that the spec of the Ingress resource is relatively simple.
Ingress is a resource definition in Kubernetes which couldn't handle any traffic by itself. So if we want to make Ingress resources effective, we must use a controller to process these Ingress resources, which is known as the Ingress controller.
The Ingress controller would keep watching any resource changes in the Kubernetes cluster and turn Ingress resource rules into the data layer's proxy rules so that the data layer could handle the traffic.
In the actual production environment, there are many different kinds of client requests. For example, Ingress resources couldn't directly describe some of the most common features like authentication and URI rewriting. So how could we satisfy these requirements?
How does Ingress-NGINX support extensions?
First, we will use an example of the Ingress-NGINX controller in the Kubernetes ecosystem to demonstrate how to use the extensions.
We could use
Annotation to describe the Ingress resources' needed extensions in Ingress-NGINX. For example:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/enable-cors: "true" nginx.ingress.kubernetes.io/cors-allow-origin: https://foo.com,https://bar.com nginx.ingress.kubernetes.io/cors-allow-headers: x-foo-1,x-foo-2 nginx.ingress.kubernetes.io/cors-allow-methods: GET,POST,PUT name: nginx-ingress spec: rules: - host: kubernetes.github.io http: paths: - path: /ingress pathType: Exact backend: service: name: nginx port: number: 80
The above configuration enables cors.
It is relatively simple to add Annotations to the Ingress resource; however, we must notice that the Ingress-NGINX controller needs full support for these Annotations; otherwise, the configuration won't take effect. If we need to use some other undeveloped features of the Ingress-NGINX controller, we need to do some custom developments.
The following sections describe how to use the Apache APISIX Ingress controller to meet these requirements.
Use plugins in APISIX Ingress
Apache APISIX Ingress utilizes plugins to satisfy customers' various requirements for handling traffic and any specific scenarios. Currently, there are more than 80 ready-to-use plugins, and users can develop custom plugins to support new features.
There are many different ways to develop custom plugins:
- Users could use Lua to develop plugins, and these plugins could run internally at the APISIX
- Users could also use other programming languages to develop plugins; this mechanism is called "plugin runner", and the plugins developed by this mechanism are called "External Plugin"
Please refer to the official docs about plugin development:
- Plugin Development: https://apisix.apache.org/docs/apisix/plugin-develop/
- External Plugin Development: https://apisix.apache.org/docs/apisix/external-plugin/
We will introduce three ways to develop plugins using Lua in the APISIX Ingress.
Pure CRD Mode
The APISIX Ingress controller supports its self-designed CRD specification, and you could directly enable plugins in the routing rules (whether an internal plugin or a custom plugin). For example:
apiVersion: apisix.apache.org/v2beta3 kind: ApisixRoute metadata: name: httpbin-route spec: http: - name: rule1 match: hosts: - apisix.apache.org paths: - /apisix-ingress backends: - serviceName: apisix-gateway servicePort: 80 plugins: - name: cors enable: true config: allow_origins: http://foo.bar.org allow_methods: "GET,POST" max_age: 3600 expose_headers: x-foo,x-baz allow_headers: x-from-ingress allow_credential: true
Users could use the above configuration to create routing rules, and cors plugin would be enabled in this route.
This way is the most native support in APISIX Ingress, and it fits in well with APISIX. Also, when users add new custom plugins, APISIX Ingress doesn't need to be redeveloped so that users can use it directly.
CRD + Ingress Annotations Mode
We could also use CRD + Ingress Annotations to extend the features in APISIX Ingress, for example:
apiVersion: apisix.apache.org/v2 kind: ApisixPluginConfig metadata: name: cors-plugin spec: plugins: - name: cors enable: true config: allow_origins: http://foo.bar.org allow_methods: "GET,POST" max_age: 3600 expose_headers: x-foo,x-baz allow_headers: x-from-ingress allow_credential: true --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: kubernetes.io/ingress.class: apisix k8s.apisix.apache.org/plugin-config-name: cors-plugin name: apisix-ingress spec: rules: - host: apisix.apache.org http: paths: - path: /apisix-ingress pathType: Exact backend: service: name: apisix-gateway port: number: 80
Using the above configuration, we could independently create a plugin config called
cors-plugin and use Ingress resource's
k8s.apisix.apache.org/plugin-config-name: cors-plugin to refer to it. The actual effect is almost the same as the first configuration; they would both enable the cors plugin for the corresponding routes.
Under this mode, the plugin configuration could be considered as an independent resource, and multiple Ingress resources could also share them. Similarly, there is no need to make any custom developments.
Ingress Annotations Mode
Due to the limited semantics of Ingress resources, we usually would use
annotations to add some additional information for the resource objects, which is also the most common way to extend Ingress capabilities. For example:
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: kubernetes.io/ingress.class: apisix k8s.apisix.apache.org/enable-cors: "true" k8s.apisix.apache.org/cors-allow-origin: https://foo.com,https://bar.com k8s.apisix.apache.org/cors-allow-headers: x-foo-1,x-foo-2 k8s.apisix.apache.org/cors-allow-methods: GET,POST,PUT name: apisix-ingress spec: rules: - host: apisix.apache.org http: paths: - path: /apisix-ingress pathType: Exact backend: service: name: apisix-gateway port: number: 80
The above configuration will add some additional information related to cors into the Ingress resource. The APISIX Ingress controller could then identify this information and turn it into the configuration of the data layer to extend the features of the Ingress resource.
However, under this mode, we must ensure that the APISIX Ingress controller has already been able to process these Annotations. Otherwise, we need to do some custom developments.
If you need to carry out custom developments, please refer to the following doc:
- How to use APISIX Ingress controller to develop: https://apisix.apache.org/docs/ingress-controller/contribute/
This article mainly introduces the specs related to Ingress resources and how to extend features for Ingress resources. For example, in Ingress-NGINX, we usually could only use Annotations to extend features. However, in the Apache APISIX Ingress, we have three modes of configurations to satisfy advanced requirements and more scenarios. Besides, users don't need any custom developments for custom plugins in most cases.