Why Is APISIX Ingress Controller Better than NGINX Ingress Controller?
Ingress exposes services in Kubernetes, where traffic routing is controlled by rules configured on the Ingress resource. To satisfy an ingress, you must also have an Ingress controller.
In this article, we will compare two popular Ingress controllers to give the readers insights into selecting ingress controllers.
- Ingress-NGINX is an Ingress controller implemented by the Kubernetes community and is widely used in the community.
- APISIX Ingress controller takes Apache APISIX, an open-source project under the ASF (Apache Software Foundation), as its data plane.
Ingress NGINX vs. APISIX Ingress
The table below compares the basic features of Ingress NGINX and APISIX Ingress, including protocols, upstream probe/resiliencies, load balancer strategies, authentication, Kubernetes integration, etc. The table is extracted from learnk8s.io.
|Product/Project||Ingress NGINX||Apache APISIX Ingress|
|1. General info|
|Rate limiting (L7)||✔️||✔️|
|4. Traffic routing|
|5. Upstream probes/resiliency|
|6. Load balancer strategies|
|Custom load balancing||✖︎||✔️|
|Client certificate - mTLS||✔️||✔️|
|9. Kubernetes Integration|
|Support for the Gateway API||✖︎||Preview|
|Integrates with service meshes||✔️||✔️|
|10. Traffic shaping|
|Wildcard certificate support||✔️||✔️|
|Configure hot reloading||Preview||✔️|
We can see from the figure below that there are more built-in functions and features in APISIX Ingress than in Ingress NGINX, including protocol support, health checks and circuit breakers, authentication, Kubernetes integration, and so on.
In the microservices architecture, the application is divided into many microservices. Whenever a microservice fails, or a service is scaled up or down, the caller needs to be notified as soon as possible to avoid call failures. Therefore, the service registration and service discovery mechanism are vital in the microservice architecture, usually maintained by the service registry.
|Service Discovery||Ingress NGINX||Apache APISIX Ingress|
When a node fails or migrates, it is inevitable that the node will be unavailable. If a large number of requests access these unavailable nodes, it will cause traffic loss and business interruption. Therefore, it is necessary to perform health checks on nodes by using probes, and direct requests to healthy nodes, thereby reducing traffic loss.
The health check functionality is not yet supported in Ingress NGINX, but APISIX Ingress provides this functionality:
- Active check: Use probes to ensure that the Pods in the backend are healthy. When
Nconsecutive probes sent to a node fail, the node will be marked as unhealthy. The load balancer will ignore the unhealthy nodes so they cannot receive requests. Enabling active health checks can effectively disable unhealthy Pods to avoid traffic loss.
- Passive check: The passive health check does not need to initiate additional probes. Each request sent to the node acts as a probe. If
Nconsecutive requests to a healthy node fail, the node will be marked as unhealthy. Since it cannot sense the node status in advance, there may be a certain amount of failed requests. This situation is relatively common during rolling updates, and we can use service downgrades to avoid the amount of failed requests.
Example of APISIX Ingress configuring health checks for httpbin service:
apiVersion: apisix.apache.org/v2 kind: ApisixUpstream metadata: name: httpbin spec: healthCheck: passive: unhealthy: httpCodes: - 500 httpFailures: 3 active: type: http httpPath: /healthz healthy: successes: 3 interval: 2s httpCodes: - 200
The gateway is an entry point for requests; it initiates calls to the backend service. When traffic peaks and the load is heavy, the backend service may fail to call due to timeout or exception. When the failure occurs, requests cannot be stacked on the gateway. It needs to return quickly, which requires a break on the gateway.
Similar to the health check feature, the feature of service circuit breaking is not supported in Ingress NGINX. In APISIX Ingress, the api-breaker plugin helps to implement this.
Sample configuration of the circuit breaker in APISIX Ingress:
apiVersion: apisix.apache.org/v2 kind: ApisixRoute metadata: name: httpbin-route spec: http: - name: rule1 match: hosts: - httpbin.org paths: - /status/* backends: - serviceName: httpbin servicePort: 80 plugins: - name: api-breaker enable: true config: break_response_code: 502 unhealthy: http_statuses: - 505 failures: 2 healthy: http_statuses: - 200 successes: 2
Support More Plugins and Authentication Methods
Ingress NGINX mainly uses Annotations and ConfigMap for configuration, and the supported plugins are relatively limited. If you want to use JWT, HAMC, or other authentication methods, you must integrate them yourself.
APISIX Ingress supports 80+ plugins in APISIX natively, which supports multiple authentication methods such as JWT, HAMC, wolf-rbac, etc. These plugins provide a wide variety of functionalities that cover most usage scenarios.
Example of HMAC authentication on APISIX route:
apiVersion: apisix.apache.org/v2 kind: ApisixConsumer metadata: name: hmac-value spec: authParameter: hmacAuth: value: access_key: papa secret_key: fatpa algorithm: "hmac-sha256" clock_skew: 0 --- apiVersion: apisix.apache.org/v2 kind: ApisixRoute metadata: name: httpbin-route spec: http: - name: rule1 match: hosts: - httpbin.org paths: - /ip backends: - serviceName: httpbin servicePort: 80 authentication: enable: true type: hmacAuth
Extensibility of Ingress NGINX and APISIX Ingress
When the Ingress controller's basic features cannot meet enterprise users' needs, it can only be customized through extension. Next, we will explain how Ingress NGINX and APISIX Ingress extend their features.
How Ingress NGINX Extends Its Features
Ingress NGINX can only be extended by embedding Lua programs.
Ingress NGINX plugin development example:
- Write Lua program example-plugin
- Install the plugin to
/etc/nginx/lua/plugins/<your plugin name>$\rightarrow$
/etc/nginx/lua/plugins/example-pluginin the ingress-nginx pod
- Enable the example-plugin in ConfigMap, and reference this ConfigMap object when installing Ingress NGINX
apiVersion: v1 kind: ConfigMap metadata: name: ingress-nginx-controller namespace: ingress-nginx data: plugins: "example-plugin"
How APISIX Ingress Extends Its Features
APISIX Ingress provides a variety of methods to extend features, and enterprise users can freely choose the method according to their own needs. APISIX supports:
- Plugin development via Lua: simple and has almost no performance loss
- Development via plugin-runner: programming languages such as JAVA/Python/Go are supported for development, so users can customize their plugins without learning new languages
- Plugins via WASM: any language that supports building WASM can be used for plugin development
In addition, you can directly write Lua code through the serverless plugin to quickly meet business needs.
Why Does APISIX Ingress Support CustomResourceDefinition (CRD)?
Currently, APISIX Ingress supports three declarative configurations: Ingress, CRD, and Gateway API. We will mainly compare Ingress and CRD here. And we will explain Gateway API after.
Ingress is more suitable for enterprise users migrating from Ingress NGINX for its low migration cost. Its shortcomings are also apparent, such as weak semantic capabilities, no standard specifications, etc. And ingress can only be extended through annotations, but annotations cannot support complex scenarios. In comparison, CRD has the following advantages:
- Easier to use since it's more in line with the design semantics of the data plane
- Reusability of configurations to reduce redundancy
- Improved functionality and extensibility
- The data plane of APISIX has an active community, with quick updates and releases, and CRD can support more capabilities of the data plane easily
Pain Point of Ingress NGINX: Hot Reloading Not Supported
Problems Caused by Static Configuration
Ingress NGINX is implemented mainly based on NGINX configuration files. Although NGINX and Lua are used to achieve feature extensions, it only partially solves the problem of static configuration files. It shows insufficiencies in its routing and reloading capabilities. For example, the NGINX configuration must be reloaded when adding or modifying a rule. With more and more routing rules and certificates, the load operation will take more time, from a few seconds to more than ten seconds. Every reloads of NGINX resets the load balancing state, negatively impacting online traffic, causing short interruptions, increasing response latency, and reducing load balancing quality.
Situations that Trigger an NGINX Reload
- Creating a new Ingress resource
- Adding TLS section to existing Ingress
- Changing Ingress Annotations may also affect upstream configuration (e.g., load-balance annotations do not need to be reloaded)
- Adding or deleting a path in Ingress
- Deleting Ingress, Service, or Secret resources
- Updating Secret
The situations listed above cover most usage scenarios of the Ingress controller. In a cluster environment with applications being frequently deployed, operations (creation, update, deletion, etc.) of resources such as Ingress and Secret will be continuously triggered. It will result in a sharp increase in the number of NGINX reloads, significantly impacting the production environment.
The architecture of Ingress NGINX determines that it must generate NGINX configuration first and reloads to complete the configuration update. The problem of reloading cannot be solved without adjusting the architecture. To solve this problem, APISIX Ingress no longer depends on NGINX configuration in its routing implementation and changed to a pure memory architecture. Dynamic routing is realized through hot reloading, and NGINX no longer needs to be restarted.
Advantages of Kubernetes Gateway API
Kubernetes Gateway API is more functional than Ingress and is designed to evolve the Kubernetes service networking through expressive, extensible, and role-oriented interface implemented by many vendors and with broad industry support. Gateway API has the following characteristics:
Role-oriented: Gateway is composed of API resources. Different API resources represent different roles for using and configuring Kubernetes network resources
Strong expressiveness: Gateway API supports core functionality, including header-based matching, traffic weighting, and other capabilities that were only possible in Ingress through customized non-standard annotations
Extensible: Gateway API allows different resources to be linked at various API layers. This enables granular customization within the API structure
Support Status of Gateway API
Kubernetes Gateway API is a standard for extending the Kubernetes service mesh. Its Gateway resource can be used as a Kubernetes API to manage the life cycle of the gateway, which is very powerful. Many Ingress controllers actively support it, including Istio, Kong, Traefik, etc. Unfortunately, Ingress NGXIN is not planning to support Gateway API yet (Gateway API Implementation Status), whereas APISIX Ingress already supports most of the features of Gateway API: including HTTPRoute, TCPRoute, TLSRoute, UDPRoute, etc.
After a thorough comparison of APISIX Ingress and Ingress NGINX, we can conclude that both open-source software is excellent and the essential functions of the two are similar. In the microservice architecture, however, APISIX Ingress shows more advantages in supporting service governance and service discovery by providing health checks and circuit breaking.
Ingress NGINX is characterized by its simplicity and ease of use, with some obvious shortcomings, such as difficulties in feature extensions. APISIX Ingress, who joined later, solved NGINX's hot reloading pain point and provided better extensibilities and functionalities. For example, supporting Gateway API and CRD enriches the capabilities of the Ingress controller in terms of project development.
In short, if you want to select an Ingress controller with richer features and better extensibilities, APISIX Ingress is strongly recommended. If you are new to the Ingress controller and don’t have many functional requirements, Ingress NGINX is also a good choice for its simplicity.