Why Is APISIX Ingress Controller Better than NGINX Ingress Controller?

Ingress exposes services in Kubernetes, where traffic routing is controlled by rules configured on the Ingress resource. To satisfy an ingress, you must also have an Ingress controller.

In this article, we will compare two popular Ingress controllers to give the readers insights into selecting ingress controllers.

• Ingress-NGINX is an Ingress controller implemented by the Kubernetes community and is widely used in the community.
• APISIX Ingress controller takes Apache APISIX, an open-source project under the ASF (Apache Software Foundation), as its data plane.

Ingress NGINX vs. APISIX Ingress

Feature Comparison

The table below compares the basic features of Ingress NGINX and APISIX Ingress, including protocols, upstream probe/resiliencies, load balancer strategies, authentication, Kubernetes integration, etc. The table is extracted from learnk8s.io.

Differences

We can see from the figure below that there are more built-in functions and features in APISIX Ingress than in Ingress NGINX, including protocol support, health checks and circuit breakers, authentication, Kubernetes integration, and so on.

Service Discovery

In the microservices architecture, the application is divided into many microservices. Whenever a microservice fails, or a service is scaled up or down, the caller needs to be notified as soon as possible to avoid call failures. Therefore, the service registration and service discovery mechanism are vital in the microservice architecture, usually maintained by the service registry.

Protocol Support

While both ingresses support HTTP/HTTPS protocol, APISIX supports more protocols. It supports TLS to encrypt TCP traffic. It also supports MQTT, Dubbo, and kafka for proxying.

Upstream Probes/Resiliency

Health Checks

When a node fails or migrates, it is inevitable that the node will be unavailable. If a large number of requests access these unavailable nodes, it will cause traffic loss and business interruption. Therefore, it is necessary to perform health checks on nodes by using probes, and direct requests to healthy nodes, thereby reducing traffic loss.

The health check functionality is not yet supported in Ingress NGINX, but APISIX Ingress provides this functionality:

• Active check: Use probes to ensure that the Pods in the backend are healthy. When N consecutive probes sent to a node fail, the node will be marked as unhealthy. The load balancer will ignore the unhealthy nodes so they cannot receive requests. Enabling active health checks can effectively disable unhealthy Pods to avoid traffic loss.
• Passive check: The passive health check does not need to initiate additional probes. Each request sent to the node acts as a probe. If N consecutive requests to a healthy node fail, the node will be marked as unhealthy. Since it cannot sense the node status in advance, there may be a certain amount of failed requests. This situation is relatively common during rolling updates, and we can use service downgrades to avoid the amount of failed requests.

Example of APISIX Ingress configuring health checks for httpbin service:

apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
  name: httpbin
spec:
  healthCheck:
    passive:
      unhealthy:
        httpCodes:
          - 500
        httpFailures: 3
    active:
      type: http
      httpPath: /healthz
      healthy:
        successes: 3
        interval: 2s
        httpCodes:
          - 200

Circuit Breaker

The gateway is an entry point for requests; it initiates calls to the backend service. When traffic peaks and the load is heavy, the backend service may fail to call due to timeout or exception. When the failure occurs, requests cannot be stacked on the gateway. It needs to return quickly, which requires a break on the gateway.

Similar to the health check feature, the feature of service circuit breaking is not supported in Ingress NGINX. In APISIX Ingress, the api-breaker plugin helps to implement this.

Sample configuration of the circuit breaker in APISIX Ingress:

apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
 name: httpbin-route
spec:
 http:
 - name: rule1
   match:
     hosts:
     - httpbin.org
     paths:
       - /status/*
   backends:
   - serviceName: httpbin
     servicePort: 80
   plugins:
   - name: api-breaker
     enable: true
     config:
       break_response_code: 502
       unhealthy:
         http_statuses:
         - 505
         failures: 2
       healthy:
         http_statuses:
         - 200
         successes: 2

Support More Plugins and Authentication Methods

Ingress NGINX mainly uses Annotations and ConfigMap for configuration, and the supported plugins are relatively limited. If you want to use JWT, HAMC, or other authentication methods, you must integrate them yourself.

APISIX Ingress supports 80+ plugins in APISIX natively, which supports multiple authentication methods such as JWT, HAMC, wolf-rbac, etc. These plugins provide a wide variety of functionalities that cover most usage scenarios.

Example of HMAC authentication on APISIX route:

apiVersion: apisix.apache.org/v2
kind: ApisixConsumer
metadata:
  name: hmac-value
spec:
  authParameter:
    hmacAuth:
      value:
        access_key: papa
        secret_key: fatpa
        algorithm: "hmac-sha256"
        clock_skew: 0
---

apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
 name: httpbin-route
spec:
 http:
 - name: rule1
   match:
     hosts:
     - httpbin.org
     paths:
       - /ip
   backends:
   - serviceName: httpbin
     servicePort: 80
   authentication:
     enable: true
     type: hmacAuth

Extensibility of Ingress NGINX and APISIX Ingress

When the Ingress controller's basic features cannot meet enterprise users' needs, it can only be customized through extension. Next, we will explain how Ingress NGINX and APISIX Ingress extend their features.

How Ingress NGINX Extends Its Features

Ingress NGINX can only be extended by embedding Lua programs.

Ingress NGINX plugin development example:

1. Write Lua program example-plugin
2. Install the plugin to /etc/nginx/lua/plugins/<your plugin name> $\rightarrow$ /etc/nginx/lua/plugins/example-plugin in the ingress-nginx pod
3. Enable the example-plugin in ConfigMap, and reference this ConfigMap object when installing Ingress NGINX

apiVersion: v1
kind: ConfigMap
metadata:
  name: ingress-nginx-controller
  namespace: ingress-nginx
data:
  plugins: "example-plugin"

How APISIX Ingress Extends Its Features

APISIX Ingress provides a variety of methods to extend features, and enterprise users can freely choose the method according to their own needs. APISIX supports:

• Plugin development via Lua: simple and has almost no performance loss
• Development via plugin-runner: programming languages such as JAVA/Python/Go are supported for development, so users can customize their plugins without learning new languages
• Plugins via WASM: any language that supports building WASM can be used for plugin development

In addition, you can directly write Lua code through the serverless plugin to quickly meet business needs.

Why Does APISIX Ingress Support CustomResourceDefinition (CRD)?

Currently, APISIX Ingress supports three declarative configurations: Ingress, CRD, and Gateway API. We will mainly compare Ingress and CRD here. And we will explain Gateway API after.

Ingress is more suitable for enterprise users migrating from Ingress NGINX for its low migration cost. Its shortcomings are also apparent, such as weak semantic capabilities, no standard specifications, etc. And ingress can only be extended through annotations, but annotations cannot support complex scenarios. In comparison, CRD has the following advantages:

• Easier to use since it's more in line with the design semantics of the data plane
• Reusability of configurations to reduce redundancy
• Improved functionality and extensibility
• The data plane of APISIX has an active community, with quick updates and releases, and CRD can support more capabilities of the data plane easily

Pain Point of Ingress NGINX: Hot Reloading Not Supported

Problems Caused by Static Configuration

Ingress NGINX is implemented mainly based on NGINX configuration files. Although NGINX and Lua are used to achieve feature extensions, it only partially solves the problem of static configuration files. It shows insufficiencies in its routing and reloading capabilities. For example, the NGINX configuration must be reloaded when adding or modifying a rule. With more and more routing rules and certificates, the load operation will take more time, from a few seconds to more than ten seconds. Every reloads of NGINX resets the load balancing state, negatively impacting online traffic, causing short interruptions, increasing response latency, and reducing load balancing quality.

Situations that Trigger an NGINX Reload

• Creating a new Ingress resource
• Adding TLS section to existing Ingress
• Changing Ingress Annotations may also affect upstream configuration (e.g., load-balance annotations do not need to be reloaded)
• Adding or deleting a path in Ingress
• Deleting Ingress, Service, or Secret resources
• Updating Secret

The situations listed above cover most usage scenarios of the Ingress controller. In a cluster environment with applications being frequently deployed, operations (creation, update, deletion, etc.) of resources such as Ingress and Secret will be continuously triggered. It will result in a sharp increase in the number of NGINX reloads, significantly impacting the production environment.

The architecture of Ingress NGINX determines that it must generate NGINX configuration first and reloads to complete the configuration update. The problem of reloading cannot be solved without adjusting the architecture. To solve this problem, APISIX Ingress no longer depends on NGINX configuration in its routing implementation and changed to a pure memory architecture. Dynamic routing is realized through hot reloading, and NGINX no longer needs to be restarted.

Gateway API

Advantages of Kubernetes Gateway API

Kubernetes Gateway API is more functional than Ingress and is designed to evolve the Kubernetes service networking through expressive, extensible, and role-oriented interface implemented by many vendors and with broad industry support. Gateway API has the following characteristics:

• Role-oriented: Gateway is composed of API resources. Different API resources represent different roles for using and configuring Kubernetes network resources

• Strong expressiveness: Gateway API supports core functionality, including header-based matching, traffic weighting, and other capabilities that were only possible in Ingress through customized non-standard annotations

• Extensible: Gateway API allows different resources to be linked at various API layers. This enables granular customization within the API structure

Support Status of Gateway API

Kubernetes Gateway API is a standard for extending the Kubernetes service mesh. Its Gateway resource can be used as a Kubernetes API to manage the life cycle of the gateway, which is very powerful. Many Ingress controllers actively support it, including Istio, Kong, Traefik, etc. Unfortunately, Ingress NGXIN is not planning to support Gateway API yet (Gateway API Implementation Status), whereas APISIX Ingress already supports most of the features of Gateway API: including HTTPRoute, TCPRoute, TLSRoute, UDPRoute, etc.

Summary

After a thorough comparison of APISIX Ingress and Ingress NGINX, we can conclude that both open-source software is excellent and the essential functions of the two are similar. In the microservice architecture, however, APISIX Ingress shows more advantages in supporting service governance and service discovery by providing health checks and circuit breaking.

Ingress NGINX is characterized by its simplicity and ease of use, with some obvious shortcomings, such as difficulties in feature extensions. APISIX Ingress, who joined later, solved NGINX's hot reloading pain point and provided better extensibilities and functionalities. For example, supporting Gateway API and CRD enriches the capabilities of the Ingress controller in terms of project development.

In short, if you want to select an Ingress controller with richer features and better extensibilities, APISIX Ingress is strongly recommended. If you are new to the Ingress controller and don’t have many functional requirements, Ingress NGINX is also a good choice for its simplicity.