Dynamic

Rich Features

Cluster Management

Go for Apache APISIX

Looking for the best API Gateway?

Built on NGINX, with more features than NGINX.

 Many companies have migrated from NGINX to Apache APISIX, immediately achieving fast growth and reducing operational costs.

Apache APISIX vs NGINX:


## Overview

As a world-leading enterprise-level consortium blockchain infrastructure supplier and solution provider, [Hyperchain Technologies](https://www.hyperchain.cn/en/about/company) is committed to building innovative infrastructure and empowering the digital development of society. During its rapid growth, Hyperchain met a lot of problems when establishing its Hyperchain blockchain platform, as follows:

- No standardized traffic management with the potential risk of system collapse
- Incomplete security control and authentication management
- Inconvenient permission control
- High cost on public network IP addresses
- Unstable blockchain nodes: single node vulnerable to being attacked
- Lack of unified management of multiple protocols

Hyperchain tried Kong and NGINX, but unfortunately, they can’t satisfy Hyperchain’s business scenarios. **However, after adopting Apache APISIX, all the problems mentioned above are solved, endowing the Hyperchain blockchain platform with endless vitality.**

### About Hyperchain Blockchain Platform

Hyperchain blockchain platform refers to embedding the blockchain framework into the cloud computing platform, taking advantage of cloud service infrastructure’s deployment and management advantages. It is an open blockchain platform that can provide developers with a convenient and high-performance blockchain ecosystem and related support services and also support developers’ business expansion and operation in web3.

**Hyperchain blockchain platform can build the blockchain network quickly and flexibly. With the platform, enterprises can manage the blockchain business uniformly.** For example, through the platform, we can sign contracts on the integrated development environment and then deploy the contracts to our created blockchain network. The upper service module can call blockchain-related contracts to continue the business process.

As there are many nodes in the chain, ranging from dozens to thousands, it becomes difficult for us to monitor and maintain the operation of the chain without the support of the platform. **By using the Hyperchain blockchain platform, users can not only save costs but also manage the blockchain more conveniently and enhance the security of the entire system.**

Let’s take Hyperchain as an example to figure out how and why blockchain companies should select Apache APISIX among many API gateways. Below we will analyze APISIX’s application in both the Hyperchain blockchain platform and blockchain node.

## APISIX’s Application in the Hyperchain Blockchain Platform

Below is an application interaction diagram of APISIX in the Hyperchain blockchain platform. The backend service registers the service information with etcd according to its business characteristics and then registers the service with APISIX by the route registration module.

APISIX works as the unified entry of internal microservices in the system. External requests are first sent to APISIX; then, they visit the access layer through the API after passing the authentication and later visit the core services through RPC (remote procedure call).

![Hyperchain application interaction diagram](https://static.apiseven.com/2022/11/15/637335a53d73c.png)

### Route Forwarding

Sometimes, the APIs are expected to be exposed under the same domain name. In this case, we can add some prefixes to the API path of the same service, such as `/baas-core` or `/baas-other`. When the client requests these APIs, the API gateway needs to strip these added prefixes and then forward the request to the backend service. APISIX’s [proxy-rewrite plugin](https://apisix.apache.org/docs/apisix/plugins/proxy-rewrite/) can help us to handle such cases conveniently.

For example:
When visit: http://apisix:8080/baas-{service}/api/v1/…
The request can be forwarded to http://{service}/api/v1/…
by writing a regular expression: `^/baas-core/(.*)$,/$1`

![proxy-rewrite plugin edit](https://static.apiseven.com/2022/11/15/637335b538dc0.png)

### Traffic Limit Management

Users can choose consortium blockchains according to their needs, not only that of Hyperchain but also IBM data Fabric, Baidu XuperChain, etc. Therefore, Hyperchain needs to conduct lifecycle management of all the consortium blockchains in the system.

When creating a consortium chain, Hyperchain only needs to write the hard code on the blockchain platform and upload the pluggable drive components to the blockchain platform for calling to create the consortium chain. In some private deployment cases, the pluggable drive components can support fastly.

Every call to the drive components is a process that needs to be limited, especially when the number is great. Consequently, APISIX’s [limit-req plugin](https://apisix.apache.org/docs/apisix/plugins/limit-req/) can be beneficial in restricting the traffic input and output of the platform to secure its stability.

The limit-req plugin allows customized configuration about rate and burst.

![limit-req plugin model](https://static.apiseven.com/2022/11/15/637335b790f49.jpeg)

### Security Control and Authority Management

To collaborate with APISIX, Hyperchain developed a plugin for private deployment circumstances. Party A often prefers to use its authentication services or service account system. When the frontend traffic visits the website, it will first go through the Access-auth plugin, and it can access the backend BFF (Backend for Frontend) only if it passes the authentication.

![Authentication process diagram](https://static.apiseven.com/2022/11/15/637335b994030.png)

According to the Restful standard specification’s three key factors (the authentication information, Restful path, and HTTP verbs like GET, POST, PUT, DELETE, PATCH, etc.), account-role-authority authentication is performed. If the authentication is passed, the user information will be returned in the header; if not, it will return to 403.

![API authentication path](https://static.apiseven.com/2022/11/15/637335bd3e080.png)

### Hot Reloading

APISIX offers [hot reloading](https://apisix.apache.org/docs/java-plugin-runner/hot-reloading/) of plugins developed internally. It saves time during development and enables users to change parts of their code without restarting the entire plugin runner. In this way, the developers can make adjustments to the interface in the backend, taking effect immediately, which is convenient and friendly for online release.

### Why not Kong?

Hyperchain had used Kong before but finally replaced it with Apache APISIX mainly because

- **High Deployment and Maintenance Cost**

    Kong cluster needs to cooperate with the Postgresql database and requires a specific database administrator for high availability. The cluster deployment of the Postgresql database is relatively difficult to implement and increases the cost of later operation and maintenance.

- **Increase the System Complexity and the Failure Rate**

    The Hyperchain blockchain platform uses My SQL database, resulting in two relational databases in the system if adopting Kong, which makes the system more complicated. It also increases the failure rate in improving the compatibility between two sets of databases.

## APISIX’s Application in Blockchain Node

![Hyperchain blockchain diagram](https://static.apiseven.com/2022/11/15/637335bfa08dc.png)

### Save Cost on Public Network IP Addresses

The user purchases blockchains through the Hyperchain blockchain platform. Then the upper-level businesses and developer clients can connect to the nodes. The services may connect to one or more nodes, and the users may access one or more nodes, which brings a problem: almost all nodes will be visited, burdening the visiting pressure of a single node.

It is relatively easy to handle in some private employment environments. However, too many nodes and public network IPs are required for those systems targeting internet users. The price of each public network IP of 4 megabytes of traffic can reach almost 200 CNY per month. There are thousands of nodes on the Hyperchains platform, which consumes high public IP costs.

APISIX manages all the visiting requests and distributes traffic to corresponding blockchain nodes. In this way, Hyperchain saved a high cost.

![Hyperchain's mode with APISIX](https://static.apiseven.com/2022/11/15/637335c23f4f8.png)

### Convenient Permission Control

There are various blockchains on the Hyperchain blockchain platform, and each chain has various complicated RBAC permission control. Therefore, different kinds of certificates are required to be added on the client side, such as TLS certificates, which also puzzle users.

At present, APISIX’s [key-auth plugin](https://apisix.apache.org/docs/apisix/plugins/key-auth/) can be used to solve this problem efficiently. Authorized users can directly access the blockchain without worrying about permission configuration, as APISIX unifies the underlying chain.

### Clustering Improves Nodes Stability

As mentioned above, the nodes are visited frequently. High concurrency exists in most bank users as the TPS (Transactions Per Second) can reach 40,000-50,000 times.

It turns out that a single node on the blockchain is vulnerable because each transaction will impact the visited node.

Hyperchain deployed Apache APISIX on K8s with the Horizontal Pod Autoscaler. Since APISIX uses etcd with good dynamic scalability, it can effectively solve the problem of single-point traffic impact.

### Support Multiple Protocols

The protocols of heterogeneous chains on the Hyperchain blockchain platform are diverse, such as HTTP protocol, Websocket protocol, gRPC protocol, TCP protocol, UDP protocol, etc.

APISIX supports multiple protocols, flexibly adapting to the underlying layers of different blockchains, thus reducing development costs.

### Why not NGINX?

It seems that Hyperchain can utilize NGINX to solve its problems. However, after the trial, Hyperchain gave it up because

- **Unsuitable for Dynamic Expansion**

    Hyperchain needs to add and delete nodes in the blockchain platform frequently, which can only take effect after restarting, which is inconvenient for developers. Besides, the developers must write complex rule code for plugins in NGINX’s conf file.

- **Difficult to Cluster**

    NGINS supports no cluster, which is relatively complex and unfriendly for users.

- **No Direct Support for TCP and UDP**

    NGINX can only proxy the 7-layer protocol rather than the 4-layer protocol and cannot directly support TCP and UDP protocols. Furthermore, the module is not compiled by default, requiring additional processing.

- **No Dashboard**

    There is no NGINX visualized interface, which is difficult for developers and operators to manage so many nodes.

## Summary

From the contents mentioned above, we know that Apache APISIX is a dynamic API gateway that can be applied in the blockchain area. Moreover, Apache APISIX provides rich traffic management features like Load Balancing, Dynamic Upstream, Canary Release, Circuit Breaking, Authentication, Observability, etc.

We hope APISIX can support more blockchain enterprises to enter another development phase.
Welcome to learn more about Apache APISIX.
You can contact us at https://api7.ai/contact.


How the blockchain unicorn company-Hyperchain uses APISIX, an open-source api gateway, to empower the Hyperchain Blockchain Platform to manage blockchain traffic in web3, mentioning the comparison among APISIX, Kong, and NGINX.

API Gateway for Web3: How APISIX Empowers Hyperchain


## Overview

Since May 2021, [vivo](https://www.vivo.com/en) has introduced [Apache APISIX](https://apisix.apache.org/) as its API Gateway. After more than a year of practice in vivo, APISIX has solved many technical and business pain points and has been used on a large scale.

### Pain Points before Using APISIX

- **Complex Management of Business Scenarios and System Maintenance**

    Due to the rapid growth of business, there are various scenarios and systems that serve them, which vivo needs a unified way to manage.

- **Interact between the Data Plane and the Control Plane**

    For medium and large-sized companies like vivo, it’s unexpected that minor trouble that occurs in the data plane will impact the control plane.

- **No Support for Multi-Dimension Resources**

    Diverse projects leads to various domain names and URLs. The business department needs to search according to different resource dimensions.

- **Uncontrollable Impact of Problems**

    As vivo’s projects are complex, and the effect of problems encountered are uncontrollable. The use of some complicated plugins intensifies this.

By replacing APISIX with NGINX, vivo finally successfully made a series of achievements as below.

### Achievements after Using APISIX

- **High Availability**

    No major failure has occurred since APISIX is launched in vivo, and the system availability exceeds **99.99%**.

- **High Performance**

    Undertaking significant online traffic and serving a large number of services, the current online forwarding traffic reaches close to **one million QPS** (Queries-per-second).

- **Rich Features**

    Thanks to APISIX's [rich features](https://apisix.apache.org/blog/2022/11/02/apache-apisix-v3-preview/), APISIX can cover almost common NGINX proxy scenarios. About 50% of the vivo projects is migrated from the NGINX to the APISIX clusters.

- **Supporting the Construction and Development of Cloud-Native**

    The K8s bare metal supporting the containerization has reached a scale of **10,000**. About 40% of the projects has been migrated from bare metal and virtual machines to the K8s container platform, supporting and promoting vivo’s containerization progress.

## vivo's System Design Based on APISIX

Next, let’s look at the vivo’s system design after adopting APISIX.

### Customized Architecture Display on APISIX

![vivo's API gateway architecture with APISIX](https://static.apiseven.com/2022/11/23/637dc2f51318e.jpg)

From the diagram above, we can analyze that vivo has:

- Completed the construction of **Layer 4** and **Layer 7** traffic gateways, which are supported by APISIX
- Realized the traffic access and mixed deployment of bare metal, virtual machines, and containers
- Implemented APISIX cluster management
- Connected internal DevOps platform and business deployment services to access traffic quickly and automatically
- Improved monitoring construction

### Configuration Management and Launching Improvements

In order to better meet the actual needs of the business department, vivo has carried out a series of adaptations on APISIX. Below are some of those adjustments, including control plane alteration, cluster separation management, and data forwarding.

#### Control Plane Alteration

The whole process can be this:

After the data is configured in the A6 changes platform, the information will be delivered through RPC notify to the ManagerAPI, which is built by vivo based on the open-sourced [APISIX Dashboard](https://github.com/apache/apisix-dashboard).

Then the traffic will be sent to `apisix-agent`. APISIX polls apisix-agent regularly through the privileged process to obtain change tasks in batches. Next, the privileged process notifies the `worker` through shared queues to realize the change in memory.

In the meantime, APISIX informs the `apisix-agent` of the result of tasks and then delivers them to ManagerAPI. Additionally, the A6 changes platform can poll the ManagerAPI to get task results.

![vivo's management and publishing mode](https://static.apiseven.com/2022/11/25/638036da44e7f.jpeg)

The [etcd](https://etcd.io/docs/v3.5/learning/why/#comparison-chart "etcd") is a highlight of APISIX, allowing the independent operation of the control and data planes. Considering the uniqueness of its architecture, vivo discarded etcd in the above process. Here are some reasons.

Due to vivo’ projects diversity, there are various domain names and URLs. Besides, the business departments need to query different dimensions. Thanks to APISIX’s adaptability with not only etcd but also varying kinds of databases, vivo could easily utilize databases like MongoDB, to work together with APISIX.

In addition, vivo made the below contributions to be compatible with Apache APISIX.

- **Developing the Agent Component**

    Since May 2021, vivo introduced Apache APISIX. Considering the technical background and context, vivo was unconfident to be incapable of adopting APISIX because vivo has no experience with OpenResty and Lua. In addition, there are many non-forwarding tasks, like log collection and monitoring handling, which might increase the management complexity of the data plane. Consequently, vivo developed the agent component to reduce the complexity of development.

- **Writing Data to Disk**

    To make the system adjustable and enable the data plane runs independently, thus reducing the reliance on the control plane, vivo wrote the configuration file into the disk. When APISIX is started, it supports complete pulling from the configuration center and also supports directly obtaining configuration resources from the file directory of the local disk. This way dramatically improves the independence of data and the robustness of the system. Moreover, it is very intuitive to understand the configured route and upstream information on the disk that is placed on the disk, which is helpful for troubleshooting.

- **Callback Change Task Result**

    As a large-sized company, vivo needs to make sure that changes to resources such as routers and upstreams can be guaranteed to be effective and successful, and the system can report the error even if these changes fail. Such logic of ACK (Acknowledgement Code) ensures that NGINX workers on a machine can call back. When the callback tasks are successful, all workers on APISIX will update resource changes to the relevant memory.

#### Cluster Separation Management

![Cluster Split Management](https://static.apiseven.com/2022/11/25/6380381595b00.jpeg)

The open-source version of APISIX provides [etcd](https://apisix.apache.org/docs/apisix/FAQ/#why-does-apache-apisix-use-etcd-for-the-configuration-center) for everyone to share. However, the company’s projects are complex, and the problems encountered are uncontrollable. In addition, it is inevitable to use complicated plugins, affecting the system’s performance.

Therefore, it is managed by cluster separation to realize the isolation of cluster configuration on APISIX, which can

- Control the fault domain and effectively support projects complexity without affecting other projects
- Effectively reduce the load caused by the APISIX non-forwarding layer when the container nodes change frequently
- Reduce the load impact caused by the health check

#### Increasing the QPS Carried by HTTPS

According to the relevant requirements of the Ministry of Industry and Information Technology in China, external network traffic must go through the HTTPS protocol. As a TLS-based HTTP encryption protocol, HTTPS heavily burdens the CPU in the encryption and decryption process.

When the routing and other configurations are the same, the traffic that HTTPS can carry is about **1/8 - 1/10** of that of HTTP.

After patching the Intel® QAT (QuickAssist Technology) accelerator card, vivo delivers the decryption handling to the QAT accelerator card, which frees up the CPU, thereby increasing the QPS carried by HTTPS on a single machine. As can be seen from the figure below, **the HTTPS load capacity of a single machine is about doubled**.

![data showing vivo's improvment on carrying traffic](https://static.apiseven.com/2022/11/23/637dc306a5fa1.jpg)

## How Does vivo Combine Businesses with APISIX

### Supporting Containerization Development

To support containerization development, vivo self-developed a K8s ingress controller. Below are some functions of it.

1. Adapting to vivo’s modified asynchronous push configuration change mechanism

2. Conducting multi-K8s cluster event processing notifications to APISIX

3. Coping with complex projetcs scenarios such as:

- One server with multiple ports

- When other RPC framework servers, such as Dubbo and gRPC, are connected to K8s, a unified set of processing logic is required to notify APISIX or other frameworks of port information according to the configuration characteristics of the projects

4. Adapting to the particular needs of the company’s internal DevOps and other automation scenarios, facilitating rapid deployment, and enabling traffic

### Helping Projects Migrate from NGINX to APISIX

Vivo’s projects are deployed on the existing NGINX cluster and has been running stably for a long time. However, it brings non-business workloads and instability to projects. Consequently, it’s challenging to conduct migration. So how to promote the projects migration to APISIX?

- First, find a project of a cooperative department, serve the business department well to set a benchmark, and provide technical guidance and training

- Build an easy-to-use control plane system to facilitate business access and multi-dimensional management of business departments
  
- Provide automatic conversion capability and basic configuration converting from NGINX configuration to APISIX configuration

### Upgrading APISIX and Supporting Its Open-Source Version

Based on APISIX 2.4 version, vivo made some adjustments and released the new version, which was upgraded to a newer one in Q2 this year.

On the one hand, thanks to the modular architecture of APISIX, it is relatively easy to integrate vivo’s modified Lua code into the branch of the higher version of APISIX. On the other hand, vivo also keeps upgrading OpenResty section, with about one version per year. Since vivo utilizes a lot of PATCH and some useful functions like QAT, upgrading this component is difficult and laborious.

The free version of NGINX community features is slow to update and is inactive. Vivo is considering whether to build jointly with APISIX. To reduce the manpower needed to do related system testing, vivo adopted Robot Framework, a generic test automation framework for system integration testing. They are promoting the relevant components for the unit test coverage and the development model of TDD (Test-driven development).

## vivo's Future Planning

Next year, vivo plans to extend [APISIX](https://apisix.apache.org/blog/2022/09/13/why-is-apache-apisix-the-best-api-gateway/) as a traffic gateway into an API gateway, utilizing its advantages of [rate limiting](https://apisix.apache.org/docs/apisix/plugins/limit-count/#attributes), [authentication](https://apisix.apache.org/docs/ingress-controller/tutorials/mtls/), circuit breaking, etc. Considering combining APISIX with DPDK-NGINX, vivo will also cultivate technical personnel and join the community establishment. Furthermore, it will consolidate the basic skills to lay a good foundation, building traffic and service governance.

Welcome to learn more about Apache APISIX.

You can contact us at [https://api7.ai/contact](https://api7.ai/contact).


How does the famous mobile manufacturer vivo adopt APISIX as its API gateway and adjust it to fit its businesses?

How Does vivo Integrate with APISIX


## Background

In recent years, many well-known online conferencing software came into being with the development of online meetings and remote working. The typical one, Zoom, has become a popular meeting tool for home office, online teaching, and social scenarios. It has 350 million daily meeting participants and nearly 470,000 paying business customers using the platform. Furthermore, recent data shows that the amount of minutes used for meeting facilitation has reached more than 3.3 trillion per year.

However, like other SaaS and internet companies, Zoom has encountered technical challenges as its business rapidly expands.

- **A significant number of microservices**: Due to Zoom’s rapid growth of business and teams, over 100 backend services need to be delivered. However, it is hard to manage a large number of microservices efficiently.

- **Various deployment environments**: SaaS enterprises often encounter scenarios where customers must deploy on dedicated clouds, private clouds, and multi-clouds. Zoom’s business services are all over the world, so there are technical challenges with a large number of hybrid cloud environments.

- **Complex infrastructure**: Medium-and-large-sized internet enterprises generally have dedicated infrastructure teams responsible for API gateways, configuration centers, key management, logs, monitoring alarms, databases, etc.
As that Zoom’s managing R&D teams are distributed globally, it is a great challenge to integrate these complex middleware and infrastructure into the continuous delivery pipeline.

**The above challenges are not a simple additive relationship but a multiplicative relationship. In other words, the actual situation is far more complex.** However, the open-source tools that Zoom used previously can no longer meet Zoom’s current requirements. That’s why a reliable continuous delivery pipeline is so important.

Below are detailed reasons why Zoom did not continue to choose these previous open-source tools.

| Helm/Kustomize  | Terraform/Pulumi  | KubeVela + Crossplane  |
| ------------ | ------------ | ------------ |
| Unable to connect systems except Kubernetes  | Difficult to use Terraform in the Kubernetes and the middleware layers unless developing extra Providers  | Too brand-new technology stack with fast update, not safe and stable enough to be adopted in the production environment  |
| Difficult to centrally control the platform in the Git sub-repository mode  | Increase the cost of learning another configuration management language: hcl  | Kubevela’s definition of Trait + Cuelang templates and its programming expression capabilities are not enough to cover various non-cloud-native middleware platforms outside the Kubernetes system  |
| Difficult to maintain and debug due to Helm Chart Template’s complex logic  | Mono Repo centralized control mode is unsuitable for bulk teams  | /  |
| Not powerful parameter system to manage dynamic parameters in a large number of environments  | /  | /  |

Due to the limitation of the above open-source tools, Zoom finally investigated some mainstream solutions for building the underlying continuous delivery pipeline.

After a few rounds of comparison and verification, Zoom ultimately chose APISIX Ingress Controller to support its new continuous delivery pipeline.

## Apache APISIX Ingress Controller

### What is Apache APISIX Ingress Controller?

[Apache APISIX Ingress Controller](https://github.com/apache/apisix-ingress-controller) is a cloud-native ingress controller that uses Apache APISIX as the data plane to carry traffic and extends the functionality of Kubernetes utilizing CRD (CustomResourceDefinition). It is responsible for interacting with the Kubernetes API server, applying for role-based access control (RBAC), monitoring changes, implementing object conversion within the Ingress controller, comparing the changes, and then synchronizing to Apache APISIX. Moreover, it can support custom resources, including APISIX Route, APISIX Upstream, and other native ingresses of Kubernetes, to control external traffic to access services deployed in Kubernetes.

Below is the [Timing Diagram of APISIX Ingress Controller](https://apisix.apache.org/docs/ingress-controller/design/#timing-diagram).

![Timing Diagram of APISIX Ingress](https://static.apiseven.com/2022/10/27/635a5613af56a.jpeg)

## Why Zoom Chooses APISIX Ingress Controller?

According to the above background, here comes the question: what kind of continuous delivery pipeline does Zoom want to build, and how does it adopt APISIX Ingress to establish its underlying continuous delivery pipeline?

Zoom previously used NGINX as an API gateway. However, with the rapid development of business and the increase in microservices, the current NGINX solution’s limitations are becoming more apparent.

Different teams must maintain NGINX separately, and thousands of lines of NGINX configuration files make maintenance difficult. Besides, NGINX cannot quickly scale when deploying many lines on the cloud server. About NGINX’s reloading mode, you can check [this blog](https://api7.ai/blog/how-nginx-reload-work). Zoom’s businesses began to evolve towards Ingress Controller.

The API Gateway team researched some open-source solutions. After simulating migration and analysis of actual business configuration in NGINX and a large number of benchmark tests of performance and plugin ecological comparison, Zoom finally chooses the Apache Software Foundation’s APISIX Ingress Controller project, exploring a more advanced Cloud-Native gateway.

Taking its business scenarios into consideration, Zoom attached more emphasis to below two parts, which can be satisfied by APISIX Ingress.

- **Data Security**: Zoom takes customer privacy and service security seriously; thus, mTLS authentication and verification are widely used in online meeting rooms and phone calls. Nevertheless, many similar API gateways can only provide such a service in their enterprise version, while APISIX Ingress provides great feasibility and convenience to achieve this goal.

- **Service Stability**: Zoom’s backend services require Multi-AZ (Multi-Availability Zones) deployments for high availability across different regions. Generally, it puts businesses in other DC. If an error occurs in the original DC, the client traffic needs to be transformed to another one, in which APISIX Ingress can successfully meet the requirement.

### Features of Apache APISIX Ingress Controller

Taking Apache APISIX as its data plane to carry business traffic, Apache APISIX Ingress Controller inherits the following advantages from Apache APISIX.

- **High Performance and Stability**: As a high-performance dynamic open-source API gateway, Apache APISIX is resilient and stable in its performance, being used in many [large-scale traffic scenarios of enterprises](https://apisix.apache.org/blog/tags/case-studies/).

- **Active Community**: As a top-level and the most active open-source API gateway project, Apache APISIX possesses an active community and has maintained an excellent growth rate from the first day.

- **Rich Ecosystem**: Apache APISIX supports L7 protocols, including HTTP(S), HTTP2, Dubbo, IoT protocol MQTT, etc. In addition, APISIX supports L4 protocols such as TCP/UDP. In addition, full support of ARM64 is supported in Apache APISIX 3.0.

- **Multiple Plugins**: There are almost 100 plugins released from the APISIX official that users can use by simple drag. The hot reloading and dynamic orchestration of plugins provide users with great convenience.

In addition, the APISIX Ingress Controller also has the following unique advantages:

- **Good Compatibility**: APISIX Ingress Controller supports multiple versions of ingress resources and can be compatible with different Kubernetes versions.

- **Dynamic Update**: There is no need to reload service when modifying routes, certificates, and other configurations, ensuring the smooth operation of the business.

- **Flexible Scaling**: Since the APISIX Ingress Controller adopts a structure that separates the control plane and the data plane, the data plane cluster of Apache APISIX can be expanded independently without scaling the APISIX Ingress Controller.

![APISIX Ingress Controller Architecture](https://static.apiseven.com/2022/10/27/635a56148940c.jpeg)

- **Friendly to Operation and Maintenance**: Under the current architecture, users can choose to deploy the data plane APISIX cluster in the Kubernetes cluster or the physical bare-metal machine environment according to the actual situation. Furthermore, due to the architectural separation of the APISIX ingress, the APISIX on the data plane carries the traffic while the APISIX ingress controller is the control plane component. Therefore, the APISIX Ingress Controller’s failure will not impact business traffic.

After the gateway selection was completed, the API Gateway team faced a new challenge: how to migrate the original API gateway configuration of hundreds of services to APISIX Ingress? Zoom’s Infrastructure team was developing the continuous delivery pipeline, which can significantly reduce the migration cost of converting from nginx.conf and other Ingress configurations to APISIX Ingress.

## The Process and Functions of Building the Continuous Delivery Pipeline

### Zoom’s Continuous Delivery Pipeline

The continuous delivery pipeline is an end-to-end application delivery system that implements a single model for declaring all application delivery requirements and arranging and executing all continuous delivery steps in one line.

![Zoom's Continuous Delivery Pipeline](https://static.apiseven.com/2022/10/27/635a56149b270.jpg)

There are six parts in the continuous delivery pipeline:

1. **Prepare**: prepare preset resources, including infrastructure, middleware, cloud service resources, etc.;

2. **Config**: prepare the configuration files and keys required by the application;

3. **Deploy**: use K8s for deployment in cloud-native scenarios (including information about containers, container images, parameters, versions, and instances);

4. **Access**: create a Kubernetes Service and automatically configure the routing rules of the Apache APISIX Ingress if the deployment requires external access;

5. **Observe**: Perform observability-related configurations, such as monitoring, alarming, logging, and tracing analysis;

6. **Scale**: Declare KEDA (Kubernetes Event-driven Autoscaling)’s dynamic scaling rules through monitoring metrics.

### How to Adopt APISIX Ingress in the Pipeline

![Zoom's Pipeline Architecture Overview](https://static.apiseven.com/2022/10/27/635a5616254ed.jpg)

**Project Management**

Project Managers pay more attention to R&D iterations and how to better manage the iterative release progress and personnel efficiency on the timeline corresponding to the iterations. Zoom uses a GitOps workflow internally to build API Gateway configuration into the application delivery model. Under this model, the definition of APISIX routing rules is integrated with “Deploy” and other links, handing over the change control to GitHub. With the creation and merging of GitHub branches, the pipeline realizes consistent timelines between application and gateway configuration release and rollback.

```yaml
# Code Snippet of Zoom's CD Pipeline in Git Repositary
deploy:
  type: Deployment
  replicas: ~{ replicas, 2 }
  version: "latest"
  containers:
    - name: my-app
      image: "busybox"
      command: "echo 'Demo' && sleep 99d"
access:
  - protocol: https
    host: my-domain.my-org.com
    cert: my-tls-cert
    apisix:
      routes:
        http:
          - name: my-api
            authentication:
              # ......
            match:
              paths:
                - /my-api/*
```

This way simplifies release management into workflow management within GitHub and solves the problem of the time lag between upstream and downstream systems matching changes when multiple iterations process simultaneously.

**Application Development**

Developers mainly focus on APIs’ routing and authentication capabilities, which are strongly related to business services and should be filled in by developers to achieve the automatic effect. In addition, they pay attention to developing and implementing business functions and the upper-level business, hoping to build an out-of-the-box infrastructure.

From the above-mentioned codes, we can see developers only need to define `Authentication` and `Match` in this continuous delivery pipeline. They do not need to know the underlying logic:

- First, translate it into Kubernetes Deployment and Service.
- Then, call the platform API to verify the path’s correctness.
- Last, translate it into ApisixRoute objects.

Integrating APISIX’s configuration and the continuous delivery pipeline workflow provides developers with a more labor-saving way.

**Environment Management**

Complex environmental management and control requirements often occur in toB scenarios, and delivery issues in some private cloud, dedicated cloud, and hybrid cloud scenarios must be considered.

Some configurations of APISIX ingress were implemented to meet the requirement of shielding environmental differences. In this way, system managers can comprehensively control the differences in some heterogeneous environments. All services deployed to the environment are effective, avoiding the cognitive burden and special deployment operations caused by environmental differences to application and ops developers. For example, Zoom can use customized configuration to disable tracing for some environments, convert ApisixRoute objects to native Ingress objects and NGINX Ingress Annotation, and use different images to pull secrets.

Furthermore, multi-tenant isolation is required when multiple lines of business are using the APISIX environment. **APISIX Ingress provides an Annotation selector** that enables different ApisixRoute objects to be picked up by other instances of the APISIX Ingress Controller.

**Infrastructure Management**

The API Gateway team need to control all APISIX instances, configure security policies comprehensively, implement multi-availability zones, and so on.

Each plugin of the pipeline provides configuration items for infrastructure engineers. In the plugin `ingress-apisix`, there is a `defaultPlugins` property.

After the API gateway team configures the property, the setting will take effect for all the services, which is suitable for a unified security and risk control strategy.

## Conclusion

APISIX Ingress Controller plays an important role in Zoom’s continuous delivery pipeline, releasing the pressure on project management, application development, environment management, and middleware and infrastructure management. The case of Zoom is worth learning for other companies, and we look forward to APISIX Ingress Controller contributing to the innovation of more companies.

Besides, Apache APISIX Ingress officially released V1.5 in August 2022, which unifies the proposal of all resource API Versions and upgrades all CRD API Versions to V2. At the same time, it supports most of the Gateway API resources. Apache APISIX Ingress has enabled Ingress resources to use arbitrary APISIX plugins by adding a new Annotation “k8s.apisix.apache.org/plugin-config-name” to Ingress resources. In this way, it will significantly increase the ease of use of the APISIX Ingress Controller and reduce the cost for users to migrate from other Ingress Controllers to APISIX Ingress Controller.

For further information, please check [Apache APISIX Ingress V1.5](https://apisix.apache.org/blog/2022/08/05/apisix-ingress-1.5rc1-release/).


Zoom integrates APISIX Ingress Controller in its continuous delivery pipeline to realize the overall management of project, application, environment and infrastructure management.

Apache APISIX vs NGINX:
Dynamic Configuration,
Cluster Management

Looking for the best API Gateway?

Go for Apache APISIX

Dynamic

Rich Features

Cluster Management

Apache APISIX vs NGINX

Case Studies

API Gateway for Web3: How APISIX Empowers Hyperchain

How Does vivo Integrate with APISIX

How Does Zoom Use APISIX Ingress in Its Continuous Delivery Pipeline?

Ready to manage your APIs?

Apache APISIX vs NGINX:.css-zzhrfm{background-image:linear-gradient(273.1deg, var(--chakra-colors-brand-400) 1.57%, #4790FF 118.62%);color:transparent;-webkit-background-clip:text;background-clip:text;}Dynamic Configuration,Cluster Management

Looking for the best API Gateway?

Go for Apache APISIX

Dynamic

Rich Features

Cluster Management

Apache APISIX vs NGINX

Case Studies

API Gateway for Web3: How APISIX Empowers Hyperchain

How Does vivo Integrate with APISIX

How Does Zoom Use APISIX Ingress in Its Continuous Delivery Pipeline?

Ready to manage your APIs?

Apache APISIX vs NGINX:
Dynamic Configuration,
Cluster Management