APISIX Greatly Improved Junrunrenli's R&D Efficiency

Overview

Challenges

1. Too many business systems, CLB (configurable logic block) traffic forwarding, and frequent configuration changes consume a lot of development and maintenance time.
2. High-concurrency business, unstable database when exporting large billions of data, and the problem can only be solved by restarting the service or releasing a new version.
3. Lack of a unified platform to supervise the accessed business data.
4. Large and complex traffic, lacking efficient traffic management strategy.
5. There are tens of millions of API requests per day, a substantial amount of log data, and limited disk load capacity.

Results

1. Apache APISIX provides Junrunrenli with a variety of observability and security protection methods, efficiently supporting tens of millions of visits per day.
2. Greatly improved the delivery efficiency of R&D: generic domain name configuration takes effect in seconds, and the control plane reduces the DevOps’ configuration burden.
3. Reduced the load balancers from 200+ to only 10+, saving a lot of money.

About Junrunrenli Human Resources

Junrunrenli Human Resources is a technology-driven human resources solution service provider, mainly providing one-stop human resources services for the blue-collar employment market.

Established in 2019, Junrunrenli has provided services for 1000+ customers, covering 300W+ blue-collars. In three years, the R&D team has grown from less than 20 people to 250+ and self-developed 15+ human resource service platforms. Besides, Junrunrenli has built an integrated human resource service ecosystem, including SaaS applications for the B-side and C-side applications for the terminal blue-collar group.

The pace of innovation leads to higher requirements for the system architecture. In addition to high availability, concurrency, and performance, Junrunrenli requires elastic scalability.

So, how does Junrunrenli choose an API gateway?

How Does APISIX Stand Out?

Junrunrenli team created a shortlist and subsequently made an investigation of many API gateways. Several of these were subsequently ruled out as they did not fully meet Junrunrenli’s needs. We can take a close look at the below picture of Junrunrenli’s API gateway selection.

The picture above indicates Apache APISIX’s features, while those in red are the points Junrunrenli emphasizes most because

1. Junrunrenli has a lot of internal business systems and a self-built internal ecosystem. Therefore, a robust API gateway is needed to support Junrun’s rapid business changes because Junrunrenli needs to configure and modify routes frequently. APISIX allows Junrunrenli to slice and dice the different methods to control API access.

2. Junrunrenli’s business focuses on doing the same thing at the same time, for example, paying billions of salaries or withdrawing that amount of funds. Hundreds of thousands of blue-collar users clock in, sign contracts, and receive tasks and wages simultaneously. Hence, the concurrent traffic is huge, especially during China’s "Black Friday", which doubles compared to usual.

3. Various systems and personalized needs result in Junrunrenli’s self-development. "APISIX can meet 99% of our needs, but the remaining 1% needs to be developed by ourselves." Yuan, Junrunrenli’s technical expert, said. Everyone knows that Kong and APISIX are developed based on OpenResty NGINX + Lua. There must be an extra cost if Junrunrenli develops plugins with Lua. Fortunately, APISIX’s external plugin supports java programming language to save learning costs. Furthermore, APISIX’s community is particularly active.

Junrunrenli Bolsters Its Systems with APISIX

Below is the overall overview of Junrunrenli's architecture diagram.

Junrunrenli has made below four achievements after using APISIX.

1. Created innovative routes strategies for efficient routes management

Based on Radixtree and etcd, Apache APISIX is capable of high-speed routing matching and fast synchronizing configuration. All these are designed to realize fast performance and ultra-low latency response.

During the peak use period of the systems, the MySQL database could not respond when exporting millions of report data, causing unavailable service, which can only resume regular operation after restarts. What’s more, the case will be intensified if the exporting continues and can only be solved by new releases.

APISIX enables intelligent management of routes by supporting Junrunrenli with routes priority configuration and API emergency shutdown with serverless plugin within several minutes configuration.

Besides, Junrunrenli’s business systems, especially SaaS systems, need to support customer-defined domain name access. Consequently, the Junrunrenli team unified the entry by configuring multiple domain names for the same service. The configurations can be utilized in the overall systems by configuring only once.

2. Separated a PaaS platform for overall security control

Junrunrenli uses the characteristics of the APISIX, such as canary release, security control, and identity recognition, to set the PaaS gateway for the security control of the upper-level business system.

The picture below shows that Junrunrenli has made a two-layer gateway architecture based on APISIX and logically isolated a gateway on it - the PaaS platform.

The user accesses CLB then APISIX forwards the request to the business system. If the function used by the user needs to use the PaaS capability, it will be accessed through the PaaS service gateway. The PaaS platform is a closed area inside k8s, containing many essential services.

3. APISIX’s traffic-split plugin enabled Junrunrenli to automatic traffic management

APISIX’s traffic-split plugin endows Junrunrenli capability to manage the traffic of its core services, such as SSO (Single Sign On), PaaS service, and payroll service.

There are two scenarios:

• Tag Filtering: The testing user traffic can be forwarded to the pre-production services based on tags like header and some other parameters. Then the testing engineers can testify first in the pre-production environment and then in the later production environment.

After passing the verification, the engineers can cut in the traffic based on the weight and then switch all the traffic to the new version after a period of observation.

• Multi-Version Coexistence: There exist many versions of one service. In this way, different business systems can visit different versions. The traffic can be forwarded to the right services by leveraging tags.

4. APISIX’s Kafka plugin accomplished Junrunrenli’s transparent log data monitoring

As can be seen from this picture, APISIX and the pod services are both deployed on k8s, and all backend routes are bound to the same service. APISIX’s Kafka plugin seems to be tailored for collecting log data.

The observation data demonstrates that there are tens of millions of API requests every day, generating 30GB of log data daily, and the total amount of logs reaches over 1TB.

The Skywalking and Sky Agent were also configured in the service startup script on the pod. Combined with the Kafka plugin, the entire call link on the Skywalking and log cloud can be observed according to the requestId and traceId, and the log records and the consuming time of API requests of each link became transparent for log data monitoring.

Benefits after Junrunrenli Using APISIX

1. Supported business development

After using APISIX, Junrunrenli’s system features are more affluent, and the performance is more powerful. It provides a variety of observability and security protection methods for API services, efficiently supporting tens of millions of daily access traffic.

2. Improved R&D delivery efficiency

Before using APISIX, Junrunrenli used to take 10 minutes to configure each DNS resolution, but now it takes only seconds to configure pan-domain names. There are more than 10 systems and over 100 services with many configurations. The developers need to modify the configuration in both CLB and NGINX. APISIX provides a central data plane for configuration modification, significantly reducing the workload of DevOps.

3. Saved cost

Saved a high cost of Load Balancers and reduced 200+ services to only 10+.

Junrunrenli’s Future Plan

• Junrunrenli will customize the java-plugin-runner and integrate sentinel to enable the service of pluggable dynamic rate-limiting and also support daily and monthly traffic control.

• Junrunrenli team is also considering storing the API request log data and then carrying out risk control identification based on the rule engine, pushing alarm information, and quickly intervening in operation and maintenance.

• Junrunrenli will conduct layered governance. Currently, there is only one entry, and more multiple sets of APISIX clusters will be provided later. For example, the open platform can access the internal services of Junrunrenli’s k8s from a separate entrance.

• Further, Junrunrenli will combine the log and sky plugins with full-chain log analysis.

Looking for APISIX Support?

Apache APISIX is an open-source, dynamic, scalable, and high-performance cloud-native API gateway for all your APIs and microservices. Being donated to Apache Software Foundation by API7.ai, APISIX has grown into a top-level open-source Apache project.

Do you want to accelerate your development with confidence like Junrunrenli Human Resources? To maximize APISIX support, you need API7. We provide in-depth support for APISIX and API management solutions based on your needs!

Contact us whenever you want: https://api7.ai/contact.