API7 Enterprise v3.5.0: Multi-Upstream Configuration

Upstream: Precise Navigation of Requests

To better understand the concept of upstream, let's use a metaphor: Imagine a bustling airport where people are constantly coming and going. Travelers, like API requests, arrive in droves, eager to find their boarding gates to embark on their journeys. In the world of API7 Gateway, upstreams are akin to these boarding gates. However, they are not physical locations but logical groupings. These logical groupings clearly define where incoming API requests should be sent.

An upstream might represent a single backend service, like a dedicated boarding gate, where all related requests can accurately reach their destination. It could also represent a pool of identical services for load balancing, similar to multiple functionally identical boarding gates that efficiently distribute passenger traffic. Alternatively, it could point to a service registry where the corresponding backends dynamically change to adapt to evolving business needs. In most cases, a single upstream within a service can connect multiple routes within the service, enabling efficient request flow.

At its core, upstreams cleverly create an abstraction layer between routes and actual backend services. This abstraction is highly significant. It greatly simplifies configuration management, making the cumbersome configuration process clear and orderly. It also seamlessly enables load balancing, ensuring that the load on each backend service is reasonable and improving overall system performance.

Multi-Upstream Configuration: A New Era of Traffic Management

Multi-upstream configuration is a standout feature of API7 Enterprise. It breaks traditional limitations, allowing a service to freely configure multiple upstream services anytime, anywhere. This is not limited to canary scenarios but further extends to various advanced traffic management scenarios. Combined with plugins, it can flexibly configure routing rules to precisely distribute requests to different upstream services.

Using Scenarios

1. Canary Deployment: During canary testing, create a new upstream pointing to a new version of the same service. Then, cleverly route a portion of the traffic to the new version of the microservice. This allows you to verify the correctness of new features on a small scale, effectively reducing the risk, much like a small-scale rehearsal before the official performance.

2. Blue-Green Deployment: First, smoothly switch all traffic to the new version. After confirming that the new version is running stably, safely take the old version offline. This achieves zero-downtime deployment, ensuring continuous service availability and making users almost unaware of the system update.

3. A/B Testing: Divide traffic into multiple groups and route them to different upstream services. This allows you to intuitively compare the effects of different solutions, providing strong data support for decision-making, much like comparing different product solutions in market research.

4. Failover: When the primary upstream service unfortunately fails, the system can quickly switch traffic to the backup upstream service. This acts as an "insurance" for the service, ensuring its availability and maintaining a good user experience.

5. Multi-Cluster Management: In multi-data center or multi-cloud environments, multi-upstream configuration can reasonably distribute traffic to different clusters, such as regular and VIP clusters. This not only improves system availability and disaster recovery capabilities but also achieves load balancing and resource isolation, ensuring stable system operation in complex environments.

However, it is important to note that while multi-upstream configuration brings powerful functionality, it also significantly increases complexity and management difficulty. This requires meticulous planning of the business and a thorough understanding of API gateway configuration.

Tips for Using Multi-Upstream Configuration Effectively

1. Gradual Introduction: When introducing multi-upstream configuration, start with simple scenarios, such as basic traffic splitting tests. Accumulate experience before gradually expanding to more complex scenarios to avoid being overwhelmed by complexity from the outset.

2. Thorough Testing: Comprehensive testing before publishing is essential. Simulate various possible scenarios to ensure the correctness of the configuration, much like conducting strict quality checks before launching a new product.

3. Monitoring and Alerts: Establish a robust monitoring and alerting mechanism to continuously monitor system operation. If any anomalies are detected, timely alerts and measures can be taken to address issues, safeguarding the stable operation of the system.

Powerful Tool: The traffic-split Plugin

To achieve the powerful functionality of multi-upstream configuration, the traffic-split plugin is indispensable. This plugin is highly functional, dynamically distributing traffic to different upstream services based on predefined conditions and weights.

In terms of condition matching, it can formulate complex traffic distribution rules based on information such as the request's URL, Header, Cookie, and even external factors like time and date. For example, it can direct requests from specific VIP users to the upstream service corresponding to the VIP cluster or route a portion of traffic to the upstream service corresponding to the new version during a specific time period. In terms of weight configuration, by setting different weights, it can precisely control the distribution ratio of traffic among various upstream services. For instance, during a canary release, it can direct 90% of the traffic to the upstream corresponding to the production environment and 10% to the upstream corresponding to the testing environment.

By effectively utilizing the multi-upstream configuration feature and the traffic-split plugin, you can greatly enhance the flexibility and reliability of the API gateway. This lays a solid foundation for the successful implementation of microservice architecture, allowing us to steadily move forward on the digital path and tackle various complex business challenges.

Conclusion

Serving as a precise navigation system for requests, upstreams efficiently route API requests to target services through logical groupings. API7 Enterprise's multi-upstream configuration further enhances the flexibility of traffic management, supporting advanced scenarios such as canary deployments, blue-green deployments, A/B testing, failover, and multi-cluster management. With the aid of the traffic-split plugin, traffic can be dynamically allocated based on predefined conditions and weights, ensuring the system's high performance and stability.