Hybrid and Multi-Cloud API Management: Challenges and Choices

1.Multi-Cloud & Hybrid Cloud

Microservices has become a popular software architecture method where organizations break down their products into individual microservices based on their understanding of their own business and using scientific methods such as domain-driven design. The organizational structure is also adjusted to align with the microservice architecture, following Conway's Law, to support the iterative development of the business.

In the past, organizations built their own data centers to deploy their products. These data centers could be located in leased or purchased facilities, and organizations were responsible for managing the complex infrastructure, such as switches, servers, storage, racks and other hardware. They also need to deal with the problems caused by power outages, high rack temperatures, server crashes, etc. Dealing with such issues often requires a lot of human and financial resources without achieving good results. As a result, the SLA (service level agreement) of the organizations' own products often fails to meet the promise to customers, resulting in compensation.

With the rise of the cloud, people have become increasingly accustomed to deploying their businesses on public clouds. Public clouds help users abstract the details of hardware infrastructure, allowing engineers to focus on deploying, maintaining, and developing their business. However, in addition to the convenience provided by the cloud, its rise and use also bring other issues for users:

• Lock-in: When a product from a cloud provider is deeply integrated into a business, moving the business to another cloud or leaving the cloud altogether becomes quite challenging. This is particularly common for PaaS or SaaS products that have a strong binding with the business. Unfortunately, this often happens when a business is in a period of rapid growth and decision-makers overlook the binding effect of using cloud products.
• Availability issues: The principle of diversification applies here, which means we do not put all eggs in one basket. During the scaling phase of a business, organizations often prioritize quickly launching products and capturing market share, neglecting the infrastructure. This can result in power outages or cable cuts in a cloud region, negatively impacting the business.

As a result, people are becoming more accustomed to using multiple public clouds or building private clouds in addition to using public clouds to avoid the aforementioned issues. This approach is commonly referred to as "multi-cloud" and "hybrid cloud."

"Multi-cloud" refers to the simultaneous use of multiple public clouds, deploying business mirrorly or heterogeneously on these clouds. It will use standard services as much as possible (to avoid vendor lock-in). Due to the usage of multi-cloud, when one cloud becomes unavailable, the impact on the business can be minimized, and it could ensure business continuity by modifying DNS resolution and enabling a backup cloud.

"Hybrid cloud" refers to organizations that, in addition to using one or more public clouds, also have their own private clouds (or data centers). In this scenario, some services could be deployed on private clouds, while others may be deployed on public clouds. Or all services are deployed on the cloud, and the private cloud is responsible for management and monitoring. In any case, software deployment's flexibility and availability are greatly improved after adopting the "multi-cloud" or "hybrid cloud" model.

However, implementing "multi-cloud" and "hybrid cloud" strategies can make managing cloud-based microservices more complex. One common challenge is API management, as many microservices rely on APIs for communication. Therefore, when deploying microservices, their APIs must be exposed externally to allow connections with external parties and provide services.

2.The Need of API Management under Multi-Cloud & Hybrid Cloud Scenarios

As we know, a good API gateway is crucial for managing microservices APIs. The API gateway enables secure and efficient exposure of microservices APIs. However, when implementing "multi-cloud" and "hybrid cloud" strategies, the need for an API gateway goes beyond its basic functionality. Specifically, considerations such as:

Whether to support multi-cluster management

As previously mentioned, when implementing "multi-cloud" or "hybrid cloud" strategies, the services deployed in each cloud or private data center may vary significantly. As a result, users may need to deploy separate API gateway clusters in each cloud with different configurations, certificates, and API keys. Suppose the chosen gateway does not support multi-cluster management. In that case, it can cause significant difficulties in managing APIs, especially during periods of business expansion when the number and scale of gateway clusters increase.

In such cases, an API gateway product that supports multi-cluster management can significantly reduce the management costs for administrators.

1. Users have access to a unified console to select the cluster they wish to configure and monitor. They also can bring a gateway cluster online or offline, depending on the current business deployment situation.
2. Users can view the running status of all these API gateway clusters on the console, including common QPS, latency, gateway cluster CPU and memory usage, etc.

Whether to support collaboration

With the rapid development of business, a small number of API gateway administrators may need help managing and maintaining all API gateway clusters on their own. Meanwhile, many gateway configurations, such as adding routes and configuring plugins, can be handled by developers, reducing the need for extensive administrator involvement. Therefore, whether it supports "collaboration" becomes essential for management.

Specifically, administrators can invite other members of the organization to join the management of the API gateway cluster using RBAC (Role-based access control) or other strategies to assign different permissions to team members. For example, setting the role of Organization Admin (able to perform any operation) for the administrator and Service Admin (able to maintain only a few services and routes) for the general developer. This approach ensures the security of operations while implementing collaboration. It also facilitates the timely recovery of accounts or modification of permissions in case of employee turnover or changes in job positions.

Whether to be able to run on any infrastructure

As containerization and container orchestration technologies mature, many microservices are moving from virtual machines to running on Kubernetes. This means that users may use Kubernetes, traditional virtual machines or even physical machines, such as in their private data centers. Suppose the API gateway product chosen by the user is feature-rich and meets business needs but is limited by the underlying infrastructure or lacks mature installation tools. In that case, it can create a challenge for the user, either to give up using it or to carry out additional development to enable the API gateway to run on certain infrastructure. Either way, this will result in additional usage costs.

3. Decisions

As discussed, it becomes extremely vital to choose the right API gateway in the context of "multi-cloud" and "hybrid cloud" scenarios. Some common options are listed, and users should analyze them based on their actual scenarios and future plans.

Different solutions on different clouds

Typically, each public cloud provider offers its own built-in API gateway solution, and users can consider purchasing an API gateway solution for each cloud.

Pros include:

1. Extremely low cost of use, with no need for deployment or maintenance.
2. Usually supports pay-as-you-go; early use may require only minimal costs.

Cons include:

1. The usage of API gateways on different clouds is often incompatible with each other, so completing the same configuration requires different paths.
2. Product features are not symmetrical among different providers. For example, one provider may support mTLS while another does not.
3. In the "hybrid cloud" scenario, choosing another open-source or commercial API gateway may be necessary and deploying it on the user's private cloud.

When using this approach, achieving a consistent user experience across different cloud providers may require product customization, the development of a configuration syncing tool, and the abstraction of underlying details. This may involve additional research, analysis, and development on the part of the user. Users may encounter compatibility issues and limited functionality and could only use a few common features from API gateway products. Additionally, the possibility of synchronization failure must also be taken into account.

Of course, users can also manually repeat the configuration of each API gateway on each cloud (if they can tolerate it).

Open-source API gateway solution

As an alternative to using different solutions across different clouds, users can consider using open-source API gateway solutions such as Apache APISIX, Kong, Tyk, Traefik, etc. This approach allows users to use the same API gateway across all clouds, including private data centers, thus avoiding compatibility issues between different solutions. A mature and robust open-source API gateway can help users solve many problems in different scenarios. Even if it may not cover all scenarios, these API gateways usually allow users to extend them in various ways. For example, Apache APISIX allows users to extend it using languages and technologies such as Go, Java, Python, Lua, WASM, etc.

However, these open-source API gateways usually adopt Open Core open-source strategy, where core features of the product are open-sourced, but enterprise-level management capabilities such as visualized console, multi-cluster management, auditing, SSO (Single Sign-On), etc., are often paid (integrated into their commercial products). Suppose users don't want to pay them. In that case, they can only choose to develop their own management platform (also known as the gateway's control plane) to manage these API gateways, meaning users need to hire a full-time engineer to take on this development work.

Purchase enterprise-level API gateway SaaS service

Suppose open-source API gateways meet the requirements in terms of functionality, but the cost of self-research control is not acceptable. In that case, users can also consider contacting the original manufacturers of these open-source software (such as API7.ai behind Apache APISIX, Kong Inc behind Kong, and Tyk Inc behind Tyk, etc.). These manufacturers often provide different enterprise-level API gateway products and support services. Especially in the scenario of "multi-cloud" and "hybrid cloud," these manufacturers have successively released their SaaS products. The SaaS products of the API gateway often focus on the management side, providing a ready-to-use console without worrying about where the API gateway instance is deployed (of course, some SaaS services also support hosting the API gateway instance, but this often also introduces other problems, such as latency when proxying the API).

SaaS services typically provide a private gateway control panel for each tenant, connecting it to user-deployed (or SaaS-hosted) gateway instances using strategies such as mTLS to ensure data security and privacy, and provide management capabilities. While purchasing a SaaS service may require an investment, it can be more cost-effective than hiring a dedicated engineer to maintain an API gateway and its control panel.

Of course, purchasing SaaS services requires caution. Therefore, before confirming an order, users should evaluate a SaaS service from the following aspects:

1. Can this SaaS service meet the current and future usage needs?
2. Is the SaaS service provider professional and trustworthy? What kind of use cases do they have?
3. Is this SaaS service compliant, compliant with GDPR, and has SOC2 audit reports?
4. Does this SaaS service have clear SLA terms?
5. How is this SaaS service charged? Can users estimate future spending for one year or a few years?

Completely self-developed

Suppose open-source API gateway solutions do not meet the user's actual needs. In that case, users may consider developing their own exclusive API gateway product, which can be time-consuming and resource-intensive, and the gateway's stability is also a major concern. Developing an API gateway is not as difficult as implementing a relational database or browser, but it is not something that can be done overnight; therefore, self-development can be considered "the worst strategy." As a result, it is often only considered a last resort.

4. Conclusion

In a cloud-native environment, managing APIs in "multi-cloud" and "hybrid cloud" scenarios will be a challenge for every organization, even before adopting these strategies. Therefore, while this article presents various options, it's essential to keep in mind that there is no one-size-fits-all solution, and users should select the option that best fits their specific business needs and development goals.

For more information about API gateway, please visit our blogs or contact us.