Elevate AI Model Development with Machine Learning
“As a mature and out-of-the-box API gateway, APISIX helps us with high development efficiency. From starting from scratch to passing POC, it only took us one week.”
Diao Shoujun
,
Senior Developer at Tencent Cloud
Take only 1 week
from 0 to finish POC
Save > 80% time and costs
with an out-of-box API gateway
Challenges
Tencent Cloud TI Platform is a one-stop machine learning service platform designed for AI engineers. It empowers AI development throughout the process, from data preprocessing to model building, training, evaluation, and service.
Tencent Cloud TI Platform encountered several technical challenges due to its rapid business growth.
-
Dispersed platform-level services across different components or services
-
Coupled platform-level services with the backend services reduced development efficiency
-
Difficulty in handling high concurrency scenarios and performance bottlenecks
-
Insufficient monitoring capabilities blocked operational efficiency and system observability
Solution
To address its technical bottlenecks, Tencent Cloud TI Platform uses APISIX as a unified gateway module to receive and process all incoming traffic, simplifying the architecture and improving efficiency. It leverages APISIX's flexible plugin system to configure features like routing, access control, and monitoring. Additionally, APISIX's support for diverse protocols enhances the platform's adaptability and scalability.
Competitive Edge of Our Solution
Ultra-High Performance
Apache APISIX supports a single-core Queries Per Second (QPS) of 23,000 with an average delay of only 0.6 milliseconds, which enables Tencent Cloud TI Platform to handle high concurrency scenarios.
High Availability
Tencent Cloud TI Platform chose APISIX because it uses etcd as its configuration center, which is more conducive to the lightweight and high availability of the system.
Developer-Friendly
Taking OpenResty and Lua as the technical basis, APISIX is easy to use and friendly to developers for customization. Developers with no prior experience in OpenResty and Lua can quickly onboard and leverage APISIX, empowering their teams to rapidly address customized development requirements for business projects within a short timeframe.
Flexible Routing
The outstanding routing flexibility and efficient routing matching algorithm of APISIX are key factors for Tencent Cloud TI Platform to choose it. APISIX has an optimized routing matching algorithm, which takes effect within 1 ms. It also supports using any NGINX variable as a routing condition, enabling developers to set up routing rules to meet the needs of different application scenarios.
Results
Improved Development Efficiency
Developers can focus on developing core features by consolidating platform-level services into the API gateway and decoupling with the backend services.
Comprehensive Observability
Comprehensive request-level monitoring was implemented, including detailed logging and monitoring of various core metrics such as QPS, latency, and error rates.
Excellent API Performance
APISIX empowered Tencent Cloud TI Platform to allocate resources dynamically based on actual traffic, easily meeting the scenarios of direct API calls by multiple users.
More Customer Stories
Establish Streaming Platform with 513 Million MAU
After comparing some mainstream gateway products, we find that the response latency of Apache APISIX is more than 50% lower than other gateways, and Apache APISIX can still run stably when the CPU reaches more than 70%.
Build All-in-One Digital Assets Platform
API7 commercial support is great for security control and comprehensive support. We chose API7 also because it is marvelous at supporting k8s.
Build a Diverse Content Social Media Platform
To fix the problem of inconvenient service management and costly R&D investment, we introduced APISIX as our API gateway. APISIX's dynamic performance with etcd aligns most with Beeto's product requirements.