Compare options including AISIX, LiteLLM, Portkey, Envoy AI Gateway, TrueFoundry, Vercel AI Gateway, and Cloudflare AI Gateway across provider coverage, multi-model routing, guardrails, token budgets, caching, and governance — and find the platform that fits how your team runs LLM and AI-agent traffic.
Educational Guide
Map routing, rate limiting, security, observability, cost, and governance requirements before evaluating products.
Introduction
At a Glance
Gateway Profiles
Our Take
In-Depth Comparisons
FAQ
An AI gateway sits in front of your LLM providers and gives teams centralized control over how applications send traffic to AI models in real time. From that control point, teams can route requests, protect sensitive data, enforce compliance policies, cache responses, and observe LLM and AI-agent workloads. These platforms add multi-model routing, token-based budgets, prompt and response guardrails, and per-model cost tracking on top of the reliability controls of a traditional API gateway.
This page consolidates the leading platforms so you can evaluate their AI services side by side before reading a single head-to-head. The fastest way to narrow the field is to decide how and where you want to run the data plane.
1. Open-source and self-hosted
AISIX OSS, LiteLLM, the Portkey gateway, and Envoy AI Gateway run on infrastructure you control. You get control over the gateway deployment in exchange for operating it yourself. Calls to external model providers and telemetry destinations still need a separate data-residency review.
2. Managed / SaaS services
Vercel AI Gateway and Cloudflare AI Gateway are fully managed: one endpoint reaches many models with built-in analytics. Their caching models differ. Onboarding is fast and there is nothing to operate, but traffic routes through the vendor's network rather than your own.
3. Products with a separate control plane
AISIX Enterprise, AISIX Cloud, TrueFoundry, and Portkey pair a governance control plane with a data plane you can run yourself — with different hosting options for the control plane. The AISIX gateway stays in your infrastructure. AISIX OSS runs directly without a control plane; AISIX Enterprise uses a customer-hosted control plane, and AISIX Cloud uses an API7-hosted control plane. Model provider calls still follow the configured upstream destination.
AISIX details verified: October 21, 2026 (1.6.0 documentation). Competitor entries last reviewed: October 16, 2026. Product capabilities and availability can change; check current vendor documentation before making a decision.
✓ documented support · ✗ documented absence · Ent. enterprise tier. AISIX cells name the applicable product. Not verified means the reviewed sources did not establish support, not that the feature is impossible.
Provider integrations and model selection are separate dimensions; support varies by endpoint and release. Provider prompt caching reuses input prefixes, whereas response caching reuses outputs. Dashboard SSO and SCIM describe administrative identity, not request authentication. Beta, pre-release, and plan restrictions are shown explicitly. Check the official sources in each profile for the selected deployment.
AISIX Enterprise and AISIX Cloud support SCIM provisioning for API keys and usage attribution, not dashboard SSO. Enterprise trusts the organization's IdP for every email address; Cloud requires a verified organization domain for immediate account creation or attachment. Other Cloud addresses wait to join through an invitation or SSO. Directory sync itself does not sign members in. Creating a SCIM user whose email already belongs to that organization returns 409. Dashboard-managed members appear in SCIM lists, but SCIM cannot modify, deactivate, or delete them. SCIM directory sync documentation
| Capability | AISIX | LiteLLM | Portkey | Envoy / Agent Router | TrueFoundry | Vercel | Cloudflare |
|---|---|---|---|---|---|---|---|
| Foundation | |||||||
| License | OSS: Apache-2.0; Enterprise / Cloud: commercial | MIT core | MIT gateway | Apache-2.0 | Proprietary | Proprietary | Proprietary |
| Hosting model | Gateway in your infrastructure; control-plane host varies | Self-host | Self-host + hosted | Self-host | Self-host or SaaS | SaaS | SaaS |
| Can run in your network | ✓ | ✓ | ✓ | ✓ | ✓ | ✗ | ✗ |
| Routing & traffic | |||||||
| Provider integrations | Provider adapters | Provider adapters | Provider adapters | Provider integrations | Provider integrations | Managed catalog + BYOK | Provider endpoints |
| Model selection | Model aliases | Model groups | Model catalog | Model-name mapping | Virtual models | Model catalog | Model IDs + dynamic routes (Beta) |
| Multi-model routing & fallback | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | Dynamic routing (Beta) |
| Caching model | Exact + semantic | Exact + semantic | Semantic (OSS 2.0 pre-release) | Provider prompt cache | Exact + semantic | Provider prompt cache | Exact response cache |
| MCP gateway | ✓ | ✓ | ✓ | ✓ | ✓ | Separate integration | Separate product |
| Governance & cost | |||||||
| Budget / usage controls | OSS limits; Enterprise / Cloud spend budgets | Spend budgets | Budgets (OSS 2.0 pre-release) | Token quotas | Spend + token limits | Spend budgets | Spend limits (Beta) |
| Prompt & response guardrails | ✓ | ✓ | ✓ | Not verified | ✓ | Not verified | Beta |
| Dashboard SSO | Enterprise / Cloud SAML; all plans | Free up to 5 users; then Ent. | Ent. | Not verified | OIDC / SAML; confirm plan | Pro add-on / Ent. | All plans |
| SCIM provisioning | Enterprise / Cloud members (not SSO) | Ent. | Ent. | Not verified | Supported IdPs; confirm plan | Ent. platform | Ent. platform |
AISIX
Foundation
License
OSS: Apache-2.0; Enterprise / Cloud: commercial
Hosting model
Gateway in your infrastructure; control-plane host varies
Can run in your network
✓
Routing & traffic
Provider integrations
Provider adapters
Model selection
Model aliases
Multi-model routing & fallback
✓
Caching model
Exact + semantic
MCP gateway
✓
Governance & cost
Budget / usage controls
OSS limits; Enterprise / Cloud spend budgets
Prompt & response guardrails
✓
Dashboard SSO
Enterprise / Cloud SAML; all plans
SCIM provisioning
Enterprise / Cloud members (not SSO)
LiteLLM
Foundation
License
MIT core
Hosting model
Self-host
Can run in your network
✓
Routing & traffic
Provider integrations
Provider adapters
Model selection
Model groups
Multi-model routing & fallback
✓
Caching model
Exact + semantic
MCP gateway
✓
Governance & cost
Budget / usage controls
Spend budgets
Prompt & response guardrails
✓
Dashboard SSO
Free up to 5 users; then Ent.
SCIM provisioning
Ent.
Portkey
Foundation
License
MIT gateway
Hosting model
Self-host + hosted
Can run in your network
✓
Routing & traffic
Provider integrations
Provider adapters
Model selection
Model catalog
Multi-model routing & fallback
✓
Caching model
Semantic (OSS 2.0 pre-release)
MCP gateway
✓
Governance & cost
Budget / usage controls
Budgets (OSS 2.0 pre-release)
Prompt & response guardrails
✓
Dashboard SSO
Ent.
SCIM provisioning
Ent.
Envoy / Agent Router
Foundation
License
Apache-2.0
Hosting model
Self-host
Can run in your network
✓
Routing & traffic
Provider integrations
Provider integrations
Model selection
Model-name mapping
Multi-model routing & fallback
✓
Caching model
Provider prompt cache
MCP gateway
✓
Governance & cost
Budget / usage controls
Token quotas
Prompt & response guardrails
Not verified
Dashboard SSO
Not verified
SCIM provisioning
Not verified
TrueFoundry
Foundation
License
Proprietary
Hosting model
Self-host or SaaS
Can run in your network
✓
Routing & traffic
Provider integrations
Provider integrations
Model selection
Virtual models
Multi-model routing & fallback
✓
Caching model
Exact + semantic
MCP gateway
✓
Governance & cost
Budget / usage controls
Spend + token limits
Prompt & response guardrails
✓
Dashboard SSO
OIDC / SAML; confirm plan
SCIM provisioning
Supported IdPs; confirm plan
Vercel
Foundation
License
Proprietary
Hosting model
SaaS
Can run in your network
✗
Routing & traffic
Provider integrations
Managed catalog + BYOK
Model selection
Model catalog
Multi-model routing & fallback
✓
Caching model
Provider prompt cache
MCP gateway
Separate integration
Governance & cost
Budget / usage controls
Spend budgets
Prompt & response guardrails
Not verified
Dashboard SSO
Pro add-on / Ent.
SCIM provisioning
Ent. platform
Cloudflare
Foundation
License
Proprietary
Hosting model
SaaS
Can run in your network
✗
Routing & traffic
Provider integrations
Provider endpoints
Model selection
Model IDs + dynamic routes (Beta)
Multi-model routing & fallback
Dynamic routing (Beta)
Caching model
Exact response cache
MCP gateway
Separate product
Governance & cost
Budget / usage controls
Spend limits (Beta)
Prompt & response guardrails
Beta
Dashboard SSO
All plans
SCIM provisioning
Ent. platform
AI gateway suite from the team behind Apache APISIX. Run AISIX OSS directly, add a customer-hosted control plane with AISIX Enterprise, or use the API7-hosted control plane with AISIX Cloud.
The AISIX gateway is a separate Rust-native runtime from Apache APISIX and API7 Gateway. It runs in your infrastructure with all three AISIX products, and live model requests do not pass through the Enterprise or Cloud control plane. AISIX OSS provides model aliases, routing, request and token limits, guardrails, semantic response caching, MCP tool access control, and usage telemetry. AISIX Enterprise and AISIX Cloud add centralized usage reporting, spend budgets, and SCIM directory sync for members. Both products support dashboard SAML SSO on every plan; SCIM provisions members and does not sign them in. API7 reports about 28,300 req/s on 4 vCPUs in a no-policy, mock-upstream benchmark; test real provider traffic and enabled policies before sizing production.
AISIX benchmark methodology and test conditionsBest for: Teams that want a dedicated AI gateway in their own infrastructure, with optional centralized management. Data residency also depends on the chosen model providers, logs, and telemetry destinations.
Use these AISIX 1.6.0 documentation checks to evaluate the capabilities above. Apply the same workload and acceptance criteria to other candidates before choosing a gateway.
AISIX OSS configures caller tool grants in resources.yaml. AISIX Enterprise and AISIX Cloud add environment and team access policies, plus server review and approval workflows.
These runtime checks apply to AISIX OSS, Enterprise, and Cloud. Size the gateway with representative provider traffic, streaming, and enabled policies, rather than treating a mock-upstream benchmark as a production capacity guarantee.
MIT-licensed, Python-based proxy that calls 100+ LLM providers behind an OpenAI-compatible API.
LiteLLM focuses on breadth and portability. The open-source proxy supports semantic caching, per-key/team spend budgets, and an MCP gateway, and self-hosts via Docker, Helm, or Terraform. SSO is free for up to 5 users; larger SSO deployments and SCIM require an enterprise license.
Best for: Teams that want a self-hosted proxy with broad provider integrations; verify each required endpoint and enterprise feature separately.
MIT-licensed gateway with self-hosted and managed deployment options; check the gateway release and control-plane arrangement separately.
Portkey supports provider routing, guardrails, and MCP traffic. Its Production Gateway announcement adds semantic caching, budget limits, and a model catalog to open source, while the gateway repository labels 2.0 as pre-release. Do not assume those features are present in an older release. SSO and SCIM remain enterprise capabilities. Confirm which gateway and control-plane components run in your environment and which provider, log, and telemetry destinations they use.
Best for: Teams evaluating managed governance or a self-hosted gateway who can validate the selected release and deployment scope.
The Apache-2.0 project formerly named Envoy AI Gateway is now Agent Router, built on Envoy Proxy with local and Kubernetes deployment paths.
Agent Router documents provider integrations, model-name mapping, fallback, token quotas, MCP routing, and observability. Token quotas are not the same as a currency-denominated spend dashboard. Provider prompt caching is documented; a semantic response cache and an administrative SSO/SCIM console were not verified in the reviewed documentation. Envoy Gateway security policies can integrate request authentication and authorization, which is a different scope from dashboard SSO.
Best for: Platform teams that want an Envoy-based data plane and can operate its policies and supporting services.
Proprietary AI gateway with SaaS, hybrid, and self-hosted deployment options.
TrueFoundry documents provider integrations, virtual models, routing and fallback, exact and semantic caching, MCP access controls, spend limits, and guardrails. Its platform supports OIDC/SAML sign-in and SCIM provisioning for supported identity providers; confirm the commercial plan and deployment scope. A private gateway does not make external model calls private: review provider, storage, log, and telemetry paths separately.
Best for: Enterprises that want a governance plane they can self-host and can verify the full provider and data path against compliance requirements.
Proprietary managed service that exposes hundreds of models through a single Vercel-hosted endpoint.
Vercel AI Gateway provides a managed model catalog, provider selection, fallback, BYOK, and spend budgets. Its documented automatic caching reuses provider-side prompt prefixes, not semantically similar responses. Requests pass through Vercel. Platform SAML SSO is available on Enterprise or as a Pro add-on; Directory Sync requires Enterprise. These identity features manage the Vercel team, not model-request authorization. MCP integrations belong to the application/platform layer; native gateway prompt/response guardrails were not verified in the reviewed documentation.
Best for: Teams building on Vercel that want zero-ops model access and spend visibility without running infrastructure.
Proprietary managed service that proxies LLM traffic through Cloudflare's edge network.
Cloudflare AI Gateway supports provider endpoints, exact-match response caching, and analytics. Dynamic routing, spend limits, and prompt/response Guardrails are documented as Beta. Its cache serves identical requests, not semantically similar ones. Traffic passes through Cloudflare; MCP hosting is a separate product scope. Cloudflare dashboard SSO is available on all plans, while dashboard SCIM provisioning requires Enterprise. Neither should be confused with authentication on a model request.
Best for: Teams already on Cloudflare that want a fast, edge-cached gateway with built-in content moderation.
AISIX fits teams that want a dedicated gateway for AI traffic in their own infrastructure. AISIX OSS provides the open-source Rust runtime; AISIX Enterprise and AISIX Cloud add customer-hosted and API7-hosted control planes respectively. Model aliases, routing, guardrails, caching, and MCP tool access address AI-specific traffic; external provider calls and telemetry still need a separate data-residency review.
The other gateways fit different operating models: evaluate LiteLLM for broad provider integrations, Envoy AI Gateway (now Agent Router) for an Envoy estate, and vendor-hosted gateways when their traffic path and operating model meet your requirements. AISIX OSS provides gateway request and token limits; AISIX Enterprise and AISIX Cloud provide centralized spend budgets, SAML SSO, and SCIM member provisioning. Verify the product, version, and deployment option behind each feature before selecting a platform.
Read the full, side-by-side breakdowns: