---
title: "AI Gateway Comparison 2026: AISIX vs LiteLLM vs Portkey & More - API7.ai"
description: "Compare AI gateways — AISIX, LiteLLM, Portkey, Envoy, TrueFoundry, Vercel, and Cloudflare — across provider coverage, multi-model routing, guardrails, token budgets, semantic caching, and governance. Find the AI gateway that fits how your team runs LLM and AI-agent traffic."
url: "https://api7.ai/ai-gateway-comparison"
---

# AI Gateway Comparison 2026: Choose the Right Platform for LLM Traffic

Compare options including AISIX, LiteLLM, Portkey, Envoy AI Gateway, TrueFoundry, Vercel AI Gateway, and Cloudflare AI Gateway across provider coverage, multi-model routing, guardrails, token budgets, caching, and governance — and find the platform that fits how your team runs LLM and AI-agent traffic.

Educational Guide

## Define Your Requirements before Comparing AI Gateways

Map routing, rate limiting, security, observability, cost, and governance requirements before evaluating products.

[Read the AI Gateway Guide](https://api7.ai/learning-center/ai-gateway-guide)

Introduction

At a Glance

Gateway Profiles

Our Take

In-Depth Comparisons

FAQ

## How to Choose an AI Gateway in 2026

An AI gateway sits in front of your LLM providers and gives teams centralized control over how applications send traffic to AI models in real time. From that control point, teams can route requests, protect sensitive data, enforce compliance policies, cache responses, and observe LLM and AI-agent workloads. These platforms add multi-model routing, token-based budgets, prompt and response guardrails, and per-model cost tracking on top of the reliability controls of a traditional API gateway.

This page consolidates the leading platforms so you can evaluate their AI services side by side before reading a single head-to-head. The fastest way to narrow the field is to decide how and where you want to run the data plane.

### Three Operating Models

- 1\. Open-source and self-hosted

  AISIX OSS, LiteLLM, the Portkey gateway, and Envoy AI Gateway run on infrastructure you control. You get control over the gateway deployment in exchange for operating it yourself. Calls to external model providers and telemetry destinations still need a separate data-residency review.

- 2\. Managed / SaaS services

  Vercel AI Gateway and Cloudflare AI Gateway are fully managed: one endpoint reaches many models with built-in analytics. Their caching models differ. Onboarding is fast and there is nothing to operate, but traffic routes through the vendor's network rather than your own.

- 3\. Products with a separate control plane

  AISIX Enterprise, AISIX Cloud, TrueFoundry, and Portkey pair a governance control plane with a data plane you can run yourself — with different hosting options for the control plane. The AISIX gateway stays in your infrastructure. AISIX OSS runs directly without a control plane; AISIX Enterprise uses a customer-hosted control plane, and AISIX Cloud uses an API7-hosted control plane. Model provider calls still follow the configured upstream destination.

## AI Gateways at a Glance

AISIX details verified: October 21, 2026 (1.6.0 documentation). Competitor entries last reviewed: October 16, 2026. Product capabilities and availability can change; check current vendor documentation before making a decision.

✓ documented support · ✗ documented absence · Ent. enterprise tier. AISIX cells name the applicable product. Not verified means the reviewed sources did not establish support, not that the feature is impossible.

Provider integrations and model selection are separate dimensions; support varies by endpoint and release. Provider prompt caching reuses input prefixes, whereas response caching reuses outputs. Dashboard SSO and SCIM describe administrative identity, not request authentication. Beta, pre-release, and plan restrictions are shown explicitly. Check the official sources in each profile for the selected deployment.

AISIX Enterprise and AISIX Cloud support SCIM provisioning for API keys and usage attribution, not dashboard SSO. Enterprise trusts the organization's IdP for every email address; Cloud requires a verified organization domain for immediate account creation or attachment. Other Cloud addresses wait to join through an invitation or SSO. Directory sync itself does not sign members in. Creating a SCIM user whose email already belongs to that organization returns 409. Dashboard-managed members appear in SCIM lists, but SCIM cannot modify, deactivate, or delete them. [SCIM directory sync documentation](https://docs.api7.ai/ai-gateway/cloud/scim-directory-sync)

| Capability                     | AISIX                                                     | LiteLLM                       | Portkey                        | Envoy / Agent Router  | TrueFoundry                  | Vercel                 | Cloudflare                        |
| ------------------------------ | --------------------------------------------------------- | ----------------------------- | ------------------------------ | --------------------- | ---------------------------- | ---------------------- | --------------------------------- |
| Foundation                     |                                                           |                               |                                |                       |                              |                        |                                   |
| License                        | OSS: Apache-2.0; Enterprise / Cloud: commercial           | MIT core                      | MIT gateway                    | Apache-2.0            | Proprietary                  | Proprietary            | Proprietary                       |
| Hosting model                  | Gateway in your infrastructure; control-plane host varies | Self-host                     | Self-host + hosted             | Self-host             | Self-host or SaaS            | SaaS                   | SaaS                              |
| Can run in your network        | ✓                                                         | ✓                             | ✓                              | ✓                     | ✓                            | ✗                      | ✗                                 |
| Routing & traffic              |                                                           |                               |                                |                       |                              |                        |                                   |
| Provider integrations          | Provider adapters                                         | Provider adapters             | Provider adapters              | Provider integrations | Provider integrations        | Managed catalog + BYOK | Provider endpoints                |
| Model selection                | Model aliases                                             | Model groups                  | Model catalog                  | Model-name mapping    | Virtual models               | Model catalog          | Model IDs + dynamic routes (Beta) |
| Multi-model routing & fallback | ✓                                                         | ✓                             | ✓                              | ✓                     | ✓                            | ✓                      | Dynamic routing (Beta)            |
| Caching model                  | Exact + semantic                                          | Exact + semantic              | Semantic (OSS 2.0 pre-release) | Provider prompt cache | Exact + semantic             | Provider prompt cache  | Exact response cache              |
| MCP gateway                    | ✓                                                         | ✓                             | ✓                              | ✓                     | ✓                            | Separate integration   | Separate product                  |
| Governance & cost              |                                                           |                               |                                |                       |                              |                        |                                   |
| Budget / usage controls        | OSS limits; Enterprise / Cloud spend budgets              | Spend budgets                 | Budgets (OSS 2.0 pre-release)  | Token quotas          | Spend + token limits         | Spend budgets          | Spend limits (Beta)               |
| Prompt & response guardrails   | ✓                                                         | ✓                             | ✓                              | Not verified          | ✓                            | Not verified           | Beta                              |
| Dashboard SSO                  | Enterprise / Cloud SAML; all plans                        | Free up to 5 users; then Ent. | Ent.                           | Not verified          | OIDC / SAML; confirm plan    | Pro add-on / Ent.      | All plans                         |
| SCIM provisioning              | Enterprise / Cloud members (not SSO)                      | Ent.                          | Ent.                           | Not verified          | Supported IdPs; confirm plan | Ent. platform          | Ent. platform                     |

AISIX

Foundation

License

OSS: Apache-2.0; Enterprise / Cloud: commercial

Hosting model

Gateway in your infrastructure; control-plane host varies

Can run in your network

✓

Routing & traffic

Provider integrations

Provider adapters

Model selection

Model aliases

Multi-model routing & fallback

✓

Caching model

Exact + semantic

MCP gateway

✓

Governance & cost

Budget / usage controls

OSS limits; Enterprise / Cloud spend budgets

Prompt & response guardrails

✓

Dashboard SSO

Enterprise / Cloud SAML; all plans

SCIM provisioning

Enterprise / Cloud members (not SSO)

LiteLLM

Foundation

License

MIT core

Hosting model

Self-host

Can run in your network

✓

Routing & traffic

Provider integrations

Provider adapters

Model selection

Model groups

Multi-model routing & fallback

✓

Caching model

Exact + semantic

MCP gateway

✓

Governance & cost

Budget / usage controls

Spend budgets

Prompt & response guardrails

✓

Dashboard SSO

Free up to 5 users; then Ent.

SCIM provisioning

Ent.

Portkey

Foundation

License

MIT gateway

Hosting model

Self-host + hosted

Can run in your network

✓

Routing & traffic

Provider integrations

Provider adapters

Model selection

Model catalog

Multi-model routing & fallback

✓

Caching model

Semantic (OSS 2.0 pre-release)

MCP gateway

✓

Governance & cost

Budget / usage controls

Budgets (OSS 2.0 pre-release)

Prompt & response guardrails

✓

Dashboard SSO

Ent.

SCIM provisioning

Ent.

Envoy / Agent Router

Foundation

License

Apache-2.0

Hosting model

Self-host

Can run in your network

✓

Routing & traffic

Provider integrations

Provider integrations

Model selection

Model-name mapping

Multi-model routing & fallback

✓

Caching model

Provider prompt cache

MCP gateway

✓

Governance & cost

Budget / usage controls

Token quotas

Prompt & response guardrails

Not verified

Dashboard SSO

Not verified

SCIM provisioning

Not verified

TrueFoundry

Foundation

License

Proprietary

Hosting model

Self-host or SaaS

Can run in your network

✓

Routing & traffic

Provider integrations

Provider integrations

Model selection

Virtual models

Multi-model routing & fallback

✓

Caching model

Exact + semantic

MCP gateway

✓

Governance & cost

Budget / usage controls

Spend + token limits

Prompt & response guardrails

✓

Dashboard SSO

OIDC / SAML; confirm plan

SCIM provisioning

Supported IdPs; confirm plan

Vercel

Foundation

License

Proprietary

Hosting model

SaaS

Can run in your network

✗

Routing & traffic

Provider integrations

Managed catalog + BYOK

Model selection

Model catalog

Multi-model routing & fallback

✓

Caching model

Provider prompt cache

MCP gateway

Separate integration

Governance & cost

Budget / usage controls

Spend budgets

Prompt & response guardrails

Not verified

Dashboard SSO

Pro add-on / Ent.

SCIM provisioning

Ent. platform

Cloudflare

Foundation

License

Proprietary

Hosting model

SaaS

Can run in your network

✗

Routing & traffic

Provider integrations

Provider endpoints

Model selection

Model IDs + dynamic routes (Beta)

Multi-model routing & fallback

Dynamic routing (Beta)

Caching model

Exact response cache

MCP gateway

Separate product

Governance & cost

Budget / usage controls

Spend limits (Beta)

Prompt & response guardrails

Beta

Dashboard SSO

All plans

SCIM provisioning

Ent. platform

## Gateway Profiles

### AISIX

AI gateway suite from the team behind Apache APISIX. Run AISIX OSS directly, add a customer-hosted control plane with AISIX Enterprise, or use the API7-hosted control plane with AISIX Cloud.

The AISIX gateway is a separate Rust-native runtime from Apache APISIX and API7 Gateway. It runs in your infrastructure with all three AISIX products, and live model requests do not pass through the Enterprise or Cloud control plane. AISIX OSS provides model aliases, routing, request and token limits, guardrails, semantic response caching, MCP tool access control, and usage telemetry. AISIX Enterprise and AISIX Cloud add centralized usage reporting, spend budgets, and SCIM directory sync for members. Both products support dashboard SAML SSO on every plan; SCIM provisions members and does not sign them in. API7 reports about 28,300 req/s on 4 vCPUs in a no-policy, mock-upstream benchmark; test real provider traffic and enabled policies before sizing production.

[AISIX benchmark methodology and test conditions](https://docs.api7.ai/ai-gateway/deployment/performance-and-sizing)

Best for: Teams that want a dedicated AI gateway in their own infrastructure, with optional centralized management. Data residency also depends on the chosen model providers, logs, and telemetry destinations.

- [Enterprise and Cloud SAML SSO availability](https://docs.api7.ai/ai-gateway/cloud/single-sign-on)
- [Provider and endpoint support](https://docs.api7.ai/ai-gateway/endpoints/overview)
- [AISIX documentation](https://docs.api7.ai/ai-gateway)
- [OSS, Enterprise, and Cloud rate limits](https://docs.api7.ai/ai-gateway/traffic-controls/rate-limits)
- [Enterprise and Cloud spend budgets](https://docs.api7.ai/ai-gateway/traffic-controls/budgets)

[Explore AISIX AI Gateway product details](https://api7.ai/ai-gateway)[Verify AISIX deployment options in the docs](https://docs.api7.ai/ai-gateway/cloud/control-plane)

Use these AISIX 1.6.0 documentation checks to evaluate the capabilities above. Apply the same workload and acceptance criteria to other candidates before choosing a gateway.

#### MCP governance: verify access and management scope

AISIX OSS configures caller tool grants in resources.yaml. AISIX Enterprise and AISIX Cloud add environment and team access policies, plus server review and approval workflows.

- [Verify caller tool access](https://docs.api7.ai/ai-gateway/mcp-gateway/tool-access-control): check that permitted tools can be listed and called, while a denied call is rejected before reaching its upstream server.
- [Check environment and team policies](https://docs.api7.ai/ai-gateway/mcp-gateway/access-policies) and the [server approval workflow](https://docs.api7.ai/ai-gateway/mcp-gateway/server-review) in Enterprise or Cloud. Confirm that a broader caller-key grant cannot override a restrictive policy, and an unapproved registration is not published to gateways.
- [Test MCP traffic controls](https://docs.api7.ai/ai-gateway/mcp-gateway/traffic-controls): use request and concurrency limits for tool-call volume. Token limits alone do not cap MCP calls; spend budgets require Enterprise or Cloud.

#### Production readiness: verify the complete request path

These runtime checks apply to AISIX OSS, Enterprise, and Cloud. Size the gateway with representative provider traffic, streaming, and enabled policies, rather than treating a mock-upstream benchmark as a production capacity guarantee.

- [Follow the production checklist](https://docs.api7.ai/ai-gateway/deployment/production): test replica failure, configuration recovery, and graceful shutdown. Use shared Redis when rate-limit counters or cached responses must be consistent across gateway instances.
- [Validate readiness with real requests](https://docs.api7.ai/ai-gateway/deployment/health-checks): liveness alone does not prove provider access. Send an authorized, provider-backed request through each required endpoint family, confirm invalid credentials are rejected, and locate the request in telemetry.
- [Review network and security boundaries](https://docs.api7.ai/ai-gateway/deployment/network-and-security): keep metrics/status and configuration-store surfaces private, protect provider credentials, and verify the destinations of model requests, logs, and telemetry for the chosen deployment.

### LiteLLM

MIT-licensed, Python-based proxy that calls 100+ LLM providers behind an OpenAI-compatible API.

LiteLLM focuses on breadth and portability. The open-source proxy supports semantic caching, per-key/team spend budgets, and an MCP gateway, and self-hosts via Docker, Helm, or Terraform. SSO is free for up to 5 users; larger SSO deployments and SCIM require an enterprise license.

Best for: Teams that want a self-hosted proxy with broad provider integrations; verify each required endpoint and enterprise feature separately.

- [Proxy capabilities](https://docs.litellm.ai/docs/simple_proxy)
- [SSO and enterprise scope](https://docs.litellm.ai/docs/enterprise)

### Portkey

MIT-licensed gateway with self-hosted and managed deployment options; check the gateway release and control-plane arrangement separately.

Portkey supports provider routing, guardrails, and MCP traffic. Its Production Gateway announcement adds semantic caching, budget limits, and a model catalog to open source, while the gateway repository labels 2.0 as pre-release. Do not assume those features are present in an older release. SSO and SCIM remain enterprise capabilities. Confirm which gateway and control-plane components run in your environment and which provider, log, and telemetry destinations they use.

Best for: Teams evaluating managed governance or a self-hosted gateway who can validate the selected release and deployment scope.

- [Gateway license and release status](https://github.com/Portkey-AI/gateway)
- [Production Gateway capabilities](https://portkey.ai/blog/gateway-2-0/)

### Envoy AI Gateway / Agent Router

The Apache-2.0 project formerly named Envoy AI Gateway is now Agent Router, built on Envoy Proxy with local and Kubernetes deployment paths.

Agent Router documents provider integrations, model-name mapping, fallback, token quotas, MCP routing, and observability. Token quotas are not the same as a currency-denominated spend dashboard. Provider prompt caching is documented; a semantic response cache and an administrative SSO/SCIM console were not verified in the reviewed documentation. Envoy Gateway security policies can integrate request authentication and authorization, which is a different scope from dashboard SSO.

Best for: Platform teams that want an Envoy-based data plane and can operate its policies and supporting services.

- [Project and deployment overview](https://theagentrouter.ai/docs/)
- [Provider and caching support](https://theagentrouter.ai/docs/capabilities/llm-integrations/)
- [Token quota scope](https://theagentrouter.ai/docs/capabilities/traffic/quota-policy/)
- [Security policy integration](https://theagentrouter.ai/docs/capabilities/security/)

### TrueFoundry AI Gateway

Proprietary AI gateway with SaaS, hybrid, and self-hosted deployment options.

TrueFoundry documents provider integrations, virtual models, routing and fallback, exact and semantic caching, MCP access controls, spend limits, and guardrails. Its platform supports OIDC/SAML sign-in and SCIM provisioning for supported identity providers; confirm the commercial plan and deployment scope. A private gateway does not make external model calls private: review provider, storage, log, and telemetry paths separately.

Best for: Enterprises that want a governance plane they can self-host and can verify the full provider and data path against compliance requirements.

- [Gateway capabilities and deployment options](https://www.truefoundry.com/docs/ai-gateway/intro-to-llm-gateway)
- [SSO and SCIM scope](https://www.truefoundry.com/docs/platform/sso/overview)

### Vercel AI Gateway

Proprietary managed service that exposes hundreds of models through a single Vercel-hosted endpoint.

Vercel AI Gateway provides a managed model catalog, provider selection, fallback, BYOK, and spend budgets. Its documented automatic caching reuses provider-side prompt prefixes, not semantically similar responses. Requests pass through Vercel. Platform SAML SSO is available on Enterprise or as a Pro add-on; Directory Sync requires Enterprise. These identity features manage the Vercel team, not model-request authorization. MCP integrations belong to the application/platform layer; native gateway prompt/response guardrails were not verified in the reviewed documentation.

Best for: Teams building on Vercel that want zero-ops model access and spend visibility without running infrastructure.

- [Gateway capabilities](https://vercel.com/docs/ai-gateway)
- [Prompt caching scope](https://vercel.com/docs/ai-gateway/models-and-providers/automatic-caching)
- [SAML SSO plans](https://vercel.com/docs/saml)
- [Directory Sync plans](https://vercel.com/docs/directory-sync)

### Cloudflare AI Gateway

Proprietary managed service that proxies LLM traffic through Cloudflare's edge network.

Cloudflare AI Gateway supports provider endpoints, exact-match response caching, and analytics. Dynamic routing, spend limits, and prompt/response Guardrails are documented as Beta. Its cache serves identical requests, not semantically similar ones. Traffic passes through Cloudflare; MCP hosting is a separate product scope. Cloudflare dashboard SSO is available on all plans, while dashboard SCIM provisioning requires Enterprise. Neither should be confused with authentication on a model request.

Best for: Teams already on Cloudflare that want a fast, edge-cached gateway with built-in content moderation.

- [Gateway feature scope](https://developers.cloudflare.com/ai-gateway/features/)
- [Response caching scope](https://developers.cloudflare.com/ai-gateway/features/caching/)
- [Dashboard SSO availability](https://developers.cloudflare.com/fundamentals/manage-members/dashboard-sso/)
- [Dashboard SCIM requirements](https://developers.cloudflare.com/fundamentals/account/account-security/scim-setup/)

## Our Take

AISIX fits teams that want a dedicated gateway for AI traffic in their own infrastructure. AISIX OSS provides the open-source Rust runtime; AISIX Enterprise and AISIX Cloud add customer-hosted and API7-hosted control planes respectively. Model aliases, routing, guardrails, caching, and MCP tool access address AI-specific traffic; external provider calls and telemetry still need a separate data-residency review.

The other gateways fit different operating models: evaluate LiteLLM for broad provider integrations, Envoy AI Gateway (now Agent Router) for an Envoy estate, and vendor-hosted gateways when their traffic path and operating model meet your requirements. AISIX OSS provides gateway request and token limits; AISIX Enterprise and AISIX Cloud provide centralized spend budgets, SAML SSO, and SCIM member provisioning. Verify the product, version, and deployment option behind each feature before selecting a platform.

## In-Depth Comparisons

Read the full, side-by-side breakdowns:

- [AISIX vs LiteLLM](https://api7.ai/aisix-vs-litellm)
- [Portkey vs LiteLLM](https://api7.ai/portkey-vs-litellm)
- [LiteLLM vs TrueFoundry](https://api7.ai/litellm-vs-truefoundry)
- [LiteLLM vs Vercel AI Gateway](https://api7.ai/litellm-vs-vercel-ai-gateway)
- [LiteLLM vs Cloudflare AI Gateway](https://api7.ai/litellm-vs-cloudflare-ai-gateway)
- [Portkey vs Vercel AI Gateway](https://api7.ai/portkey-vs-vercel-ai-gateway)
- [Portkey vs Cloudflare AI Gateway](https://api7.ai/portkey-vs-cloudflare-ai-gateway)
- [Vercel vs Cloudflare AI Gateway](https://api7.ai/vercel-vs-cloudflare-ai-gateway)
- [OpenRouter vs LiteLLM](https://api7.ai/openrouter-vs-litellm)
- [OpenRouter vs Portkey](https://api7.ai/openrouter-vs-portkey)
- [OpenRouter vs Vercel AI Gateway](https://api7.ai/openrouter-vs-vercel-ai-gateway)
- [OpenRouter vs Cloudflare AI Gateway](https://api7.ai/openrouter-vs-cloudflare-ai-gateway)
- [Kong AI Gateway vs LiteLLM](https://api7.ai/kong-ai-gateway-vs-litellm)
- [Kong AI Gateway vs Portkey](https://api7.ai/kong-ai-gateway-vs-portkey)
- [LiteLLM Alternatives](https://api7.ai/litellm-alternative)

## Frequently Asked Questions

### What is an AI gateway?

An AI gateway is a specialized gateway that sits in front of LLM providers (OpenAI, Anthropic, AWS Bedrock, Google Vertex, Azure OpenAI, and others) and gives teams one place to route, govern, secure, cache, and observe LLM and AI-agent traffic. On top of the routing and reliability features of a traditional API gateway, it adds AI-specific controls such as multi-model and semantic routing, token-based rate limits and budgets, prompt and response guardrails, and per-model cost tracking.

### What is the difference between an AI gateway and an API gateway?

An API gateway manages traffic to general backend services with routing, authentication, rate limiting, and protocol handling. An AI gateway applies a similar entry-point pattern to model and agent traffic, adding model aliases, provider routing, token-aware controls, and optional caching and guardrails. Apache APISIX and API7 Gateway can add AI features to existing API routes through plugins. AISIX is a separate product suite built around a Rust-native gateway for AI traffic; it does not share their data plane.

### Which AI gateways are open source, and which are managed services?

AISIX OSS (Apache-2.0), the LiteLLM core (MIT), the Portkey gateway (MIT), and Envoy AI Gateway, now Agent Router (Apache-2.0), can be self-hosted. AISIX Enterprise adds a customer-hosted control plane; AISIX Cloud adds an API7-hosted control plane. Open-source licensing does not include every commercial control-plane feature. Vercel and Cloudflare AI Gateway are managed services. TrueFoundry offers proprietary self-hosted and SaaS options; Portkey also offers managed and enterprise deployments. Verify the selected release, control-plane placement, providers, logs, and telemetry before making a data-residency claim.

### Should I use an open-source or a hosted (SaaS) AI gateway?

Open-source AI gateways such as AISIX OSS, LiteLLM, and Envoy AI Gateway can run inside your own environment, giving you control over the gateway deployment. That alone does not keep every prompt inside your network: calls to external model providers, telemetry exports, and hosted control planes need their own residency review. Vendor-hosted services operate the gateway for you but route traffic through their infrastructure. Compare the full request and data path, not only the hosting label.

### Which AI gateway is best for self-hosting and data residency?

If the gateway must run in your own VPC, evaluate AISIX OSS, LiteLLM, Envoy AI Gateway, and self-hosted options from other vendors against your deployment requirements. Every AISIX product keeps the gateway in your infrastructure; AISIX Enterprise and AISIX Cloud add customer-hosted and API7-hosted control planes respectively. A private gateway does not make an external model provider private. Confirm the destinations of provider calls, prompts, responses, logs, telemetry, and cached data before making a residency claim.

### What should I look for when comparing AI gateways?

Check the required provider, endpoint, and model features rather than comparing provider counts with model counts. Then evaluate routing and fallback, token limits versus monetary budgets, guardrails, and the difference between response caching and provider prompt caching. Assess dashboard SSO separately from SCIM provisioning and request authentication. Finally, verify the release, paid tier, full deployment/data path, and observability required for your workload.
