AISIX AI Gateway
for LLMs and AI agents.
AISIX is the open-source AI gateway for LLMs and agents. Put one OpenAI-compatible API in front of every model — ship AI features to production in days, then scale the ones that move your numbers.
Apache-2.0 core · 100+ providers · built in Rust · Docs · See pricing
from openai import OpenAI client = OpenAI( base_url="https://YOUR_AISIX_GATEWAY/v1", api_key="YOUR_CALLER_API_KEY", ) resp = client.chat.completions.create( model="YOUR_MODEL_ALIAS", messages=[{"role": "user", "content": "Hello"}], )
Don't let AI plumbing slow your roadmap
The first prompt is easy. The 50th service, the 10th model, and the first incident in front of customers are not. A gateway is where that complexity goes — so your team keeps shipping.
Every team reinvents the plumbing
- A different SDK per provider — every integration is a project
- No shared limits — one runaway loop takes everyone down
- Spend you can't see until the invoice lands
- Provider keys copied across every service
- No guardrails or audit trail — afraid to ship AI to customers
One platform to ship AI faster
- One OpenAI-compatible API — integrate once, never rewrite
- Org-wide limits & budgets — scale without surprises
- Every call logged — see what's working
- Keys centralized & encrypted — ship without leaking secrets
- Guardrails, retries & failover — put AI in front of customers
Switch models without switching code
Point your existing OpenAI SDK at AISIX and reach every major provider — no rewrites, no per-vendor SDKs, no lock-in.
- OpenAI-compatible chat, streaming (SSE), embeddings, rerank, audio & images — plus an Anthropic-compatible
/v1/messages. - Virtual / routing models let one model name fan out across providers and regions.
- Cloud providers — AWS Bedrock, Azure OpenAI & GCP Vertex AI — alongside direct APIs.
One policy point between applications and models
Applications and agents send model requests through one AISIX gateway using a caller key and model alias. AISIX authenticates the caller, applies traffic policy, records telemetry, and routes each approved request to the configured model provider.
Evaluate capabilities against published behavior
Use the official documentation to verify the request path, configuration boundary, and operational limits behind each capability.
| Production requirement | AISIX capability | Documentation evidence |
|---|---|---|
| One client API | OpenAI-compatible endpoints with caller keys and model aliases | OpenAI-compatible API |
| Provider resilience | Multi-target routing, eligible retries, and pre-stream failover | Routing and failover |
| Shared traffic policy | Request, token, and concurrency limits scoped by policy | Rate limit policies |
| Input and output controls | Monitor or block guardrails before and after provider calls | Guardrail behavior |
| Operational evidence | Gateway metrics for requests, tokens, limits, models, and policy | Metrics reference |
Continue with the AISIX documentation, inspect the open-source gateway, or review the supported endpoint matrix.
Your whole AI gateway, in one console
Requests, latency, errors, spend, and model health — live across every environment, with no dashboards to build yourself.
Everything you need to take AI to production
From first prototype to production scale — one platform, every control built in.
Ship and switch models without rewrites. Alias any model to any provider — OpenAI, Anthropic, Bedrock, Vertex, Groq — with weighted load balancing, automatic failover, health checks, and semantic & cost-optimal routing.
Scale to production traffic without surprises. Sliding-window request (RPM / RPD) and token (TPM / TPD) limits plus concurrency caps — scoped per key, team, or provider account, and synchronized across replicas through Redis.
Put AI in front of customers, confidently. Pre-input and post-output checks — keyword / regex blocklists, PII redaction (Presidio), prompt-injection and moderation (Lakera, OpenAI Moderation, Llama-Guard), and per-key model access control.
Invest in what's working. Month-to-date spend across every environment, key, and member — with per-key and org budgets, alerts at 75 / 90 / 100%, per-model cost tracking, and hard-stop caps.
A real gateway — not a Python proxy
Most AI gateways are a scripting-language shim in your hot path. AISIX is gateway infrastructure, rebuilt for AI.
Built in Rust
Sub-millisecond overhead, no garbage-collection pauses, and a stateless data plane that scales horizontally with your traffic.
Open source, Apache-2.0
Self-host the full gateway as a single binary — free, forever, no lock-in. Or run it fully managed on AISIX Cloud.
From the Apache APISIX team
Five-plus years of production gateway engineering from API7.ai, the creators of Apache APISIX — now rebuilt for LLMs and agents.
Cloud or your own VPC
Start on the managed cloud in minutes, or deploy the control plane and data planes entirely inside your own infrastructure.
Ship AI that grows your business
OpenAI-compatible — point your SDK at AISIX and start shipping. Free to start, no credit card.