AISIX AI Gateway
for LLMs and AI agents.
AISIX is an open-source AI gateway for LLMs and agents. Connect applications to supported models through one OpenAI-compatible API. Run the gateway in your infrastructure and add AISIX Cloud when you need centralized management.
Apache-2.0 core · 100+ providers · built in Rust · Docs · See pricing
from openai import OpenAI client = OpenAI( base_url="https://YOUR_AISIX_GATEWAY/v1", api_key="YOUR_CALLER_API_KEY", ) resp = client.chat.completions.create( model="YOUR_MODEL_ALIAS", messages=[{"role": "user", "content": "Hello"}], )
Don't let AI plumbing slow your roadmap
The first prompt is easy. The 50th service, the 10th model, and the first incident in front of customers are not. A gateway is where that complexity goes — so your team keeps shipping.
Every team reinvents the plumbing
- A different SDK per provider — every integration is a project
- No shared limits — one runaway loop takes everyone down
- Spend you can't see until the invoice lands
- Provider keys copied across every service
- No guardrails or audit trail — afraid to ship AI to customers
One platform to ship AI faster
- One OpenAI-compatible API — verify required endpoints and provider features
- Shared limits and AISIX Cloud budgets — control traffic and spend
- Every call logged — see what's working
- Provider keys at the gateway boundary — AISIX Cloud encrypts managed keys
- Guardrails, retries & failover — put AI in front of customers
Route supported models through one gateway
Point your OpenAI SDK at AISIX for supported endpoints and providers. Validate tools, streaming, and provider-specific features before switching a production workflow.
- OpenAI-compatible chat, streaming (SSE), embeddings, rerank, audio & images — plus an Anthropic-compatible
/v1/messages. - Virtual / routing models let one model name fan out across providers and regions.
- Cloud providers — AWS Bedrock, Azure OpenAI & GCP Vertex AI — alongside direct APIs.
One policy point between applications and models
Applications and agents send model requests through one AISIX gateway using a caller key and model alias. AISIX authenticates the caller, applies traffic policy, records telemetry, and routes each approved request to the configured model provider.
Evaluate capabilities against published behavior
Use the official documentation to verify the request path, configuration boundary, and operational limits behind each capability.
| Production requirement | AISIX capability | Documentation evidence |
|---|---|---|
| One client API | OpenAI-compatible endpoints with caller keys and model aliases | OpenAI-compatible API |
| Provider resilience | Multi-target routing, eligible retries, and pre-stream failover | Routing and failover |
| Shared traffic policy | Request, token, and concurrency limits scoped by policy | Rate limit policies |
| Input and output controls | Monitor or block guardrails before and after provider calls | Guardrail behavior |
| Operational evidence | Gateway metrics for requests, tokens, limits, models, and policy | Metrics reference |
Continue with the AISIX documentation, inspect the open-source gateway, or review the supported endpoint matrix.
Your whole AI gateway, in one console
AISIX Cloud brings request, latency, error, usage, and cost views together for the gateways you operate.
Everything you need to take AI to production
Choose the routing, traffic controls, and management options your deployment needs.
Configure aliases for supported upstream models. Use weighted load balancing, failover, health checks, and semantic & cost-optimal routing where supported. Validate each client workflow before changing its provider.
Scale to production traffic without surprises. Sliding-window request (RPM / RPD) and token (TPM / TPD) limits plus concurrency caps — scoped per key, team, or provider account, and synchronized across replicas through Redis.
Put AI in front of customers, confidently. Pre-input and post-output checks — keyword / regex blocklists, PII redaction (Presidio), prompt-injection and moderation (Lakera, OpenAI Moderation, Llama-Guard), and per-key model access control.
Track usage and spend with AISIX Cloud. Review model costs across environments and set budgets and alerts for centrally managed gateways.
A real gateway — not a Python proxy
AISIX is a Rust gateway built for model and agent traffic, with routing, policy, and telemetry on the request path.
Built in Rust
A compiled Rust gateway with no garbage-collection pauses. Published tests report sub-millisecond p50 proxy overhead under their stated conditions; measure your own provider traffic and enabled policies.
Open source, Apache-2.0
Run the open-source gateway in your infrastructure without a control plane, or connect your gateway to AISIX Cloud for centralized management.
From the Apache APISIX team
Five-plus years of production gateway engineering from API7.ai, the creators of Apache APISIX — now rebuilt for LLMs and agents.
Control plane hosting options
Run the gateway in your own environment. Host the AISIX Cloud control plane yourself, or have API7 host the control plane.
Related Guide
AI Gateway Guide
Plan deployment, routing, security, observability, cost controls, and operations for production AI traffic.
Ship AI that grows your business
Use an OpenAI-compatible API with a gateway you run in your infrastructure. Deploy open-source AISIX yourself or contact API7 to evaluate AISIX Cloud.