API Gateways in Microservices Architecture

Microservices let teams deploy and scale services independently, but they also expose clients to a changing set of endpoints, protocols, failure modes, and access policies. An API gateway provides a controlled entry point for north-south API traffic: requests from browsers, mobile apps, partners, and other networks into services.

That role is important but bounded. A gateway can route requests, balance traffic, verify credentials, apply quotas, transform messages, and emit telemetry when the selected product and configuration support those features. It does not automatically make every service secure or highly available, and it should not become a hidden business-workflow engine.

This guide explains what an API gateway does in a microservices architecture, how it differs from adjacent components, and how to deploy it without creating a new single point of failure.

Key Takeaways

• An API gateway is primarily an API traffic and policy layer at a trust boundary.
• Keep object-level authorization and business rules in the services that own the data.
• Response aggregation usually belongs in a backend-for-frontend (BFF) or aggregation service unless the gateway has an explicitly supported, tested feature for that use case.
• Timeouts, retries, rate limits, and caching require API-specific policies; unsafe defaults can amplify failures or expose stale data.
• Deploy multiple stateless data-plane instances and separate their runtime path from configuration management where the product architecture allows it.
• Measure user-visible success and latency, then use gateway, upstream, and trace data to locate failures.

flowchart LR
    Browser[Browser] --> Edge[CDN or external load balancer]
    Mobile[Mobile app] --> Edge
    Partner[Partner client] --> Edge
    Edge --> Gateway[API gateway data plane]
    Gateway --> Catalog[Catalog service]
    Gateway --> Orders[Orders service]
    Gateway --> Identity[Identity provider or auth service]
    Gateway --> BFF[BFF or aggregation service]
    BFF --> Catalog
    BFF --> Orders

What Is an API Gateway in Microservices?

An API gateway is a reverse proxy specialized for publishing and governing APIs. It matches an incoming request to a route, applies configured policies, selects an upstream target, forwards the request, and returns or streams the response.

The gateway normally handles traffic that crosses an application or organizational boundary. Internal service-to-service traffic can also pass through a gateway in selected architectures, but routing every call through one central hop can add latency, operational coupling, and a large failure domain. A service mesh or platform networking layer may be a better fit for uniform east-west traffic policy.

The gateway is not the entire API management lifecycle. A developer portal, API catalog, analytics system, control plane, and monetization service may be separate components even when a commercial platform presents them as one product.

Core Gateway Responsibilities

Route Matching and Upstream Selection

The gateway maps a client-facing host, path, method, header, or other supported attribute to an upstream service. This keeps clients independent of individual service instances and gives operators one place to roll out route changes.

For example, /catalog/* can go to a catalog service while /orders/* goes to an order service. A weighted route can move a controlled share of eligible traffic to a new version. Route priority and overlap must be tested so a broad rule does not capture requests intended for a more specific API.

Upstream selection may use static nodes, DNS, a service registry, or Kubernetes resources, depending on the gateway. Health checks and load-balancing algorithms help avoid unhealthy targets, but they cannot guarantee that the application dependency behind a healthy process is working.

Apache APISIX documents Routes and Upstreams as separate objects. This lets multiple routes reuse an upstream definition when that model suits the deployment.

Authentication and Edge Authorization

A gateway can authenticate a calling application or validate a token before traffic reaches a service. Typical mechanisms include API keys, mTLS, JWT validation, and OAuth 2.0 or OpenID Connect integration.

Authentication at the edge reduces duplicated checks, but the service must still enforce authorization that depends on business data. The gateway may know that a token is valid and has an orders:read scope; the orders service knows whether that subject may view order 1234. Passing a trusted identity context downstream also requires a protected network path and a policy that removes spoofable client headers before setting authoritative ones.

TLS protects data in transit when certificates, protocol versions, trust stores, and hostname validation are configured correctly. If the gateway terminates client TLS, consider whether the gateway-to-service hop also needs TLS or mTLS based on the threat model.

Rate Limiting, Quotas, and Traffic Policy

Rate limiting can protect scarce resources and enforce product contracts. A useful policy identifies the resource, the key being limited, the time model, and the behavior after the limit is exceeded. Per-IP limits alone can group many users behind a shared address or be bypassed by distributed sources. Authenticated consumer, route, and tenant dimensions may be more meaningful when available.

A gateway commonly rejects an over-limit request with 429 Too Many Requests; queueing it is not a safe universal alternative. Unbounded queues consume memory, increase latency, and can process stale work. If work must be asynchronous, accept it through an application workflow designed with bounded queues, expiration, idempotency, and status reporting.

Rate limiting is not complete DDoS protection. Large volumetric attacks should be handled before they consume the gateway's network or compute capacity, using an upstream provider, CDN, or network-layer protection appropriate to the environment.

Timeouts, Retries, and Failure Containment

Gateway timeouts should reflect the API contract and the upstream's behavior. Separate connect, send, and read timeouts where supported. A timeout that is much longer than the user's deadline wastes resources; one that is too short creates false failures.

Retries need strict boundaries. Retry only when the failure is transient and the operation is safe to repeat. A timed-out POST may already have changed state upstream. Use idempotency keys or an application-level deduplication design before retrying such operations, limit the number of attempts, and add jittered backoff outside an interactive request path where appropriate.

Circuit breaking and passive health checks can reduce repeated traffic to a failing target, but settings must be validated against real failure patterns. Aggressive ejection can remove healthy capacity during a temporary network problem.

Protocol Handling and Message Transformation

Gateways can publish HTTP, gRPC, WebSocket, or streaming interfaces when the selected product supports them. Some gateways can translate between protocols or transform headers, paths, query parameters, and bodies. These capabilities are product- and plugin-specific; an API gateway is not inherently a universal protocol converter.

Prefer small, deterministic transformations at the boundary. Large data mappings and business-dependent response composition are easier to test, version, and observe in an application service or BFF.

Observability

The gateway sees every request that crosses its boundary, making it a valuable telemetry source. Record normalized route, response outcome, gateway and upstream duration, policy decisions, and a trace context that can be correlated with services.

Do not put raw paths, user IDs, API keys, tokens, or request IDs in metric labels. They create high cardinality and may expose sensitive data. Logs should be structured, redacted, access-controlled, and retained according to policy. Distributed traces are needed to explain what happened inside the service after the gateway forwarded a request.

What an API Gateway Should Not Own

Object and Business Authorization

The service that owns a resource should decide whether the caller can read or modify that resource. A gateway can enforce coarse route or scope policy, but it typically cannot evaluate ownership, account state, transaction limits, or workflow rules safely without duplicating business data.

Durable Workflow Orchestration

A multi-step transaction needs persisted workflow state, idempotency, timeout and retry policy per activity, compensation behavior, and operator visibility. A general API gateway request path is not the right place for a workflow that must survive process restarts or continue for minutes or days.

Use a workflow engine or application orchestrator for durable processes. Use a BFF or aggregation service for client-specific read composition. The gateway can authenticate, route, limit, and observe calls to those components.

Database Access and Domain Logic

Do not turn route plugins or scripts into a second application layer. Domain validation, database transactions, pricing, entitlements, and data ownership belong in maintainable services with suitable tests and release controls.

API Gateway, Load Balancer, Service Mesh, and BFF

These components overlap, but their primary concerns differ:

An L7 load balancer can match paths and terminate TLS, while a gateway can balance upstream traffic. Choose based on the policy model, lifecycle, protocols, extensibility, and operating responsibility rather than a rigid feature checklist.

A service mesh and gateway can coexist. The gateway handles north-south API concerns, and the mesh handles east-west workload traffic. The exact boundary depends on the platform; avoid applying the same authentication or retry policy twice without understanding the interaction.

Deployment Architecture and Availability

Deploying one gateway process creates an obvious failure point. Production deployments normally use multiple instances across failure domains behind a network load balancer or equivalent entry layer. Keep runtime configuration consistent and make health probes reflect whether an instance can safely receive traffic.

Many gateway products distinguish a control plane, where configuration is managed, from a data plane, which handles API traffic. The data plane should have a defined behavior when the control plane or configuration store is unavailable. Test startup, restart, configuration propagation, rollback, and stale-configuration behavior rather than assuming the last known configuration will always be available.

flowchart TB
    Operator[Operator or CI] --> Control[Gateway control plane]
    Control --> Config[(Configuration store)]
    Control -. configuration .-> DP1[Data plane A]
    Control -. configuration .-> DP2[Data plane B]
    LB[External load balancer] --> DP1
    LB --> DP2
    DP1 --> Services[Microservices]
    DP2 --> Services
    DP1 --> Telemetry[Metrics, logs, and traces]
    DP2 --> Telemetry

Capacity planning must include policy cost. TLS handshakes, token verification, transformations, detailed logging, and tracing can change throughput and latency. Load-test the deployed plugins, payload sizes, connection patterns, and upstream behavior, leaving headroom for instance loss.

A Minimal Apache APISIX Routing Example

Apache APISIX is an open-source API gateway with a data plane based on NGINX and OpenResty. Its documented deployment modes include traditional and decoupled modes that use etcd, plus standalone modes that load declarative configuration without etcd. Select the mode supported by your APISIX release and operating model.

The following Admin API request creates a simple GET route to an orders service. Replace the Admin API key and upstream address, and keep the Admin API off the public data-plane network.

export APISIX_ADMIN_KEY='replace-with-a-protected-admin-key'

curl --fail --request PUT \
  'http://127.0.0.1:9180/apisix/admin/routes/orders-read' \
  --header "X-API-KEY: ${APISIX_ADMIN_KEY}" \
  --header 'Content-Type: application/json' \
  --data '{
    "uri": "/orders/*",
    "methods": ["GET"],
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "orders.internal:8080": 1
      }
    }
  }'

This route does not add authentication, authorization, quotas, or monitoring by itself. Add only the plugins required by the API contract, validate their configuration in the documentation for the deployed release, and test both allowed and rejected requests.

Implementation Checklist

Define the Boundary

Inventory which clients and APIs cross the gateway. Decide which traffic remains internal, where TLS terminates, who owns identity, and which service owns each authorization decision. Document the BFF, workflow, and service-mesh boundaries.

Design Routes and Policies as Code

Use stable route names and normalized paths. Review overlapping rules and default routes. Store declarative configuration or Admin API automation in version control where the deployment model permits it, run validation before rollout, and keep a rollback path.

Set Failure Policies Per API

Define timeouts, retries, rate limits, payload limits, and caching from the API's semantics. Test dependency slowness, partial failure, instance loss, malformed tokens, limit exhaustion, and configuration-store unavailability.

Protect the Management Plane

Restrict the Admin API and control plane to authorized networks and identities. Rotate credentials, audit configuration changes, use least privilege, and keep runtime and management endpoints separate. Back up the configuration according to the product's recovery model.

Establish Observability and SLOs

Measure request volume, user-visible success, latency distributions, policy rejection, upstream health, saturation, retries, and telemetry-pipeline health. Connect gateway traces to services and write runbooks for common failure modes.

Roll Out Gradually

Validate configuration in a representative environment, then use canary or staged deployment when possible. Compare error, latency, and rejection signals before increasing traffic. Avoid changing routing, authentication, rate limits, and upstream versions in one unobservable release.

Choosing an API Gateway

Evaluate products with a workload-based test rather than a generic ranking. Important questions include:

• Does it support the protocols, identity systems, discovery mechanisms, and deployment model you actually use?
• Can configuration be reviewed, promoted, rolled back, and audited?
• How does the data plane behave when management dependencies fail?
• Which policies are built in, officially maintained, or dependent on custom code?
• Can it expose low-cardinality metrics, structured logs, and trace context?
• What is the operational cost of upgrades, plugins, control-plane availability, and support?
• Can the team test and operate it within the required security and compliance controls?

Claims such as “enterprise ready,” “unlimited scalability,” or “zero downtime” do not answer these questions. Test the precise configuration under expected and failure traffic.

Conclusion

An API gateway can simplify microservices clients and provide a consistent point for routing, authentication, quotas, transformations, and telemetry. Its value comes from a clear boundary and disciplined policies, not from putting every distributed-system responsibility into one component.

Keep business authorization in the owning service, response composition in a BFF or aggregation service, and durable processes in a workflow system. Deploy the gateway redundantly, protect its management plane, and connect user-facing SLOs to gateway and upstream evidence. That design makes the gateway a controlled API boundary instead of a bottleneck or hidden monolith.