--- title: "Asynchronous API Gateway Patterns: Queues, Callbacks, and Performance Boundaries - API7.ai" description: "Design safe asynchronous APIs behind a gateway with durable acceptance, job status resources, callbacks, idempotency, and backpressure." url: "https://api7.ai/learning-center/api-gateway-guide/asynchronous-api-gateway-patterns" --- [Skip to article](#column-content) Column contentsAPI Gateway Guide [API Gateway Guide](https://api7.ai/learning-center/api-gateway-guide) Search this column 01 Gateway Fundamentals [View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=fundamentals)[1. What Is an API Gateway? API Gateway Software and Use Cases](https://api7.ai/learning-center/api-gateway-guide/what-is-an-api-gateway)[2. Why Do Modern Architectures Need an API Gateway?](https://api7.ai/learning-center/api-gateway-guide/why-do-modern-architectures-need-api-gateway)[3. Reverse Proxy vs Load Balancer vs API Gateway](https://api7.ai/learning-center/api-gateway-guide/api-gateway-vs-reverse-proxy-vs-load-balancer)[4. API Gateway and Microservices: Decoupling and Traffic Management](https://api7.ai/learning-center/api-gateway-guide/api-gateway-and-microservices)[5. Understanding Core API Gateway Features](https://api7.ai/learning-center/api-gateway-guide/core-api-gateway-features)[6. Open Source vs. Commercial API Gateways: How to Choose the Right One?](https://api7.ai/learning-center/api-gateway-guide/open-source-vs-commercial-api-gateway)[7. API Gateway vs Service Mesh: How to Choose?](https://api7.ai/learning-center/api-gateway-guide/api-gateway-vs-service-mesh)[8. Envoy vs NGINX, Java, and Go for API Gateways](https://api7.ai/learning-center/api-gateway-guide/nginx-vs-envoy-vs-java-vs-go)[9. Core Concepts of API Gateway: Service, Route, Upstream, Plugin, and Consumer](https://api7.ai/learning-center/api-gateway-guide/core-concepts-of-api-gateway)[10. API Gateway Comparison: Apache APISIX vs. Kong vs. Traefik vs. KrakenD vs. Tyk](https://api7.ai/learning-center/api-gateway-guide/api-gateway-comparison-apisix-kong-traefik-krakend-tyk) 02 Gateway Architecture and Design [View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=architecture)[11. Deep Dive into API Gateway Architecture: Core Components and Design Principles](https://api7.ai/learning-center/api-gateway-guide/api-gateway-architecture)[12. Choosing the Right API Gateway for Your Architecture](https://api7.ai/learning-center/api-gateway-guide/api-gateway-monolithic-vs-microservices)[13. How to Architect an API Gateway for High Availability (HA)?](https://api7.ai/learning-center/api-gateway-guide/api-gateway-high-availability)[14. API Gateway Plugin Architecture: How to Extend and Customize Your API Gateway](https://api7.ai/learning-center/api-gateway-guide/api-gateway-plugin-system)[15. How Event-Driven Architecture (EDA) Works with API Gateway?](https://api7.ai/learning-center/api-gateway-guide/api-gateway-event-driven-architecture)[16. API Gateway Request Lifecycle Explained](https://api7.ai/learning-center/api-gateway-guide/api-gateway-request-lifecycle-explained)[17. Cloud-Native API Gateway on Kubernetes](https://api7.ai/learning-center/api-gateway-guide/cloud-native-api-gateway-kubernetes)[18. API Gateway Control Plane vs. Data Plane](https://api7.ai/learning-center/api-gateway-guide/api-gateway-control-plane-vs-data-plane)[19. How API Gateways Work with GitOps: Automating API Management](https://api7.ai/learning-center/api-gateway-guide/api-gateway-gitops-integration)[20. How to Perform A/B Testing and Canary Releases with an API Gateway](https://api7.ai/learning-center/api-gateway-guide/ab-testing-canary-release-api-gateway) 03 Gateway Setup and Traffic Handling [View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=traffic-protocols)[21. How to Deploy an API Gateway on Kubernetes: Step-by-Step Guide for Engineers](https://api7.ai/learning-center/api-gateway-guide/deploy-api-gateway-on-kubernetes)[22. Hands-on Setup with Apache APISIX API Gateway](https://api7.ai/learning-center/api-gateway-guide/api-gateway-getting-started-guide)[23. API Gateway Auth: OAuth 2.0, JWT & OIDC with APISIX](https://api7.ai/learning-center/api-gateway-guide/api-gateway-authentication-apache-apisix-oauth-jwt-oidc)[24. Mastering Traffic Control: Efficiently Managing API Requests with Gateways](https://api7.ai/learning-center/api-gateway-guide/api-gateway-traffic-control-strategies)[25. Building Reliable API Gateways with Logging and Monitoring](https://api7.ai/learning-center/api-gateway-guide/api-gateway-logging-monitoring-best-practices)[26. Dynamic Routing in API Gateways: Best Practices for Flexible Traffic Management](https://api7.ai/learning-center/api-gateway-guide/api-gateway-dynamic-routing-management)[27. How API Gateways Support WebSocket](https://api7.ai/learning-center/api-gateway-guide/how-api-gateways-support-websocket)[28. API Gateway for GraphQL: Security, Cost, and Caching](https://api7.ai/learning-center/api-gateway-guide/api-gateway-handle-graphql)[29. How API Gateways Handle HTTP/3 QUIC Requests](https://api7.ai/learning-center/api-gateway-guide/api-gateway-http3-quic)[30. How API Gateways Handle MQTT Requests](https://api7.ai/learning-center/api-gateway-guide/api-gateway-mqtt) 04 Gateway Integrations, AI, and MCP [View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=ai-mcp)[31. How API Gateways Handle gRPC Requests](https://api7.ai/learning-center/api-gateway-guide/api-gateway-grpc-support)[32. How API Gateways Implement Custom Plugins](https://api7.ai/learning-center/api-gateway-guide/api-gateway-custom-plugins)[33. How API Gateways Integrate with Kafka](https://api7.ai/learning-center/api-gateway-guide/api-gateway-kafka-integration)[34. How to Implement API Aggregation in API Gateways](https://api7.ai/learning-center/api-gateway-guide/api-gateway-api-aggregation)[35. How API Gateways Enable Serverless Architecture](https://api7.ai/learning-center/api-gateway-guide/api-gateway-serverless-architecture)[36. How API Gateways Proxy LLM Requests](https://api7.ai/learning-center/api-gateway-guide/api-gateway-proxy-llm-requests)[37. How API Gateways Enhance MCP Servers](https://api7.ai/learning-center/api-gateway-guide/api-gateway-enhance-mcp-server)[38. MCP Gateway vs MCP Proxy vs AI Gateway vs API Gateway](https://api7.ai/learning-center/api-gateway-guide/ai-gateway-vs-mcp-gateway-vs-api-gateway)[39. What Is an AI Gateway? Architecture and Benefits](https://api7.ai/learning-center/api-gateway-guide/what-is-an-ai-gateway)[40. What Is an MCP Gateway? Architecture, Use Cases & How It Works (2026 Guide)](https://api7.ai/learning-center/api-gateway-guide/what-is-mcp-gateway) 05 Gateway Performance [View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=performance)[41. How API Gateway Reduces Latency: Performance Optimization Strategies](https://api7.ai/learning-center/api-gateway-guide/how-api-gateway-reduces-latency)[42. API Gateway Caching Strategy: Comprehensive Analysis and Implementation Guide](https://api7.ai/learning-center/api-gateway-guide/api-gateway-caching-strategy-analysis)[43. How to Optimize API Gateway Load Balancing: Advanced Strategies and Best Practices](https://api7.ai/learning-center/api-gateway-guide/how-to-optimize-api-gateway-load-balancing)[44. How API Gateway Efficiently Handles Large-Scale Traffic: Architecture and Strategies](https://api7.ai/learning-center/api-gateway-guide/how-api-gateway-handles-large-scale-traffic)[45. How to Improve API Gateway QoS: Traffic Classes, Capacity Protection, and SLOs](https://api7.ai/learning-center/api-gateway-guide/api-gateway-quality-of-service)[46. API Gateway Log Storage: Pipelines, Retention, Security, and Cost](https://api7.ai/learning-center/api-gateway-guide/api-gateway-log-storage)[47. API Gateway Timeouts and Retries: Safe Policies and APISIX Configuration](https://api7.ai/learning-center/api-gateway-guide/api-gateway-timeouts-retries)[48. API Gateway Concurrency Control: Capacity Budgets, Backpressure, and Load Shedding](https://api7.ai/learning-center/api-gateway-guide/api-gateway-concurrency-control)[49. Asynchronous API Gateway Patterns: Queues, Callbacks, and Performance Boundaries](https://api7.ai/learning-center/api-gateway-guide/asynchronous-api-gateway-patterns)[50. API Gateway TLS Performance: Handshakes, Session Reuse, and Measurement](https://api7.ai/learning-center/api-gateway-guide/api-gateway-tls-performance) 06 Gateway Security [View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=security)[51. DDoS Defense at the API Gateway: Layered Controls and Failure Planning](https://api7.ai/learning-center/api-gateway-guide/api-gateway-ddos-defense)[52. Distributed API Gateway Rate Limiting: Local vs Redis Counters and Accuracy Trade-offs](https://api7.ai/learning-center/api-gateway-guide/distributed-api-gateway-rate-limiting)[53. API Gateway IP Allowlist and Denylist Management: Trust, Proxies, and Operations](https://api7.ai/learning-center/api-gateway-guide/api-gateway-ip-allowlist-denylist-management)[54. API Gateway Security Scanning: What to Test and How to Operationalize Findings](https://api7.ai/learning-center/api-gateway-guide/api-gateway-security-scanning)[55. SQL Injection and XSS at the API Gateway: Detection Limits and Defense in Depth](https://api7.ai/learning-center/api-gateway-guide/sql-injection-xss-api-gateway)[56. API Access Log Auditing: Evidence, Integrity, Retention, and Review](https://api7.ai/learning-center/api-gateway-guide/api-access-log-auditing)[57. Preventing API Scraping and Abuse: Identity, Behavioral Signals, and Rate Controls](https://api7.ai/learning-center/api-gateway-guide/prevent-api-scraping-abuse)[58. API Gateway mTLS: Client Identity, Certificate Rotation, and APISIX Configuration](https://api7.ai/learning-center/api-gateway-guide/api-gateway-mtls)[59. Fine-Grained API Gateway Authorization: RBAC, ABAC, and External Policy Decisions](https://api7.ai/learning-center/api-gateway-guide/fine-grained-api-gateway-authorization)[60. API Gateway and WAF Reference Architecture: Ownership, Header Trust, and Failure Modes](https://api7.ai/learning-center/api-gateway-guide/api-gateway-waf-reference-architecture) 07 Gateway Ecosystem and Comparisons [View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=ecosystem)[61. Apache APISIX vs Kong: Migration, Operations, and Team Fit](https://api7.ai/learning-center/api-gateway-guide/apache-apisix-vs-kong)[62. Traefik vs Kong vs APISIX: A Reproducible API Gateway Benchmark Method](https://api7.ai/learning-center/api-gateway-guide/traefik-kong-apisix-benchmark-method)[63. Amazon API Gateway Explained: Deployment Models, Trade-offs, and Use Cases](https://api7.ai/learning-center/api-gateway-guide/amazon-api-gateway-explained)[64. API Gateway and Istio: Clear North-South and East-West Boundaries](https://api7.ai/learning-center/api-gateway-guide/api-gateway-istio)[65. API Gateway with NGINX: Layering, Ownership, and Migration Patterns](https://api7.ai/learning-center/api-gateway-guide/api-gateway-nginx)[66. API Gateway with Envoy: Deployment Patterns and Policy Ownership](https://api7.ai/learning-center/api-gateway-guide/api-gateway-envoy)[67. API Gateway with Cloudflare API Shield: Edge and Origin Security Boundaries](https://api7.ai/learning-center/api-gateway-guide/api-gateway-cloudflare-api-shield)[68. Apigee vs MuleSoft vs Amazon API Gateway: A Decision Framework](https://api7.ai/learning-center/api-gateway-guide/apigee-mulesoft-amazon-api-gateway)[69. Should an API Gateway Connect Directly to a Database? Patterns and Risks](https://api7.ai/learning-center/api-gateway-guide/api-gateway-database-access-patterns)[70. API Gateway Architecture for Large-Scale SaaS Platforms](https://api7.ai/learning-center/api-gateway-guide/api-gateway-large-scale-saas-architecture) 08 Industry Applications and Integrations [View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=applications)[71. API Gateway CI/CD: Validation, Promotion, and Safe Rollback](https://api7.ai/learning-center/api-gateway-guide/api-gateway-ci-cd)[72. API Gateways for AI and ML Inference: Traffic, Security, and Observability](https://api7.ai/learning-center/api-gateway-guide/api-gateways-ai-ml-inference)[73. API Gateway Architecture for Financial Services: Controls and Failure Boundaries](https://api7.ai/learning-center/api-gateway-guide/api-gateway-financial-services)[74. API Gateway Architecture for E-Commerce: Checkout Reliability and Traffic Peaks](https://api7.ai/learning-center/api-gateway-guide/api-gateway-ecommerce)[75. API Gateways for Mobile Applications: Authentication, Payloads, and Versioning](https://api7.ai/learning-center/api-gateway-guide/api-gateways-mobile-applications)[76. API Gateways for Web3 Applications: RPC Traffic, Identity, and Abuse Controls](https://api7.ai/learning-center/api-gateway-guide/api-gateways-web3-applications)[77. OpenAPI-Driven API Gateway Operations: Import, Validation, and Drift Control](https://api7.ai/learning-center/api-gateway-guide/openapi-driven-api-gateway-operations)[78. gRPC-Web and gRPC Transcoding Through an API Gateway](https://api7.ai/learning-center/api-gateway-guide/grpc-web-grpc-transcoding-api-gateway)[79. API Gateway Pricing and Cost Optimization: A Workload-Based Model](https://api7.ai/learning-center/api-gateway-guide/api-gateway-pricing-cost-optimization)[80. Evaluating API Gateway Directions: Evidence and Architectural Impact](https://api7.ai/learning-center/api-gateway-guide/api-gateway-trends) 09 Advanced Gateway Applications [View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=operations)[81. API Gateways in Zero Trust Architecture: Identity, Policy, and Limits](https://api7.ai/learning-center/api-gateway-guide/api-gateways-zero-trust-architecture)[82. API Gateway Log Analytics: Correlation, Queries, and Operational Signals](https://api7.ai/learning-center/api-gateway-guide/api-gateway-log-analytics)[83. From API Sprawl to Governed Delivery: The Gateway's Role in Modernization](https://api7.ai/learning-center/api-gateway-guide/api-gateway-modernization)[84. API Monetization at the Gateway: Plans, Metering, Entitlements, and Billing Boundaries](https://api7.ai/learning-center/api-gateway-guide/api-monetization-gateway)[85. Safe AI-Assisted API Gateway Optimization: Recommendations, Guardrails, and Approval](https://api7.ai/learning-center/api-gateway-guide/ai-assisted-api-gateway-optimization)[86. API Gateways for Low-Code and No-Code Integration: Governance and Safety](https://api7.ai/learning-center/api-gateway-guide/api-gateways-low-code-no-code-integration)[87. API Gateway and CIAM Integration: Token Validation, Consent, and Identity Context](https://api7.ai/learning-center/api-gateway-guide/api-gateway-ciam-integration)[88. Elastic API Gateway Scaling: Signals, Capacity, and Safe Scale-In](https://api7.ai/learning-center/api-gateway-guide/elastic-api-gateway-scaling)[89. Multi-Tenant SaaS API Gateways: Isolation, Quotas, and Tenant-Aware Policy](https://api7.ai/learning-center/api-gateway-guide/multi-tenant-saas-api-gateway-policy)[90. API Gateway DevSecOps: Policy as Code, Testing, and Runtime Feedback](https://api7.ai/learning-center/api-gateway-guide/api-gateway-devsecops) 10 Platform Engineering and Planning [View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=platform-governance)[91. API Discovery and Documentation at the Gateway: Inventory, Ownership, and Drift](https://api7.ai/learning-center/api-gateway-guide/api-discovery-documentation-gateway)[92. WebAssembly Plugins for API Gateways: Portability, Isolation, and APISIX Boundaries](https://api7.ai/learning-center/api-gateway-guide/api-gateway-webassembly-plugins)[93. Monitoring API Gateways with Prometheus and Grafana: Metrics, Dashboards, and Alerts](https://api7.ai/learning-center/api-gateway-guide/monitor-api-gateways-prometheus-grafana)[94. API Gateway Reliability Engineering: SLOs, Failure Isolation, and Incident Response](https://api7.ai/learning-center/api-gateway-guide/api-gateway-reliability-engineering)[95. API Gateways for Edge AI Inference: Routing, Constraints, and Observability](https://api7.ai/learning-center/api-gateway-guide/api-gateways-edge-ai-inference)[96. Reducing Cloud Cost with API Gateway Controls: Measurement Before Optimization](https://api7.ai/learning-center/api-gateway-guide/reducing-cloud-cost-api-gateway-controls)[97. Improving API Developer Experience with Gateway Standards and Self-Service](https://api7.ai/learning-center/api-gateway-guide/improving-api-developer-experience-gateway-self-service)[98. API Gateway Change Management: Compatibility, Progressive Delivery, and Rollback](https://api7.ai/learning-center/api-gateway-guide/api-gateway-change-management-rollback)[99. Operating Serverless APIs: Cold Starts, Idempotency, and Asynchronous Handoffs](https://api7.ai/learning-center/api-gateway-guide/operating-serverless-apis-gateway)[100. Planning an API Gateway Roadmap for the Next Five Years](https://api7.ai/learning-center/api-gateway-guide/planning-api-gateway-roadmap-next-five-years) [API Gateway Guide](https://api7.ai/learning-center/api-gateway-guide)[Gateway Performance](https://api7.ai/learning-center/api-gateway-guide?topic=performance) On this page On this page - [Key Takeaways](#key-takeaways) - [When an Asynchronous API Helps](#when-an-asynchronous-api-helps) - [Define the Acceptance Contract](#define-the-acceptance-contract) - [Choose a Completion Pattern](#choose-a-completion-pattern) - [Poll a job-status resource](#poll-a-job-status-resource) - [Deliver a callback or webhook](#deliver-a-callback-or-webhook) - [Publish an event](#publish-an-event) - [Make Duplicate Submission Safe](#make-duplicate-submission-safe) - [Put Backpressure at Intake](#put-backpressure-at-intake) - [Configure the APISIX Intake Route](#configure-the-apisix-intake-route) - [Operate the Whole Workflow](#operate-the-whole-workflow) - [Test Failure at Every Handoff](#test-failure-at-every-handoff) - [Decision Checklist](#decision-checklist) - [Summary](#summary) - [FAQ](#faq) - [Can an API gateway return `202` and enqueue the work itself?](#can-an-api-gateway-return-202-and-enqueue-the-work-itself) - [Is polling worse than a webhook?](#is-polling-worse-than-a-webhook) - [Does a queue remove the need for rate limits?](#does-a-queue-remove-the-need-for-rate-limits) - [Next Steps](#next-steps) Article 49 # Asynchronous API Gateway Patterns: Queues, Callbacks, and Performance Boundaries API7.ai September 8, 2026 An API gateway supports asynchronous processing by securing, validating, limiting, and routing an acceptance request—but the application must durably record the work before returning success. A reliable design returns `202 Accepted` with a job identifier and status location, makes duplicate submissions safe, exposes a terminal outcome, and applies backpressure when the queue or workers are saturated. Returning `202` from the gateway without a durable handoff does not make work asynchronous. It only makes failure harder for the client to see. ## Key Takeaways - A `202 Accepted` response means processing has not finished; it does not promise eventual success. - Persist the job or message before acknowledging acceptance. - Give clients a status resource, callback, event, or another explicit completion channel. - Make submission idempotent because clients cannot always distinguish a lost response from a lost request. - Limit queue age, depth, request size, and per-tenant intake; asynchronous work can move overload rather than remove it. - Keep gateway and application responsibilities separate: policy at the edge, workflow state in a durable owner. ## When an Asynchronous API Helps The synchronous request-response model is appropriate when work can predictably finish within the caller's deadline. An asynchronous contract is useful when an operation: - takes seconds or minutes, such as media conversion or report generation; - experiences bursts that workers should absorb gradually; - depends on slow or rate-limited external services; - needs independent retry and recovery after the client disconnects; - produces a result too late for one open HTTP connection. The performance benefit is specific: the client and gateway release the acceptance connection sooner, and a durable queue can decouple intake from worker throughput. Total work does not disappear. Queue storage, scheduling, status reads, callbacks, and worker processing add their own latency and cost. Do not convert a fast operation to asynchronous merely to conceal an overloaded dependency. First determine whether the operation needs a long-running workflow or simply needs [concurrency control](https://api7.ai/learning-center/api-gateway-guide/api-gateway-concurrency-control) and capacity correction. ## Define the Acceptance Contract [RFC 9110 section 15.3.3](https://www.rfc-editor.org/rfc/rfc9110.html#section-15.3.3) defines `202 Accepted` for a request accepted for processing whose processing is not complete and might ultimately be disallowed. The response ought to describe the request's current status and point to a status monitor when one is available. A practical submission looks like this: ``` POST /v1/report-jobs HTTP/1.1 Host: api.example.com Authorization: Bearer Idempotency-Key: 6d28a698-11e4-47bc-a85b-5427fbd89261 Content-Type: application/json {"account_id":"acct-42","format":"csv"} ``` ``` HTTP/1.1 202 Accepted Location: /v1/report-jobs/job-8f31 Retry-After: 3 Content-Type: application/json { "id": "job-8f31", "status": "queued", "status_url": "/v1/report-jobs/job-8f31" } ``` The application should issue this response only after it has committed enough state to recover and process the job. A safe sequence is: ``` sequenceDiagram participant C as Client participant G as API Gateway participant A as Intake Service participant Q as Durable Queue/Store participant W as Worker C->>G: POST job + idempotency key G->>G: Authenticate, validate, admit G->>A: Forward accepted request A->>Q: Atomically record job/message Q-->>A: Durable acknowledgement A-->>G: 202 + job ID + status URL G-->>C: 202 Accepted W->>Q: Claim and process job C->>G: GET status URL G->>A: Read authorized status A-->>G: queued/running/succeeded/failed G-->>C: Authorized status response ``` If the service sends `202` before the durable acknowledgement, a crash can silently lose accepted work. If it waits for the worker to finish, it has recreated a synchronous operation. ## Choose a Completion Pattern ### Poll a job-status resource Polling is simple for clients that can make outbound HTTP requests but cannot expose a public callback endpoint. Model the job as a resource with a small state machine: ``` queued -> running -> succeeded -> failed queued/running -> cancelled ``` Return stable terminal states, timestamps, a safe error category, and an authenticated result link. Use `Retry-After`, exponential polling backoff, or server-documented intervals to prevent the status endpoint from becoming the dominant load. ### Deliver a callback or webhook A callback reduces polling but creates another distributed-system boundary. Require HTTPS, authenticate the delivery, sign the body, include a timestamp and delivery identifier, and define replay protection. Treat redirects and DNS changes carefully to reduce server-side request forgery risk. Use bounded retries and a dead-letter path; do not retry forever. Callback delivery is normally at least once. The receiver must deduplicate events and query the authoritative status resource if event order is uncertain. ### Publish an event For controlled service-to-service environments, publish a completion event to a broker or event stream. Define the message schema, partitioning/order expectation, retention, consumer authentication, and replay behavior. The public HTTP gateway can still govern job intake and status access; it does not need to impersonate the broker. One API may offer both polling and callbacks, but one durable job record should remain the source of truth. ## Make Duplicate Submission Safe After the application commits a job, the response can be lost. The client sees a timeout and cannot know whether to submit again. An idempotency key gives the server a way to return the existing job instead of creating a second one. A robust implementation: 1. scopes the key to the authenticated tenant and operation; 2. stores a fingerprint of the relevant request payload; 3. atomically records the key, job identifier, and response state; 4. returns the same job for an exact replay; 5. rejects reuse of the key with a different payload; 6. retains the record longer than the documented client retry window. Forwarding an `Idempotency-Key` header through the gateway is useful, but it is not enforcement. The application or durable acceptance component must provide the atomic behavior. Workers also need idempotent effects or deduplication because queue delivery can be repeated after a crash. ## Put Backpressure at Intake A queue smooths a burst only while its depth and job age remain acceptable. If intake continuously exceeds completion, the queue is an expanding outage. Define limits for: - accepted jobs per tenant and per operation; - concurrent jobs and maximum queued jobs; - maximum payload and result size; - maximum queue age before work is no longer useful; - worker attempts and total processing deadline; - external-service calls and spending; - retention for job records, results, and dead letters. Reject before durable acceptance when the system cannot honor the contract. A tenant-specific intake limit generally maps to `429`. Shared saturation can map to `503`. Do not return `202` for a job placed into a queue that has no plausible processing capacity. Large inputs are often better uploaded directly to object storage with a short-lived, scoped upload credential. Submit a reference plus integrity metadata to the job API rather than buffering the entire object through the gateway and queue. ## Configure the APISIX Intake Route Apache APISIX should enforce edge policy while an intake service owns durable workflow state. The following APISIX 3.18 example authenticates the caller, validates a small JSON body, bounds the arrival rate, and exports metrics. It does not claim that APISIX itself enqueues the job. Prerequisites are Apache APISIX 3.18, an Admin API key, a reachable `job-intake.internal:8080` service, and an APISIX Consumer with a `key-auth` credential. The block is a parseable route example; replace the upstream address, schema, and numeric policy with values from your environment. ``` curl "http://127.0.0.1:9180/apisix/admin/routes/report-jobs" \ -X PUT \ -H "X-API-KEY: ${admin_key}" \ -d '{ "uri": "/v1/report-jobs", "methods": ["POST"], "plugins": { "key-auth": {}, "request-validation": { "body_schema": { "type": "object", "required": ["account_id", "format"], "properties": { "account_id": {"type": "string", "maxLength": 80}, "format": {"type": "string", "enum": ["csv", "json"]} }, "additionalProperties": false } }, "limit-req": { "rate": 5, "burst": 10, "key_type": "var", "key": "consumer_name", "rejected_code": 429, "policy": "local" }, "prometheus": {} }, "upstream": { "type": "roundrobin", "nodes": { "job-intake.internal:8080": 1 } } }' ``` The [`request-validation` plugin](https://apisix.apache.org/docs/apisix/plugins/request-validation/) checks the configured schema before the request reaches the intake service. Keep business invariants in the application as well. The numeric limits above are illustrative and must be replaced with measured tenant and worker capacity. This example explicitly uses `policy: local`, so each APISIX node keeps its own counter. In a multi-node fleet, the aggregate admitted rate can exceed five requests per second for one Consumer. Divide the budget conservatively across nodes or evaluate a supported Redis-backed policy when the intake service requires a shared fleet-wide limit; include that dependency's latency and availability in the design. With a test Consumer credential stored in `consumer_api_key`, send a schema-valid request: ``` curl "http://127.0.0.1:9080/v1/report-jobs" \ -X POST \ -H "apikey: ${consumer_api_key}" \ -H "Idempotency-Key: 6d28a698-11e4-47bc-a85b-5427fbd89261" \ -H "Content-Type: application/json" \ -d '{"account_id":"acct-42","format":"csv"}' ``` The success condition is a `202` response with the intake service's job identifier and status location after durable acceptance. An invalid body should be rejected before reaching the service, and an invalid credential should fail authentication. Requests above the steady rate but within the burst allowance may be delayed; requests that exhaust the configured burst allowance should return `429`. Exact response bodies depend on the application and plugin configuration. Create a separate route for status reads so its authorization, cache behavior, and rate policy can differ from submission. Verify that a caller can access only jobs in its own tenant. Never place credentials or sensitive input in a predictable job identifier or status URL. Clients can express a preference for asynchronous handling with `Prefer: respond-async`, defined by [RFC 7240](https://www.rfc-editor.org/rfc/rfc7240.html#section-4.1). It is a preference, not a command. Only advertise or return `Preference-Applied: respond-async` when the application actually implements that negotiation. ## Operate the Whole Workflow Gateway latency alone is a misleading success metric. Track: - acceptance rate, rejection rate, and acceptance latency; - queue depth and age of the oldest ready job; - time from acceptance to start and to terminal outcome; - worker concurrency, success, failure, retry, and dead-letter rate; - duplicate submissions and idempotency conflicts; - status-read and callback delivery load; - expired, cancelled, and abandoned results; - cost per accepted and completed job. Propagate a correlation identifier from the submission to the durable job, worker attempts, result, and callback. Do not propagate one live tracing span for hours; link workflow events using stable identifiers and trace relationships supported by your telemetry system. ## Test Failure at Every Handoff Before launch, test these ambiguous moments: 1. Crash before durable commit: the client must not receive `202`. 2. Crash after commit but before response: a retry with the same idempotency key must return the existing job. 3. Deliver the same queue message twice: externally visible effects must not duplicate. 4. Lose or reorder callbacks: the receiver must deduplicate and reconcile from status. 5. Stop workers while intake continues: queue-age alerts and admission limits must activate. 6. Expire a result while a client polls: the API must return a documented terminal representation rather than an ambiguous absence. 7. Revoke a tenant while jobs remain: define whether work is cancelled, quarantined, or allowed to finish. Also load-test status reads and callbacks. Moving work out of the original request often creates two or more new API traffic streams. ## Decision Checklist - Does the operation truly outlive a reasonable HTTP deadline? - What durable event allows the service to return `202`? - How does a client discover success, failure, cancellation, and expiration? - What makes resubmission and worker redelivery safe? - Who may read the status or result? - When does intake reject because the queue is too deep or too old? - What is the retention and deletion policy? - Which gateway metrics join to queue, worker, and application signals? ## Summary An asynchronous API is a workflow contract, not a status-code shortcut. The API gateway should authenticate, validate, meter, and observe intake, while a durable application component records the job before acknowledging it. Combine `202 Accepted` with a status or notification channel, atomic idempotency, bounded queueing, and explicit overload behavior. That design releases request connections quickly without turning lost work and growing queues into hidden failure modes. ## FAQ ### Can an API gateway return `202` and enqueue the work itself? Only when a documented gateway capability provides a durable handoff with the required delivery and recovery semantics. In the common design, the gateway forwards to an intake service, and that service commits the job before returning `202`. ### Is polling worse than a webhook? Not universally. Polling is easier for clients that cannot receive inbound requests; webhooks reduce repeated reads but require authenticated delivery, replay protection, retry, and dead-letter handling. ### Does a queue remove the need for rate limits? No. A queue absorbs a bounded mismatch between intake and completion. If arrivals continuously exceed workers, job age and storage grow until the workflow fails or becomes too expensive. ## Next Steps Define the durable-acceptance point and job state machine before configuring the gateway. Then use the [APISIX request-validation documentation](https://apisix.apache.org/docs/apisix/plugins/request-validation/) to validate the intake schema supported by your release. For centrally governed Apache APISIX deployments, explore [API7 Enterprise](https://api7.ai/enterprise). [Previous article**API Gateway Concurrency Control: Capacity Budgets, Backpressure, and Load Shedding**](https://api7.ai/learning-center/api-gateway-guide/api-gateway-concurrency-control)[Next article**API Gateway TLS Performance: Handshakes, Session Reuse, and Measurement**](https://api7.ai/learning-center/api-gateway-guide/api-gateway-tls-performance) On this page - [Key Takeaways](#key-takeaways) - [When an Asynchronous API Helps](#when-an-asynchronous-api-helps) - [Define the Acceptance Contract](#define-the-acceptance-contract) - [Choose a Completion Pattern](#choose-a-completion-pattern) - [Poll a job-status resource](#poll-a-job-status-resource) - [Deliver a callback or webhook](#deliver-a-callback-or-webhook) - [Publish an event](#publish-an-event) - [Make Duplicate Submission Safe](#make-duplicate-submission-safe) - [Put Backpressure at Intake](#put-backpressure-at-intake) - [Configure the APISIX Intake Route](#configure-the-apisix-intake-route) - [Operate the Whole Workflow](#operate-the-whole-workflow) - [Test Failure at Every Handoff](#test-failure-at-every-handoff) - [Decision Checklist](#decision-checklist) - [Summary](#summary) - [FAQ](#faq) - [Can an API gateway return `202` and enqueue the work itself?](#can-an-api-gateway-return-202-and-enqueue-the-work-itself) - [Is polling worse than a webhook?](#is-polling-worse-than-a-webhook) - [Does a queue remove the need for rate limits?](#does-a-queue-remove-the-need-for-rate-limits) - [Next Steps](#next-steps)