---
title: "API Gateway Change Management: Compatibility, Progressive Delivery, and Rollback - API7.ai"
description: "Manage gateway changes as versioned release units with compatibility checks, progressive traffic, verified application, and rehearsed rollback."
url: "https://api7.ai/learning-center/api-gateway-guide/api-gateway-change-management-rollback"
---

[Skip to article](#column-content)

Column contentsAPI Gateway Guide

[API Gateway Guide](https://api7.ai/learning-center/api-gateway-guide)

Search this column

01 Gateway Fundamentals

[View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=fundamentals)[1. What Is an API Gateway? API Gateway Software and Use Cases](https://api7.ai/learning-center/api-gateway-guide/what-is-an-api-gateway)[2. Why Do Modern Architectures Need an API Gateway?](https://api7.ai/learning-center/api-gateway-guide/why-do-modern-architectures-need-api-gateway)[3. Reverse Proxy vs Load Balancer vs API Gateway](https://api7.ai/learning-center/api-gateway-guide/api-gateway-vs-reverse-proxy-vs-load-balancer)[4. API Gateway and Microservices: Decoupling and Traffic Management](https://api7.ai/learning-center/api-gateway-guide/api-gateway-and-microservices)[5. Understanding Core API Gateway Features](https://api7.ai/learning-center/api-gateway-guide/core-api-gateway-features)[6. Open Source vs. Commercial API Gateways: How to Choose the Right One?](https://api7.ai/learning-center/api-gateway-guide/open-source-vs-commercial-api-gateway)[7. API Gateway vs Service Mesh: How to Choose?](https://api7.ai/learning-center/api-gateway-guide/api-gateway-vs-service-mesh)[8. Envoy vs NGINX, Java, and Go for API Gateways](https://api7.ai/learning-center/api-gateway-guide/nginx-vs-envoy-vs-java-vs-go)[9. Core Concepts of API Gateway: Service, Route, Upstream, Plugin, and Consumer](https://api7.ai/learning-center/api-gateway-guide/core-concepts-of-api-gateway)[10. API Gateway Comparison: Apache APISIX vs. Kong vs. Traefik vs. KrakenD vs. Tyk](https://api7.ai/learning-center/api-gateway-guide/api-gateway-comparison-apisix-kong-traefik-krakend-tyk)

02 Gateway Architecture and Design

[View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=architecture)[11. Deep Dive into API Gateway Architecture: Core Components and Design Principles](https://api7.ai/learning-center/api-gateway-guide/api-gateway-architecture)[12. Choosing the Right API Gateway for Your Architecture](https://api7.ai/learning-center/api-gateway-guide/api-gateway-monolithic-vs-microservices)[13. How to Architect an API Gateway for High Availability (HA)?](https://api7.ai/learning-center/api-gateway-guide/api-gateway-high-availability)[14. API Gateway Plugin Architecture: How to Extend and Customize Your API Gateway](https://api7.ai/learning-center/api-gateway-guide/api-gateway-plugin-system)[15. How Event-Driven Architecture (EDA) Works with API Gateway?](https://api7.ai/learning-center/api-gateway-guide/api-gateway-event-driven-architecture)[16. API Gateway Request Lifecycle Explained](https://api7.ai/learning-center/api-gateway-guide/api-gateway-request-lifecycle-explained)[17. Cloud-Native API Gateway on Kubernetes](https://api7.ai/learning-center/api-gateway-guide/cloud-native-api-gateway-kubernetes)[18. API Gateway Control Plane vs. Data Plane](https://api7.ai/learning-center/api-gateway-guide/api-gateway-control-plane-vs-data-plane)[19. How API Gateways Work with GitOps: Automating API Management](https://api7.ai/learning-center/api-gateway-guide/api-gateway-gitops-integration)[20. How to Perform A/B Testing and Canary Releases with an API Gateway](https://api7.ai/learning-center/api-gateway-guide/ab-testing-canary-release-api-gateway)

03 Gateway Setup and Traffic Handling

[View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=traffic-protocols)[21. How to Deploy an API Gateway on Kubernetes: Step-by-Step Guide for Engineers](https://api7.ai/learning-center/api-gateway-guide/deploy-api-gateway-on-kubernetes)[22. Hands-on Setup with Apache APISIX API Gateway](https://api7.ai/learning-center/api-gateway-guide/api-gateway-getting-started-guide)[23. API Gateway Auth: OAuth 2.0, JWT & OIDC with APISIX](https://api7.ai/learning-center/api-gateway-guide/api-gateway-authentication-apache-apisix-oauth-jwt-oidc)[24. Mastering Traffic Control: Efficiently Managing API Requests with Gateways](https://api7.ai/learning-center/api-gateway-guide/api-gateway-traffic-control-strategies)[25. Building Reliable API Gateways with Logging and Monitoring](https://api7.ai/learning-center/api-gateway-guide/api-gateway-logging-monitoring-best-practices)[26. Dynamic Routing in API Gateways: Best Practices for Flexible Traffic Management](https://api7.ai/learning-center/api-gateway-guide/api-gateway-dynamic-routing-management)[27. How API Gateways Support WebSocket](https://api7.ai/learning-center/api-gateway-guide/how-api-gateways-support-websocket)[28. API Gateway for GraphQL: Security, Cost, and Caching](https://api7.ai/learning-center/api-gateway-guide/api-gateway-handle-graphql)[29. How API Gateways Handle HTTP/3 QUIC Requests](https://api7.ai/learning-center/api-gateway-guide/api-gateway-http3-quic)[30. How API Gateways Handle MQTT Requests](https://api7.ai/learning-center/api-gateway-guide/api-gateway-mqtt)

04 Gateway Integrations, AI, and MCP

[View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=ai-mcp)[31. How API Gateways Handle gRPC Requests](https://api7.ai/learning-center/api-gateway-guide/api-gateway-grpc-support)[32. How API Gateways Implement Custom Plugins](https://api7.ai/learning-center/api-gateway-guide/api-gateway-custom-plugins)[33. How API Gateways Integrate with Kafka](https://api7.ai/learning-center/api-gateway-guide/api-gateway-kafka-integration)[34. How to Implement API Aggregation in API Gateways](https://api7.ai/learning-center/api-gateway-guide/api-gateway-api-aggregation)[35. How API Gateways Enable Serverless Architecture](https://api7.ai/learning-center/api-gateway-guide/api-gateway-serverless-architecture)[36. How API Gateways Proxy LLM Requests](https://api7.ai/learning-center/api-gateway-guide/api-gateway-proxy-llm-requests)[37. How API Gateways Enhance MCP Servers](https://api7.ai/learning-center/api-gateway-guide/api-gateway-enhance-mcp-server)[38. MCP Gateway vs MCP Proxy vs AI Gateway vs API Gateway](https://api7.ai/learning-center/api-gateway-guide/ai-gateway-vs-mcp-gateway-vs-api-gateway)[39. What Is an AI Gateway? Architecture and Benefits](https://api7.ai/learning-center/api-gateway-guide/what-is-an-ai-gateway)[40. What Is an MCP Gateway? Architecture, Use Cases & How It Works (2026 Guide)](https://api7.ai/learning-center/api-gateway-guide/what-is-mcp-gateway)

05 Gateway Performance

[View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=performance)[41. How API Gateway Reduces Latency: Performance Optimization Strategies](https://api7.ai/learning-center/api-gateway-guide/how-api-gateway-reduces-latency)[42. API Gateway Caching Strategy: Comprehensive Analysis and Implementation Guide](https://api7.ai/learning-center/api-gateway-guide/api-gateway-caching-strategy-analysis)[43. How to Optimize API Gateway Load Balancing: Advanced Strategies and Best Practices](https://api7.ai/learning-center/api-gateway-guide/how-to-optimize-api-gateway-load-balancing)[44. How API Gateway Efficiently Handles Large-Scale Traffic: Architecture and Strategies](https://api7.ai/learning-center/api-gateway-guide/how-api-gateway-handles-large-scale-traffic)[45. How to Improve API Gateway QoS: Traffic Classes, Capacity Protection, and SLOs](https://api7.ai/learning-center/api-gateway-guide/api-gateway-quality-of-service)[46. API Gateway Log Storage: Pipelines, Retention, Security, and Cost](https://api7.ai/learning-center/api-gateway-guide/api-gateway-log-storage)[47. API Gateway Timeouts and Retries: Safe Policies and APISIX Configuration](https://api7.ai/learning-center/api-gateway-guide/api-gateway-timeouts-retries)[48. API Gateway Concurrency Control: Capacity Budgets, Backpressure, and Load Shedding](https://api7.ai/learning-center/api-gateway-guide/api-gateway-concurrency-control)[49. Asynchronous API Gateway Patterns: Queues, Callbacks, and Performance Boundaries](https://api7.ai/learning-center/api-gateway-guide/asynchronous-api-gateway-patterns)[50. API Gateway TLS Performance: Handshakes, Session Reuse, and Measurement](https://api7.ai/learning-center/api-gateway-guide/api-gateway-tls-performance)

06 Gateway Security

[View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=security)[51. DDoS Defense at the API Gateway: Layered Controls and Failure Planning](https://api7.ai/learning-center/api-gateway-guide/api-gateway-ddos-defense)[52. Distributed API Gateway Rate Limiting: Local vs Redis Counters and Accuracy Trade-offs](https://api7.ai/learning-center/api-gateway-guide/distributed-api-gateway-rate-limiting)[53. API Gateway IP Allowlist and Denylist Management: Trust, Proxies, and Operations](https://api7.ai/learning-center/api-gateway-guide/api-gateway-ip-allowlist-denylist-management)[54. API Gateway Security Scanning: What to Test and How to Operationalize Findings](https://api7.ai/learning-center/api-gateway-guide/api-gateway-security-scanning)[55. SQL Injection and XSS at the API Gateway: Detection Limits and Defense in Depth](https://api7.ai/learning-center/api-gateway-guide/sql-injection-xss-api-gateway)[56. API Access Log Auditing: Evidence, Integrity, Retention, and Review](https://api7.ai/learning-center/api-gateway-guide/api-access-log-auditing)[57. Preventing API Scraping and Abuse: Identity, Behavioral Signals, and Rate Controls](https://api7.ai/learning-center/api-gateway-guide/prevent-api-scraping-abuse)[58. API Gateway mTLS: Client Identity, Certificate Rotation, and APISIX Configuration](https://api7.ai/learning-center/api-gateway-guide/api-gateway-mtls)[59. Fine-Grained API Gateway Authorization: RBAC, ABAC, and External Policy Decisions](https://api7.ai/learning-center/api-gateway-guide/fine-grained-api-gateway-authorization)[60. API Gateway and WAF Reference Architecture: Ownership, Header Trust, and Failure Modes](https://api7.ai/learning-center/api-gateway-guide/api-gateway-waf-reference-architecture)

07 Gateway Ecosystem and Comparisons

[View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=ecosystem)[61. Apache APISIX vs Kong: Migration, Operations, and Team Fit](https://api7.ai/learning-center/api-gateway-guide/apache-apisix-vs-kong)[62. Traefik vs Kong vs APISIX: A Reproducible API Gateway Benchmark Method](https://api7.ai/learning-center/api-gateway-guide/traefik-kong-apisix-benchmark-method)[63. Amazon API Gateway Explained: Deployment Models, Trade-offs, and Use Cases](https://api7.ai/learning-center/api-gateway-guide/amazon-api-gateway-explained)[64. API Gateway and Istio: Clear North-South and East-West Boundaries](https://api7.ai/learning-center/api-gateway-guide/api-gateway-istio)[65. API Gateway with NGINX: Layering, Ownership, and Migration Patterns](https://api7.ai/learning-center/api-gateway-guide/api-gateway-nginx)[66. API Gateway with Envoy: Deployment Patterns and Policy Ownership](https://api7.ai/learning-center/api-gateway-guide/api-gateway-envoy)[67. API Gateway with Cloudflare API Shield: Edge and Origin Security Boundaries](https://api7.ai/learning-center/api-gateway-guide/api-gateway-cloudflare-api-shield)[68. Apigee vs MuleSoft vs Amazon API Gateway: A Decision Framework](https://api7.ai/learning-center/api-gateway-guide/apigee-mulesoft-amazon-api-gateway)[69. Should an API Gateway Connect Directly to a Database? Patterns and Risks](https://api7.ai/learning-center/api-gateway-guide/api-gateway-database-access-patterns)[70. API Gateway Architecture for Large-Scale SaaS Platforms](https://api7.ai/learning-center/api-gateway-guide/api-gateway-large-scale-saas-architecture)

08 Industry Applications and Integrations

[View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=applications)[71. API Gateway CI/CD: Validation, Promotion, and Safe Rollback](https://api7.ai/learning-center/api-gateway-guide/api-gateway-ci-cd)[72. API Gateways for AI and ML Inference: Traffic, Security, and Observability](https://api7.ai/learning-center/api-gateway-guide/api-gateways-ai-ml-inference)[73. API Gateway Architecture for Financial Services: Controls and Failure Boundaries](https://api7.ai/learning-center/api-gateway-guide/api-gateway-financial-services)[74. API Gateway Architecture for E-Commerce: Checkout Reliability and Traffic Peaks](https://api7.ai/learning-center/api-gateway-guide/api-gateway-ecommerce)[75. API Gateways for Mobile Applications: Authentication, Payloads, and Versioning](https://api7.ai/learning-center/api-gateway-guide/api-gateways-mobile-applications)[76. API Gateways for Web3 Applications: RPC Traffic, Identity, and Abuse Controls](https://api7.ai/learning-center/api-gateway-guide/api-gateways-web3-applications)[77. OpenAPI-Driven API Gateway Operations: Import, Validation, and Drift Control](https://api7.ai/learning-center/api-gateway-guide/openapi-driven-api-gateway-operations)[78. gRPC-Web and gRPC Transcoding Through an API Gateway](https://api7.ai/learning-center/api-gateway-guide/grpc-web-grpc-transcoding-api-gateway)[79. API Gateway Pricing and Cost Optimization: A Workload-Based Model](https://api7.ai/learning-center/api-gateway-guide/api-gateway-pricing-cost-optimization)[80. Evaluating API Gateway Directions: Evidence and Architectural Impact](https://api7.ai/learning-center/api-gateway-guide/api-gateway-trends)

09 Advanced Gateway Applications

[View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=operations)[81. API Gateways in Zero Trust Architecture: Identity, Policy, and Limits](https://api7.ai/learning-center/api-gateway-guide/api-gateways-zero-trust-architecture)[82. API Gateway Log Analytics: Correlation, Queries, and Operational Signals](https://api7.ai/learning-center/api-gateway-guide/api-gateway-log-analytics)[83. From API Sprawl to Governed Delivery: The Gateway's Role in Modernization](https://api7.ai/learning-center/api-gateway-guide/api-gateway-modernization)[84. API Monetization at the Gateway: Plans, Metering, Entitlements, and Billing Boundaries](https://api7.ai/learning-center/api-gateway-guide/api-monetization-gateway)[85. Safe AI-Assisted API Gateway Optimization: Recommendations, Guardrails, and Approval](https://api7.ai/learning-center/api-gateway-guide/ai-assisted-api-gateway-optimization)[86. API Gateways for Low-Code and No-Code Integration: Governance and Safety](https://api7.ai/learning-center/api-gateway-guide/api-gateways-low-code-no-code-integration)[87. API Gateway and CIAM Integration: Token Validation, Consent, and Identity Context](https://api7.ai/learning-center/api-gateway-guide/api-gateway-ciam-integration)[88. Elastic API Gateway Scaling: Signals, Capacity, and Safe Scale-In](https://api7.ai/learning-center/api-gateway-guide/elastic-api-gateway-scaling)[89. Multi-Tenant SaaS API Gateways: Isolation, Quotas, and Tenant-Aware Policy](https://api7.ai/learning-center/api-gateway-guide/multi-tenant-saas-api-gateway-policy)[90. API Gateway DevSecOps: Policy as Code, Testing, and Runtime Feedback](https://api7.ai/learning-center/api-gateway-guide/api-gateway-devsecops)

10 Platform Engineering and Planning

[View this topic · 10](https://api7.ai/learning-center/api-gateway-guide?topic=platform-governance)[91. API Discovery and Documentation at the Gateway: Inventory, Ownership, and Drift](https://api7.ai/learning-center/api-gateway-guide/api-discovery-documentation-gateway)[92. WebAssembly Plugins for API Gateways: Portability, Isolation, and APISIX Boundaries](https://api7.ai/learning-center/api-gateway-guide/api-gateway-webassembly-plugins)[93. Monitoring API Gateways with Prometheus and Grafana: Metrics, Dashboards, and Alerts](https://api7.ai/learning-center/api-gateway-guide/monitor-api-gateways-prometheus-grafana)[94. API Gateway Reliability Engineering: SLOs, Failure Isolation, and Incident Response](https://api7.ai/learning-center/api-gateway-guide/api-gateway-reliability-engineering)[95. API Gateways for Edge AI Inference: Routing, Constraints, and Observability](https://api7.ai/learning-center/api-gateway-guide/api-gateways-edge-ai-inference)[96. Reducing Cloud Cost with API Gateway Controls: Measurement Before Optimization](https://api7.ai/learning-center/api-gateway-guide/reducing-cloud-cost-api-gateway-controls)[97. Improving API Developer Experience with Gateway Standards and Self-Service](https://api7.ai/learning-center/api-gateway-guide/improving-api-developer-experience-gateway-self-service)[98. API Gateway Change Management: Compatibility, Progressive Delivery, and Rollback](https://api7.ai/learning-center/api-gateway-guide/api-gateway-change-management-rollback)[99. Operating Serverless APIs: Cold Starts, Idempotency, and Asynchronous Handoffs](https://api7.ai/learning-center/api-gateway-guide/operating-serverless-apis-gateway)[100. Planning an API Gateway Roadmap for the Next Five Years](https://api7.ai/learning-center/api-gateway-guide/planning-api-gateway-roadmap-next-five-years)

[API Gateway Guide](https://api7.ai/learning-center/api-gateway-guide)[Platform Engineering and Planning](https://api7.ai/learning-center/api-gateway-guide?topic=platform-governance)

On this page

On this page

- [Key Takeaways](#key-takeaways)

- [Build a Release Unit](#build-a-release-unit)

- [Classify the Change Before Deployment](#classify-the-change-before-deployment)

- [Use an Evidence-Gated Release Flow](#use-an-evidence-gated-release-flow)

- [Check Contract Compatibility in Both Directions](#check-contract-compatibility-in-both-directions)

- [Example: Weighted Canary with APISIX](#example-weighted-canary-with-apisix)

- [Define Promotion Gates](#define-promotion-gates)

- [Verify Configuration Application, Not Only Acceptance](#verify-configuration-application-not-only-acceptance)

- [Design Rollback by Change Type](#design-rollback-by-change-type)

- [Rehearse Failure Paths](#rehearse-failure-paths)

- [Change Management Checklist](#change-management-checklist)

- [FAQ](#faq)

  - [Is a successful Admin API response proof that a gateway change is live?](#is-a-successful-admin-api-response-proof-that-a-gateway-change-is-live)
  - [Should every change use a weighted canary?](#should-every-change-use-a-weighted-canary)
  - [Is rollback always safer than rolling forward?](#is-rollback-always-safer-than-rolling-forward)

- [Next Steps](#next-steps)

Article 98

# API Gateway Change Management: Compatibility, Progressive Delivery, and Rollback

API7.ai

October 29, 2026

Safe API gateway change management treats the contract, gateway policy, upstream release, credentials, and rollback plan as one versioned release unit. A configuration write that returns success is only one step: teams still need compatibility evidence, data-plane application evidence, user-visible health, and a tested way back.

Rollback is not “restore the previous file” unless the previous application, schema, secrets, and external dependencies still support it. Design reversibility before the rollout begins.

## Key Takeaways

- Classify changes by consumer, security, traffic, data, and operational impact before choosing the release path.
- Pin the API contract, generated gateway resources, upstream artifact, and policy profile in one release record.
- Separate configuration acceptance, worker application, traffic exposure, and business correctness.
- Use progressive delivery to reduce blast radius, but do not promise exact ratios over small samples.
- Prefer forward-compatible expansion and delayed removal when data or contracts cannot roll back cleanly.
- Rehearse rollback with the same automation, permissions, and evidence used in production.

## Build a Release Unit

Record these items together:

- source commit and reviewed OpenAPI revision;
- generated Route, Service, Upstream, Consumer, certificate, and plugin resources;
- policy-profile version and approved exceptions;
- upstream application image or function version;
- database or event-schema compatibility state;
- secret and certificate references, never secret values;
- rollout stages, metrics, owners, and decision times;
- last known compatible rollback artifact;
- verification evidence and final disposition.

A gateway-only snapshot can be unusable if it points to an application version that no longer accepts the request or a credential that has already been revoked.

## Classify the Change Before Deployment

| Change                                  | Main risk                               | Minimum evidence                                                 |
| --------------------------------------- | --------------------------------------- | ---------------------------------------------------------------- |
| Add optional operation or field         | Route conflict, unexpected exposure     | Contract validation, authorization test, non-production call     |
| Tighten authentication or authorization | Legitimate clients rejected             | Consumer inventory, staged enforcement, denied-and-allowed tests |
| Change timeout, retry, or body limit    | Hidden latency or replay failure        | Failure test, total deadline, idempotency proof                  |
| Change upstream or traffic weight       | Wrong target or partial incompatibility | Contract test on both versions, health and business metrics      |
| Remove or rename an operation           | Breaking consumers                      | Usage evidence, deprecation window, migration confirmation       |
| Change cache or identity key            | Cross-user data or quota drift          | Key provenance, isolation tests, invalidation plan               |
| Rotate certificate or secret            | Handshake or credential outage          | Overlap window, both-direction verification, expiry alarm        |

“Configuration-only” does not mean low risk. A one-line identity, cache, or retry change can alter every request.

## Use an Evidence-Gated Release Flow

```
flowchart LR
    S[Versioned contract app and policy] --> V[Static compatibility and security checks]
    V --> N[Non production verification]
    N --> A[Gateway control plane accepts change]
    A --> D[Data planes confirm application]
    D --> C[Canary traffic]
    C --> G{Reliability security and business gates}
    G -->|pass| P[Progressive promotion]
    G -->|fail| R[Execute compatible rollback]
    P --> O[Observe and close release]
    R --> O
```

The control plane accepts configuration. Data planes apply it. The gateway then executes routing and policy. Application metrics and business tests decide whether the release is useful. Do not collapse those actors into one “deployed” state.

## Check Contract Compatibility in Both Directions

Compatibility depends on who changes first:

- **New client to old server:** can the client avoid new fields or operations until rollout completes?
- **Old client to new server:** does the server preserve required fields, status codes, and semantics?
- **New gateway to old upstream:** will transformed paths, headers, authentication, and retries still work?
- **Old gateway to new upstream:** can rollback still reach and authenticate to the new application?

Schema compatibility is necessary but insufficient. Test defaults, nullability, enum expansion, pagination, error shape, idempotency, authorization, and business meaning. A response can validate against JSON Schema and still break a consumer.

## Example: Weighted Canary with APISIX

Apache APISIX 3.19.0's [`traffic-split` plugin](https://apisix.apache.org/docs/apisix/3.19/plugins/traffic-split/) can send weighted traffic to alternate Upstreams. The example below keeps the Route's existing Upstream at weight 90 and sends weight 10 to `orders-v2-canary`.

```
{
  "uri": "/orders/*",
  "upstream_id": "orders-v1-stable",
  "plugins": {
    "traffic-split": {
      "rules": [
        {
          "weighted_upstreams": [
            {"weight": 90},
            {"upstream_id": "orders-v2-canary", "weight": 10}
          ]
        }
      ]
    }
  }
}
```

This is a schema-valid Route body, but both referenced Upstream objects must exist before traffic is sent. The schema validates an `upstream_id` as an ID; a missing alternate object surfaces when APISIX handles a matching request. The entry with no `upstream_id` represents the Route's default Upstream. APISIX uses weighted round robin for this split and documents that the observed ratio can be less accurate, especially after state resets. Do not promise that exactly one of every ten requests reaches the canary.

Weighted traffic is also not sticky by default. If one user's workflow must stay on one version, use an evidence-backed affinity design and test first, later, absent, and failover paths. Do not derive release assignment from an untrusted caller header.

## Define Promotion Gates

Use a minimum sample and observation time. Compare canary and stable for:

- request and business success rate;
- latency by gateway, upstream, and application stage;
- authentication and authorization denials;
- retry attempts, timeouts, and ambiguous outcomes;
- cache, quota, and idempotency behavior;
- resource saturation and dependency errors;
- application correctness and support signals.

Choose absolute stop conditions as well as relative comparisons. A canary that is “no worse” than an already failing stable version should not promote.

## Verify Configuration Application, Not Only Acceptance

APISIX 3.19.0 [standalone API-driven mode](https://apisix.apache.org/docs/apisix/3.19/deployment-modes/) distinguishes acceptance from worker application for `PUT /apisix/admin/configs`:

| Request path                                           | Result           | What it proves                                                                       |
| ------------------------------------------------------ | ---------------- | ------------------------------------------------------------------------------------ |
| No `?wait` or `?wait=0`                                | `202 Accepted`   | Configuration was accepted; it does not confirm every worker applied it              |
| `?wait=<milliseconds>` and all workers confirm in time | `200 OK`         | All workers, including stream workers when enabled, confirmed application            |
| `?wait=<milliseconds>` expires first                   | `202 Accepted`   | Accepted, but full worker application is not yet confirmed                           |
| Matching `X-Digest` skips the update                   | `204 No Content` | Submitted digest matched the loaded configuration; `?wait` does not change this path |

The wait is capped at 60 seconds. This mode is intended for defined integrations such as APISIX Ingress Controller and ADC; do not adopt it directly without understanding its full-replacement and version semantics. In other deployment modes, use the evidence exposed by that control path and verify a data-plane request.

## Design Rollback by Change Type

| Change                | Rollback approach                                | Irreversible or delayed concern                      |
| --------------------- | ------------------------------------------------ | ---------------------------------------------------- |
| Traffic split         | Set canary weight to zero or restore prior Route | In-flight and long-lived connections may remain      |
| Authentication policy | Restore compatible policy and keys               | Revoked credentials may not be safe to re-enable     |
| Timeout or rate limit | Restore prior values                             | Queued, retried, or rejected work cannot be undone   |
| Cache policy          | Disable/restore and purge affected entries       | Incorrect data may already have reached clients      |
| API field removal     | Restore compatible response                      | Consumers may have migrated or data may have changed |
| Database/event schema | Prefer expand-and-contract and forward fix       | Destructive migration may prevent binary rollback    |

Keep rollback artifacts immutable and permissioned. “Edit until it works” destroys provenance and makes the incident harder to reconstruct.

## Rehearse Failure Paths

Before production, test:

1. invalid generated gateway configuration;
2. accepted change that has not reached every worker;
3. canary Upstream unhealthy before and after receiving traffic;
4. gateway policy compatible with only one application version;
5. expired or revoked credential during rollback;
6. partial region or cluster promotion;
7. metrics unavailable during the decision window;
8. rollback while long-lived connections remain;
9. forward-only data migration;
10. release controller restart during promotion.

Record the expected client response, traffic owner, configuration revision, application version, and operator action for each case.

## Change Management Checklist

- Does one release record pin contract, gateway, application, policy, and rollback artifacts?
- Is the change classified by consumer, security, data, and operational impact?
- Are old/new client, gateway, and upstream combinations tested?
- Are acceptance, worker application, traffic exposure, and business correctness separate states?
- Does the canary have a minimum sample, observation window, and absolute stop gate?
- Are weighted-routing accuracy and affinity limits explicit?
- Can rollback still authenticate to and communicate with the retained application version?
- Are cache, queue, retry, and data side effects handled as non-reversible evidence?
- Has rollback been rehearsed with production-equivalent automation and permissions?
- Can operators reconstruct who approved, applied, promoted, and rolled back the release?

## FAQ

### Is a successful Admin API response proof that a gateway change is live?

No. It proves only the semantics of that response path. Verify data-plane application and then test user-visible behavior.

### Should every change use a weighted canary?

No. Some authentication, schema, or data changes need shadow tests, explicit cohorts, parallel versions, or an expand-and-contract migration instead.

### Is rollback always safer than rolling forward?

No. When data, credentials, events, or consumers have moved forward, restoring old binaries or configuration may be incompatible. Prepare a forward fix and containment path too.

## Next Steps

Build the automated path in [API gateway CI/CD](https://api7.ai/learning-center/api-gateway-guide/api-gateway-ci-cd), enforce it through [gateway DevSecOps](https://api7.ai/learning-center/api-gateway-guide/api-gateway-devsecops), and bound recovery with [safe timeout and retry policy](https://api7.ai/learning-center/api-gateway-guide/api-gateway-timeouts-retries).

[Previous article**Improving API Developer Experience with Gateway Standards and Self-Service**](https://api7.ai/learning-center/api-gateway-guide/improving-api-developer-experience-gateway-self-service)[Next article**Operating Serverless APIs: Cold Starts, Idempotency, and Asynchronous Handoffs**](https://api7.ai/learning-center/api-gateway-guide/operating-serverless-apis-gateway)

On this page

- [Key Takeaways](#key-takeaways)

- [Build a Release Unit](#build-a-release-unit)

- [Classify the Change Before Deployment](#classify-the-change-before-deployment)

- [Use an Evidence-Gated Release Flow](#use-an-evidence-gated-release-flow)

- [Check Contract Compatibility in Both Directions](#check-contract-compatibility-in-both-directions)

- [Example: Weighted Canary with APISIX](#example-weighted-canary-with-apisix)

- [Define Promotion Gates](#define-promotion-gates)

- [Verify Configuration Application, Not Only Acceptance](#verify-configuration-application-not-only-acceptance)

- [Design Rollback by Change Type](#design-rollback-by-change-type)

- [Rehearse Failure Paths](#rehearse-failure-paths)

- [Change Management Checklist](#change-management-checklist)

- [FAQ](#faq)

  - [Is a successful Admin API response proof that a gateway change is live?](#is-a-successful-admin-api-response-proof-that-a-gateway-change-is-live)
  - [Should every change use a weighted canary?](#should-every-change-use-a-weighted-canary)
  - [Is rollback always safer than rolling forward?](#is-rollback-always-safer-than-rolling-forward)

- [Next Steps](#next-steps)
