Batch Request Processing with API Gateway
April 27, 2023
Batch request processing is a powerful technique used in web development to improve the performance of APIs. It allows developers to group multiple API requests into a single HTTP request/response cycle. In other words, a single API request from a client can be turned into multiple API requests to a set of backend servers, and the responses are aggregated into a single response to the client. It can significantly reduce the number of round trips between the client and server.
In this article, we'll explore how to implement batch request processing in Apache APISIX and look at some use cases where it can be beneficial.
It is similar to the API Composition pattern which is widely applied in Microservices architecture.
Batch request processing vs multiple API calls.
Why use batch request processing?
When a client sends multiple API requests to a server, each request requires a separate HTTP request/response cycle. This can result in increased latency, reduced performance, and increased server load. By grouping multiple requests into a single batch request, the number of HTTP request/response cycles can be reduced, resulting in improved performance and reduced latency.
Batch request processing can be particularly useful in scenarios where you need to retrieve or update multiple records in a single transaction, such as:
- Retrieving multiple records from a database
- Updating multiple records in a database
- Executing multiple API requests to complete a task
Real-world examples for batch request processing
Example 1:
Suppose you have a social media application that displays a user's feed, which includes posts from their friends and pages they follow. To populate this feed, you need to make multiple API requests to retrieve the necessary data, such as:
- Retrieve a list of the user's friends and the pages they follow.
- For each friend or page, retrieve their recent posts.
In a traditional approach, you would perform each of these API requests separately. First, you retrieve a list of users’ friends and in the second request, you get recent posts for each user’s friend. This can result in increased latency, especially when the user has a large number of friends and follows many pages.
Example 2:
Another example, you have a mobile application that displays a list of products for users to browse. To populate this list, you need to make multiple API requests to retrieve product data from a remote server, such as:
- Retrieve a list of product IDs.
- For each product ID, retrieve the product details (name, description, image, etc.)
Example 3:
Imagine that you have a web app for conference management where there are multiple speakers in the system and you want to display a speaker’s sessions and related topics on a single web page. A backend Conference API service has two different endpoints /speaker/{speakerId}/sessions and /speaker/{speakerId}/topics to expose this information. To show both sessions and topics belong to a single speaker, you can send two requests from the frontend app which is not an ideal solution. Instead, you can use an API Gateway to group all of these requests into a single HTTP request, as it is explained in the next section.
Batch request processing with Apache APISIX API Gateway
To implement batch request processing in APISIX, you can use the batch-requests
plugin. This plugin allows you to define a set of API requests in a single HTTP POST request payload. Each request can have its own HTTP method, URL path, set of headers, and payload. See a curl request command below for example 3 (Conference API requests):
curl -i http://{API_GATEWAY_HOST_ADDRESS}/speaker -X POST -d \
'{
"pipeline": [
{
"method": "GET",
"path": "/speaker/1/topics"
},
{
"method": "GET",
"path": "/speaker/1/sessions"
}
]
}'
When a batch request is received by APISIX, the batch-requests plugin will parse the payload and execute each request in the batch in parallel. The plugin will also aggregate the responses from each request and return them in a single HTTP response to the client. See the next demo section to learn how to achieve this step-by-step.
Batch requests plugin demo
Before you can use the batch-requests plugin, you'll need to install Apache APISIX.
Prerequisites
- Docker is used to install the containerized etcd and APISIX.
- curl is used to send requests to APISIX Admin API. You can also use easy tools such as Postman to interact with the API.
APISIX can be easily installed and started with the following quickstart script:
curl -sL <https://run.api7.ai/apisix/quickstart> | sh
Configure the backend service (upstream)
You will need to configure the backend service for Conference API that you want to route requests to. This can be done by adding an upstream server in the Apache APISIX through the Admin API.
curl http://127.0.0.1:9180/apisix/admin/upstreams/1 -X PUT -d '
{
"name": "Conferences API upstream",
"desc": "Register Conferences API as the upstream",
"type": "roundrobin",
"scheme": "https",
"nodes": {
"conferenceapi.azurewebsites.net:443": 1
}
}'
Create a Route for batch processing API
We need to create a new route that intercepts requests to /speaker and exposes a public virtual endpoint for batch processing using [public-api](https://apisix.apache.org/docs/apisix/plugins/public-api/) plugin.
curl http://127.0.0.1:9180/apisix/admin/routes/a -X PUT -d '
{
"uri": "/speaker",
"plugins": {
"public-api": {
"uri": "/apisix/batch-requests"
}
}
}'
Create a Route for the speaker’s topics and sessions endpoints
Next, we create another route for the speaker’s topics and sessions endpoints (/speaker/*/topics and /speaker/*/topics paths matching) so that individual requests extracted by the API Gateway from batch requests to retrieve the speaker’s topics or sessions are forwarded to the responsible Conference API endpoints and we referenced to the existing upstream service.
curl http://127.0.0.1:9180/apisix/admin/routes/b -X PUT -d '
{
"methods": ["GET"],
"uris": ["/speaker/*/topics","/speaker/*/sessions"],
"plugins": {
"proxy-rewrite":{
"host":"conferenceapi.azurewebsites.net"
}
},
"upstream_id":"1"
}'
You may notice that we are using another proxy-rewrite plugin and specifying implicitly host address for Conference API. Otherwise, API Gateway can make DNS conversion and request the Conference API by its IP address.
Test batch request processing
Here's an example of how to use the batch-requests plugin in APISIX:
curl -i http://127.0.0.1:9080/speaker -X POST -d \
'{
"pipeline": [
{
"method": "GET",
"path": "/speaker/1/topics"
},
{
"method": "GET",
"path": "/speaker/1/sessions"
}
]
}'
In this example, the route is defined for the /speaker endpoint, which supports batch request processing via the batch-requests plugin. The plugin is configured with a set of two requests, each retrieving a speaker record by ID with topics and sessions. If you run this command, you will get a merged response back from the API Gateway:
[
{
"body":"{\r\n \"collection\": {\r\n \"version\": \"1.0\",\r\n \"links\": [],\r\n \"items\": [\r\n {\r\n \"href\": \"https://conferenceapi.azurewebsites.net/topic/8\",\r\n \"data\": [\r\n {\r\n \"name\": \"Title\",\r\n \"value\": \"Microsoft\"\r\n }\r\n ],\r\n \"links\": [\r\n {\r\n \"rel\": \"http://tavis.net/rels/sessions\",\r\n \"href\": \"https://conferenceapi.azurewebsites.net/topic/8/sessions\"\r\n }\r\n ]\r\n },\r\n {\r\n \"href\": \"https://conferenceapi.azurewebsites.net/topic/10\",\r\n \"data\": [\r\n {\r\n \"name\": \"Title\",\r\n \"value\": \"Mobile\"\r\n }\r\n ],\r\n \"links\": [\r\n {\r\n \"rel\": \"http://tavis.net/rels/sessions\",\r\n \"href\": \"https://conferenceapi.azurewebsites.net/topic/10/sessions\"\r\n }\r\n ]\r\n }\r\n ],\r\n \"queries\": [],\r\n \"template\": {\r\n \"data\": []\r\n }\r\n }\r\n}",
"status":200,
"headers":{
"Expires":"-1",
"Connection":"keep-alive",
"Pragma":"no-cache",
"Content-Length":"953",
"Server":"APISIX/3.2.0",
"Content-Type":"application/vnd.collection+json",
"X-AspNet-Version":"4.0.30319",
"Cache-Control":"no-cache",
"X-Powered-By":"ASP.NET"
},
"reason":"OK"
},
{
"body":"{\r\n \"collection\": {\r\n \"version\": \"1.0\",\r\n \"links\": [],\r\n \"items\": [\r\n {\r\n \"href\": \"https://conferenceapi.azurewebsites.net/session/206\",\r\n \"data\": [\r\n {\r\n \"name\": \"Title\",\r\n \"value\": \"\\r\\n\\t\\t\\tjQuery Mobile and ASP.NET MVC\\r\\n\\t\\t\"\r\n },\r\n {\r\n \"name\": \"Timeslot\",\r\n \"value\": \"05 December 2013 09:00 - 10:00\"\r\n },\r\n {\r\n \"name\": \"Speaker\",\r\n \"value\": \"Scott Allen\"\r\n }\r\n ],\r\n \"links\": [\r\n {\r\n \"rel\": \"http://tavis.net/rels/speaker\",\r\n \"href\": \"https://conferenceapi.azurewebsites.net/speaker/16\"\r\n },\r\n {\r\n \"rel\": \"http://tavis.net/rels/topics\",\r\n \"href\": \"https://conferenceapi.azurewebsites.net/session/206/topics\"\r\n }\r\n ]\r\n }\r\n ],\r\n \"queries\": [],\r\n \"template\": {\r\n \"data\": []\r\n }\r\n }\r\n}",
"status":200,
"headers":{
"Expires":"-1",
"Connection":"keep-alive",
"Pragma":"no-cache",
"Content-Length":"961",
"Server":"APISIX/3.2.0",
"Content-Type":"application/vnd.collection+json",
"X-AspNet-Version":"4.0.30319",
"Cache-Control":"no-cache",
"X-Powered-By":"ASP.NET"
},
"reason":"OK"
}
]
The maximum size of a batch request is limited by API Gateway. You can check the API Gateway documentation for the current limits and request timeout configuration.
Takeaways
- Batch request processing with API Gateway can be a useful technique for improving the performance of your API.
- Apache APISIX provides a plugin called
batch-requeststhat allows developers to implement batch request processing easily.
Next steps
With API Gateway, it is also possible to provide some form of custom aggregation in the response data to your users. You can use the serverless-function plugin to execute custom code and merge the response from backend services and return it to the API consumer in a different structure.