Send Apache APISIX Logs to Splunk with HEC

Apache APISIX can send gateway access logs to Splunk through the splunk-hec-logging plugin. The plugin serializes request context into Splunk's event format, buffers entries through the APISIX batch processor, and posts each batch to a Splunk HTTP Event Collector (HEC) endpoint.

This guide configures the integration on one APISIX Route, sends a test request, and verifies that the event reaches Splunk.

How the Integration Works

The request and logging paths are separate:

1. A client sends a request to an APISIX Route.
2. APISIX proxies the request to the configured upstream.
3. During the log phase, splunk-hec-logging creates an event from the request context.
4. The batch processor sends buffered events to Splunk HEC.
5. Splunk indexes the events for search, dashboards, and alerting.

Logging happens after request processing. A successful API response does not by itself prove that Splunk accepted or indexed the corresponding event, so verify both paths.

Prerequisites

Before configuring APISIX, prepare:

• a running Apache APISIX instance and a reachable test upstream;
• a Splunk Enterprise or Splunk Cloud deployment with HEC enabled; for Splunk Cloud, use an HEC token without indexer acknowledgment for this APISIX integration;
• an enabled HEC token with access to the intended index;
• network connectivity from every APISIX data-plane node to the HEC endpoint;
• the APISIX Admin API key for a non-production test Route.

Use HTTPS for HEC outside an isolated local test. Treat the HEC token as a secret and do not commit it to source control or paste it into shared logs.

Configure Splunk HEC

Create and enable an HEC token by following Splunk's current HEC setup documentation.

Record these values:

• the HEC base URL, commonly https://<splunk-host>:8088 for Splunk Enterprise, or the assigned http-inputs endpoint, normally on port 443, for Splunk Cloud;
• the token value;
• the destination index;
• whether indexer acknowledgment is enabled on Splunk Enterprise.

If indexer acknowledgment is enabled for a Splunk Enterprise token, generate a unique channel identifier in GUID format. Splunk requires that channel on every event request made with the token. The APISIX plugin accepts it through endpoint.channel. This guide verifies that events are indexed through Splunk search; it does not poll the separate HEC acknowledgment endpoint.

Splunk Cloud supports acknowledgment-enabled HEC tokens only with Amazon Data Firehose. When sending APISIX logs directly to Splunk Cloud, use a token with indexer acknowledgment disabled.

Test the token directly before adding APISIX. The following command uses the JSON event endpoint and should return a success response from Splunk:

export SPLUNK_HEC_URL='https://splunk.example.com:8088'
export SPLUNK_HEC_TOKEN='<hec-token>'

curl --fail-with-body "$SPLUNK_HEC_URL/services/collector/event" \
  -H "Authorization: Splunk $SPLUNK_HEC_TOKEN" \
  -H 'X-Splunk-Request-Channel: <hec-channel-guid>' \
  -H 'Content-Type: application/json' \
  -d '{
    "source": "apisix-hec-connectivity-test",
    "sourcetype": "_json",
    "event": {"message": "HEC connectivity test"}
  }'

For Splunk Enterprise, replace <hec-channel-guid> with the channel identifier when indexer acknowledgment is enabled. Omit the X-Splunk-Request-Channel header when it is disabled. For Splunk Cloud, use its assigned HEC endpoint and a token with indexer acknowledgment disabled for this direct integration.

Do not disable certificate verification to make this test pass. Install the appropriate CA certificate or use a certificate trusted by the APISIX hosts.

Configure the APISIX Plugin

Save the APISIX Admin API key in your shell environment. Adjust the configuration path if your installation stores config.yaml elsewhere:

export APISIX_ADMIN_KEY="$(yq -r '.deployment.admin.admin_key[0].key' /usr/local/apisix/conf/config.yaml)"

Create a test Route with the splunk-hec-logging plugin. Replace the HEC URI, token, and upstream with values from your environment:

curl --fail-with-body http://127.0.0.1:9180/apisix/admin/routes/1 \
  -H "X-API-KEY: $APISIX_ADMIN_KEY" \
  -H 'Content-Type: application/json' \
  -X PUT \
  -d '{
    "uri": "/splunk.do",
    "plugins": {
      "splunk-hec-logging": {
        "endpoint": {
          "uri": "https://splunk.example.com:8088/services/collector/event",
          "token": "<hec-token>",
          "channel": "<hec-channel-guid>",
          "timeout": 10
        },
        "ssl_verify": true
      }
    },
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "127.0.0.1:1980": 1
      }
    }
  }'

The channel field is required when a Splunk Enterprise HEC token has indexer acknowledgment enabled. Remove it when acknowledgment is disabled. For Splunk Cloud, remove the field and use a token with indexer acknowledgment disabled for this APISIX integration.

The HEC token is shown inline only to make the example complete. In an operational workflow, inject sensitive configuration through an approved secret-management process and restrict access to the APISIX configuration store and Admin API.

The current plugin supports additional endpoint, retry, batch, and log-format settings. Check the versioned splunk-hec-logging documentation for the APISIX release you run instead of copying an attribute list from another version.

Send a Test Request

Call the Route through the APISIX proxy port:

curl -i 'http://127.0.0.1:9080/splunk.do?message=hello'

Confirm that the upstream response is successful. The plugin submits logs in batches, so the event may not appear in Splunk immediately after the response.

Verify the Event in Splunk

Open Search & Reporting in Splunk and search for the default APISIX source:

source="apache-apisix-splunk-hec-logging"

Narrow the search to the Route path or request time. Verify at least:

• the request URI and method;
• the response status;
• the APISIX Route or service identifier when present;
• the event timestamp and target index.

If the API request succeeds but no event appears, check APISIX error logs, DNS and TLS connectivity to the HEC host, token status, index permissions, and Splunk HEC health. Also allow for the configured batch flush interval.

Customize the Log Format

APISIX supports a global plugin metadata configuration for splunk-hec-logging. The metadata applies to every Route and Service using this plugin, so review the impact before changing it.

The following example records a compact set of request and response fields:

curl --fail-with-body \
  http://127.0.0.1:9180/apisix/admin/plugin_metadata/splunk-hec-logging \
  -H "X-API-KEY: $APISIX_ADMIN_KEY" \
  -H 'Content-Type: application/json' \
  -X PUT \
  -d '{
    "log_format": {
      "host": "$host",
      "@timestamp": "$time_iso8601",
      "client_ip": "$remote_addr",
      "request": {
        "method": "$request_method",
        "uri": "$request_uri"
      },
      "response": {
        "status": "$status"
      }
    }
  }'

Avoid logging authorization headers, session cookies, credentials, personal data, or complete request bodies unless there is a documented need and an appropriate retention and access policy.

Remove the Plugin

To stop sending logs for this Route, update the Route without the splunk-hec-logging configuration. APISIX applies Route changes dynamically and does not require a gateway restart.

curl --fail-with-body http://127.0.0.1:9180/apisix/admin/routes/1 \
  -H "X-API-KEY: $APISIX_ADMIN_KEY" \
  -H 'Content-Type: application/json' \
  -X PUT \
  -d '{
    "uri": "/splunk.do",
    "plugins": {},
    "upstream": {
      "type": "roundrobin",
      "nodes": {
        "127.0.0.1:1980": 1
      }
    }
  }'

Delete the test Route instead if it was created only for validation.

Production Checklist

Before enabling this integration broadly:

1. use a trusted HTTPS HEC endpoint and keep ssl_verify enabled;
2. use a dedicated token and index with the minimum required permissions;
3. define which fields may be logged and redact sensitive values;
4. monitor APISIX logging errors, queue pressure, and dropped entries;
5. test Splunk unavailability and recovery without assuming access-log delivery is guaranteed;
6. estimate event volume, index retention, and Splunk ingestion cost;
7. roll out to a limited Route before enabling the plugin across services.

The plugin provides a direct path from APISIX access logs to Splunk, but reliable observability still requires monitoring the exporter, network, HEC service, and indexing pipeline.

Related Resources

• Apache APISIX splunk-hec-logging plugin documentation
• Splunk HEC REST API endpoints
• API gateway logging strategies
• Apache APISIX HTTP Logger plugin