Using SOAP-to-REST to Simplify Migration and Integration

1. What Is Web Service?

A web service is defined by the World Wide Web Consortium (W3C) as a software system designed to support interoperable machine-to-machine interaction over a network.

A web service performs specific tasks or a set of tasks and is described by a standard XML representation of a service called Web Services Description Language (WSDL). The service description provides all the necessary details for interacting with the service, including message formats (used to describe operations in detail), transport protocols, and location.

Other systems would interact with the web service using SOAP messages, usually by using HTTP in conjunction with XML serialization and other web-related standards.

The architecture diagram of a web service (note that the service broker is optional) is:

*Image source (licensed under CC 3.0 BY-SA): https://en.wikipedia.org/wiki/Web_service

The WSDL interface hides detailed information about how the service is implemented so that the service's use is independent of the hardware or software platform that implements the service, as well as the programming language used to write the service.

Applications based on web services are loosely coupled, component-oriented, and cross-technology implementations. Web services can be used individually or with other web services to perform complex aggregation or business transactions.

Web services are the implementation units of Service-oriented architecture (SOA), a design method used to replace monolithic systems. An extensive system can be broken down into multiple web services, which can then be combined into a large black box to provide business logic externally. The popular container-based microservices, are the latest replacement for web services. However, many legacy systems are already based on web services for implementation and operation, so compatibility with these systems remains a challenge despite the rapid evolution of technology.

WSDL (Web Services Description Language)

WSDL is an XML notation used to describe web services. The WSDL definition instructs clients on how to write web service requests and defines the interface provided by the web service provider.

The WSDL definition is divided into multiple parts, specifying the web service's logical interface and physical details. Physical details include endpoint information, such as the HTTP port number, as well as binding information that specifies how to represent SOAP message content and which transport method to use.

Image source (licensed under CC 3.0 BY-SA): https://en.wikipedia.org/wiki/Web_Services_Description_Language

• A WSDL file can contain multiple services.
• A service can contain multiple ports.
• A port defines a URL address (which can vary for each port) and can contain multiple operations.
• Each operation includes an input type and an output type.
• Types define the message structure, including which fields compose the message, each field's data type (which can be nested), and the number of fields allowed.

1.1 What Is SOAP

SOAP is an XML message format used in web service interactions. SOAP messages are typically sent via HTTP or JMS, but other transport protocols can also be used. The WSDL definition describes SOAP usage in a specific web service.

There are two commonly used versions of SOAP: SOAP 1.1 and SOAP 1.2.

Image source (licensed under CC 3.0 BY-SA): https://en.wikipedia.org/wiki/SOAP

A SOAP message contains the following parts:

• Header metadata, which is usually empty
• Body
  • The message type defined in WSDL
  • In addition to successful responses, there are also structured error messages for response types.

For example:

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
  <SOAP-ENV:Header></SOAP-ENV:Header>
  <SOAP-ENV:Body>
    <ns2:getCountryResponse xmlns:ns2="http://spring.io/guides/gs-producing-web-service">
      <ns2:country>
        <ns2:name>Spain</ns2:name>
        <ns2:population>46704314</ns2:population>
        <ns2:capital>Madrid</ns2:capital>
        <ns2:currency>EUR</ns2:currency>
      </ns2:country>
    </ns2:getCountryResponse>
  </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

1.2 What Is REST

A web service is an abstract concept that can be implemented in any way; for example, REST is a popular implementation method.

REST, short for Representational State Transfer, literally means state transfer of the presentation layer. Roy Thomas Fielding proposed the term REST in his doctoral dissertation in 2000. At that time, the Internet was booming, and there was a need for a practical definition of software development and interaction with networks.

For a long time, software research has mainly focused on the classification of software design and the evolution of design methods. It rarely objectively evaluates the impact of different design choices on system behaviour. On the other hand, network research mainly focuses on the details of communication behaviour between systems and how to improve the performance of specific communication mechanisms, often overlooking that changing the application's interaction style has a greater impact on overall performance than changing the interaction protocol. My article aims to understand and evaluate the architecture design of network-based application software that is functional, performs well, and is suitable for communication while complying with architectural principles.

Accessing a website represents an interaction process between the client and the server. During this process, there must be changes in data and status. The HTTP protocol is stateless, meaning all states are stored on the server side. Therefore, if the client wants to operate the server, it must use some means to cause "state transfer" on the server side. Since this transfer is built on the presentation layer, called "presentation layer state transfer."

The four basic principles of REST are:

1. Use HTTP verbs: GET, POST, PUT, DELETE;
2. Stateless connection, the server should not store too much contextual state; that is, each request is independent;
3. Set a URI for each resource;
4. Use x-www-form-urlencoded or JSON as the data format;

Converting SOAP to REST can facilitate users to access traditional web services in a RESTful manner, reducing the development cost of SOAP clients. Furthermore, if it can dynamically adapt to any web service with zero-code development, it would be even more perfect.

The most significant advantage of REST is that it has no schema, making it convenient to develop, and JSON has higher readability and lower redundancy.

2. Traditional Implementations of SOAP-to-REST Proxy

2.1 Manual Template Conversion

This approach requires providing conversion templates for the request and response for each operation of the web service, which is also used by many other gateway products.

We can use the APISIX body transformer plugin to create a simple SOAP-to-REST proxy and try this approach.

For example, we construct an XML formatted request template for the getCountry operation of the CountriesPortService in the WSDL file based on the type definition.

Here, we fill the name field in the JSON into the name field in the getCountryRequest.

req_template=$(cat <<EOF | awk '{gsub(/"/,"\\\"");};1' | awk '{$1=$1};1' | tr -d '\r\n'
<?xml version="1.0"?>
<soap-env:Envelope xmlns:soap-env="http://schemas.xmlsoap.org/soap/envelope/">
 <soap-env:Body>
  <ns0:getCountryRequest xmlns:ns0="http://spring.io/guides/gs-producing-web-service">
   <ns0:name>{{_escape_xml(name)}}</ns0:name>
  </ns0:getCountryRequest>
 </soap-env:Body>
</soap-env:Envelope>
EOF
)

An XML-to-JSON template needs to be provided for the response, which can be a bit more complex (especially if SOAP version differences need to be considered). This is because it needs to be determined whether the response was successful:

• For a successful response, the fields can be directly mapped to JSON.
• For a failed response (i.e., a fault), a separate JSON structure is needed, and it must be determined whether specific optional fields are present.

rsp_template=$(cat <<EOF | awk '{gsub(/"/,"\\\"");};1' | awk '{$1=$1};1' | tr -d '\r\n'
{% if Envelope.Body.Fault == nil then %}
{
   "currency":"{{Envelope.Body.getCountryResponse.country.currency}}",
   "population":{{Envelope.Body.getCountryResponse.country.population}},
   "capital":"{{Envelope.Body.getCountryResponse.country.capital}}",
   "name":"{{Envelope.Body.getCountryResponse.country.name}}"
}
{% else %}
{
   "message":{*_escape_json(Envelope.Body.Fault.faultstring[1])*},
   "code":"{{Envelope.Body.Fault.faultcode}}"
   {% if Envelope.Body.Fault.faultactor ~= nil then %}
   , "actor":"{{Envelope.Body.Fault.faultactor}}"
   {% end %}
}
{% end %}
EOF
)

Configure APISIX routing and perform testing:

curl http://127.0.0.1:9180/apisix/admin/routes/1 \
    -H 'X-API-KEY: xxx' -X PUT -d '
{
    "methods": ["POST"],
    "uri": "/ws/getCountry",
    "plugins": {
        "body-transformer": {
            "request": {
                "template": "'"$req_template"'"
            },
            "response": {
                "template": "'"$rsp_template"'"
            }
        }
    },
    "upstream": {
        "type": "roundrobin",
        "nodes": {
            "localhost:8080": 1
        }
    }
}'
curl -s http://127.0.0.1:9080/ws/getCountry \
    -H 'content-type: application/json' \
    -X POST -d '{"name": "Spain"}' | jq
{
  "currency": "EUR",
  "population": 46704314,
  "capital": "Madrid",
  "name": "Spain"
}
# Fault response
{
  "message": "Your name is required.",
  "code": "SOAP-ENV:Server"
}

As we can see, this method requires a manual understanding of the definition of each operation in the WSDL file and the web service address corresponding to each operation. If the WSDL file is large, contains a large number of operations, and has complex nested type definitions, then this approach can be very cumbersome, difficult to debug, and prone to errors.

2.2 Apache Camel

https://camel.apache.org/

Camel is a well-known Java integration framework used for implementing routing pipelines that convert different protocols and business logic, and SOAP-to-REST is just one of its use cases.

Using Camel requires downloading and importing the WSDL file, generating stub code for the SOAP client, and writing Java code:

• Defining REST endpoints
• Defining protocol conversion routes, such as how JSON fields map to SOAP fields

As an example, let's consider a web service for temperature unit conversion:

https://apps.learnwebservices.com/services/tempconverter?wsdl

1. Generate SOAP client code based on WSDL file using Maven.

cxf-codegen-plugin will generate SOAP client endpoints for us to access the web service.

<build>
  <plugins>
    <plugin>
      <groupId>org.apache.cxf</groupId>
      <artifactId>cxf-codegen-plugin</artifactId>
      <executions>
        <execution>
          <id>generate-sources</id>
          <phase>generate-sources</phase>
          <configuration>
            <wsdlOptions>
              <wsdlOption>
                <wsdl>src/main/resources/TempConverter.wsdl</wsdl>
              </wsdlOption>
            </wsdlOptions>
          </configuration>
          <goals>
            <goal>wsdl2java</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

2. Writing SOAP client bean.

Note that we remember the name of the bean as cxfConvertTemp, which will be used when defining Camel routes later on.

import com.learnwebservices.services.tempconverter.TempConverterEndpoint;
@Configuration
public class CxfBeans {
    @Value("${endpoint.wsdl}")
    private String SOAP_URL;
    @Bean(name = "cxfConvertTemp")
    public CxfEndpoint buildCxfEndpoint() {
        CxfEndpoint cxf = new CxfEndpoint();
        cxf.setAddress(SOAP_URL);
        cxf.setServiceClass(TempConverterEndpoint.class);
        return cxf;
    }
}

3. Write the downstream REST route first.

From this route, we can see that it defines RESTFul-style URLs and their parameter definitions and defines the next hop route for each URL. For example, /convert/celsius/to/fahrenheit/{num} takes the last part of the URL as a parameter (of type double) and provides it to the next hop route direct:celsius-to-fahrenheit.

rest("/convert")
    .get("/celsius/to/fahrenheit/{num}")
    .consumes("text/plain").produces("text/plain")
    .description("Convert a temperature in Celsius to Fahrenheit")
    .param().name("num").type(RestParamType.path).description("Temperature in Celsius").dataType("int").endParam()
    .to("direct:celsius-to-fahrenheit");

4. Finally, write the upstream SOAP route and the transformation between upstream and downstream.

from("direct:celsius-to-fahrenheit")
    .removeHeaders("CamelHttp*")
    .process(new Processor() {
        @Override
        public void process(Exchange exchange) throws Exception {
            // Initialize SOAP request
            // Fill the downstream parameter num into the body, which is a simple double type.
            CelsiusToFahrenheitRequest c = new CelsiusToFahrenheitRequest();
            c.setTemperatureInCelsius(Double.valueOf(exchange.getIn().getHeader("num").toString()));
            exchange.getIn().setBody(c);
        }
    })
    // Specify SOAP operation and namespace
    // Defined in the application.properties file
    .setHeader(CxfConstants.OPERATION_NAME, constant("{{endpoint.operation.celsius.to.fahrenheit}}"))
    .setHeader(CxfConstants.OPERATION_NAMESPACE, constant("{{endpoint.namespace}}"))
    // Send the package using the SOAP client bean generated by WSDL.
    .to("cxf:bean:cxfConvertTemp")
    .process(new Processor() {
        @Override
        public void process(Exchange exchange) throws Exception {
            // Handle SOAP response
            // Fill the body, which is a double type value, into the string
            // Return the string to the downstream
            MessageContentsList response = (MessageContentsList) exchange.getIn().getBody();
            CelsiusToFahrenheitResponse r = (CelsiusToFahrenheitResponse) response.get(0);
            exchange.getIn().setBody("Temp in Farenheit: " + r.getTemperatureInFahrenheit());
        }
    })
    .to("mock:output");

5. Test

curl localhost:9090/convert/celsius/to/fahrenheit/50
Temp in Farenheit: 122.0

As we can see, using Camel for SOAP-to-REST requires defining routes and conversion logic for all operations using Java code, which incurs development costs.

Similarly, if the WSDL contains many services and operations, using Camel for proxying can also be painful.

2.3 Conclusion

Let's summarize the drawbacks of traditional methods.

Both of these approaches have development costs, and for every new web service, this development cost needs to be repeated.

The development cost is proportional to the complexity of the web service.

3. APISIX's SOAP-to-REST Proxy

Traditional proxy approaches either require providing conversion templates or writing conversion code, both of which require users to deeply analyze WSDL files and incur significant development costs.

APISIX provides an automated approach that automatically analyzes WSDL files and provides conversion logic for each operation, eliminating development costs for users.

This is an Enterprise plugin, contact us for more information.

3.1 Codeless Automatic Conversion

Using the APISIX SOAP proxy:

• No manual parsing or importing of WSDL files is required
• No need to define conversion templates
• No need to write any conversion or coupling code.

Users only need to configure the URL of the WSDL, and APISIX will automatically perform the conversion. It is suitable for any web service, is a generic program, and does not require secondary development for specific needs.

3.2 Dynamic Configuration

• The WSDL URL can be bound to any route and, like other APISIX resource objects, can be updated at runtime. Configuration changes take effect dynamically without requiring restarting APISIX.
• The service URL(s) contained in the WSDL file (which may have multiple URLs), i.e., the upstream address, will be automatically recognized and used as the SOAP upstream without the need for users to parse and configure them.

3.3 Implementation Mechanism

• Obtain the contents of the WSDL file from the WSDL URL and automatically generate a proxy object after analysis.
• The proxy object is responsible for protocol conversion:
  • Generate compliant SOAP XML requests based on JSON input.
  • Convert SOAP XML responses to JSON responses.
  • Access the web service and automatically handle SOAP protocol details, such as fault-type responses.
  • Support SOAP1.1 and SOAP1.2 and several extension features, such as WS-Addressing.

3.4 Example Configuration

Explanation of configuration parameters for the SOAP plugin:

Test:

# Configure APISIX routing using the SOAP plugin
# Note that a single route can execute all operations, with the operation name specified using URL parameters
# This also reflects the benefits of the dynamic proxy, eliminating the need for manual analysis of each operation in the WSDL file.
curl http://127.0.0.1:9180/apisix/admin/routes/1 \
    -H 'X-API-KEY: xxx' -X PUT -d '
{
    "methods": ["POST"],
    "uri": "/getCountry",
    "plugins": {
        "soap": {
            "wsdl_url": "http://localhost:8080/ws/countries.wsdl"
        }
    }
}'
curl 'http://127.0.0.1:9080/getCountry' \
    -X POST -d '{"name": "Spain"}'
# Call succeeded
HTTP/1.1 200 OK
Date: Tue, 06 Dec 2022 08:07:48 GMT
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Server: APISIX/2.99.0
{"currency":"EUR","population":46704314,"capital":"Madrid","name":"Spain"}
# Call failed
HTTP/1.1 502 Bad Gateway
Date: Tue, 03 Jan 2023 13:43:33 GMT
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Server: APISIX/2.99.0
{"message":"Your name is required.","actor":null,"code":"SOAP-ENV:Server","subcodes":null,"detail":null}

4. Conclusion

In today's world, web services have advanced to the point where many enterprise users rely on traditional SOAP-based web services to provide their services. Due to historical reasons and cost considerations, these services are not always suitable for a complete refactoring into RESTful services. As a result, there is a significant demand for SOAP-to-REST among many enterprise users.

The SOAP-to-REST plugin provided by APISIX can achieve zero-code proxy functionality, can be dynamically configured, and does not require secondary development, which is beneficial for enterprise users' zero-cost business migration and integration.