An APISIX Service Mesh with Istio and Amesh
Apache APISIX is primarily used to handle north-south traffic and often sits at the boundary between client applications and backend services.
With the APISIX Ingress Controller, APISIX can also control the ingress-egress traffic in Kubernetes clusters with native configuration.
But as organizations embrace microservices, there is a new challenge to handle the east-west traffic between these microservices.
Service meshes like Istio solve this by removing the networking responsibility from the microservice developer and providing an additional L4/L7 networking layer.
With the new Amesh library and Istio, Apache APISIX can also be used as a service mesh, specifically as a data plane for Istio, bringing all its traffic management capabilities to service-to-service communication.
In this article, we will examine what Amesh is, how it was developed, and how it is used to bring APISIX into a service mesh.
Istio and the xDS Protocol
Istio is one of the most widely used service meshes.
Under the hood, Istio uses Envoy as the reverse proxy in its sidecar containers.
Istio manages traffic by dynamically configuring the sidecars using Envoy's xDS APIs.
The xDS APIs are a way to configure Envoy with incremental changes instead of simple configurations with static files.
Although these APIs were initially intended for configuring Envoy, the APIs have evolved to become a universal data plane API. Any data plane proxy can implement these APIs, and any control plane can use this API to work with these data plane proxies.
In Istio, this means that you can replace the default Envoy data plane with any data plane that implements the xDS APIs. So you can replace Envoy with APISIX to get its traffic management capabilities in a service mesh.
But APISIX does not support the xDS APIs out-of-the-box. And that's where Amesh comes in.
Amesh is a library that translates the data from Istio's control plane to the APISIX configuration.
APISIX replaces Envoy as the data plane for Istio.
Istio communicates to the data plane through the xDS APIs. Amesh supports these APIs and then converts them to APISIX configuration.
This is similar to how APISIX and the APISIX Ingress controller work. The Ingress controller converts the configurations defined using the Ingress or the Gateway API to APISIX format.
Since the xDS APIs are supported by more service meshes like Linkerd and Open Service Mesh, APISIX can also work with them using the Amesh library. Amesh is still in the early stages of development and currently works with Istio v1.13.1.
With Amesh + APISIX, you can use Istio as you would typically. Once you configure traffic rules with Istio's virtual service, APISIX can implement these rules.
APISIX's extended capabilities come through its 80+ plugins. To use APISIX plugins with Istio, we deploy an Amesh control plane component called the Amesh controller.
The Amesh controller takes a plugin configuration defined with the
AmeshPluginConfig CRD and converts it to an APISIX plugin configuration.
All of this will enable us to leverage the full capabilities of APISIX within sidecar containers.
APISIX + Istio Mesh
Let's put everything we learned about in action.
We will build the Amesh images, configure Istio to use APISIX sidecars, deploy Istio, and test everything by running a sample application.
Building the Images
We will build three images:
- amesh-iptables: used for creating an init container to set up some iptables rules. These rules are to direct all traffic through APISIX.
- amesh-sidecar: used for creating the sidecar container.
- amesh-controller: used for creating the Amesh controller control plane component. This controller is used to configure APISIX plugins.
First, clone the Amesh repo:
git clone https://github.com/api7/Amesh.git cd Amesh
You can build and push these images to your own registry.
Add the address of your registry in an environment variable before you run the build, as shown below:
export REGISTRY="docker.io/navendu" make prepare-images
If you don't want to build your own images, you can use these images:
docker pull navendup/amesh-iptables:dev docker pull navendup/amesh-sidecar:dev docker pull navendup/amesh-controller:latest
Deploy Amesh Controller and Install CRDs
We will use Helm to deploy everything to the Kubernetes cluster. I use minikube in these examples.
We will start by creating a new namespace
kubectl create namespace istio-system
To deploy the Amesh controller, run:
helm install amesh-controller -n istio-system \ ./controller/charts/amesh-controller
You also need to install CRDs to work with the Amesh controller:
kubectl apply -k controller/config/crd/
Configure and Deploy Istio + APISIX
Before deploying the service mesh, we will set some environment variables:
export ISTIO_RELEASE=1.13.1 export REGISTRY="docker.io/navendup"
Then we will use Helm to deploy the service mesh:
helm install amesh \ --namespace istio-system \ --set pilot.image=istio/pilot:"$ISTIO_RELEASE" \ --set global.proxy.privileged=true \ --set global.proxy_init.image="$REGISTRY"/amesh-iptables:dev \ --set global.proxy.image="$REGISTRY"/amesh-sidecar:dev \ --set global.imagePullPolicy=IfNotPresent \ --set global.hub="docker.io/istio" \ --set global.tag="$ISTIO_RELEASE" \ ./charts/amesh
Now we have the service mesh and Amesh controller deployed. Next, let's deploy a sample application to test our service mesh.
We will use Istio's Bookinfo sample app.
First, we will add a label to the default namespace to automatically inject sidecars to any pod in the namespace:
kubectl label ns default istio-injection=enabled
Then we can deploy Bookinfo by running:
kubectl apply -f e2e/bookinfo/bookinfo.yaml
This will spin up the Bookinfo application, and each of the pods will have APISIX sidecars:
$ kubectl get pods NAME READY STATUS RESTARTS AGE details-v1-79f774bdb9-cbn87 2/2 Running 0 55s productpage-v1-6b746f74dc-tntc8 2/2 Running 0 55s ratings-v1-b6994bb9-r5j45 2/2 Running 0 55s reviews-v1-545db77b95-n657s 2/2 Running 0 55s reviews-v2-7bf8c9648f-zn97s 2/2 Running 0 55s reviews-v3-84779c7bbc-wn8k2 2/2 Running 0 55s
Testing the Mesh
To access the Bookinfo application, we would need an ingress gateway.
You can use APISIX for this ingress gateway, but that's for another time. For now, we can just use
port-forward to access the
kubectl port-forward productpage-v1-6b746f74dc-tntc8 9080:9080
Now if we open up localhost:9080, we will be able to see our sample application.
Each time you refresh the page, the reviews are pulled from a different version of the reviews service (we deployed three versions).
Now let's apply a rule using virtual services that routes all traffic to v1 versions of the services.
The rule is self-explanatory, and it would look like this:
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: productpage spec: hosts: - productpage http: - route: - destination: host: productpage subset: v1 --- apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: reviews spec: hosts: - reviews http: - route: - destination: host: reviews subset: v1 --- apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: ratings spec: hosts: - ratings http: - route: - destination: host: ratings subset: v1 --- apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: details spec: hosts: - details http: - route: - destination: host: details subset: v1 ---
You can apply it to your cluster by running:
kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.18/samples/bookinfo/networking/virtual-service-all-v1.yaml
Now if we go back to our application and hit refresh, it will stop cycling through the multiple versions of the reviews service and will only route to the v1 version.
Notice how the look of the reviews section changed here. It will remain the same even if you refresh the page.
To summarize, we configure a rule in Istio, and Istio implements it using its sidecar containers with Apache APISIX. Neat!
Amesh is an experimental project, and it is still in its infancy.
Future versions of the project aim to support more features through virtual services.
You can contribute to improving the project or keep track of new versions on GitHub.