ZYLK Achieves Streamlined IoT Services With APISIX Optimization
May 26, 2023
Overview
Alfonso González is an AI & IoT Developer at ZYLK, a leader in optimizing open-source solutions and driving digital transformation in Spain. Alfonso shared their experience of how ZYLK used APISIX for MQTT communications and implemented cybersecurity inside this implementation. He said, "Secure, reliable, and high-performance, APISIX is probably the best gateway for APIs."
Challenges
- Troubled with issues regarding MQTT routing, K8S networking, and HTTP communication.
- Managing and maintaining every accessible endpoint and routing was challenging, which resulted in significant time and effort costs.
- Accommodating new users on the network was challenging as they needed to become familiar with the entire architecture before accessing any service or endpoint.
Results
- ZYLK fixed their HTTP communication issues and MQTT problem by implementing APISIX.
- ZYLK used APISIX as a TCP proxy to create a secure stream route from Raspberry Pi to the RabbitMQ queuing system in the cloud, ensuring the confidentiality and integrity of data transmission.
- ZYLK experienced significant infrastructure enhancements that were more user-friendly, resulting in substantial savings in time and effort and ultimately leading to improved productivity.
Background Information
ZYLK is an experienced IT consultancy specializing in Big Data, Liferay websites and portals, and Artificial Intelligence for over 15 years. The company has been a pioneer in optimizing open-source solutions and driving digital transformation for diverse clients, including private enterprises, public administrations, and government bodies across Spain and Europe since 2004.
ZYLK relies exclusively on open-source software to provide solutions, as they believe that open-source software offers numerous advantages over proprietary alternatives, such as cost-effectiveness, flexibility, transparency, and community support. Before implementing APISIX, the team had to handle their communications endpoint by endpoint and directly reference each endpoint from the client. This led to significant time and effort costs associated with managing and maintaining every accessible endpoint and routing.
The primary pain points of the previous process included accommodating new users on the network who had to become familiar with the entire architecture before accessing any service or endpoint. Additionally, the team faced MQTT routing issues, K8S networking-related issues, and HTTP communication issues.
Technical Selection
ZYLK worked completely with open-source software, including many Apache projects. Therefore, an Apache solution would be nice to try when looking for an API gateway. After searching for a solution to their problems for about six months and comparing it with Kong Gateway and HAProxy, ZYLK ultimately chose to implement APISIX. The reasons are as follows:
1. APISIX is simpler and has more comprehensive security plugins with a user-friendly Dashboard.
APISIX provides a straightforward and easy-to-use interface with a comprehensive set of security plugins, which makes it easier for users to manage and secure their API Gateway. The user-friendly Dashboard also allows for easier configuration and management of the API Gateway.
2. APISIX has support for multiple protocols, not only HTTP.
APISIX supports a wide range of protocols, making it a versatile tool for managing different types of API traffic. This flexibility allows users to manage all their API traffic in one place, simplifying their infrastructure and reducing complexity.
3. APISIX has the ability to redirect incoming MQTT messages with mTLS between the client and server.
APISIX's support for MQTT messaging and mTLS encryption provides an added layer of security for communication between clients and servers. With this feature, incoming MQTT messages can be securely redirected through APISIX, ensuring that they are encrypted and authenticated before reaching their destination. This enhances the overall security of the system and safeguards sensitive information from unauthorized access or tampering.
APISIX proved to be a valuable addition to ZYLK's solutions, providing a simpler and more secure way to manage their API Gateway needs.
Architecture Introduction
1. Industrial devices：PLCs register climate control machinery data (at remote buildings)
Firstly, the focus is on industrial devices such as PLCs, which are used to register data generated by climate control machinery. The climate control system is located in a remote building, which is considered the edge computing infrastructure. The data generated by these devices need to be stored in the cloud servers. The cloud has two parts that share data with each other, and the data is connected by industrial protocols.
2. Industrial protocols：Modbus and BACnet are used to read data from the PLCs
To read the data directly from the PLCs, modules on BACnet are utilized. ZYLK plans to translate these modules and BACnet information into TCP to make it easier to manage. Once the data is in a suitable format, it is sent to the cloud servers using MQTT. While MQTT is straightforward to use for IoT devices, dealing with it in the cloud has presented some challenges for ZYLK, particularly with regard to implementing cybersecurity measures.
3. MQTT communication：TCP conversion is done and data is sent to a Cloud server via MQTT
ZYLK uses APISIX as their solution for secure MQTT communications in their use case. The data generated by the industrial air conditioning machinery is collected by PLCs and translated into TCP language by a TCP gateway for Modbus, then sent to a Raspberry Pi. The data must be sent through the MQTT protocol to reach a queuing system, which could be either Mosquito or RabbitMQ, and ZYLK uses APISIX to accomplish this.
The goal of storing this data is to have historical records of the machinery's functioning, which is used to build machine learning models in the cloud. ZYLK uses NTL models to train and feed on this data. The edge infrastructure is built using a simple diagram, with the industrial air conditioning machinery generating data on the left and being collected by PLCs.
The Raspberry Pi was sending MQTT messages in a non-secure way, which posed a problem for ZYLK since they needed the messages to be secure, at least from APISIX to the exterior because the Raspberry Pi is located in a remote building. ZYLK did not care as much about the security once the messages reached APISIX because the queuing system, RabbitMQ, was located on their own server. However, securing the first part of the communication was crucial to ensure cybersecurity and encrypted messages.
4. ML Models：Once in the Cloud, data are stored and used to train and feed LSTM models
Finally, in the cloud, there is a Kubernetes cluster with the database NIFI. The data generated by the industrial devices is stored in the cloud and used to train and feed LSTM models.
Implementing Process
Step 1：Enable the Plugin
ZYLK describes the steps they took to enable APISIX to work as a TCP proxy and create a stream route from APISIX to the RabbitMQ queuing system. First, they enabled the MQTT proxy plugin by un-commenting a few lines in the config file and allowing the specified port to work as a stream process for TCP messages. It's important to note that by setting this up, they could still use all other APISIX features and work with REST APIs and the HTTP protocol without any problems.
Step 2: Create the Stream Route
Next, ZYLK created the stream route using a special object in APISIX that is similar to roads but does not appear on the dashboard. They used a simple command to specify the plugin they were using, the MQTT protocol name and version, and the SNI (server name indication) to match their certificate for secure communication. They specified the upstream node hosting the RabbitMQ service or any other message queuing system they preferred to use. Enabling APISIX as a TCP proxy and setting upstream routes allowed ZYLK to securely send MQTT messages from the Raspberry Pi to the RabbitMQ queuing system in the cloud. This ensured the cybersecurity and encrypted messages necessary for their use case. The ability to still use other APISIX features and work with REST APIs and HTTP protocol made the solution flexible and adaptable to their needs. ZYLK has successfully tested their solution with the Mosquitto Broker and found it to work perfectly fine.
Step 3: Sign and Load Certificates
After creating the stream route, the next step was to create and sign certificates and load them into the APISIX instance. They used a Python script to create SSL objects with the certificate, key, and SNI, ensuring the SNI matched the one specified in the stream route.
ZYLK emphasizes the importance of ensuring the SNI is correct. Once the certificates were loaded, mutual TLS was established between the client and server, with both showing their certificates. With this setup, ZYLK had secure communication from APISIX to the exterior. They could also encrypt communication between APISIX and their upstream if necessary but chose to leave it unencrypted. ZYLK has used APISIX for other uses and found the plugin gave them additional functionality. They redirected incoming traffic to MQTT brokers, using a client ID to divide messages into different topics if needed. And MQTT authentication was also possible.
The use of APISIX and MQTT allowed ZYLK to solve unique problems and expand their capabilities beyond typical HTTP-related issues. By creating a secure communication channel, they could store and analyze data generated by industrial equipment and use it to train machine learning models for predictive maintenance.
Achievements after Using APISIX
ZYLK highly endorses APISIX and is very willing to recommend it to their business partners. When describing APISIX to a friend, Alfonso González would say, "APISIX is an Apache API gateway service that provides many features for routing, including authentication, rate limiting, and traceability." Alfonso believes that APISIX is the best gateway for APIs.
ZYLK's use case involved industrial devices, industrial protocols, MQTT communication, and ML models. The benefits that ZYLK gained from using APISIX include:
1. Securely Sending MQTT Messages with APISIX in Industrial Settings
Industrial devices such as PLCs register climate control machinery data, which are translated into TCP and sent to the cloud servers using MQTT. APISIX was used to securely send MQTT messages from the Raspberry Pi to the RabbitMQ queuing system in the cloud, ensuring cybersecurity and encrypted messages necessary for their use case.
2. Enabling secure communication with MQTT Proxy Plugin and Mutual TLS
ZYLK enabled the MQTT proxy plugin and created a stream route using a special object in APISIX. They also signed and loaded certificates with a Python script, ensuring the SNI matched the one specified in the stream route. By establishing mutual TLS between the client and server, ZYLK was able to have secure communication from APISIX to the exterior.
3. Expanding Capabilities with APISIX
ZYLK's successful implementation of APISIX and MQTT allowed them to solve unique problems and expand their capabilities beyond typical HTTP-related issues. They could store and analyze data generated by industrial equipment and use it to train machine learning models for predictive maintenance.
ZYLK is committed to staying at the forefront of their industry and providing their clients with the best possible solutions. As they continue to tackle the challenges of the future, they will undoubtedly find new and innovative ways to leverage technologies like APISIX to achieve their goals.
Summary
Before implementing APISIX, ZYLK had to manage their communications endpoint by endpoint, which was time-consuming and inefficient. They faced several issues related to MQTT routing, K8S networking, and HTTP communication.
After researching various API gateway solutions, ZYLK ultimately chose to implement APISIX due to its comprehensive security features, user-friendly dashboard, and simplicity. With APISIX, ZYLK was able to securely expose a port of the router for MQTT messaging communications and internal API routing. They were also able to redirect incoming MQTT messages with mTLS between the client and server. ZYLK also experienced significant improvements in their infrastructure, resulting in time and effort savings.
In terms of future goals, ZYLK sees cybersecurity, IT and OT convergence, and explainable deep learning as the biggest challenges on the horizon for their industry. Over the next six months, the team plans to continue working on R&D projects and developing their own IoT service.