Comparison of Message Queue Technologies

for Highly Available Microservices in IoT

Marko Milosavljević, Milica Matić, Neven Jović, Marija Antić

Abstract— Internet of Things (IoT) solutions connect large messages generated by the end devices [2]. Message queuing
numbers of devices, which generate various data and control technologies which are available today differ in terms of the
messages asynchronously. In the IoT system cloud, these performance guarantees they offer, and depending on the
messages need to be queued in order to control the processing actual use-case, metrics such as latency, disk space, RAM
load and prevent the overload in cases of traffic bursts. On the memory or processor usage may be a limiting factor [2], [3].
other hand, one of the requirements the IoT cloud needs to fulfill
is the high availability. Therefore, multiple instances of services The comparison of Kafka and Apache Pulsar has been
accepting and processing the messages generated by the devices performed by the authors in [4], and it has been shown that,
are needed. There are various message queue technologies although Apache Pulsar may achieve better results in terms of
available today, but they all have their limitations. In this paper, resource usage, the maturity of the solution, available
we compare the performance of Apache Kafka and RabbitMQ in documentation, and possibility to integrate with other data
the scenario of the highly available IoT cloud data processing. processing tools, may be a reason to favor Kafka in the
commercial deployment scenarios. On the other hand, Kafka
Index Terms— message queue; high availability; load
balancing; internet of things. and RabbitMQ have been compared in [5], to show that
RabbitMQ has its advantages in terms of the achieved
throughput on a single server instance, but the scaling options
I. INTRODUCTION are on Kafka’s side.
In this paper, we explore the possibility of replacing the
In the past decade, the world is witnessing the expansion of
already implemented RabbitMQ message queueing within the
Internet of Things (IoT) solutions. Within IoT systems,
smart home system cloud [6],[7], with Apache Kafka. Within
different devices are connected to perform a certain function
the deployed smart home cloud, messages generated by end
together. IoT use-cases are various, such as smart transport,
devices are processed by multiple cloud services. As the
smart fabrics, smart cities, smart homes, etc.
number of supported features is growing, so is the number of
In order to collaborate, the devices need to be able to
the cloud services that process these messages. Also, some of
exchange data such as commands and state change reports.
the messages need to be processed by multiple of these
Although the expansion of IoT has led to the development of
services. Additionally, as the number of users grows, the
technologies such as ZigBee, Z-Wave, WiFi or Bluetooth
system needs to be scaled up, and, as already said, Kafka has
Low Energy, which enabled the connection of many different
its advantages in this domain. The paper is organized as
actuators and sensors into large local mesh networks, in order
follows: in Section II, the elements of smart home system and
for an IoT solution to achieve its purpose, the existence of the
its cloud architecture are introduced, then the overview of
cloud component is also needed. The cloud allows remote
RabbitMQ and Kafka is given in Section III and Section IV.
control and monitoring of the local networks, but it can also
Finally, the performed tests and their results are presented in
provide advanced features which require processing of larger
Section V.
quantities of historical system data, or the interaction with
components responsible for customer management, software
II. SMART HOME CLOUD DATA BUFFERING
update and third-party services.
As the data from the IoT system is generated In the existing smart home solution, the end devices within
asynchronously [1], and processing it requires a certain the household use technologies such as ZigBee, Z-Wave and
amount of time, mechanisms are needed to control the cloud ONVIF/IP to connect to the home gateway – Fig. 1. The
load. Usually, this control is achieved by deploying various gateway is responsible to execute the core system logic: it
message queueing systems, that allow to communicate implements the middleware which represents all of the
between different components of the cloud, and react to devices in the same way, regardless of the communication
technology they use in the local network, and allows them to
work together, according to the automation rules set up by the
Marko Milosavljević is with OBLO Living, Novi Sad, Narodnog fronta
21a, Serbia (e-mail: marko.a.milosavljevic@ obloliving.com).
user. To communicate with the user applications and cloud
Milica Matić is with the Faculty of Technical Sciences, University of Novi backend, the gateway uses MQTT protocol. MQTT conveys
Sad, Serbia (e-mail: [email protected]). commands issued by the user, system control messages, and
Neven Jović is with OBLO Living, Novi Sad, Narodnog fronta 21a, Serbia
reports about device state changes. Control messages are
(e-mail: neven.jovic@ obloliving.com).
Marija Antić is with the Faculty of Technical Sciences, University of Novi processed on the cloud side, for the purpose of system
Sad, Serbia (e-mail: [email protected]).

RTI2.6 Page 1 of 4
administration, upgrade, backup and restore. Also, reports it for processing. Message queuing allows web servers to
about device state changes are stored to provide user with the avoid the overload, as they can control the number of the
information about the history of system usage [7]. messages that are processed simultaneously. It is also useful
for distributing messages to multiple consumers sharing the
load and providing fault tolerance.

Fig. 2. RabbitMQ message delivery mechanism.

Producer applications create the messages, but the

messages are not published directly to a queue. First, the
Fig. 1. Smart home system components and communication between them. producer sends the message to the RabbitMQ exchange
running on the broker – Fig. 2. The exchange is responsible
The observed smart home cloud system solution has the for routing the messages to different queues, based on the
microservice-based architecture. It is highly available (HA), configured bindings and routing keys. Four types of
which means that the entire system is fault tolerant, i.e. that exchanges exist - direct, topic, fanout and headers exchange.
there are multiple instances of every microservice running [6]. In the direct exchange, the message is routed to the queue
In order to prevent problems with MQTT messages whose binding key matches the routing key of the message.
processing due to the overload of cloud system, or the failure The topic exchange does a wildcard match between the
of some instances, temporary data buffering is necessary. In routing key and the routing pattern specified in the binding.
the temporary data buffering module all important messages The fanout exchange routes messages to all of the queues
are first queued, allowing relevant microservices to process bound to it. The headers exchange uses the message header
them at their own pace. attributes for routing. Consumers subscribe to the queues and
In the current implementation, RabbitMQ is used for the process the messages from them. All consumers subscribed to
purpose of data buffering. The incoming MQTT messages are the same queue will share the load of processing the messages
parsed by the B2Q (Broker to Queue) microservice, and from that queue. The messages are deleted from the queue
directed to the appropriate RabbitMQ queues, based on the after processing.
information they contain. All of the instances of one cloud
microservice share the load of processing the messages from B. Apache Kafka
the RabbitMQ queue they are associated with. The problem Apache Kafka is an event streaming platform. It is elastic,
here represents the fact that if one message needs to be distributed, highly scalable and fault-tolerant. Similar to
processed in multiple ways (i.e. it is relevant as the input for RabbitMQ, Kafka has the client and server side. Kafka clients
multiple cloud microservices), it has to be replicated to and servers communicate using TCP protocol.
multiple queues. Therefore, in this paper we explore the Kafka implements the publish/subscribe mechanism, and
possibility of replacing RabbitMQ with Apache Kafka. We allows processing streams of events as they arrive into the
implement the B2K (Broker to Kafka) microservice, which system or retrospectively, but also allow to store streams of
publishes messages to Kafka queues, that the processing events as long as they are needed.
microservices are subscribed to, and we compare the
performance of the two implementations.
A. RabbitMQ
RabbitMQ is a message queue manager, which has
originally implemented the Advanced Message Queuing
Protocol (AMQP). Later it was extended to support Streaming
Text Oriented Messaging Protocol (STOMP), Message Queue
Telemetry Transport (MQTT), and other protocols, but
AMQP remains the default and the most widely used one.
RabbitMQ messages can convey any kind of information,
from a simple text message to a message with information
about processes important for the system. Message broker Fig. 3. Kafka message processing mechanism.
stores the message into the queue, until the application fetches

RTI2.6 Page 2 of 4
 Similar to RabbitMQ, the Apache Kafka clients can act as presented in Table I.
producers and consumers – Fig. 3. Producers represent client RabbitMQ reached CPU limit after 16 consumers, but was
applications that write (publish) events to Kafka. On the other able to continue working stably, while the setup with 32
hand, consumers are subscribing to topics, reading and writing consumers stopped working after ten minutes. The throughput
events. Producers and consumers are not aware of each other. of the system was approximately 11000 messages per second.
They work completely independently, and that is a key design Maximum CPU usage was 800%, i.e. all eight cores were
to achieve high scalability. Therefore, producers will never used 100%.
need to wait for consumers. To test Kafka performance, 16 producers were created,
When data is written to Kafka, it is written in the form of an which published to the variable number of partitions (32, 64,
event containing the key, value, timestamp and optional 128). Since Kafka allows only one consumer per partition, the
metadata. Events are stored in topics. The durability of events number of consumers was also varied from 0 to 128. Test
inside Kafka’s topic is configurable. Unlike RabbitMQ, Kafka results are presented in Table II.
events can be read whenever they are needed, because events In any of test cases limit of Kafka maximum CPU load was
are not deleted after consumptions. Events can be stored as not reached. It can be observed that the CPU usage deviation
long as needed. Storing data for a long time does not affect is smaller than in RabbitMQ case. Therefore, the server stays
Kafka. stable, even as the number of messages that are stored in
Topics in Kafka are partitioned, and one Kafka topic can Kafka increases with time.
have any number of partitions defined in the Kafka
configuration file. Events are ordered inside the partition in TABLE II
KAFKA TEST RESULTS
the exactly same order as they were written, and one
consumer can process data from one partition only. However,
the data stored in one partition can be processed by multiple CPU usage on 8 cores [%]
Setup
average maximum deviation
consumers belonging to different consumer groups, i.e. one
16 producers
message can be processed multiple times, without the need to 32 partitions 210 573 65
duplicate it. Offset is an integer number that is used to 0 consumers
maintain the current position of a consumer inside partition. 16 producers
Every topic can be replicated, so that there are dozens of 32 partitions 202 347 43
32 onsumers
brokers that have a copy of data. This makes data fault- 16 producers
tolerant and highly-available. 64 partitions 186 473 90
0 consumers
III. TESTING AND RESULTS 16 producers
64 partitions 208 360 37
Tests were designed to measure CPU load of smart home 64 consumers
system servers when RabbitMQ and Apache Kafka are used 16 producers
for data buffering. RabbitMQ and Kafka brokers were run on 128 partitions 150 300 93
0 consumers
the 8-core Intel i7 processor with 8 GB of RAM memory. 16 producers
128 partitions 480 553 53
TABLE I 128 consumers
RABBITMQ TEST RESULTS

CPU usage on 8 cores [%]

Setup IV. CONCLUSION
average maximum deviation
16 producers This paper gave a brief description of some of the message
8 queues 324 640 108 queueing technologies that can be used for flow control and
0 consumers
load balancing in the IoT scenario. RabbitMQ and Apache
16 producers
8 queues 410 794 197 Kafka were deployed within the smart home system cloud,
8 consumers and their performance was tested for a variable number of
16 producers consumers.
8 queues 486 800 167 The presented test results indicate that data buffering in
16 consumers
16 producers
Kafka is highly stable and has the lower average CPU usage.
8 queues 553 800 147 At any point of testing, maximum CPU usage was never
32 consumers reached. Therefore, in our further work we will focus on
integrating Kafka in the data collection and storage module of
To test the RabbitMQ buffering, 16 producer B2Q the smart home system. Using Kafka will allow us to process
processes were created, that published messages to 8 queues. the same messages multiple times, without the need to
The messages from these queues were processed by a variable duplicate data. This, in turn, opens the possibility to create
number of consumers (0, 8, 16, 32). Producers were advanced data processing scenarios which may bring added
configured to publish messages every 1 ms. Test results are value to the users of the smart home system.

RTI2.6 Page 3 of 4
 V. ACKNOWLEDGMENT [4] S. Intorruk and T. Numnonda, “A Comparative Study on Performance
and Resource Utilization of Real-time Distributed Messaging Systems
This research (paper) has been supported by the Ministry of for Big Data,” Proc. of IEEE/ACIS International Conference on
Education, Science and Technological Development through Software Engineering, Artificial Intelligence, Networking and
Parallel/Distributed Computing (SNPD), July 2019
the project no. 451-03-68/2020-14/200156: “Innovative [5] P. Dobbelaere and K. S. Esmaili, “Kafka versus RabbitMQ: A
scientific and artistic research from the FTS (activity) comparative study of two industry reference publish/subscribe
domain”. implementations: Industry Paper,” Proceedings of the 11th ACM
International Conference on Distributed and Event-based Systems
(DEBS '17), June 2017
REFERENCES [6] M. Matić, E. Nan, M. Antić, S. Ivanović and R. Pavlović, “Model-
[1] F. Metzger, T. Hoßfeld, A. Bauer, S. Kounev and P. E. Heegaard, Based Load Testing in the IoT System,” Proc. of International
“Modeling of Aggregated IoT Traffic and Its Application to an IoT Conference on Consumer Electronics (ICCE-Berlin), Sept. 2019
Cloud,” Proceedings of the IEEE, vol. 107, no. 4, pp. 679-694, April [7] S. Ivanović, M. Antić, I. Papp, N. Jović, “Data Acquisition, Collection
2019 and Storage in Smart Home Solutions,” Proc. of 6th International
[2] G. Fu, Y. Zhang and G. Yu, “A Fair Comparison of Message Queuing Conference on Electrical, Electronic and Computing Engineering
Systems,” IEEE Access, vol. 9, pp. 421-432, Jan. 2021 (IcETRAN), May 2019
[3] H. Wu, Z. Shang and K. Wolter, “Performance Prediction for the
Apache Kafka Messaging System,” Proc. of IEEE
HPCC/SmartCity/DSS, Aug. 2019

RTI2.6 Page 4 of 4