Chapitre 2 :

File de messages pour les applications

réparties
(Message Queue for Distributed
applications)

52
Sofiane Ouni 53
Modèles des communications :
Publish-subscribe

54
Sofiane Ouni 55
56
Rabbit MQ

57
In 2018, it has been used for more than 35 000 companies

58
 Advanced Message Queuing Protocol (AMQP)
• The Advanced Message Queuing Protocol (AMQP) is an open standard
application layer protocol for message-oriented middleware.
• The defining features of AMQP are message orientation, queuing, routing
(including point-to-point and publish-and-subscribe), reliability and
security
These are the known open protocol specifications that cover the same or
similar space as AMQP:
• Streaming Text Oriented Messaging Protocol (STOMP), a text-based protocol
developed at Codehaus; uses the JMS-like semantics of 'destination'.
• Extensible Messaging and Presence Protocol (XMPP), the Extensible Messaging and
Presence Protocol.
• MQTT, a lightweight publish-subscribe protocol.
• OpenWire as used by ActiveMQ.

OASIS (Organization for the Advancement of Structured

Information Standards) international open standards consortium :
• cybersecurity, blockchain, IoT, emergency management,
cloud computing, legal data exchange,…
• American Bar Association, Collabora, Dell, EclecticIQ,
General Motors, IBM, ISO/IEC, KDE e.V., Microsoft, Novell, 59
Oracle, Red Hat,….
 RabbitMQ basic
• RabbitMQ is a message-queueing software called a message broker or queue
manager.
• It is a software where queues can be defined, applications may connect to the
queue and transfer a message onto it.
• The queue-manager software stores the messages until a receiving application
connects and takes a message off the queue.
• The receiving application then processes the message in an appropriate
manner.
• there are client applications called producers that create messages and deliver
them to the broker (the message queue). Other applications, called
consumers, connects to the queue and subscribes to the messages to be
processed

60
1. The user sends a PDF creation request to the web application.
2. The web application (the producer) sends a message to RabbitMQ,
including data from the request, like name and email.
3. An exchange accepts the messages from a producer application and
routes them to correct message queues for PDF creation.
4. The PDF processing worker (the consumer) receives the task and starts
the processing of the PDF. 61
Message flow in RabbitMQ
1. The producer publishes a message to an
exchange. When you create the exchange,
you have to specify the type of it.
2. The exchange receives the message and is
now responsible for the routing of the
message. The exchange takes different
message attributes into account, such as
routing key, depending on the exchange
type.
3. Bindings have to be created from the
exchange to queues. In this case, we see
two bindings to two different queues from
the exchange. The Exchange routes the
message into the queues depending on
message attributes.
4. The messages stay in the queue until they
are handled by a consumer
5. The consumer handles the message.

62
63
 Types of exchanges
• Direct: A direct exchange delivers messages to
queues based on a message routing key. In a
direct exchange, the message is routed to the
queues whose binding key exactly matches the
routing key of the message. If the queue is
bound to the exchange with the binding key
pdfprocess, a message published to the
exchange with a routing key pdfprocess is
routed to that queue.
• Fanout: A fanout exchange routes messages to
all of the queues that are bound to it.
• Topic: The topic exchange does a wildcard
match between the routing key and the
routing pattern specified in the binding.
• Headers: Headers exchanges use the message
header attributes for routing.

64
65
the message headers as opposed to the routing key, may occur in any
order, and may be specified as matching any or all of the specified
headers.
they are represented in the bindings as { "x-match", "any" ..} or
{ “x-match”, “all” ..}.

66
RabbitMQ and server concepts
• Producer: Application that sends the messages.
• Consumer: Application that receives the messages.
• Queue: Buffer that stores messages.
• Message: Information that is sent from the producer to a consumer through RabbitMQ.
• Connection: A connection is a TCP connection between your application and the RabbitMQ
broker.
• Channel: A channel is a virtual connection inside a connection. When you are publishing or
consuming messages from a queue - it's all done over a channel.
• Exchange: Receives messages from producers and pushes them to queues depending on rules
defined by the exchange type. To receive messages, a queue needs to be bound to at least one
exchange.
• Binding: A binding is a link between a queue and an exchange.
• Routing key: The routing key is a key that the exchange looks at to decide how to route the
message to queues. The routing key is like an address for the message.
• AMQP: AMQP (Advanced Message Queuing Protocol) is the protocol used by RabbitMQ for
messaging.

67
 Send-receive RabbitMQ Simple
receive hello message
Send hello message

68
69
70
Lambda Expressions in Java 8 : interface implementation

71
Publish/subscribe pattern

fanout exchange

72
Publish/subscribe pattern : Fanout

73
Using Exchange, routing key, Binding Key

74
Emitting logs simple
• We'll use this model for our logging system. We will supply the log severity as a
routing key. That way the receiving program will be able to select the severity it
wants to receive.
• We assume that 'severity' can be one of 'info', 'warning', 'error'.

75
• we need to create an exchange first. we'll send messages to a direct
exchange :

• we're ready to send a message with severity routingKey ('info',

'warning', 'error’):

• Subscribe and create a new binding for each severity:

76
EmitLogDirect info some_Informations

77
ReceiveLogsDirect info warning

78
Read message
without
DeliverCallBack
function

rabbitmq-java-client API
> 4.x.x

79
 Example TD1
Réaliser un broker qui permet de classifier les messages
arrivant dans une file en deux files : une prioritaire et une autre
moins prioritaire.

80
81
Réception des messages prioritaires avant !

82
Header exchange simple

83
84
85
86
87
88
89
 Acknowledgment in RabbitMQ

• If a consumer disconnects for various reasons before

managing to send an acknowledgment to confirm that the
message has been received, the broker will redeliver it to
another consumer. If none are available at the moment,
the broker will wait until at least one consumer is registered
for the same queue before attempting re-delivery.

• When a consumer receives a message, processing it may or

may not succeed. It can signal to the broker l by explicitly
rejecting a message. In this case, an application can request
the broker to discard or add it back to the queue.

90
 Delivery guarantee problem
• How to make the delivery
guarantee form publisher to
subscriber ?
• Delivery means that the
message is received for one or
may and processed correctly

91
 Transaction support with RabbitMQ
A consumed message from an RabbitMQ AMQP queue,
an unexpected error occurred.
=> This will result in a state that the message was consumed from the
queue but it was not used for the intended purpose.

To avoid these kind of

situation, RabbitMQ
Client provide inbuilt
support with
auto acknowledgement
and transactions. Exception!

92
93
• setting autoAcknowledgement as false when consuming the
message, so that message remains in the queue and then start to
process the message.
• During processing, I’m checking for a condition. If that condition is
not satisfied I roll back the transaction, and if it is satisfied, I commit
the transaction by sending an acknowledgement.

94
ACK management

95
 Acknowledging Multiple Deliveries at Once
• Manual acknowledgements can be batched to reduce network traffic. This is done by setting the multiple field of
acknowledgement methods (see above) to true.
• For example, given that there are delivery tags 5, 6, 7, and 8 unacknowledged on channel Ch, when an acknowledgement
frame arrives on that channel with delivery_tag set to 8 and multiple set to true, all tags from 5 to 8 will be acknowledged.
If multiple was set to false, deliveries 5, 6, and 7 would still be unacknowledged.
• To acknowledge multiple deliveries with RabbitMQ Java client, pass true for the multiple parameter to Channel#basicAck :

96
 Negative Acknowledgement and Requeuing of Deliveries
• Sometimes a consumer cannot process a delivery immediately but other instances
might be able to. In this case it may be desired to requeue it and let another
consumer receive and handle it

97
 Channel Prefetch Setting (QoS)

• using the basic.qos method: defines the max number of unacknowledged

deliveries that are permitted on a channel. Once the number reaches the
configured count, RabbitMQ will stop delivering more messages on the
channel unless at least one of the outstanding ones is acknowledged.

98
Security : TLS(Transport Layer Security) Support

99
This very basic example will show a simple client
connecting to a RabbitMQ server over TLS :
- Without validating the server certificate,
- and without presenting any client certificate
to the server.

100
https://www.rabbitmq.com/ssl.html
 RabbitMQ Multi-tenant system and Access Control
• RabbitMQ is multi-tenant system:
• connections, exchanges, queues, bindings, user permissions, policies and some other things belong to
virtual hosts, logical groups of entities.
• virtual hosts : If you are familiar with virtual hosts in Apache or server blocks in Nginx, the idea is similar.
• Different users can be granted access only to specific virtual hosts. Their permissions in each virtual host
also can be limited.

1. Creating a vhost :

Virtual hosts can have

metadata associated

2. Adding a User:

3. Granting Permissions to a User:

https://en.wikipedia.org/wiki/Regular_expression
^ : Matches the starting position within the string.
$ : Matches the ending position of the string
101
https://stackoverflow.com/questions/7840283/how-can-queues-be-made-private-secure-in-rabbitmq-in-a-multitenancy-system
Static RabbitMQ Cluster Reference Architecture
• Each RabbitMQ broker runs on a Host node that could be a vm with
network connectivity to the other Host nodes in the cluster.

102
 RabbitMQ in cluster

• We can run some instances of RabbitMQ

provided in docker containers in the
cluster with highly available (HA)
queues.
• message broker handles a large number
of incoming messages.

103
 RabbitMQ Cluster running in Docker
Containers
• We can run the RabbitMQ brokers
clustered across Docker Containers on
a single host. One of the bigger
differences is that Docker is handling
network routing at the Host-level to
take traffic from a set of TCP ports:
5672, 5673, and 5674 and maps them
to each internal Container’s port
where the RabbitMQ broker listens on
TCP port 5672.

https://docs.docker.com/compose/gettingstarted/ 104
• docker run -d --hostname rabbit1 --name rabbit1 -e RABBITMQ_ERLANG_COOKIE='rabbitcluster' -
p 30000:5672 -p 30001:15672 rabbitmq:management

now there are three RabbitMQ running instances. We can go to the UI management console
for all of those instances available as docker containers, for example
http://192.168.99.100:30001 (rabbitmq). Each instance is available on its independent cluster

https://tanzu.vmware.com/developer/guides/messaging-and-integration/rabbitmq-gs/ 105
 Reverse proxy for load distribution
• The idea here is to create a RabbitMQ cluster composed by 3 nodes, having a
reverse proxy sitting in front of the cluster distributing the load between the
nodes.

http://throughaglass.io/technology/RabbitMQ-cluster-with-Docker-and-Docker-Compose.html
106
 Additional Features
• Routing: Directing messages to specific queues based on criteria.
• Persistence: Storing messages even if RabbitMQ restarts.
• Clustering: Scaling horizontally by distributing queues across nodes.
• Message Acknowledgement: Ensuring that messages are successfully processed.
• Priority Queues: Handling high-priority messages first.
• Plugins: Extending functionality as needed.

Guarantees for Message Ordering:

•RabbitMQ guarantees message ordering under specific conditions:
• Messages published in one channel, passing through one exchange, one queue, and one
outgoing channel will be received in the same order they were sent.
• However, with multiple parallel consumers, strict order of processing cannot be guaranteed

it a reliable choice for message delivery in distributed systems

107
 Exercice
• Soit le schéma suivant représentant les opérations et les services invoqués lors d’achat à partir d’un site
marchand de e-commerce. Soit les modules/micro-services répartis : site e-commerce (eCommerce site),
achat et inventaire (Sales and inventory), facturation et paiement (billing), livraison (shipping). A partir
du site, on peut réaliser une commande (PlaceOrder), modifier la commande, annuler la commande. Ces
opérations seront suivi par une suite de traitement comme le montre la figure suivante.

108
1-Réaliser le schéma représentant les files de messages et les exchanges de broker RabbitMQ. Dans ce cas
on suppose, l’utilisation du mode Fanout pour l’exchange. (l’hexagone pour représenter l’exchange, le
losange pour représenter une file). Pour la solution, on réalise une seule file par application ou module et
le mode fanout d’exchange par événement.

109
2- La même question que la précédente. Utiliser le mode topics pour l’exchange du broker RabbitMQ.

110
Other MOM (message-oriented middleware) :
ZeroMQ
• ZeroMQ is an open-source, high-performance messaging library.
• operates without a dedicated message broker
• ZeroMQ is a service provider of messaging. Using these providers, a
messaging API is required to send and receive messages.
• ZeroMQ is especially suited for high performance and low latency
scenarios.

111
 Components of ZeroMQ
• Sockets: Using sockets, the user communicates with ZeroMQ. The
sockets are much like TCP sockets; the difference between them is
that each socket is able to handle multiple peer communication.
• Worker Thread: Various objects are residing in the worker thread.

112
Basis of Comparison ZeroMQ RabbitMQ

To implement
Developed by a broad AMQP(Advanced
Origin contributor group, which iMatix Messaging Queueing
created. Protocol).

Slower when compared to

Performance It is Faster.
ZeroMQ.

Throughput 66K messages/sec 4K-10K messages/sec

Message Retention No retention. Acknowledgement based.

ZeroMQ does not support

Persistence Supports persistence.
persistence.

113
114