Fr

ee

RabbitMQ is one of the most powerful pieces of open

source Message Broker software and is widely used in
companies such as Mozilla, VMware, Google, and AT&T.
RabbitMQ gives you lots of fantastic and easy-to-manage
functionalities to control and manage its messaging facility,
with lots of community support.

Who this book is written for

If you are an intermediate-level RabbitMQ developer who
wants to achieve professional-level expertise in the subject,
this book is for you. You'll also need to have a decent
understanding of message queuing.

Administer RabbitMQ using different tools

Configure the RabbitMQ Server using its
powerful configuration structures
Understand the roots and details of
messaging, message brokers, and the
AMQP protocol

Monitor the RabbitMQ server using different

open source tools such as Nagios, Munin,
and Zabbix
Ensure your RabbitMQ's security using SSL,
SASL, and access control

Yusuf Ayta

P U B L I S H I N G

C o m m u n i t y

Extend RabbitMQ by developing Erlang

OTP-based applications that use the
RabbitMQ API

$ 54.99 US
34.99 UK

community experience distilled

Scale the RabbitMQ Server using clusters

and high availability techniques

Emrah Ayanoglu
Dotan Nahum

You'll begin with the installation and configuration of the

RabbitMQ Server. Next, you'll study the major problems that
your server faces, including scalability and high availability,
and try to find solutions by using RabbitMQ mechanisms.
You'll then get to design and develop your own plugins
using the Erlang language and RabbitMQ's internal API.
This knowledge will help you to start with the management
and monitoring of messages, tools, and applications. You'll
also walk through the security and integrity of the messaging
facilities that RabbitMQ provides. In the last few chapters,
you will build and keep track of your clients using Java,
Python, and C#.

What you will learn from this book

pl

Mastering RabbitMQ

Mastering RabbitMQ

Sa
m

E x p e r i e n c e

Mastering RabbitMQ
Master the art of developing message-based applications
with RabbitMQ

Prices do not include

local sales tax or VAT
where applicable

Visit www.PacktPub.com for books, eBooks,

code, downloads, and PacktLib.

D i s t i l l e d

Emrah Ayanoglu
Dotan Nahum

Yusuf Ayta

In this package, you will find:

The authors biography

A preview chapter from the book, Chapter 1 'Getting Started'
A synopsis of the books content
More information on Mastering RabbitMQ

About the Authors

Emrah Ayanoglu has been into technology since a young age, when he was

programming with his Tandy 1000 using Basic language. His deep interest and
passion for programming lead to him pursue computer engineering at Bilkent
University, Ankara. He now works as a software engineer and heavily works on
integrating software systems using RabbitMQ.
He frequently speaks at different tech conferences about scalability and real-time
web applications where RabbitMQ has a major role. Additionally, he participates in
different open source projects.
For the future, he hopes to participate more in open source projects and work on the
real-time scalable applications.

Yusuf Ayta is a software engineer. He completed his B.S. and M.S. in computer

science from Bilkent University, Ankara. He has worked in both early stage startups
and multinational companies. He is proficient in agile methodologies, continuous
delivery, and software development best practices.

Preface
RabbitMQ is an open source messaging broker. It's often referred to as a messageoriented middleware that implements the Advanced Message Queuing Protocol
(AMQP). Fundamentally, RabbitMQ provides a common platform for sending and
receiving messages, where it guarantee the safety of messages until they are received.
By playing an intermediary role between message consumers and producers, AMQP
makes it easy to decouple applications.
Out of the box, RabbitMQ provides support for many messaging patterns.
RabbitMQ guarantees data delivery, provides non-blocking operations, and sends
push notifications. Moreover, it provides infrastructure for publish/subscribe,
asynchronous processing, and work queues.
RabbitMQ provides a variety of features, including the tuning of application
performance, clustering, flexible routing, federation, and so on. If you need specific
features, RabbitMQ has several plugins that cater different needs. The RabbitMQ
plugins extend its features in different ways, and you can also write your own plugin.
Through this book, we aim to give you a deep understanding of RabbitMQ and its
use cases by providing multiple opportunities to learn about the message-oriented
middleware, messaging architecture, messaging patterns, and solutions to real-life
scenarios using RabbitMQ.

What this book covers

This book covers many aspects of software development with RabbitMQ. It provides
thorough understanding of messaging, RabbitMQ, message-oriented software
development, and so on.
Chapter 1, Getting Started, introduces you to message queues, message brokers,
AMQP, and RabbitMQ.

Preface

Chapter 2, Configuring RabbitMQ, covers the configuration opportunities in RabbitMQ

in detail.
Chapter 3, Architecture and Messaging, goes over RabbitMQ componentsProducer,
Message Broker, Consumer and the Message. This chapter provides learning
opportunities for interoperability, heterogeneous integration, scalability, and so on.
Chapter 4, Clustering and High Availability, provides opportunities to tune RabbitMQ
for high availability, federation, and much more.
Chapter 5, Plugins and Plugin Development, highlights several important features of
RabbitMQ and gives an insight into creating your own plugin.
Chapter 6, Managing Your RabbitMQ Server, covers in detail the management of
RabbitMQ using the command-line tools, management plugin, and rest API.
Chapter 7, Monitoring, discusses the methods to monitor RabbitMQ instances through
a command line, management plugin, and well-known monitoring software.
Chapter 8, Security in RabbitMQ, covers the details about potential security
vulnerabilities and securing RabbitMQ.
Chapter 9, Java RabbitMQ Client Programming, talks about developing RabbitMQ client
using the Java platform.
Chapter 10, Ruby Client Programming, talks about developing a RabbitMQ client using
Ruby.
Chapter 11, Python Client Programming, demonstrates developing a RabbitMQ client
using Python.

Getting Started
Scalability is one of the major problems of our time, and messaging is an integral part
of the solution. It finally comes down to the message broker software to manage and
control messaging between applications, processes, and threads. Message brokers
can help to solve scalability issues and architectural issues, such as coupling.
RabbitMQ is one of the most powerful open source message broker software that is
widely used in the tech companies such as Mozilla, VMware, Google, AT&T, and so
on. RabbitMQ is a highly configurable messaging platform developed and supported
by a knowledgeable and committed community.
Before diving into the details and technologies behind the RabbitMQ, let's introduce
you to the topics that we will cover in this chapter:

A brief introduction to message brokers and the message queue

An introduction to advanced message queue protocol

Getting started with RabbitMQ

Installing RabbitMQ

Starting RabbitMQ

Summary

[1]

Getting Started

Message brokers and message queue

Recently, software systems evolved dramatically. Applications have to communicate
with other applications, these applications can be internal and external to the
application itself. For the same application, we may have different type of
clients, such as browsers, mobile clients, and so on. Hence, we absolutely need a
communication layer between internal applications and between applications and
clients. We need to deliver different messages to different applications or clients.
Delivering messages can be a bottleneck if the communication layer isn't scalable.
Pursuing scalable systems for communication layer leads us to Message Brokers and
Message Queues. Let's now discuss what Message Brokers and Message Queues are.

Message brokers
A Message Broker is an architectural pattern that can receive messages from multiple
destinations, determine the correct destination, and route the message along the
correct route, as stated in the book Enterprise Integration Patterns by Hohpe and
Woolf. Message brokers enable systems to deal with messaging and routing by
mediating communication among components. Once applications implement a
message broker pattern, it decreases the coupling between application components.
Message Brokers are centralized, in the architectural sense, to control and manage
all messages. Therefore, all of the incoming and outgoing messages are sent
through Message Brokers, which analyze and deliver the messages to their correct
destination. This procedural step can be understood with the following diagram:

Message Broker

[2]

Chapter 1

Message Brokers address the following concerns in the communication layer:

Transforming messages to alternative formats

Routing messages to destinations

Supporting different types of patterns to send messages

Receiving and responding to events

Performing message aggregation

Persisting the message states

Ensuring the receiving and sending of message

Decoupling the destination software systems

Many tasks of the Message Broker need a Message Queue for exchanging or passing
data to the destination. The next section covers Message Queues. We will talk about
the mechanism behind Message Brokers in Chapter 3, Architecture and Messaging.

Message Queues
A Message Queue is, briefly, a queue for messaging. Queue is the basic data structure
behind the functioning of a Message Queue. Message Queue operations are similar
to Queue data structure operations, such as the enqueue and dequeu operations. An
enqueue operation leads to adding an element to the back of the queue. A dequeue
operation leads to the deletion of an element from the front of the queue.
Message Queues provide concurrent and asynchronous operations to scale
applications. In a message queue, messages wait up until a message is retrieved by
an application. Let's take a look at the following diagram:

Message Queue

Different types of standards and protocols define the Message Queuing

specifications. Some protocols are open to everyone; however, some protocols are
closed. Let's come back to our topic. RabbitMQ uses Advanced Message Queuing
Protocol (AMQP) that determines the policies of the Message Queues. The next topic
will cover detailed information on AMQP. Chapter 3, Architecture and Messaging,
covers the detailed explanation of Message Queues.
[3]

Getting Started

An introduction to the advanced

message queue protocol
John O'Hara from J. P. Morgan started AMQP in 2003. He put incredible amount of
work into it. Then, J. P. Morgan approached other firms to establish an organization
for creating open standards in messaging. According to AMOP's official website
(http://www.amqp.org), AMQP is an open standard for passing messages between
applications or organizations. So, AMQP just defines the messaging properties,
queue properties, how messages are routed between applications and clients, how
Message Brokers ensure that the message is received or sent, and other concerns such
as reliability and security.
According to the AMQP website (http://www.amqp.org), AMQP has lots of
capabilities to accomplish goals:

Security

Reliability

Interoperability

Standard

Open standard

Interoperability and reliability are very important for today's software engineering
problems. The power of AMQP comes from its features like interoperability,
reliability and so forth. Especially, with interoperability, we can use different types of
technologies in sender and receiver. The main problem for most of the Internet giants
is scalability. Scalability has direct relationship to reliability. Chapter 3, Architecture
and Messaging, covers the details and specifications of AMQP.

[4]

Chapter 1

An overview of RabbitMQ
As mentioned earlier, RabbitMQ is an open source Message Broker software that
tries to solve messaging problems by implementing the AMQP. RabbitMQ is
licensed with Mozilla Public License. RabbitMQ became part of GoPivotal in May
2013 and the community has helped in the development of RabbitMQ. Since then,
the community has been trying to improve RabbitMQ.

RabbitMQ logo

As we explained, RabbitMQ solves messaging problems; however, you may

ask yourself what kind of messaging problems RabbitMQ solves. In summary,
RabbitMQ has the following functionalities to solve messaging problems:

Ensures that messages are sent and received

Routes the messages to the correct destinations
Saves the state of the messages
Supports multiple transportation protocols (AMQP, MQTT, STOMP, HTTP)
Supports clustering
Highly scalable, reliable and available
Extendible with plugins
Supports clients for almost any of language
Large community support also provides commercial support

Now that we have discussed the concepts behind the RabbitMQ, let's try to install
and use RabbitMQ on your computer.

[5]

Getting Started

Installation of RabbitMQ
Installation of RabbitMQ is not distinctly different from other software in different
operation systems. Unix-based operating systems can build RabbitMQ from source
code and Microsoft Windows can run the standard MSI installers. RabbitMQ
installation files can be found in the download webpage of the RabbitMQ website,
as shown in the following image:

]
Download Webpage of RabbitMQ

[6]

Chapter 1

The only prerequisite for the RabbitMQ installation is the Erlang runtime
environment because RabbitMQ runs on the Erlang VM. Therefore, we have to
install Erlang before installing the RabbitMQ. Erlang can be downloaded from
the Erlang download webpage, as shown in the following image, and installation
instructions will be covered in the topics that follow:

Download Webpage of Erlang

Now, let's to talk about the installation of both RabbitMQ and Erlang on Windows,
Mac OS X, Ubuntu, Fedora, and Amazon Web Services.

[7]

Getting Started

Windows
RabbitMQ runs on both 32-bit and 64-bit machines from the same package. Erlang
is installed either as 32-bit and 64-bit. So, RabbitMQ can be easily installed on the
Windows operating system. Let's install these stuff for running RabbitMQ.
Firstly, we should install the Erlang runtime environment on Windows. Erlang has
Windows installers for 32-bit and 64-bit as shown in the previous image. We can
easily download the related binary file to our computer and install Erlang using it:

Installation of Erlang in Windows

[8]

Chapter 1

After installing the Erlang runtime environment, we've completed the requirements
of RabbitMQ installation. The next step is to download and install the RabbitMQ
binary file with related the Windows version:

Installation of Erlang in Windows

We can find the related Windows installer for RabbitMQ with the help of RabbitMQ
download webpage as shown in the screenshot showing the download webpage of
RabbitMQ. Then, we just need to click and install the RabbitMQ on our Windows
computer. Besides installing using the installer, we can install using the Windows
binary file that is served within the RabbitMQ download webpage. The following
instructions will be enough for installing RabbitMQ without the installer:

Download the binary file for RabbitMQ Windows binary files

Extract the downloaded RabbitMQ zip file to our local folder

It is possible to install the RabbitMQ on your Windows computer in both ways. Note
that you may add the RabbitMQ binaries directory to the windows system path in
the system/environment variable settings.

[9]

Getting Started

Mac OS X
As we specified, we only have one requirement to install RabbitMQ on our computers.
In Mac OS X, we have package managers and we have the opportunity to compile
from the source for both Erlang runtime environment and RabbitMQ.
Firstly, both Erlang and RabbitMQ can be easily installed on Mac OS X using
package managers. Although we have lots of package managers on Mac OS X,
Homebrew and MacPort are the ones that are mostly used in Mac OS X. So, we'll
talk about the installation using Homebrew and MacPorts.
Homebrew has both RabbitMQ and Erlang on its repository. As RabbitMQ has
a dependency with Erlang, Homebrew finds its dependent software and installs
them together.
Homebrew is just another package manager for Mac OS X. Homebrew is
quite easy to install on Mac OS X and has lots of packages. So, you would
find your application in its repository. Check it out at http://brew.sh

So, we just need to install RabbitMQ in Homebrew using the brew install
rabbitmq command on our terminal as shown in the following image:

Homebrew Installation of RabbitMQ

MacPorts has the similar method of operation with Homebrew. MacPorts also
installs the software with its dependencies. Therefore, we just need to install
RabbitMQ in MacPorts using the port install rabbitmq-server command on
our terminal, as shown in the following image:
[ 10 ]

Chapter 1

MacPorts Installation of RabbitMQ

Another way to install RabbitMQ and Erlang is by compiling the source codes in
Mac OS X. Before compiling, we need the following to compile Erlang source code:

The GNU make

The GNU C compiler

Perl 5

After downloading and unzipping the source codes of the Erlang, we just need
common commands on the Erlang folder for compiling from source code, as follows:
./configure
make
make install

Finally, we just need to download and unzip the RabbitMQ binary files.

[ 11 ]

Getting Started

Ubuntu
Ubuntu is just another Linux distribution based on Debian. Similar instructions as the
ones we discussed for the installation on the Mac OS X would be applied for Ubuntu.
Ubuntu has a package manager called Advanced Packaging Tool (apt-get) and has
a Debian package manager called dpkg. So, we are able to install RabbitMQ and
Erlang runtime environment using apt-get. Moreover, similar to Mac OS X, we can
compile from source codes of Erlang.
Firstly, as we said in the previous paragraph, we can install RabbitMQ using aptget and dpkg. Before installing RabbitMQ, we should add the RabbitMQ repository
to the APT repository using the following line (add the following line to /etc/apt/
sources.list):
deb http://www.rabbitmq.com/debian/ testing main

Now, we are ready to install RabbitMQ and its dependency Erlang runtime
environment, as shown in the following image:
sudo apt-get install rabbitmq-server

Ubuntu Installation of RabbitMQ

Also, we have the dpkg dependency management tool for installing RabbitMQ.
RabbitMQ has packages for dpkg in its download webpage. We can download it
from its website then run the following command:
dpkg i rabbitmq-server.deb

[ 12 ]

Chapter 1

Secondly, we have another option, which was explained in the Mac OS X topic.
That is, compiling from source codes. We just need to compile the downloaded
Erlang source code, and we are ready to run the downloaded binary files of
RabbitMQ. You can look at the details in the Mac OS X section.

Fedora
Fedora is yet another Linux distribution based on Red Hat. Installation instructions
of Fedora are similar to Ubuntu's installation. Fedora has package managers called
rpm and yum.
Firstly, we are able to install RabbitMQ with its dependency Erlang using package
managers. Before using yum, we should run the following command to add
RabbitMQ repository:
wget -O /etc/yum.repos.d/epel-erlang.repo
http://repos.fedorapeople.org/repos/peter/erlang/epel-erlang.repo

Then, we can install RabbitMQ using the following command as shown in the
following screenshot:
sudo yum install rabbitmq-server

Fedora Installation of RabbitMQ

[ 13 ]

Getting Started

Fedora has another package manager, which comes from Red Hat, called rpm. As
RabbitMQ publishes package as rpm, we can easily install using the rpm package
manager. After downloading the rpm package from RabbitMQ webpage, we can
install RabbitMQ with Erlang, using following command:
rpm ivh rabbitmq-server.rpm

Secondly, we have another option, which was explained in the Mac OS X topic.
That is compiling from source code. We just need to compile the downloaded Erlang
source code and we are ready to run the downloaded binary files of RabbitMQ. You
can look at the details in the Mac OS X section.

Amazon elastic compute cloud (EC2)

Amazon Web Services (AWS) is cloud-computing platform offered from Amazon.
AWS has lots of features for developers such as autoscaling. Besides RabbitMQ,
AWS also offers their own messaging service called Simple Queue Service:
however, RabbitMQ has lots of advantages over Amazon SQS. For instance,
RabbitMQ has an extendable plugin system, whereas SQS capabilities are short.
RabbitMQ implements a standard approach to message acknowledgement and
consumption, whereas SQS has its own standards.
Anyway, we can easily install our RabbitMQ to the AWS EC2 instance and can save
images of the RabbitMQ installed operating system. In AWS EC2, we choose one of
the operating systems from the list or any other instance that we used earlier.
Amazon EC2 would be a good choice for your servers. In a scalable
architecture, we need clusters of Message Brokers, databases, caches,
and so on. EC2 gives you great API, and it's really quite easy to create
clusters using EC2. We will talk about the clusters of RabbitMQ using
EC2 in Chapter 4, Clustering and High Availability.

[ 14 ]

Chapter 1

As we explained, we are able to install RabbitMQ on Windows, Linux, and Mac OS

X, we just need to follow the instructions for AWS EC2 that are explained well in the
preceding sections. Let's take a look at the following screenshot:

Amazon EC2

Starting RabbitMQ
As we can see, the installation part of RabbitMQ is quite easy and starting RabbitMQ
is similar to its installation. Some package managers in Linux, Mac OS X, and
Windows installer add configuration parameters to operation system's configuration
for automatic startup. In such a case, we don't need to run the RabbitMQ command
manually; however, if we install RabbitMQ manually, we need to run the RabbitMQ
commands manually.

[ 15 ]

Getting Started

Starting RabbitMQ on Windows

If we use the Windows installer of RabbitMQ, the installer already makes
configurations for starting automatically. Therefore, we don't need to run RabbitMQ
manually; however, whenever we'd like to control the status of the server, we just
need to run following command on the sbin folder of RabbitMQ:
rabbitmqctl status

Status of RabbitMQ in Windows

You may have installed RabbitMQ manually on your Windows and you might
wonder how you can run the RabbitMQ server. You should run the following
command to start RabbitMQ (you have to run this command with an administrative
user). Moreover, you can install the RabbitMQ server as a Windows service:
rabbitmq-server
[ 16 ]

Chapter 1

Other OSes (Linux, Mac OS X)

There's isn't much difference in running RabbitMQ on Windows and other operating
systems. If we have RabbitMQ already installed using package managers, such as
apt-get, yum, and so on, we don't need to run the RabbitMQ manually because
RabbitMQ has already started automatically. So, we'd like to check the status of the
RabbitMQ using the following command:
sudo rabbitmqctl status

Status of RabbitMQ in Unix

[ 17 ]

Getting Started

After controlling the RabbitMQ status, if we get a message that says that RabbitMQ
isn't running, then we should run RabbitMQ using the following command on the
sbin folder of RabbitMQ installation folder:
rabbitmq-server

Starting of RabbitMQ in Unix

Summary
We finally finished our first chapter, which introduces messaging concepts along
with brief details about Message Queues and Message Brokers, and the protocol
called AMQP that defines the functionalities of a Message Queue. Finally, our
chapter introduces the RabbitMQ, providing information about its installation on
different types of operating systems and how we can run the RabbitMQ server.
Now, we are ready to jump into the details of RabbitMQ, starting from its
configuration in the next chapter.

[ 18 ]

Get more information Mastering RabbitMQ

Where to buy this book

You can buy Mastering RabbitMQ from the Packt Publishing website.
Alternatively, you can buy the book from Amazon, BN.com, Computer Manuals and most internet
book retailers.
Click here for ordering and shipping details.

www.PacktPub.com

Stay Connected: