™

Manage Your Network

with Ansible and SSH

Disaster-Planning Tips
for Server Owners
AUGUST 2017 | ISSUE 280
Since 1994: The Original Magazine of the Linux Community http://www.linuxjournal.com

Filesystem Events
with inotify
+
EMACS FOR
SCIENCE

PREPARE FOR
VACATION
LIKE A
SYSADMIN

EOF: THE
ACTUALLY
DISTRIBUTED

Create an WEB

Internet WATCH:
Radio Station ISSUE
OVERVIEW

with Icecast and Liquidsoap

V

LJ280-Aug2017.indd 1 7/19/17 12:52 PM

 Practical books
for the most technical
people on the planet.

GEEK GUIDES

Download books for free with a

simple one-time registration.

http://geekguide.linuxjournal.com

LJ280-Aug2017.indd 2 7/19/17 12:52 PM

 !
NEW Managing Harnessing
Container the Power
Security and of the Cloud
Compliance with SUSE
in Docker Author:
Author: Petros Koutoupis
Petros Koutoupis Sponsor:
Sponsor: SUSE
Twistlock

DevOps for An
the Rest of Us Architect’s
Author:
Guide:
John S. Tonello Linux for
Sponsor: Enterprise IT
Puppet
Author:
Sol Lederman
Sponsor:
SUSE

Memory: Past, Cloud-Scale

Present and Automation
Future—and with Puppet
the Tools to Author:
Optimize It John S. Tonello
Author: Sponsor:
Petros Koutoupis Puppet
Sponsor:
Intel

Why Innovative Tame the

App Developers Docker Life
Love High-Speed Cycle with
OSDBMS SUSE
Author: Author:
Ted Schmidt John S. Tonello
Sponsor: Sponsor:
IBM SUSE

LJ280-Aug2017.indd 3 7/19/17 12:52 PM

 CONTENTS AUGUST 2017
ISSUE 280
FEATURES
74 Creating an 88 Linux Filesystem
Internet Radio Events with inotify
Station with A survey of OS utilities that
manipulate the Linux inotify
Icecast and system calls.
Liquidsoap Charles Fisher
Set up a full-featured internet
radio station with free software
and open standards.
Bill Dengler

Cover Image: © Can Stock Photo Inc. / dny3d

4 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 4 7/19/17 12:52 PM

 CONTENTS

COLUMNS
34 Reuven M. Lerner’s
At the Forge
Avoiding Disaster

42 Dave Taylor’s
Work the Shell
Let’s Play Bunco!

50 Kyle Rankin’s
Hack and /
Preparing for Vacation 22

56 Shawn Powers’
The Open-Source
Classroom
Ansible: the Automation
Framework That Thinks Like
a Sysadmin

114 Doc Searls’ EOF 32

The Actually Distributed Web

IN EVERY ISSUE
8 Current_Issue.tar.gz ON THE COVER

10 Letters UÊ>˜>}iÊ9œÕÀÊ iÌܜÀŽÊ܈̅Ê
˜ÃˆLiÊ>˜`Ê--]Ê«°ÊxÈ

UÊˆÃ>ÃÌiÀ‡*>˜˜ˆ˜}Ê/ˆ«ÃÊvœÀÊ-iÀÛiÀÊ"ܘiÀÃ]Ê«°ÊÎ{

14 UPFRONT UʈiÃÞÃÌi“Ê Ûi˜ÌÃÊ܈̅ʈ˜œÌˆvÞ]Ê«°Ênn

UÊ
Ài>ÌiÊ>˜Ê˜ÌiÀ˜iÌÊ,>`ˆœÊ-Ì>̈œ˜Ê܈̅ÊViV>ÃÌÊÊ

32 Editors’ Choice Ê >˜`ʈµÕˆ`Ü>«]Ê«°ÊÇ{

UÊ “>VÃÊvœÀÊ-Vˆi˜Vi]Ê«°ÊÓÈ

66 New Products UÊ*Ài«>ÀiÊvœÀÊ6>V>̈œ˜ÊˆŽiÊ>Ê-ÞÃ`>“ˆ˜]Ê«°Êxä

UÊ "\Ê̅iÊ
VÕÌ>ÞÊˆÃÌÀˆLÕÌi`Ê7iL]Ê«°Ê££{

120 Advertisers Index

LINUX JOURNAL (ISSN 1075-3583) is published monthly by Belltown Media, Inc., PO Box 980985, Houston, TX 77098 USA.
Subscription rate is $29.50/year. Subscriptions start with the next issue.

5 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 5 7/19/17 12:52 PM

 Executive Editor Jill Franklin
[email protected]
Senior Editor Doc Searls
[email protected]
Associate Editor Shawn Powers
[email protected]
Art Director Garrick Antikajian
[email protected]
Products Editor James Gray
[email protected]
Editor Emeritus Don Marti
[email protected]
Technical Editor Michael Baxter
[email protected]
Senior Columnist Reuven Lerner
[email protected]
Security Editor Mick Bauer
[email protected]
Hack Editor Kyle Rankin
[email protected]
Virtual Editor Bill Childers
[email protected]

Contributing Editors
)BRAHIM (ADDAD s 2OBERT ,OVE s :ACK "ROWN s $AVE 0HILLIPS s -ARCO &IORETTI s ,UDOVIC -ARCOTTE
0AUL "ARRY s 0AUL -C+ENNEY s $AVE 4AYLOR s $IRK %LMENDORF s *USTIN 2YAN s !DAM -ONSEN

President Carlie Fairchild

[email protected]

Publisher Mark Irgang

[email protected]

Associate Publisher John Grogan

[email protected]

Director of Digital Experience Katherine Druckman

[email protected]

Accountant Candy Beauchamp

[email protected]

Linux Journal is published by, and is a registered trade name of,

Belltown Media, Inc.
0/ "OX  (OUSTON 48  53!

Editorial Advisory Panel

Nick Baronian
Kalyana Krishna Chadalavada
"RIAN #ONNER s +EIR $AVIS
-ICHAEL %AGER s 6ICTOR 'REGORIO
$AVID ! ,ANE s 3TEVE -ARQUEZ
$AVE -C!LLISTER s 4HOMAS 1UINLAN
#HRIS $ 3TARK s 0ATRICK 3WARTZ

Advertising
% -!),: [email protected]
52,: www.linuxjournal.com/advertising
0(/.%     EXT 

Subscriptions
% -!),: [email protected]
52,: www.linuxjournal.com/subscribe
-!), 0/ "OX  (OUSTON 48  53!

LINUX IS A REGISTERED TRADEMARK OF ,INUS 4ORVALDS

LJ280-Aug2017.indd 6 7/19/17 12:52 PM

 You Manage data expansion
with SUSE Enterprise
cannot Storage.

keep up
SUSE Enterprise Storage, the leading
open source storage solution, is highly
scalable and resilient, enabling high-end

with data
functionality at a fraction of
the cost.

explosion.
suse.com/storage

Data

LJ280-Aug2017.indd 7 7/19/17 12:52 PM

 Current_Issue.tar.gz

The Wacky
World of Linux SHAWN

O
NE OF THE NIFTY THINGS ABOUT BEING A POWERS
,INUX USER IS HOW BIZARRE LIFE CAN GET /NE
Shawn Powers is the
MOMENT YOU CAN BE WRITING CUTTING EDGE
Associate Editor for
code, and the next you can get stuck in a nostalgia Linux Journal. He’s
RABBIT HOLE INSTALLING !FTERSTEP BECAUSE YOU USED also the Gadget Guy
for LinuxJournal.com,
.E843TEP MACHINES IN COLLEGE GUILTY  4HIS MONTH
and he has an
MY LIFE GOT A LITTLE CRAZY BECAUSE ) HAD TO INSTALL interesting collection
-ICROSOFT /FFICE FOR MY DAUGHTER 4HE COMPUTER ) of vintage Garfield
coffee mugs. Don’t
had to install it on? Linux. Yet, in this wacky world
let his silly hairdo
WE LIVE IN IT ENDED UP WORKING PERFECTLYˆSORT OF fool you, he’s a
4HERE SEEMS TO BE SOMETHING NEW EVERY DAY IN THE pretty ordinary guy
and can be reached
,INUX WORLD AND THIS MONTH THERE ARE LOTS OF NEW
via email at
things to talk about. [email protected].
2EUVEN - ,ERNER STARTS OFF THIS ISSUE WITH SOME Or, swing by the
#linuxjournal IRC
TIPS ON DISASTER PLANNINGˆNOT hEVIL GENIUSv SORT OF
channel on
PLANNING BUT RATHER PLANNING FOR WHAT TO DO WHEN Freenode.net.
disaster inevitably happens. Yes, the obvious answer
IS hHAVE A BACKUPv BUT ITS A BIT MORE COMPLICATED
than that, and Reuven provides sound advice.
V

$AVE 4AYLOR DECIDES YOU DESERVE A BIT OF A BREAK

and although you still will learn some awesome
coding skills, you do it while creating a dice game. VIDEO:
Shawn
4HE NAME OF THE GAME IS "UNCO WHICH SOUNDS Powers
runs
LIKE SOMETHING HE MADE UP BUT NONETHELESS ITS through
interesting. Be sure to check it out. the latest
issue.
Kyle Rankin teaches how to take a vacation
PROPERLY THIS MONTH )F YOUVE KNOWN +YLE FOR A

8 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 8 7/19/17 12:52 PM

 Current_Issue.tar.gz

WHILE YOU KNOW THAT HES BEEN IN THE AWKWARD SITUATION OF FIXING
REMOTE SERVERS FROM ATOP A SKI LIFT BEFORE SO HIS PREPARATORY SKILLS ARE
worth reading.
) START A NEW SERIES THIS ISSUE ON !NSIBLE )M A BIG FAN OF $EV/PS
TOOLS BUT SO MANY OF THEM HAVE SUCH A STEEP LEARNING CURVE IT MAKES
THEM DIFFICULT TO INTEGRATE INTO YOUR SYSTEM !NSIBLE IS ONE OF MY
FAVORITE CONFIGURATION MANAGEMENT PLATFORMS BECAUSE IT USES STANDARD
33( FOR CONNECTING TO CLIENT COMPUTERS 4HIS MAKES IT EASY TO START
SMALL AND ADD MORE CLIENTS AS YOU LEARN TO TAKE ADVANTAGE OF HOW
POWERFUL IT CAN BE
3TREAMING MUSIC IS THE METHOD MOST PEOPLE USE FOR LISTENING
NOWADAYS WHETHER ITS FROM 0ANDORA 3POTIFY OR ANY OF DOZENS OF OTHER
services. Bill Dengler shows how to create your own streaming radio
STATION USING OPEN SOURCE TOOLS )F YOUVE EVER WANTED TO STREAM A LIVE
EVENT WITHOUT GOING THROUGH THE HASSLE OF VIDEO ITS AN ARTICLE YOU
WONT WANT TO MISS
#HARLES &ISHER FINISHES OFF THE ISSUE WITH INSTRUCTIONS ON USING INOTIFY
TO FIRE OFF EVENTS ON A ,INUX SYSTEM )M GUILTY OF RESORTING TO TIMED
CRON JOBS FOR MOST THINGS RELATING TO FILESYSTEM CHANGES BUT WITH
INOTIFY ITS POSSIBLE TO HAVE A FILESYSTEM CHANGE LAUNCH A PROCESS )TS
A PROACTIVE WAY OF ACCOMPLISHING A TASK AND FAR FAR MORE EFFICIENT
#HARLES WALKS THROUGH THE PROCESS AND HELPS YOU AVOID SOME PITFALLS
along the way.
!S WITH EVERY ISSUE OF Linux Journal THIS ONE IS FULL OF TECH TIPS
PRODUCT ANNOUNCEMENTS USEFUL APPLICATIONS AND REVIEWS 7HETHER
YOU NEED TO INSTALL -ICROSOFT /FFICE ON YOUR ,INUX LAPTOP OR WANT
TO PREP YOUR DATA CENTER FOR YOUR TRIP TO (AWAII THIS ISSUE HAS YOU
COVERED 4HE WORLD OF ,INUX CAN BE A CRAZY ONE AND WERE HAPPY TO
BE A PART OF THE CRAZY
Q

RETURN TO CONTENTS

9 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 9 7/19/17 12:52 PM

 LETTERS
LETTERS
[ ]

PREVIOUS NEXT

V
V

Current_Issue.tar.gz UpFront

Another Way
2EGARDING $AVE 4AYLORS h7ATERMARKING )MAGESˆFROM THE #OMMAND
,INEv IN THE !PRIL  ISSUE AS USUAL THERE IS MORE THAN ONE WAY TO
IMPLEMENT A SOLUTION )NSTEAD OF THIS

predot=$(echo    $name    |    rev    |    cut    -­-­d.    -­-­f2-­-­    |    rev)  

postdot=$(echo    $name    |    rev    |    cut    -­-­d.    -­-­f1    |    rev)  
newname=$(echo    ${predot}-­-­wm.$postdot)

I would use this:

predot=${name%.*}  
postdot=${name##*.}  
newname=${predot}-­wm.$postdot

/R AS A ONE LINER

newname==${name%.*}-­wm.${name##*.}

4HESE PATTERN MATCHING OPERATORS ARE AVAILABLE IN "ASH AND ALL SHELLS THAT
ARE 0/3)8 COMPLIANT
—G. Allard

Dave Taylor replies: Thanks for sharing the fancy way to do those data field
chops in Bash. The reason I don’t use those sorts of notations is simply because
I believe it makes the script considerably harder to understand and edit if you
ever go back to it weeks, months or years later. I actually have two goals I’m

10 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 10 7/19/17 12:52 PM

 LETTERS

trying to attain simultaneously with my column: write interesting code and

make sure it’s as understandable as possible for the widest range of readers.

Beetle
2EGARDING THE REQUEST FROM A READER IN THE -AY ISSUES ,ETTERS SECTION
TO SEE A PHOTO OF 3HAWNS TRUCK )D RATHER SEE A PICTURE OF HIS 67 -Y
DAUGHTER AND ) ARE JUST ABOUT FINISHED WITH A  4YPE  7E HAD TO
REBUILD THE ENGINE BRAKES #6 JOINTS AND ON AND ON

I have enjoyed the magazine a lot over the years. Keep it up.
—Steve Witt

Shawn Powers replies: I actually have two Beetles.

1) 1973 Super Beetle Convertible: it’s my daily driver in the summer.

(Winters are hard on Beetles here in Michigan. The salt eats them, so I
only drive them in the summer.) It’s orange with a black top, and it is in

Figure 1. 1973 Super Beetle Convertible

11 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 11 7/19/17 12:52 PM

 LETTERS

the perfect shape for driving. The paint isn’t perfect, and the top isn’t
perfect, but they’re presentable. Mechanically, it’s solid, without any rust.
It’s not a show car, but because it’s a bug, it turns heads!

2) This was my first bug, a couple years ago. It’s a yellow 1975 Basic
Beetle, model 110. It’s a very unique model, because it was made only
in 1975, and it is the “worst” outfitted Beetle ever made! In 1975,
inflation was rising like crazy, but VW wanted to offer a Beetle for less
than $3,000, so the company stripped down the Standard Beetle as much
as it could and sent one to every dealership with a price of $2,999.

Figure 2. 1975 Basic Beetle

The car came with all black trim, no ventilation blower, no radio, a
partial inside header cover, stripped down handles on the door, no sound-
proofing in the engine compartment, and every other shortcut the VW
folks could think of. Mine has a few modifications from its original model

12 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 12 7/19/17 12:53 PM

 LETTERS

110 status. There’s an aftermarket sunroof, some

chrome trim (although the original flat black was
in the car, and I hope to put it back on), a radio
and a conversion to carburetor. I love having such At Your Service
a unique piece of Beetle history, but having a super SUBSCRIPTIONS: Linux Journal is available

stripped-down car is an odd bragging point! in a variety of digital formats, including PDF,
.epub, .mobi and an online digital edition,
as well as apps for iOS and Android devices.
Renewing your subscription, changing your
Here are some details about the model 110: email address for issue delivery, paying your
invoice, viewing your account details or other
HTTPSWWWTHESAMBACOMVWFORUMVIEWTOPICPHPT subscription inquiries can be done instantly
online: http://www.linuxjournal.com/subs.
HIGHLIGHTBASICMODEL. Email us at [email protected] or reach
us via postal mail at Linux Journal, PO Box
980985, Houston, TX 77098 USA. Please
Anyway, good luck with your daughter’s car. It remember to include your complete name
and address when contacting us.
sounds like a wonderful project! I just wish I had a
ACCESSING THE DIGITAL ARCHIVE:
garage, because working in the driveway is no fun! Your monthly download notifications
will have links to the various formats
and to the digital archive. To access the
digital archive at any time, log in at

7ÀˆÌiÊLJÊ>ÊiÌÌiÀÊ http://www.linuxjournal.com/digital.

We love hearing from our readers. Please send us your comments LETTERS TO THE EDITOR: We welcome your
letters and encourage you to submit them
and feedback via http://www.linuxjournal.com/contact. at http://www.linuxjournal.com/contact or
mail them to Linux Journal, PO Box 980985,
Houston, TX 77098 USA. Letters may be
edited for space and clarity.

*"/"- WRITING FOR US: We always are looking

for contributed articles, tutorials and
Send your Linux-related photos to real-world stories for the magazine.
[email protected], and An author’s guide, a list of topics and
due dates can be found online:
we’ll publish the best ones here.
http://www.linuxjournal.com/author.

FREE e-NEWSLETTERS: Linux Journal

editors publish newsletters on both
a weekly and monthly basis. Receive
late-breaking news, technical tips and
tricks, an inside look at upcoming issues
and links to in-depth stories featured on
http://www.linuxjournal.com. Subscribe
for free today: http://www.linuxjournal.com/
enewsletters.

ADVERTISING: Linux Journal is a great

resource for readers and advertisers alike.
Request a media kit, view our current
editorial calendar and advertising due dates,
or learn more about other advertising
and marketing opportunities by visiting
us on-line: http://www.linuxjournal.com/
advertising. Contact us directly for further
RETURN TO CONTENTS information: [email protected] or
+1 713-344-1956 ext. 2.

13 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 13 7/19/17 12:53 PM

 UPFRONT

UPFRONT NEWS + FUN

PREVIOUS NEXT

V
V

Letters Editors’ Choice

diff -u
7…>̽ÃÊ iÜʈ˜ÊÊ
iÀ˜iÊiÛiœ«“i˜Ì
Arnd Bergmann has recommended that the minimum supported
GCC VERSION BE RAISED TO  AND THE RECOMMENDED VERSION TO
 (OWEVER HE PLANS TO DOCUMENT THE FACT THAT FOLKS LIKE Geert
Uytterhoeven AND OTHERS WERE STILL USING '##  TO COMPILE RECENT
Linux kernels with success.
!CCORDING TO !RNDS ANALYSIS TESTING OLDER AND OLDER '## VERSIONS
initially would produce more and more unnecessary warnings, making
IT MORE AND MORE DIFFICULT TO SPOT LEGITIMATE BUGS !ND BEYOND A
CERTAIN '## VERSION LINKER ERRORS AND BUILD FAILURES WOULD BEGIN TO
APPEAR FOR CERTAIN ARCHITECTURES NOTABLY ARM.
(E POSTED A SERIES OF KERNEL PATCHES THAT GREW INCREASINGLY UGLY
as they had to accommodate older and older GCC versions. Some
OF THESE HE ACKNOWLEDGED PROBABLY WOULD BE TOO UGLY TO MAKE IT
into the kernel.
0ROTECTING THE KERNELS ABILITY TO COMPILE WITH ANCIENT '##
VERSIONS IS VALUABLE WORK ! LOT OF ENTERPRISE SYSTEMS CANT REALLY BE
upgraded without risking massive breakage, and so they just sit there
UNCHANGED FOR YEARS CHUGGING ALONG BRINGING IN REVENUE "UT THE
business has no choice but to upgrade its kernel in order to keep the

14 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 14 7/19/17 12:53 PM

 UPFRONT

system secure. As long as the company can still use all its old tools to
do that, it won’t have to worry about the large expenditures needed
to retool the entire system and possibly discover that it can’t be done
within the available budget.
The oldest supported version of GNU Make is being raised from
3.80 to 3.81. Masahiro Yamada inadvertently broke Linux support
for Make 3.80 in 2014 with a cleanup patch that made it into the
kernel without anyone noticing the compatibility issue.
In fact, in the intervening three years, no one else has noticed the
issue either. Instead, Masahiro himself discovered what had happened
and recently suggested updating the documentation to list Make
version 3.81 as the official minimum version.
There were no objections of any kind. Being broken for three years
with no one noticing is a fairly good argument in favor of dropping
support for a given version of a tool. Even Linus Torvalds, in his
comment to the thread, seemed to indicate that this time period was

LINUX JOURNAL
on your
e-Reader
Customized
Kindle and Nook
editions
available e-Reader
editions
FREE for
Subscribers
LEARN MORE
15 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 15 7/20/17 8:57 AM

 UPFRONT

enough to claim there were no active users of a given version.

As in the case of GCC, if even a single active user can be found
for an old version of a tool, the Linux folks will make a big effort to
continue to support that version. But as shown in the case of Make,
with no such users standing forth, no such effort will be made.
The filesystem mounting API has been needing an update to help
distinguish between a variety of actions that user code might take.
But, it’s also been needing new features to support modern ideas like
containers for virtualization. Recently, David Howells addressed the
latter, posting some patches to implement mount contexts. These
would hold various binary data, along with the mount options, for
use by the code performing the mount or by the filesystem itself.
This is valuable because it allows more detailed error reporting,
which is useful for something like filesystem mounting that can fail in
many different ways. But as Miklos Szeredi put it, David’s code didn’t
go far enough to clean up the overall mount API.
Miklos wanted the mount API to have sharp distinctions between
creating a filesystem instance, attaching a filesystem within an existing
mounted directory, editing the superblock and adjusting the mount
properties. David’s work did this a little, but not enough, said Miklos.
But, David wasn’t convinced of the necessity of having such
extremely sharp distinctions between actions that were, after all,
very closely related.
Ultimately, Miklos agreed that “everything that works now should
work the same way on the old as well as the new interfaces.” This
way at least, there would be no breakage, and other developers could
pick up where David’s work left off. —Zack Brown

16 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 16 7/20/17 8:57 AM

 INFRASTRUCTURE
AUTOMATION
SECURITY
CLOUD
DEVOPS
SCALABILITY
IOT

WOMENbINbLINUX
SUMMIT
2017

LJ280-Aug2017.indd 17 7/19/17 12:53 PM

 UPFRONT

Litebook
Linux Journal reader Kevin Bruce dropped me an email about a
TOO GOOD TO BE TRUE LAPTOP FROM THE FOLKS AT !LPHA 3TORE
https://alpha.store  4HE LAPTOP IS A  QUAD CORE P LAPTOP
WITH A '" HARD DRIVE FOR  -Y OLD $ELL $ HAS FINALLY
GIVEN UP SO ) WAS IN THE MARKET FOR A FAIRLY INEXPENSIVE LAPTOP
) DECIDED TO ORDER ONE ) OPTED FOR THE  MODEL WITH A HYBRID
drive, but everything else is the same as the base model. Rather
THAN A FULL REVIEW HERE ARE MY QUICK TAKEAWAYS

Q 4HE SCREEN IS AMAZING 2EALLY )TS BRIGHT VIBRANT AND HAS INCREDIBLE
COLORS 4HE SCREEN MAKES MY $ LOOK LIKE GARBAGE AND UPON FIRST
boot, I was giddy to see how amazing it looked.

Q 4HE KEYBOARD IS A LITTLE SPRINGY BUT TOLERABLE 4HE PLASTIC KEYS FEEL
A LITTLE CHEAP BUT THE ACTION IS FINE )T REMINDS ME OF TYPING ON A

18 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 18 7/19/17 12:53 PM

 UPFRONT

-ACBOOK 0RO BUT WITH LOOSER CHEAPER PLASTIC FOR THE ACTUAL KEYS

Q 4HE TRACKPAD IS HORRIBLEˆAND NOT JUST THE SORT OF HORRIBLE THAT IS

ANNOYING BUT THE SORT OF HORRIBLE THAT MAKES YOU UNWILLING TO USE
THE COMPUTER FOR ANYTHING OTHER THAN TYPING OR WATCHING MOVIES )T
HAS AN UNPREDICTABLE SCROLLING FEATURE POOR RESPONSE AND A CHEAP
FEEL AND ITS COMPLEMENTING BUTTONS ARE JUST AS BAD 4HE TRACKPAD
MAKES THE ENTIRE EXPERIENCE UNPLEASANT AND UNLESS YOURE OKAY
USING AN EXTERNAL MOUSE )D ABANDON SHIP

4HERE ARE ALSO A FEW STRANGE QUIRKS BUT NONE OF THEM WOULD HAVE
BEEN A SHOWSTOPPER FOR ME 4HE INCLUDED 7 I &I DRIVER HAS WEIRD
ISSUES WITH SLEEPING SO ) FOUND MYSELF REBOOTING OCCASIONALLY WHEN
THE LAPTOP LOST CONNECTIVITY 4HIS ISNT UNCOMMON WHEN YOURE TRYING
TO RUN ,INUX ON A LAPTOP THAT OBVIOUSLY WAS MADE FOR 7 INDOWS AND
)M FAIRLY CERTAIN FINDING THE RIGHT DRIVER WOULD HELP
4HE HYBRID HARD DRIVE WAS REALLY JUST A '" 33$ DEVICE WITH THE
ENTIRE %LEMENTARY /3 INSTALLED AND A '" DEVICE THAT HAD FOUR
.4&3 PARTITIONS ON IT ) COULD DELETE THE PARTITIONS AND RECLAIM THE
SPACE BUT IT WAS A STRANGE hOUT OF THE BOXv EXPERIENCE
Finally, you can see two things in the photo. One, I have a really
AWESOME CAT 4WO THE hREDv MODEL ) BOUGHT IS CLEARLY PINK .OT
JUST PINK BUT HOT PINK 4HE WHITE LABEL EVEN SAYS ITS PINK BUT THE
!LPHA 3TORE INSISTS ) BOUGHT A RED LAPTOP 4HE FORUMS SHOW OTHER
PEOPLE WITH THE SAME ISSUE SO APPARENTLY TO THE !LPHA FOLKS PINK
and red are the same thing. Honestly, I think the pink looks cooler
than red anyway, but it was odd.
-Y ADVICE TO +EVIN 5NLESS YOU WANT TO USE AN EXTERNAL MOUSE
AVOID THIS COMPUTER 4HE DISPLAY IS ABSOLUTELY GORGEOUS BUT THE
TRACKPAD MAKES IT ALMOST UNUSABLE 4HE WHOLE COMPUTER FEELS CHEAP
BUT THATS NOT SURPRISING FOR SUCH AN INEXPENSIVE MODEL )F YOURE
LOOKING FOR A CHEAP LAPTOP COMPUTER ) THINK THE BEST OPTION IS TO
GET A USED ONE LIKE THE ,ENOVO 9OGA E OR SOMETHING 4HEYRE NICE
MACHINES HAVE BETTER HARDWARE APART FROM MAYBE THE SCREEN  AND
YOU CAN GET THEM FOR ABOUT THE SAME PRICE —Shawn Powers

19 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 19 7/19/17 12:53 PM

 UPFRONT

Microsoft Office on
a Linux Machine?!
-Y MIDDLE CHILD IS HEADED TO COLLEGE THIS FALL AND ALTHOUGH THE
COLLEGE DOESNT REQUIRE A SPECIFIC TYPE OF COMPUTER IT DOES REQUIRE
STUDENTS TO HAVE -ICROSOFT /FFICE .OT -ICROSOFT /FFICE COMPATIBLE
BUT SPECIFICALLY -ICROSOFT /FFICE 4HAT BUMS ME OUT BUT ) FIGURED
/FFICE  MIGHT BE JUST THE ANSWER FOR A DAUGHTER WHO DOESNT WANT
TO GET A 7 INDOWS LAPTOP
4HE COOLEST PART ABOUT /FFICE  IS THAT COLLEGE STUDENTS CAN GET
IT FREE IF THEY HAVE A COLLEGE EMAIL ACCOUNT 4HE FREE SUBSCRIPTION
DOESNT INCLUDE A DOWNLOADABLE INSTALLABLE VERSION OF /FFICE BUT
FOR ,INUX USERS THAT DOESNT MATTER 4HE ONLINE VERSION IS ALL WED
BE ABLE TO USE ANYWAY SO FOR A COLLEGE STUDENT -ICROSOFT /FFICE
DOESNT ACTUALLY INCLUDE A -ICROSOFT TAX
4HE SURPRISING PART /FFICE  WORKS GREAT ON ,INUX .O REALLY

20 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 20 7/19/17 12:53 PM

 UPFRONT

) OPENED A FEW NATIVE -ICROSOFT FILES AND CREATED A FEW OF MY OWN

4HE /NE$RIVE STORAGE WORKED WONDERFULLY FROM THE WEB BROWSER
and even printing worked well. You can see in the screenshot that
-ICROSOFT CONVERTS THE DOCUMENT INTO A 0$& FILE SO IT CAN BE PRINTED
NATIVELY ON WHATEVER SYSTEM YOURE USING
)M SURPRISED TO REPORT THAT /FFICE  WORKS SO WELL ON ,INUX BUT
IT HONESTLY DOES &OR A COLLEGE STUDENT THE ONLINE OFFERING MIGHT BE
ENOUGH TO MEET ALL THE REQUIREMENTS MOST COLLEGES HAVE FOR SOFTWARE
%VEN IF YOU ARENT A FAN OF -ICROSOFT AND ITS PRODUCTS ) RECOMMEND AT
LEAST TRYING /FFICE  IF YOURE IN A POSITION THAT REQUIRES -ICROSOFT
/FFICE BECAUSE IT SURE BEATS INSTALLING 7 INDOWS 
—Shawn Powers

LINUX JOURNAL
on your Android device

Download the app

now from the
Google Play Store.

www.linuxjournal.com/android

For more information about advertising opportunities within Linux Journal iPhone, iPad and
Android apps, contact John Grogan at +1-713-344-1956 x2 or [email protected].

21 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 21 7/19/17 12:53 PM

 UPFRONT

Android Candy:
Clip-on Lenses
-Y 3ONY 8PERIA PHONE TAKES FAIRLY NICE PHOTOS 9OU CAN SEE IN &IGURE 
THAT ITS PRETTY CLEAR WITH NICE COLOR 4HE THING IS ) WANT TO LIVESTREAM
MY DAUGHTERS VOLLEYBALL AND BASKETBALL GAMES THIS YEAR AND MY PHONE
JUST DOESNT HAVE A WIDE ENOUGH ANGLE TO CAPTURE THE ENTIRE COURT 3O
) FIGURED )D TRY SOME OF THE CLIP ON WIDE ANGLE LENSES FOR PHONES "UT
honestly, the results are a bit disappointing.
4HE FIRST LENS ) TRIED THE LARGER ONE ON THE LEFT OF &IGURE  IS THE
 :OMEI X LENS http://a.co/3b3AdGC  4HIS LENS FEELS SOLID AND
HEAVY WITH GOOD LOOKING GLASS AND A METAL LENS CASE 4HE hCLIPPYv PART
ATTACHES WITH PLASTIC THREADS BUT STILL IT FEELS SOLID 5NFORTUNATELY IT
DIDNT PROVIDE VERY MUCH MORE COVERAGE AND THE WIDENESS IT DID GIVE
MADE FOR A PRETTY UGLY IMAGE &IGURE  
4HE OTHER LENS ) TRIED WAS A  !MIR ° wide angle lens that

Figure 1. This is my backyard taken with my Sony Xperia.

22 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 22 7/19/17 12:53 PM

 UPFRONT

Figure 2. Oddly, the cheaper lens felt nicer but performed worse, and the more expensive
lens felt cheaper but performed better.

Figure 3. This lens felt so solid, I expected much better results.

23 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 23 7/19/17 12:53 PM

 UPFRONT

Figure 4. This has a wider angle and a clearer picture, but both were disappointments.

DOES X HTTPACO/ZKCD  4HIS IS A MUCH SMALLER LENS AND IT

FEELS MUCH CHEAPER 4HE COVERAGE AND PHOTO ARE ACTUALLY FAIRLY SIMILAR
TO THE :OMEI BUT ITS ACTUALLY A BIT NICER 0LUS THE SMALLER SIZE MAKES
the phone less unwieldy when taking a photo. As you can tell in
&IGURE  ITS STILL NOT A GREAT PICTURE BUT IT IS WIDER THAN THE PHONE
can do natively.
-Y ADVICE )F YOU HAVE TO TAKE A WIDE ANGLE SHOT WITH A PHONE
STEP BACK A BIT TO GET A WIDER ANGLE THE OLD FASHIONED WAY #LIP ON
LENSES ARE A GREAT IDEA AND THEY DO TECHNICALLY WORK BUT IF YOURE
EXPECTING THE SAME QUALITY PHOTO YOUR PHONE CAN TAKE NATIVELY YOULL
be sadly disappointed. —Shawn Powers

24 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 24 7/19/17 12:53 PM

 THE LARGEST OPEN SOURCE CONFERENCE ON THE EAST COAST

October 23 & 24 | Raleigh, NC USA

FEATURING THE MOST WELL-KNOWN EXPERTS IN THE WORLD:

Jeff Atwood Sara Chipps Kelsey Hightower Yehuda Katz Angie Jones
Stack Overflow Jewelbots Google Cloud Tilde Inc Twitter

More than 3,000 technologists and decision makers are expected from all over the U.S. and the world

www.AllThingsOpen.org

LJ280-Aug2017.indd 25 7/19/17 12:53 PM

 UPFRONT

Emacs for Science

) TYPICALLY COVER SOFTWARE PACKAGES THAT DO ACTUAL CALCULATIONS TO
ADVANCE SCIENTIFIC KNOWLEDGE "UT FOR THIS ARTICLE ) AM COVERING A
SLIGHTLY STRANGER TOOL IN THE ARSENAL OF SCIENTIFIC COMPUTATION
%MACS IS A TEXT EDITOR THAT HAS ALMOST ALL THE FUNCTIONALITY OF AN
OPERATING SYSTEM ! COLLECTION OF ENHANCEMENTS AND CONFIGURATION
SETTINGS ARE AVAILABLE BUNDLED UNDER THE NAME OF SCIMAX "EING AN
%MACS USER MYSELF ) WAS SURPRISED )D NEVER HEARD OF IT BEFORE NOW
4HIS PROJECT HAS BEEN IN DEVELOPMENT FOR SOME TIME BUT IT RECENTLY
HAS STARTED TO FIND WIDER ATTENTION
"EFORE INSTALLING IT HOWEVER YOU NEED TO INSTALL THE LATEST VERSION OF
%MACS TO GET ALL OF THE NEEDED FUNCTIONALITY !S WITH MOST OF MY ARTICLES
) AM ASSUMING THAT YOU ARE USING A $EBIAN BASED DISTRIBUTION 9OU CAN
INSTALL %MACS BY USING THE DAILY SNAPSHOT PACKAGE AVAILABLE AT THE OFFICIAL
LAUNCHPAD ARCHIVE )NSTALL IT WITH THE FOLLOWING COMMANDS

sudo  add-­apt-­repository  ppa:ubuntu-­elisp/ppa  

sudo  update  
sudo  install  emacs-­snapshot

4HIS WILL ENSURE THAT YOU HAVE THE LATEST AND GREATEST VERSION
/NCE THIS IS INSTALLED GO AHEAD AND INSTALL THE SCIMAX CODE ITSELF
9OU CAN GET IT FROM THE MAIN 'IT(UB REPOSITORY WITH THE COMMAND

git  clone  https://github.com/jkitchin/scimax.git

Start it with the command:

emacs-­snapshot  -­q  -­l  path/to/scimax/init.el

4HE FIRST TIME YOU DO THIS THERE WILL BE A LOT OF ACTIVITY WHILE %MACS
DOWNLOADS AND INSTALLS THE FULL SUITE OF EXTRA PACKAGES YOU NEED IN ORDER
FOR THE SCIMAX CODE TO HAVE ALL OF THE REQUIRED DEPENDENCIES
7HEN YOU FINALLY HAVE EVERYTHING INSTALLED AND START SCIMAX YOU WILL

26 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 26 7/19/17 12:53 PM

 UPFRONT

Figure 1. You will see several new menu item entries at the top of your Emacs window.

SEE SEVERAL NEW MENU ITEMS IN YOUR %MACS SESSION

4HE REAL DRIVING NEED BEHIND ALL OF THE WORK THAT HAS GONE INTO
scimax is to make research more easily reproducible and to handle
DOCUMENTATION OF YOUR RESEARCH WITH MINIMAL EXTRA OVERHEAD
3INCE YOU WANT TO WORK WITH ORGANIZED DOCUMENTS WITHIN %MACS
THE BASE OF SCIMAX IS BUILT ON TOP OF ORG MODE /RG MODE BY ITSELF
is probably something you will want to look into as a potent tool.
3CIMAX TAKES THIS POWERFUL CORE AND MAKES IT EASIER TO USE WITH A
SERIES OF SHORTCUTS
"ECAUSE OF ORG MODES POWER IT WOULD BE TIME WELL SPENT IF YOU
LEARNED AT LEAST THE BASICS OF HOW TO USE THE MAIN PARTS OF THIS
PACKAGE /RG MODE HAS A MARKUP SYNTAX OF ITS OWN AND SCIMAX ADDS
A LAYER OF SHORTCUTS THAT MAKE IT EASIER TO WRITE
!LONG WITH THE REGULAR ORG MODE MARKUP SYNTAX SCIMAX MAKES IT
EASIER TO INCLUDE ,A4E8 SECTIONS FOR MORE ADVANCED TEXTUAL LAYOUT
-ANY PEOPLE IN SCIENTIFIC FIELDS ARE FAMILIAR WITH ,A4E8 &OR THOSE WHO

27 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 27 7/19/17 12:53 PM

 UPFRONT

ARENT ,A4E8 IS DOCUMENT LAYOUT PROGRAM WHERE YOU WRITE YOUR TEXT
AND INCLUDE LAYOUT INSTRUCTIONS FOR THE ,A4E8 ENGINE 4HE IDEA IS THAT
YOU SEPARATE OUT THE TEXT FROM THE FORMATTING OF THAT TEXT
)F YOU HAVE GRAPHICAL IMAGES AS PART OF YOUR RESEARCH SCIMAX
HAS ADDED SOME EXTRA FUNCTIONALITY TO MAKE IMAGE RESCALING AND
PRESENTATION BETTER THAN THE ORG MODE DEFAULTS BY USING EXTERNAL
PROGRAMS FROM THE IMAGEMAGICK PACKAGE
"ECAUSE ORG MODE IS DESIGNED TO BE A DOCUMENT STRUCTURING
PACKAGE FOR %MACS IT ALLOWS FOR EXPORTING YOUR TEXT INTO A GREAT MANY
OTHER FORMATS !LSO SINCE IT SEPARATES OUT THE FORMATTING FROM THE
ACTUAL TEXT IT CAN BE APPLIED TO MANY DIFFERENT DOCUMENT STRUCTURES
such as articles, books or web pages.
3CIMAX USES THE OX MANUSCRIPT %MACS PACKAGE TO HANDLE EXPORTING
TO HIGH QUALITY DOCUMENT FORMATS 4HIS IS USEFUL WHEN YOU NEED TO
PRODUCE FINAL VERSIONS OF YOUR SCIENTIFIC REPORTS OR ARTICLES IN A FORMAT
like PDF.
"IBLIOGRAPHIC REFERENCES WITHIN YOUR DOCUMENT ARE HANDLED THROUGH
bibtex entries.
3O FAR )VE COVERED A QUICK OVERVIEW OF THE DOCUMENT MANAGEMENT
ORGANIZATION AND FORMATTING TOOLS THAT ARE PROVIDED THROUGH SCIMAX
BUT %MACS AND ORG MODE HAVE MUCH MORE FUNCTIONALITY 9OU CAN
INTERACT WITH THE OUTSIDE WORLD IN A FEW DIFFERENT WAYS 4HE FIRST
IS THROUGH EMAIL 9OU CAN GRAB SELECTIONS OF YOUR TEXT OR AN ENTIRE
BUFFER AND ISSUE AN org-­mime COMMAND WITHIN %MACS TO TELL IT TO
SEND AN (4-, BASED EMAIL $EPENDING ON YOUR SYSTEM YOU MAY NEED
ADDITIONAL CONFIGURATION IN ORDER FOR THIS TO WORK AS EXPECTED
4HE OTHER WAY TO INTERACT WITH THE OUTSIDE WORLD IS THROUGH
'OOGLE SEARCHES !S SOMEONE WHO WRITES A FAIR BIT MYSELF ) CANNOT
UNDERSTATE THE NEED FOR A 'OOGLE WINDOW TO BE OPEN TO BE ABLE
TO VERIFY SOME FACT OR STATEMENT AS ) AM WRITING 7 ITH SCIMAX THE
google-­this %MACS PACKAGE GETS INSTALLED AND IS AVAILABLE AS YOU ARE
WORKING 4HIS ALLOWS YOU TO FIRE UP 'OOGLE SEARCHES BASED ON EITHER
SPECIFIC TEXT SELECTIONS OR THE CONTENTS OF ENTIRE BUFFERS IMMEDIATELY
FROM THE DOCUMENT THAT YOU ARE WORKING ON
Along with communicating with the outside world, the other
POWERFUL INTERACTION WITH EXTERNAL TOOLS IS THROUGH ORG MODES

28 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 28 7/19/17 12:53 PM

 UPFRONT

ability to run external programs and have their output inserted into
SECTIONS OF YOUR DOCUMENT 4HIS ONE PIECE OF FUNCTIONALITY MAKES THE
DREAM OF REPRODUCIBLE RESEARCH A REAL POSSIBILITY 9OU DO NEED TO BE
diligent and put it into practice, but you no longer have the excuse
OF SAYING THAT IT ISNT POSSIBLE 4HE IDEA IS THAT FROM WITHIN YOUR
ORG MODE DOCUMENT YOU CAN DEFINE A BLOCK OF CODE THAT MAKES SOME
CALCULATION OR GENERATES SOME GRAPH 9OU THEN CAN HAVE ORG MODE
FIRE THIS BLOCK SO THAT IT CAN BE RUN THROUGH AN EXTERNAL ENGINE AND
have the results pulled back in and inserted as the displayed text in
the original location.
4HE DEFAULT ENGINE CONFIGURED IN SCIMAX IS 0YTHON WHICH IS DEFINITELY
A GOOD STARTING POINT 7ITH MORE CONFIGURATION YOU CAN ADD SUPPORT
FOR SEVERAL OTHER LANGUAGES 4HE POWERFUL IDEA HERE IS THAT YOU
always can go back to the original code that generated some result
OR SOME GRAPH AND RE CREATE IT -ORE AND MORE SCIENTIFIC JOURNALS ARE
DEMANDING THIS LEVEL OF REPRODUCIBILITY SO HAVING IT AS PART OF YOUR
ARTICLE CONTENTS DIRECTLY MEANS YOU NEVER WILL LOSE TRACK OF IT
4HE LAST THING ) WANT TO COVER IS HOW TO ORGANIZE ALL OF THE WORK
THAT SCIMAX IS HELPING YOU DO 4HE HIGHEST LEVEL OF ORGANIZATION IS
the ability to set up projects. A project is essentially a directory with
ALL OF THE ASSOCIATED FILES FOR THAT GIVEN PROJECT 4HESE PROJECTS ARE
HANDLED BY THE %MACS PROJECTILE PACKAGE 4HIS PACKAGE ALLOWS YOU TO
MOVE BETWEEN PROJECTS FIND FILES WITHIN PROJECTS OR DO FULL SEARCHES
through a given project.
Projectile assumes that these project directories are under some
KIND OF VERSION CONTROL SYSTEM SUCH AS 'IT ,UCKILY SCIMAX INCLUDES
THE MAGIT %MACS PACKAGE WHICH ADDS LOTS OF EXTRA FUNCTIONS THAT
ALLOW YOU TO INTERACT WITH THE 'IT REPOSITORY THAT THE CURRENT FILE
BELONGS TO DIRECTLY FROM %MACS 9OU CAN CREATE OR CLONE REPOSITORIES
STAGE AND COMMIT CHANGES MANAGE DIFFS BETWEEN VERSIONS AND EVEN
HANDLE PUSHES TO AND PULLS FROM REMOTE REPOSITORIES !LONG WITH THE
explicit control over a Git repository, scimax includes extensions to
ORG MODE TO HANDLE TRACK CHANGES AS WELL AS TO INSERT EDIT MARKS
WITHIN YOUR ORG MODE DOCUMENT
0UTTING ALL OF THIS ORGANIZATIONAL WORK TOGETHER SCIMAX PROVIDES THE
ABILITY TO CREATE AND USE SCIENTIFIC NOTEBOOKS ! SERIES OF COMMANDS

29 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 29 7/19/17 12:53 PM

 UPFRONT

starting with nb-­ ALLOW YOU TO WRAP ALL OF THE

ORGANIZATIONAL FUNCTIONALITY TO CREATE MANAGE
THEY
and archive these notebooks. Now, you have SAID IT
NO REASON NOT TO START DOCUMENTING ALL OF YOUR
SCIENTIFIC RESEARCH IN A REPRODUCIBLE WAYˆEXCEPT The shoe that
maybe the learning curve. But, as the old saying fits one person
pinches another;
goes, nothing worth doing is easy, and I think
there is no recipe
THIS IS DEFINITELY WORTH DOING AT LEAST FOR SOME
for living that
people. —Joey Bernard
suits all cases.
—Carl Jung

Resources Courage is the

Q Scimax: https://github.com/jkitchin/scimax ladder on which
all the other
Q %MACS HTTPSWWWGNUORGSOFTWAREEMACS virtues mount.
Q ORG MODE http://orgmode.org —Clare Booth Luce

The thing women

have got to learn
is that nobody
gives you power.
You just take it.
—Roseanne Barr

One man practicing

sportsmanship
is better than
a hundred
teaching it.
—Knute Rockne

The man who

is swimming
against the
stream knows the
strength of it.
RETURN TO CONTENTS —Woodrow Wilson

30 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 30 7/19/17 12:53 PM

 THE  GLOBAL  WOMEN  IN  STEM  
CONFERENCE  &  AWARDS.

ϭϬϬ^W<Z^ͳ,hEZ^K&tKDE/E
^/EEd,EK>K'z
tomen  are  increasingly  becoming  the  engine  
driving  global  economic  growth  and  
ŝŶŶŽǀĂƟŽŶ͘:ŽŝŶƵƐĂƐǁĞĐĞůĞďƌĂƚĞƚŚĞ
SEPTEMBER
women  who  are  making  this  possible  in  spite   10th-­‐12th
ŽĨĂůůƚŚĞŽĚĚƐ͘tŝƚŚŽǀĞƌĂŚƵŶĚƌĞĚƐƉĞĂŬĞƌƐ
ĂŶĚŚƵŶĚƌĞĚƐŽĨĂƩĞŶĚĞĞƐĨƌŽŵĂůůĂĐƌŽƐƐ SAN
ƚŚĞǁŽƌůĚ͕t/^dDŝƐƉŽƐƐŝďůLJƚŚĞďŝŐŐĞƐƚ FRANCISCO

tŽŵĞŶŝŶ^dDĐŽŶĨĞƌĞŶĐĞŝŶƚŚĞtŽƌůĚ͘

EXPERIENCE  THE  THREE  AMAZING  DAYS  THAT  FLY  BY,  BUT  STAY  WITH  YOU  FOREVER.

^^^/KE^ ^dZdͳhW tZ^

ΘdZ<^ W/d,

WWW.WOM EN I N ST EM CO N FEREN C E.CO M

ĸůŝĂƚĞWĂƌƚŶĞƌWƌŽŐƌĂŵŽĨ

LJ280-Aug2017.indd 31 7/19/17 12:53 PM

 EDITORS’ CHOICE

NEXT
PREVIOUS
Reuven M. Lerner’s

V
V

UpFront
At the Forge

My Quest for ™

EDITORS’
Inbox Zero CHOICE
)VE NEVER BEEN ABLE TO ACCOMPLISH hINBOX ZEROv FOR
★
MORE THAN A COUPLE HOURS 0ART OF IT IS DUE TO POOR
ORGANIZATION ON MY PART AND PART OF IT IS DUE TO BEING TOO BUSY 4HE
PROBLEM ISNT THAT ) GET MORE EMAIL THAN ) CAN HANDLE ALTHOUGH THATS
PROBABLY TRUE ) GET A COUPLE HUNDRED MESSAGES A DAY  ITS THAT MOST
MESSAGES REQUIRE A FOLLOWUP THAT ) CANT ACCOMPLISH IMMEDIATELY )VE

32 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 32 7/19/17 12:53 PM

 EDITORS' CHOICE

TRIED TO MOVE ITEMS ) NEED TO HANDLE LATER QUICKLY INTO A TO DO LIST LIKE
7UNDERLIST BUT ITS AN EXTRA STEP THAT TAKES MORE TIME SO ) JUST LET THINGS
pile up in my inbox.
&OR SOME REASON ) RECENTLY DECIDED TO TRY 'OOGLE )NBOX )TS NOT A NEW
OFFERING FROM 'OOGLE BUT ) THINK ITS MATURED SINCE ) ORIGINALLY TRIED IT
years ago. See, Google Inbox allows me to keep inbox zero with minimal
EFFORT AND ZERO GUILT (ERES THE PREMISE

 7HEN AN EMAIL COMES IN A QUICK SWIPE TO THE RIGHT WILL ARCHIVE IT
)TS NOT EVEN A CLICKˆJUST A SWIPE AND ITS GONE

 )F ) NEED TO OPEN IT ) CAN TAP IT THEN CLICK A CHECK BOX TO ARCHIVE IT IF

) DONT NEED TO FOLLOW UP

 (ERES THE BEAUTY )F ) NEED TO FOLLOW UP ON A MESSAGE EVEN IF ITS JUST

something interesting I want to read later at my leisure, I can snooze it. In the
SCREENSHOT YOU CAN SEE THE OPTIONS FOR SNOOZING AND THEY WORK PERFECTLY

4HE IDEA OF SNOOZING EMAIL ISNT NEW ) REMEMBER USING hBOOMERANGv

FOR SUCH A THING A FEW YEARS AGO "UT 'OOGLE )NBOX HAS INTEGRATED THE
SNOOZE AND ARCHIVE FEATURES SO WELL THAT ) CAN CLEAR MY INBOX completely
IN MOMENTS AND NOT WORRY THAT )VE FORGOTTEN SOMETHING )N FACT THE
SNOOZE FEATURE WORKS MUCH BETTER THAN JUST LEAVING STUFF IN MY INBOX
BECAUSE IT ACTUALLY COMES BACK TO MY ATTENTION AFTER THE SNOOZE PERIOD
INSTEAD OF FALLING OFF THE END OF MY INBOX
4HANKS TO A FAST MINIMALIST TAKE ON ACHIEVING INBOX ZERO )M GIVING
'OOGLE )NBOX THIS MONTHS %DITORS #HOICE AWARD )TS NOT A NEW PRODUCT
BUT IT HAS REVOLUTIONIZED MY LIFE 9OU CAN CHECK IT OUT WITHOUT MESSING
up your regular Gmail inbox by logging in at http://inbox.google.com in
A WEB BROWSER OR BY DOWNLOADING THE MOBILE APP 9OU WONT REGRET IT

—Shawn Powers

RETURN TO CONTENTS

33 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 33 7/19/17 12:53 PM

 AT THE FORGE

Avoiding
Disaster REUVEN M.
Worried that your server will go down? LERNER
You should be. Here are some disaster-planning Reuven M. Lerner, a

tips for server owners. longtime Web developer,

offers training and
consulting services in
Python, Git, PostgreSQL
and data science. He has
NEXT written two programming
PREVIOUS ebooks (Practice Makes
Dave Taylor’s
V
V

Editors’ Choice Python and Practice Makes

Work the Shell Regexp) and publishes
a free weekly newsletter
for programmers, at
http://lerner.co.il/
newsletter. Reuven tweets
IF YOU OWN A CAR OR A HOUSE, YOU ALMOST at @reuvenmlerner and
lives in Modi’in, Israel, with
CERTAINLY HAVE INSURANCE. Insurance seems like
his wife and three children.
A HUGE WASTE OF MONEY 9OU PAY IT EVERY YEAR AND
MAKE SURE THAT YOU GET THE BEST POSSIBLE PRICE FOR
the best possible coverage, and then you hope you
never need to use the insurance. Insurance seems
LIKE A REALLY BAD DEALˆUNTIL YOU HAVE A DISASTER AND
REALIZE THAT HAD IT NOT BEEN FOR THE INSURANCE YOU
MIGHT HAVE BEEN IN FINANCIAL RUIN
5NFORTUNATELY DISASTERS AND MISHAPS ARE A FACT
OF LIFE IN THE COMPUTER INDUSTRY !ND SO JUST AS YOU
pay insurance and hope never to have to use it,
YOU ALSO NEED TO TAKE TIME TO ENSURE THE SAFETY AND
RELIABILITY OF YOUR SYSTEMSˆNOT BECAUSE YOU WANT
disasters to happen, or even expect them to occur,

34 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 34 7/19/17 12:53 PM

 AT THE FORGE

but rather because you have to.

)F YOUR WEBSITE IS AN ONLINE BROCHURE FOR YOUR COMPANY AND THEN GOES
DOWN FOR A FEW HOURS OR EVEN DAYS ITLL BE EMBARRASSING AND ANNOYING
BUT NOT FINANCIALLY PAINFUL "UT IF YOUR WEBSITE IS YOUR BUSINESS WHEN
YOUR SITE GOES DOWN YOURE LOSING MONEY )F THATS THE CASE ITS CRUCIAL
TO ENSURE THAT YOUR SERVER AND SOFTWARE ARE NOT ONLY UNLIKELY TO GO
DOWN BUT ALSO EASILY RECOVERABLE IF AND WHEN THAT HAPPENS
7HY AM ) WRITING ABOUT THIS SUBJECT 7ELL LETS JUST SAY THAT THIS
PARTICULAR PROBLEM HIT CLOSE TO HOME FOR ME JUST BEFORE ) STARTED
TO WRITE THIS ARTICLE !FTER YEARS OF HELPING CLIENTS AROUND THE WORLD
TO ENSURE THE RELIABILITY OF THEIR SYSTEMS ) MADE THE MISTAKE OF NOT
BEING AS THOROUGH WITH MY OWN h4HE SHOEMAKERS CHILDREN GO
BAREFOOTv AS THE SAYING GOES 4HIS MEANS THAT JUST AFTER LAUNCHING
MY NEW ONLINE PRODUCT FOR 0YTHON DEVELOPERS A SEEMINGLY TRIVIAL
UPGRADE TURNED INTO A DISASTER 4HE PRECAUTIONS ) PUT IN PLACE IT TURNS
OUT WERENT QUITE ENOUGHˆAND AS ) WRITE THIS )M STILL PUTTING MY
WEB SERVER TOGETHER )LL SURVIVE AS WILL MY SERVER AND BUSINESS BUT
THIS HAS BEEN A PAINFUL AND IMPORTANT LESSONˆONE THAT )LL DO ALMOST
ANYTHING TO AVOID REPEATING IN THE FUTURE
3O IN THIS ARTICLE ) DESCRIBE A NUMBER OF TECHNIQUES )VE USED TO
KEEP SERVERS SAFE AND SOUND THROUGH THE YEARS AND TO REDUCE THE
CHANCES OF A COMPLETE MELTDOWN 9OU CAN THINK OF THESE TECHNIQUES
AS INSURANCE FOR YOUR SERVER SO THAT EVEN IF SOMETHING DOES GO
WRONG YOULL BE ABLE TO RECOVER FAIRLY QUICKLY
) SHOULD NOTE THAT MOST OF THE ADVICE HERE ASSUMES NO REDUNDANCY
IN YOUR ARCHITECTUREˆTHAT IS A SINGLE WEB SERVER AND AT MOST A
SINGLE DATABASE SERVER )F YOU CAN AFFORD TO HAVE A BUNCH OF SERVERS
OF EACH TYPE THESE SORTS OF PROBLEMS TEND TO BE MUCH LESS FREQUENT
(OWEVER THAT DOESNT MEAN THEY GO AWAY ENTIRELY "ESIDES
ALTHOUGH PEOPLE LIKE TO TALK ABOUT HEAVY DUTY WEB APPLICATIONS THAT
REQUIRE MASSIVE IRON IN ORDER TO RUN THE FACT IS THAT MANY BUSINESSES
RUN ON SMALL ONE AND TWO COMPUTER SERVERS -OREOVER THOSE
BUSINESSES DONT NEED MORE THAN THAT THE 2/) RETURN ON INVESTMENT
THEYLL GET FROM ADDITIONAL SERVERS CANNOT BE JUSTIFIED (OWEVER THE
2/) FROM A GOOD BACKUP AND RECOVERY PLAN IS HUGE AND THUS WORTH
the investment.

35 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 35 7/19/17 12:53 PM

 AT THE FORGE

Indeed, I prefer to keep my sites in Git,

backed up on a commercial hosting service,
such as GitHub or Bitbucket, and then
deployed using a system like Capistrano.

The Parts of a Web Application

Before I can talk about disaster preparation and recovery, it’s important
to consider the different parts of a web application and what those
various parts mean for your planning.
For many years, my website was trivially small and simple. Even if it
contained some simple programs, those generally were used for sending
email or for dynamically displaying different assets to visitors. The entire
site consisted of some static HTML, images, JavaScript and CSS. No
database or other excitement was necessary.
At the other end of the spectrum, many people have full-blown web
applications, sitting on multiple servers, with one or more databases and
caches, as well as HTTP servers with extensively edited configuration files.
But even when considering those two extremes, you can see that a web
application consists of only a few parts:

n The application software itself.

n Static assets for that application.

n Configuration file(s) for the HTTP server(s).

n Database configuration files.

n Database schema and contents.

Assuming that you’re using a high-level language, such as Python, Ruby

or JavaScript, everything in this list either is a file or can be turned into

36 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 36 7/21/17 8:33 AM

 AT THE FORGE

ONE !LL DATABASES MAKE IT POSSIBLE TO hDUMPv THEIR CONTENTS ONTO DISK
INTO A FORMAT THAT THEN CAN BE LOADED BACK INTO THE DATABASE SERVER
#ONSIDER A SITE CONTAINING ONLY APPLICATION SOFTWARE STATIC ASSETS AND
CONFIGURATION FILES )N OTHER WORDS NO DATABASE IS INVOLVED )N MANY
CASES SUCH A SITE CAN BE BACKED UP RELIABLY IN 'IT )NDEED ) PREFER TO
keep my sites in Git, backed up on a commercial hosting service, such as
GitHub or Bitbucket, and then deployed using a system like Capistrano.
In other words, you develop the site on your own development
MACHINE 7HENEVER YOU ARE HAPPY WITH A CHANGE THAT YOUVE MADE YOU
COMMIT THE CHANGE TO 'IT ON YOUR LOCAL MACHINE AND THEN DO A git  
push to your central repository. In order to deploy your application, you
then use Capistrano to do a cap  deploy  WHICH READS THE DATA FROM
THE CENTRAL REPOSITORY PUTS IT INTO THE APPROPRIATE PLACE ON THE SERVERS
FILESYSTEM AND YOURE GOOD TO GO
4HIS SYSTEM KEEPS YOU SAFE IN A FEW DIFFERENT WAYS 4HE CODE ITSELF
is located in at least three locations: your development machine, the
server and the repository. And those central repositories tend to be
FAIRLY RELIABLE IF ONLY BECAUSE ITS IN THE FINANCIAL INTEREST OF THE HOSTING
company to ensure that things are reliable.
) SHOULD ADD THAT IN SUCH A CASE YOU ALSO SHOULD INCLUDE THE (440
SERVERS CONFIGURATION FILES IN YOUR 'IT REPOSITORY 4HOSE FILES ARENT
LIKELY TO CHANGE VERY OFTEN BUT ) CAN TELL YOU FROM EXPERIENCE IF YOURE
RECOVERING FROM A CRISIS THE LAST THING YOU WANT TO THINK ABOUT IS HOW
YOUR !PACHE CONFIGURATION FILES SHOULD LOOK #OPYING THOSE FILES INTO YOUR
'IT REPOSITORY WILL WORK JUST FINE

Backing Up Databases
9OU COULD ARGUE THAT THE DIFFERENCE BETWEEN A hWEBSITEv AND A hWEB
APPLICATIONv IS A DATABASE $ATABASES LONG HAVE POWERED THE BACK ENDS OF
MANY WEB APPLICATIONS AND FOR GOOD REASONˆTHEY ALLOW YOU TO STORE AND
RETRIEVE DATA RELIABLY AND FLEXIBLY 4HE POWER THAT MODERN OPEN SOURCE
DATABASES PROVIDES WAS UNTHINKABLE JUST A DECADE OR TWO AGO AND THERES
NO REASON TO THINK THAT THEYLL BE ANY LESS RELIABLE IN THE FUTURE
!ND YET JUST BECAUSE YOUR DATABASE IS PRETTY RELIABLE DOESNT MEAN
THAT IT WONT HAVE PROBLEMS 4HIS MEANS YOURE GOING TO WANT TO KEEP
A SNAPSHOT hDUMPv OF THE DATABASES CONTENTS AROUND IN CASE THE

37 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 37 7/19/17 12:53 PM

 AT THE FORGE

DATABASE SERVER CORRUPTS INFORMATION AND YOU NEED TO ROLL BACK TO A

previous version.
-Y FAVORITE SOLUTION FOR SUCH A PROBLEM IS TO DUMP THE DATABASE ON A
REGULAR BASIS PREFERABLY HOURLY (ERES A SHELL SCRIPT )VE USED IN ONE FORM
OR ANOTHER FOR CREATING SUCH REGULAR DATABASE DUMPS

#!/bin/sh  
 
BACKUP_ROOT="/home/database-­backups/"  
YEAR=`/bin/date  +'%Y'`  
MONTH=`/bin/date  +'%m'`  
DAY=`/bin/date  +'%d'`  
 
DIRECTORY="$BACKUP_ROOT/$YEAR/$MONTH/$DAY"  
USERNAME=dbuser  
DATABASE=dbname  
HOST=localhost  
PORT=3306  
 
/bin/mkdir  -­p  $DIRECTORY  
 
/usr/bin/mysqldump  -­h  $HOST  -­-­databases  $DATABASE  -­u  $USERNAME      
 ´|  /bin/gzip  -­-­best  -­-­verbose  >      
 ´$DIRECTORY/$DATABASE-­dump.gz

4HE ABOVE SHELL SCRIPT STARTS OFF BY DEFINING A BUNCH OF VARIABLES

FROM THE DIRECTORY IN WHICH ) WANT TO STORE THE BACKUPS TO THE PARTS OF
THE DATE STORED IN 9%!2 -/.4( AND $!9  4HIS IS SO ) CAN HAVE
A SEPARATE DIRECTORY FOR EACH DAY OF THE MONTH ) COULD OF COURSE GO
FURTHER AND HAVE SEPARATE DIRECTORIES FOR EACH HOUR BUT )VE FOUND THAT
) RARELY NEED MORE THAN ONE BACKUP FROM A DAY
/NCE ) HAVE DEFINED THOSE VARIABLES ) THEN USE THE mkdir command
TO CREATE A NEW DIRECTORY 4HE -­p option tells mkdir THAT IF NECESSARY
IT SHOULD CREATE ALL OF THE DIRECTORIES IT NEEDS SUCH THAT THE ENTIRE PATH
will exist.
&INALLY ) THEN RUN THE DATABASES hDUMPv COMMAND )N THIS PARTICULAR

38 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 38 7/19/17 12:53 PM

 AT THE FORGE

CASE )M USING -Y31, SO )M USING THE mysqldump COMMAND 4HE OUTPUT
FROM THIS COMMAND IS A STREAM OF 31, THAT CAN BE USED TO RE CREATE THE
DATABASE ) THUS TAKE THE OUTPUT FROM mysqldump and pipe it into gzip ,
WHICH COMPRESSES THE OUTPUT FILE &INALLY THE RESULTING DUMPFILE IS PLACED
IN COMPRESSED FORM INSIDE THE DAILY BACKUP DIRECTORY
$EPENDING ON THE SIZE OF YOUR DATABASE AND THE AMOUNT OF DISK SPACE
YOU HAVE ON HAND YOULL HAVE TO DECIDE JUST HOW OFTEN YOU WANT TO
RUN DUMPS AND HOW OFTEN YOU WANT TO CLEAN OUT OLD ONES ) KNOW FROM
experience that dumping every hour can cause some load problems.
/N ONE VIRTUAL MACHINE )VE USED THE OVERALL ADMINISTRATION TEAM WAS
unhappy that I was dumping and compressing every hour, which they saw
AS AN UNNECESSARY USE OF SYSTEM RESOURCES
)F YOURE WORRIED YOUR SYSTEM WILL RUN OUT OF DISK SPACE YOU MIGHT
WELL WANT TO RUN A SPACE CHECKING PROGRAM THATLL ALERT YOU WHEN THE
FILESYSTEM IS LOW ON FREE SPACE )N ADDITION YOU CAN RUN A CRON JOB THAT
uses find TO ERASE ALL DUMPFILES FROM BEFORE A CERTAIN CUTOFF DATE )M
always a bit nervous about programs that automatically erase backups, so
) GENERALLY PREFER NOT TO DO THIS 2ATHER ) RUN A PROGRAM THAT WARNS ME IF
THE DISK USAGE IS GOING ABOVE  WHICH IS USUALLY LOW ENOUGH TO ENSURE
THAT ) CAN FIX THE PROBLEM IN TIME EVEN IF )M ON A LONG FLIGHT  4HEN ) CAN
GO IN AND REMOVE THE PROBLEMATIC FILES BY HAND
7HEN YOU BACK UP YOUR DATABASE YOU SHOULD BE SURE TO BACK UP
THE CONFIGURATION FOR THAT DATABASE AS WELL 4HE DATABASE SCHEMA AND
DATA WHICH ARE PART OF THE DUMPFILE ARE CERTAINLY IMPORTANT (OWEVER
IF YOU FIND YOURSELF HAVING TO RE CREATE YOUR SERVER FROM SCRATCH YOULL
WANT TO KNOW PRECISELY HOW YOU CONFIGURED THE DATABASE SERVER WITH
A PARTICULAR EMPHASIS ON THE FILESYSTEM CONFIGURATION AND MEMORY
ALLOCATIONS ) TEND TO USE 0OSTGRE31, FOR MOST OF MY WORK AND ALTHOUGH
POSTGRESQLCONF IS SIMPLE TO UNDERSTAND AND CONFIGURE ) STILL LIKE TO KEEP
IT AROUND WITH MY DUMPFILES
Another crucial thing to do is to check your database dumps
occasionally to be sure that they are working the way you want. It turns
OUT THAT THE BACKUPS ) THOUGHT ) WAS MAKING WERENT ACTUALLY HAPPENING
IN NO SMALL PART BECAUSE ) HAD MODIFIED THE SHELL SCRIPT AND HADNT
DOUBLE CHECKED THAT IT WAS CREATING USEFUL BACKUPS /CCASIONALLY PULLING
OUT ONE OF YOUR DUMPFILES AND RESTORING IT TO A SEPARATE AND OFFLINE

39 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 39 7/19/17 12:53 PM

 AT THE FORGE

database to check its integrity is a good practice, both to ensure that the
DUMP IS WORKING AND THAT YOU REMEMBER HOW TO RESTORE IT IN THE CASE OF
an emergency.

Storing Backups
"UT WAIT )T MIGHT BE GREAT TO HAVE THESE BACKUPS BUT WHAT IF THE SERVER
GOES DOWN ENTIRELY )N THE CASE OF THE CODE ) MENTIONED TO ENSURE THAT IT
was located on more than one machine, ensuring its integrity. By contrast,
YOUR DATABASE DUMPS ARE NOW ON THE SERVER SUCH THAT IF THE SERVER FAILS
your database dumps will be inaccessible.
4HIS MEANS YOULL WANT TO HAVE YOUR DATABASE DUMPS STORED ELSEWHERE
PREFERABLY AUTOMATICALLY (OW CAN YOU DO THAT
4HERE ARE A FEW RELATIVELY EASY AND INEXPENSIVE SOLUTIONS TO THIS
PROBLEM )F YOU HAVE TWO SERVERSˆIDEALLY IN SEPARATE PHYSICAL LOCATIONSˆ
you can use rsync TO COPY THE FILES FROM ONE TO THE OTHER $ONT rsync
THE DATABASES ACTUAL FILES SINCE THOSE MIGHT GET CORRUPTED IN TRANSFER
AND ARENT DESIGNED TO BE COPIED WHEN THE SERVER IS RUNNING "Y CONTRAST
THE DUMPFILES THAT YOU HAVE CREATED ARE MORE THAN ABLE TO GO ELSEWHERE
3ETTING UP A REMOTE SERVER WITH A USER SPECIFICALLY FOR HANDLING THESE
BACKUP TRANSFERS SHOULDNT BE TOO HARD AND WILL GO A LONG WAY TOWARD
ENSURING THE SAFETY OF YOUR DATA
I should note that using rsync IN THIS WAY BASICALLY REQUIRES THAT YOU
SET UP PASSWORDLESS 33( SO THAT YOU CAN TRANSFER WITHOUT HAVING TO BE
physically present to enter the password.
!NOTHER POSSIBLE SOLUTION IS !MAZONS 3IMPLE 3TORAGE 3ERVER 3 
WHICH OFFERS ASTONISHING AMOUNTS OF DISK SPACE AT VERY LOW PRICES )
KNOW THAT MANY COMPANIES USE 3 AS A SIMPLE ALBEIT SLOW BACKUP
system. You can set up a cron job to run a program that copies the
CONTENTS OF A PARTICULAR DATABASE DUMPFILE DIRECTORY ONTO A PARTICULAR
SERVER 4HE ASSUMPTION HERE IS THAT YOURE NOT EVER GOING TO USE THESE
BACKUPS MEANING THAT 3S SLOW SEARCHING AND ACCESS WILL NOT BE AN
ISSUE ONCE YOURE WORKING ON THE SERVER
Similarly, you might consider using Dropbox. Dropbox is best known
FOR ITS DESKTOP CLIENT BUT IT HAS A hHEADLESSv TEXT BASED CLIENT THAT CAN
BE USED ON ,INUX SERVERS WITHOUT A '5) CONNECTED /NE NICE ADVANTAGE
OF $ROPBOX IS THAT YOU CAN SHARE A FOLDER WITH ANY NUMBER OF PEOPLE

40 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 40 7/19/17 12:53 PM

 AT THE FORGE

which means you can have Dropbox distribute your backup databases
EVERYWHERE AUTOMATICALLY INCLUDING TO A NUMBER OF PEOPLE ON YOUR TEAM
4HE BACKUPS ARRIVE IN THEIR $ROPBOX FOLDER AND YOU CAN BE SURE THAT THE
LAMP is conditional.
&INALLY IF YOURE RUNNING A 7ORD0RESS SITE YOU MIGHT WANT TO CONSIDER
6AULT0RESS A FOR PAY BACKUP SYSTEM ) MUST ADMIT THAT IN THE WEEKS
BEFORE ) TOOK MY SERVER DOWN WITH A DATABASE BACKUP ERROR ) KEPT SEEING
ADS IN 7ORD0RESS FOR 6AULT0RESS h7HO WOULD BUY THATv ) ASKED MYSELF
THINKING THAT )M SMART ENOUGH TO DO BACKUPS MYSELF /F COURSE AFTER
DISASTER OCCURRED AND MY DATABASE WAS RUINED ) REALIZED THAT YEAR TO
BACK UP ALL OF MY DATA IS CHEAP AND ) SHOULD HAVE DONE IT BEFORE

Conclusion
7HEN IT COMES TO YOUR SERVERS THINK LESS LIKE AN OPTIMISTIC PROGRAMMER
AND MORE LIKE AN INSURANCE AGENT 0ERHAPS DISASTER WONT STRIKE BUT IF IT
DOES WILL YOU BE ABLE TO RECOVER -AKING SURE THAT EVEN IF YOUR SERVER IS
COMPLETELY UNAVAILABLE YOULL BE ABLE TO BRING UP YOUR PROGRAM AND ANY
associated database is crucial.
-Y PREFERRED SOLUTION INVOLVES COMBINING A 'IT REPOSITORY FOR CODE AND
CONFIGURATION FILES DISTRIBUTED ACROSS SEVERAL MACHINES AND SERVICES &OR
THE DATABASES HOWEVER ITS NOT ENOUGH TO DUMP YOUR DATABASE YOULL
NEED TO GET THAT DUMP ONTO A SEPARATE MACHINE AND PREFERABLY TEST THE
BACKUP FILE ON A REGULAR BASIS 4HAT WAY EVEN IF THINGS GO WRONG YOULL
be able to get back up in no time. Q

Send comments or feedback via

http://www.linuxjournal.com/contact
or to [email protected].

RETURN TO CONTENTS

41 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 41 7/19/17 12:53 PM

 WORK THE SHELL

Let’s Play
Bunco! DAVE TAYLOR
Bunco—a dice game that makes Yahtzee
look complicated! Dave Taylor has been
hacking shell scripts
on UNIX and Linux
systems for a really
PREVIOUS NEXT long time. He’s the
author of Learning
Reuven M. Lerner’s Kyle Rankin’s

V
V

Unix for Mac OS X

At the Forge Hack and / and Wicked Cool
Shell Scripts. You can
find him on Twitter
as @DaveTaylor,
or reach him through
I HAVEN’T DUG INTO ANY GAME PROGRAMMING
his tech Q&A site: http://
FOR A WHILE, so I thought it was high time to do www.AskDaveTaylor.com.
SOMETHING IN THAT REALM !T FIRST ) THOUGHT h(ALO AS A
SHELL SCRIPTv BUT THEN ) CAME TO MY SENSES )NSTEAD
LETS LOOK AT A SIMPLE DICE GAME CALLED "UNCO 9OU
MAY NOT HAVE HEARD OF IT BUT ) BET YOUR -OM HASˆ
ITS A QUITE POPULAR GAME FOR GROUPS OF GALS AT A LOCAL
pub or tavern.
Played in six rounds with three dice, the game is
simple. You roll all three dice and have to match the
CURRENT ROUND NUMBER )F ALL THREE DICE MATCH THE
CURRENT ROUND NUMBER FOR EXAMPLE THREE S IN ROUND
THREE  YOU SCORE  )F ALL THREE MATCH BUT ARENT THE
CURRENT ROUND NUMBER ITS A -INI "UNCO AND WORTH FIVE
POINTS &AILING BOTH OF THOSE EACH DIE WITH THE SAME
value as the round number is worth one point.
Played properly, the game also involves teams,

42 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 42 7/19/17 12:53 PM

 WORK THE SHELL

MULTIPLE TABLES INCLUDING A WINNERS TABLE AND USUALLY CASH PRIZES FUNDED
BY EVERYONE PAYING  OR SIMILAR TO PLAY AND BASED ON SPECIFIC WINNING
SCENARIOS LIKE hMOST "UNCOSv OR hMOST POINTSv )LL SKIP THAT PART HERE
HOWEVER AND JUST FOCUS ON THE DICE PART

Let’s Do the Math

"EFORE ) GO TOO FAR INTO THE PROGRAMMING SIDE OF THINGS LET ME TALK
BRIEFLY ABOUT THE MATH BEHIND THE GAME $ICE ARE EASY TO WORK WITH
BECAUSE ON A PROPERLY WEIGHTED DIE THE CHANCE OF A PARTICULAR VALUE
COMING UP IS 
2ANDOM TIP NOT SURE WHETHER YOUR DICE ARE BALANCED 4OSS THEM IN
SALTY WATER AND SPIN THEM 4HERE ARE SOME REALLY INTERESTING 9OU4UBE
VIDEOS FROM THE $$ WORLD SHOWING HOW TO DO THIS TEST
3O WHAT ARE THE ODDS OF THREE DICE HAVING THE SAME VALUE 4HE FIRST
DIE HAS A  CHANCE OF HAVING A VALUE NO LEANERS HERE  SO THATS
EASY 4HE SECOND DIE HAS A  CHANCE OF BEING ANY PARTICULAR VALUE
AND THEN THE THIRD DIE HAS THE SAME CHANCE OF BEING THAT VALUE BUT OF
COURSE THEY MULTIPLY SO THREE DICE HAVE ABOUT A  CHANCE OF ALL
having the same value.
4HEN ITS A  CHANCE THAT THOSE THREE DICE WOULD BE THE
CURRENT ROUNDS NUMBERˆOR IN MATHEMATICAL TERMS 


  
)N OTHER WORDS YOU HAVE A  CHANCE OF ROLLING A "UNCO WHICH IS A
BIT LESS THAN ONCE OUT OF EVERY  ROLLS OF THREE DICE
)T COULD BE TOUGHER THOUGH )F YOU WERE PLAYING WITH FIVE DICE THE
CHANCE OF ROLLING A -INI "UNCO OR 9AHTZEE IS  AND IF YOU WERE
TRYING TO ACCOMPLISH A SPECIFIC VALUE SAY JUST SIXES THEN ITS 
LIKELY ON ANY GIVEN ROLLˆWHICH IS TO SAY NOT BLOODY LIKELY

And So into the Coding

As with every game, the hardest part is really having a good random
NUMBER GENERATOR THAT GENERATES TRULY RANDOM VALUES 4HATS ACTUALLY
HARD TO AFFECT IN A SHELL SCRIPT THOUGH SO )M GOING TO SIDESTEP THIS
ENTIRE ISSUE AND ASSUME THAT THE SHELLS BUILT IN RANDOM NUMBER
GENERATOR WILL BE SUFFICIENT
7HATS NICE IS THAT ITS SUPER EASY TO WORK WITH *UST REFERENCE $RANDOM ,

43 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 43 7/19/17 12:53 PM

 WORK THE SHELL

AND YOULL HAVE A RANDOM VALUE BETWEEN  AND -!8).4  

$  echo  $RANDOM  $RANDOM  $RANDOM  

10252  22142  14863

4O CONSTRAIN THAT TO VALUES BETWEEN n USE THE MODULUS FUNCTION

$  echo  $((  $RANDOM  %  6  ))  

3  
$  echo  $((  $RANDOM  %  6  ))  
0

/OPS
) FORGOT TO SHIFT IT ONE (ERES ANOTHER TRY

$  echo  $((  (  $RANDOM  %  6  )  +  1  ))  

6

4HATS THE DICE ROLLING FEATURE ,ETS MAKE IT A FUNCTION WHERE YOU CAN SPECIFY
THE VARIABLE YOUD LIKE TO HAVE THE GENERATED VALUE AS PART OF THE INVOCATION

rolldie()  
{  
     local  result=$1  
     rolled=$((  (  $RANDOM  %  6  )  +  1  ))  
     eval  $result=$rolled  
}

4HE USE OF THE eval IS TO ENSURE THAT THE VARIABLE SPECIFIED IN THE
INVOCATION IS ACTUALLY ASSIGNED THE CALCULATED VALUE )TS EASY TO WORK WITH

rolldie  die1

4HAT WILL LOAD A RANDOM VALUE BETWEEN n INTO THE VARIABLE die1  4O
ROLL YOUR THREE DICE ITS STRAIGHTFORWARD

rolldie  die1  ;;  rolldie  die2  ;;  rolldie  die3

44 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 44 7/19/17 12:53 PM

 WORK THE SHELL

.OW TO TEST THE VALUES &IRST LETS TEST FOR A "UNCO WHERE ALL THREE DICE
HAVE THE SAME VALUE AND ITS THE VALUE OF THE CURRENT ROUND TOO

if  [  $die1  -­eq  $die2  ]  &&  [  $die2  -­eq  $die3  ]  ;;  then  

   if  [  $die1  -­eq  $round  ]  ;;  then  
       echo  "BUNCO!"  
       score=25  
   else  
       echo  "Mini  Bunco!"  
       score=5  
   fi

4HATS PROBABLY THE HARDEST OF THE TESTS AND NOTICE THE UNUSUAL USE
OF TEST IN THE FIRST CONDITIONAL [  cond1  ]  &&  [  cond2  ]  )F YOURE
thinking that you could also write it as cond1  -­a  cond2  YOURE
RIGHT !S WITH SO MUCH IN THE SHELL THERES MORE THAN ONE WAY TO GET
to the solution.
4HE REMAINDER OF THE CODE IS STRAIGHTFORWARD YOU JUST NEED TO TEST FOR
whether the die matches the current round value:

if  [  $die1  -­eq  $round  ]  ;;  then  

   score=1  
fi  
if  [  $die2  -­eq  $round  ]  ;;  then  
   score=$((  $score  +  1  ))  
fi  
if  [  $die3  -­eq  $round  ]  ;;  then  
   score=$((  $score  +  1  ))  
fi

4HE ONLY THING TO CONSIDER HERE IS THAT YOU DONT WANT TO SCORE
DIE VALUE VS ROUND IF YOUVE ALSO SCORED A "UNCO OR -INI "UNCO
SO THE ENTIRE SECOND SET OF TESTS NEEDS TO BE WITHIN THE else
CLAUSE OF THE FIRST CONDITIONAL TO SEE IF ALL THREE DICE HAVE THE
SAME VALUE 
0UT IT TOGETHER AND SPECIFY THE ROUND NUMBER ON THE COMMAND LINE

45 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 45 7/19/17 12:53 PM

 WORK THE SHELL

AND HERES WHAT YOU HAVE AT THIS POINT

$  sh  bunco.sh  5  
You  rolled:  1  1  5  
score  =  1  
$  sh  bunco.sh  2  
You  rolled:  6  4  3  
score  =  0  
$  sh  bunco.sh  1  
You  rolled:  1  1  1  
BUNCO!  
score  =  25

! "UNCO SO QUICKLY 7ELL AS ) SAID THERE MIGHT BE A SLIGHT ISSUE WITH

THE RANDOMNESS OF THE RANDOM NUMBER GENERATOR IN THE SHELL
9OU CAN TEST IT ONCE YOU HAVE THE SCRIPT WORKING BY RUNNING IT A FEW
hundred times and then checking to see what percentage are Bunco or
-INI "UNCO BUT )LL LEAVE THAT AS AN EXERCISE FOR YOU DEAR READER 7ELL
MAYBE )LL COME BACK TO IT NEXT MONTH
,ETS FINISH UP THIS SCRIPT BY HAVING IT ACCUMULATE SCORE AND RUN FOR ALL
SIX ROUNDS INSTEAD OF SPECIFYING A ROUND ON THE COMMAND LINE 4HATS
EASILY DONE BECAUSE ITS JUST A WRAPPER AROUND THE ENTIRE SCRIPT OR
BETTER THE BIG CONDITIONAL STATEMENT BECOMES A FUNCTION ALL ITS OWN

BuncoRound()  
{  
     #  roll,  display,  and  score  a  round  of  bunco!  
     #  round  is  specified  when  invoked,  score  added  to  totalscore  
 
     local  score=0  ;;  local  round=$1  ;;  local  hidescore=0  
 
     rolldie  die1  ;;  rolldie  die2  ;;  rolldie  die3  
     echo  Round  $round.  You  rolled:  $die1  $die2  $die3  
 
     if  [  $die1  -­eq  $die2  ]  &&  [  $die2  -­eq  $die3  ]  ;;  then  
         if  [  $die1  -­eq  $round  ]  ;;  then  

46 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 46 7/19/17 12:53 PM

 WORK THE SHELL

             echo  "    BUNCO!"  

             score=25  
             hidescore=1  
         else  
             echo  "    Mini  Bunco!"  
             score=5  
             hidescore=1  
         fi  
     else  
         if  [  $die1  -­eq  $round  ]  ;;  then  
             score=1  
         fi  
         if  [  $die2  -­eq  $round  ]  ;;  then  
             score=$((  $score  +  1  ))  
         fi  
         if  [  $die3  -­eq  $round  ]  ;;  then  
             score=$((  $score  +  1  ))  
         fi  
     fi  
 
     if  [  $hidescore  -­eq  0  ]  ;;  then  
         echo  "    score  this  round:  $score"  
     fi  
 
     totalscore=$((  $totalscore  +  $score  ))  
}

) ADMIT ) COULDNT RESIST A FEW IMPROVEMENTS AS ) WENT ALONG

INCLUDING THE ADDITION OF IT SHOWING EITHER Bunco , Mini  Bunco or a
SCORE VALUE THATS WHAT $hidescore DOES 
)NVOKING IT IS A BREEZE AND YOULL USE A for loop:

for  round  in  {1..6}  ;;  do  

   BuncoRound  $round  
done

47 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 47 7/19/17 12:53 PM

 WORK THE SHELL

4HATS ABOUT THE ENTIRE PROGRAM AT THIS POINT ,ETS RUN IT ONCE AND SEE
what happens:

$  sh  bunco.sh  1  
Round  1.  You  rolled:  2  3  3  
   score  this  round:  0  
Round  2.  You  rolled:  2  6  6  
   score  this  round:  1  
Round  3.  You  rolled:  1  2  4  
   score  this  round:  0  
Round  4.  You  rolled:  2  1  4  
   score  this  round:  1  
Round  5.  You  rolled:  5  5  6  
   score  this  round:  2  
Round  6.  You  rolled:  2  1  3  
   score  this  round:  0  
Game  over.  Your  total  score  was  4

5GH .OT TOO IMPRESSIVE BUT ITS PROBABLY A TYPICAL ROUND !GAIN YOU
CAN RUN IT A FEW HUNDREDˆOR THOUSANDˆTIMES JUST SAVE THE h'AME
OVERv LINE THEN DO SOME QUICK STATISTICAL ANALYSIS TO SEE HOW OFTEN YOU
SCORE MORE THAN  POINTS IN SIX ROUNDS 7ITH THREE DICE TO ROLL A GIVEN
VALUE YOU SHOULD HIT THAT  OF THE TIME
)TS NOT A COMPLICATED GAME BY ANY MEANS BUT IT MAKES FOR AN
INTERESTING LITTLE PROGRAMMING PROJECT .OW WHAT IF THEY USED  SIDED
DIE AND LET YOU RE ROLL ONE DIE PER ROUND AND HAD A DOZEN ROUNDS Q

Send comments or feedback via

http://www.linuxjournal.com/contact
or to [email protected].

RETURN TO CONTENTS

48 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 48 7/19/17 12:53 PM

LiunuxJournal_Layout 1 5/2/17 9:47 AM Page 1

Discover the Future – at the World’s Largest

Commercial Drone Conference & Expo

“If you want to see the state-of-the-art and expand

your knowledge about the drone industry,
September 6-8, 2017
InterDrone is the place to be.” Las Vegas
—George Gorrill, Structural Engineer, Thomas Engineering Group
www.InterDrone.com

Register Early for the

Biggest Discount!

LJ280-Aug2017.indd 49 7/19/17 12:53 PM

 HACK AND /

Preparing
for Vacation KYLE RANKIN
What to expect when you are expecting to go on
vacation. Kyle Rankin is VP of
engineering operations
at Final, Inc., the
NEXT author of many
PREVIOUS
Shawn Powers’ books including Linux
Dave Taylor’s

V
V

The Open-Source Hardening in Hostile

Work the Shell
Classroom Networks, DevOps
Troubleshooting and
The Official Ubuntu Server
Book, and a columnist
EVERY YEAR OR TWO MY FAMILY AND I LIKE TO
for Linux Journal.
TAKE A VACATION ABROAD. Normally, vacation is
Follow him @kylerankin.
A TIME TO UNPLUG AND IF YOU ARE A SYSADMIN WHOS
ON AN ON CALL ROTATION SOMEONE ELSE ON THE TEAM
TYPICALLY TAKES OVER YOUR ON CALL DUTIES 9ET AS YOU
progress in your career, you start to gain more
expertise and responsibilities over systems, and even
WITH SOMEONE ELSE ON CALL THERES A CERTAIN CLASS
OF EMERGENCY WHERE THE TEAM MIGHT NEED TO REACH
OUT TO YOU FOR HELP EVEN WHEN YOURE ON VACATION
) RECENTLY TOOK A VACATION ABROAD AND BEFORE ) LEFT
) WENT THROUGH A SET OF TASKS TO REDUCE THE CHANCE
that I would need to jump on an emergency while I
WAS AWAY 3O IN THIS ARTICLE ) DESCRIBE SOME OF THE
STEPS ) TAKE TO PREPARE FOR A VACATION THAT WILL HELP
you unplug on your next trip.

50 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 50 7/19/17 12:53 PM

 HACK AND /

Preparing Your Computer

/NE OF THE FIRST QUESTIONS YOU SHOULD ANSWER BEFORE GOING ON
vacation is whether you will need to take your work laptop with you.
Depending on your organization and its security controls, you might
BE ABLE TO PERFORM BASIC EMERGENCY ADMINISTRATIVE TASKS FROM YOUR
personal computer, tablet or phone, or you may be able to connect to
PRODUCTION ONLY FROM YOUR WORK COMPUTER )N OTHER CASES YOU MAY NOT
need a computer, because you can just serve an advisory role over the
phone or chat with other people on the team and walk them through
WHAT TO DO IN THE EVENT OF AN EMERGENCY
)F YOU DO NEED TO TAKE YOUR COMPUTER ) HIGHLY RECOMMEND MAKING A FULL
BACKUP BEFORE THE TRIP 9OUR COMPUTER IS MORE LIKELY TO BE LOST STOLEN OR
BROKEN WHILE TRAVELING THAN WHEN SITTING SAFELY AT THE OFFICE SO ) ALWAYS
TAKE A BACKUP OF MY WORK MACHINE BEFORE A TRIP %VEN BETTER THAN TAKING
a backup, leave your expensive work computer behind and use a cheaper
MORE DISPOSABLE MACHINE FOR TRAVEL AND JUST RESTORE YOUR IMPORTANT FILES
AND SETTINGS FOR WORK ON IT BEFORE YOU LEAVE AND WIPE IT WHEN YOU RETURN
)F YOU DECIDE TO GO THE DISPOSABLE COMPUTER ROUTE ) RECOMMEND WORKING
ONE OR TWO FULL WORK DAYS ON THIS COMPUTER BEFORE THE VACATION TO MAKE
SURE ALL OF YOUR FILES AND SETTINGS ARE IN PLACE

Documentation
Good documentation is the best way to reduce or eliminate how much
YOU HAVE TO STEP IN WHEN YOU ARENT ON CALL WHETHER YOURE ON VACATION
OR NOT %VERYTHING FROM ROUTINE PROCEDURES TO EMERGENCY RESPONSE SHOULD
BE DOCUMENTED AND KEPT UP TO DATE (ONESTLY THIS FALLS UNDER STANDARD
BEST PRACTICES AS A SYSADMIN SO ITS SOMETHING YOU SHOULD HAVE WHETHER
or not you are about to go on vacation.
&IRST ALL ROUTINE PROCEDURES FROM HOW YOU DEPLOY CODE AND
CONFIGURATION CHANGES HOW YOU MANAGE TICKETS HOW YOU PERFORM
security patches, how you add and remove users, and how the overall
ENVIRONMENT IS STRUCTURED SHOULD BE DOCUMENTED IN A CLEAR STEP BY STEP
WAY )F YOU USE AUTOMATION TOOLS FOR ROUTINE PROCEDURES WHETHER ITS
AS SIMPLE AS A FEW SCRIPTS OR AS COMPLEX AS FULL ORCHESTRATION TOOLS YOU
should make sure you document not only how to use the automation
TOOLS BUT ALSO HOW TO PERFORM THE SAME TASKS MANUALLY SHOULD THE

51 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 51 7/19/17 12:53 PM

 HACK AND /

AUTOMATION TOOLS FAIL

)F YOU ARE ON CALL THAT MEANS YOU HAVE A MONITORING SYSTEM IN PLACE
THAT SCANS YOUR INFRASTRUCTURE FOR PROBLEMS AND PAGES YOU WHEN IT FINDS
ANY %VERY SINGLE SYSTEM CHECK IN YOUR MONITORING TOOL SHOULD HAVE A
CORRESPONDING PLAYBOOK THAT A SYSADMIN CAN FOLLOW TO TROUBLESHOOT
AND FIX THE PROBLEM )F YOUR MONITORING TOOL ALLOWS YOU TO CUSTOMIZE
THE ALERTS IT SENDS CREATE CORRESPONDING WIKI ENTRIES FOR EACH ALERT
name, and then customize the alert so that it provides a direct link to
the playbook in the wiki.
)F YOU HAPPEN TO BE THE SUBJECT MATTER EXPERT ON A PARTICULAR SYSTEM
MAKE SURE THAT DOCUMENTATION IN PARTICULAR IS WELL FLESHED OUT AND
UNDERSTANDABLE 4HESE ARE THE SYSTEMS THAT WILL PULL YOU OUT OF YOUR
VACATION SO LOOK THROUGH THOSE DOCUMENTS FOR ANY ASSUMPTIONS YOU
MAY HAVE MADE WHEN WRITING THEM THAT A JUNIOR MEMBER OF THE TEAM
MIGHT NOT UNDERSTAND (AVE OTHER MEMBERS OF THE TEAM REVIEW THE
DOCUMENTATION AND ASK YOU QUESTIONS
/NE SAYING ABOUT DOCUMENTATION IS THAT IF SOMETHING IS DOCUMENTED
IN TWO PLACES ONE OF THEM WILL BE OUT OF DATE %VEN IF YOU DOCUMENT
SOMETHING ONLY IN ONE PLACE THERES A GOOD CHANCE IT IS OUT OF DATE
UNLESS YOU PERFORM ROUTINE MAINTENANCE )TS A GOOD PRACTICE TO REVIEW
YOUR DOCUMENTATION FROM TIME TO TIME AND UPDATE IT WHERE NECESSARY
AND BEFORE A VACATION IS A PARTICULARLY GOOD TIME TO DO IT )F YOU ARE THE
ONLY PERSON THAT KNOWS ABOUT THE NEW WAY TO PERFORM A PROCEDURE YOU
should make sure your documentation covers it.
Finally, have your team maintain a page to capture anything that
happens while you are gone that they want to tell you about when you
GET BACK )F YOU ARE THE MAIN MAINTAINER OF A PARTICULAR SYSTEM BUT THEY
HAD TO PERFORM SOME EMERGENCY MAINTENANCE OF IT WHILE YOU WERE GONE
THATS THE KIND OF THING YOUD LIKE TO KNOW ABOUT WHEN YOU GET BACK )F
THERES A CENTRAL PLACE FOR THE TEAM TO CAPTURE THESE NOTES THEY WILL BE
MORE LIKELY TO WRITE THINGS DOWN AS THEY HAPPEN AND LESS LIKELY TO FORGET
about things when you get back.

Stable State
4HE MORE STABLE YOUR INFRASTRUCTURE IS BEFORE YOU LEAVE AND THE MORE
STABLE IT STAYS WHILE YOU ARE GONE THE LESS LIKELY YOULL BE DISTURBED ON

52 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 52 7/19/17 12:54 PM

 HACK AND /

YOUR VACATION 2IGHT BEFORE A VACATION IS A TERRIBLE TIME TO MAKE A MAJOR

CHANGE TO CRITICAL SYSTEMS )F YOU CAN FREEZE CHANGES IN THE WEEKS LEADING
UP TO YOUR VACATION 4RY TO ENCOURAGE OTHER TEAMS TO PUSH OFF ANY MAJOR
CHANGES UNTIL AFTER YOU GET BACK
"EFORE A VACATION IS ALSO A GREAT TIME TO PERFORM ANY PREVENTATIVE
MAINTENANCE ON YOUR SYSTEMS #HECK FOR ANY SYSTEMS ABOUT TO HIT A DISK
WARNING THRESHOLD AND CLEAR OUT SPACE )N GENERAL IF YOU COLLECT TRENDING
DATA SKIM THROUGH IT FOR ANY RESOURCES THAT ARE TRENDING UPWARD THAT
MIGHT GO PAST THRESHOLDS WHILE YOU ARE GONE )F YOU HAVE ANY TASKS
that might add extra load to your systems while you are gone, pause
OR POSTPONE THEM IF YOU CAN -AKE SURE ALL OF YOUR BACKUP SCRIPTS ARE
WORKING AND ALL OF YOUR BACKUPS ARE UP TO DATE

Emergency Contact Methods

Although it would be great to unplug completely while on vacation,
THERES A CHANCE THAT SOMEONE FROM WORK MIGHT WANT TO REACH YOU IN
an emergency. Depending on where you plan to travel, some contact
OPTIONS MAY WORK BETTER THAN OTHERS &OR INSTANCE SOME CELL PHONE
PLANS THAT WORK WHILE TRAVELING MIGHT CHARGE HIGH RATES FOR CALLS
BUT TEXT MESSAGES AND DATA BILL AT THE SAME RATES AS AT HOME )F YOU
plan to get a local sim card, text messages sent over the cell network
FROM HOME MIGHT COST MORE THAN THOSE SENT OVER THE DATA PLAN )N
THE EVENT OF A LOCAL SIM CARD YOU WILL HAVE TO WORK OUT SOME WAY TO
communicate that new number to your team.
Discuss with your team what escalation path they should use to contact
YOU IN AN EMERGENCY &OR INSTANCE IN MY CASE ) KNEW MY CELL PHONE
plan would provide me with unlimited text messages and the same data
PLAN AS AT HOME BUT ) ALSO DIDNT WANT WORK EMAIL TO DISTRACT ME 4HIS
PRESENTED A PROBLEM AS EMAIL IS THE PRIMARY WAY )M PAGED )N MY CASE
I disabled email syncing while I was on vacation and instructed everyone
TO CONTACT ME VIA TEXT MESSAGE IN THE CASE OF EMERGENCY ) ALSO NEEDED TO
BE ON THE SECONDARY ESCALATION PATH FOR ANY ALERTS THAT WERENT RESOLVED
WITHIN A CERTAIN AMOUNT OF TIME SO ) CONFIGURED MY MONITORING TOOL TO
USE AN EMAIL TO 3-3 GATEWAY AS MY EMAIL ADDRESS FOR ALERTS
)F THERE ARE CERTAIN DAYS WHEN YOU KNOW YOU OR YOUR ON CALL
COUNTERPART AT HOME MIGHT BE IN AREAS WITH LIMITED CELL COVERAGE WORK

53 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 53 7/19/17 12:54 PM

 HACK AND /

OUT THOSE DATES AHEAD OF TIME AND PUT THEM IN YOUR CALENDAR )F NOTHING
ELSE IT MIGHT ENCOURAGE OTHERS TO WAIT ON MAKING A RISKY CHANGE IF THEY
KNOW THEY ABSOLUTELY WILL NOT BE ABLE TO REACH YOU FOR THE NEXT TWO
days. In general, set expectations on your availability, and also make sure
EVERYONE TAKES ANY TIME ZONE DIFFERENCES INTO ACCOUNT

Conclusion
/VERALL A VACATION SHOULD BE A TIME FOR YOU TO BE COMPLETELY REMOVED
FROM YOUR WORKS ON CALL PROCESS 7HETHER THATS POSSIBLE OR NOT THE
MORE YOU PREPARE AHEAD OF TIME THE LESS LIKELY YOUR VACATION WILL BE
interrupted. Finally, when you get back, do a post mortem with your
team about anything that went wrong and any documentation that
WAS CONFUSING OR INCOMPLETE SO YOU CAN MAKE IMPROVEMENTS FOR YOUR
next vacation. Q

Send comments or feedback via

http://www.linuxjournal.com/contact
or to [email protected].

RETURN TO CONTENTS

54 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 54 7/19/17 12:54 PM

'17HPCSeptLinuxJnlad.qxp_Layout 1 7/9/17 11:05 AM Page 1

14th Annual
2017 H IGH PERFORMANCE COMPUTING
FOR WALL STREET- CLOUD, AI AND DATA CENTERS
Show and Conference
September 12, 2017 (Tuesday) Roosevelt Hotel, NYC
Madison Ave and 45th St, next to Grand Central Station

Meetup September 12, Tuesday, for Finance, Trading, Banking,

Exchanges, Brokerage, Funds, at one time and one place.

Register Today: HPC, Cloud, AI, Machine Learning, Data Centers, Big Data,
Linux, Low Latency, Networks, Cost Savings.
Capital Markets, Systems, Architecture, Cloud, Machine Learning and AI is
driving solutions for large data centers and HPC computing.
Go online for the full conference program and save
$100. Includes general sessions, drill down sessions,
an industry luncheon, $295 in advance. $395 on site.
Qualified end-users are invited to register at no
charge. for the full conference. HPC sponsors and
Don’t have time for the full Conference? Register exhibitors to show and
demonstrate all new
for the free Show. at: www.flaggmgmt.com/hpc HPC systems at the
Show.
Register online: www.flaggmgmt.com/hpc

Show Hours: Tues, Sept 12 8:00 - 4:00 Wall Street IT speakers and Gold
Conference Hours: 8:30 - 4:50 Sponsors will lead drill-down ses-
sions in the Grand Ballroom program.

2016 Sponsors

Need Sponsorship and Exhibit Information?

Show & Conference: Flagg Management Inc
353 Lexington Avenue, New York 10016
(212) 286 0333 fax: (212) 286 0086
[email protected] Visit: www.flaggmgmt.com/hpc
LJ280-Aug2017.indd 55 7/19/17 12:54 PM
 THE OPEN-SOURCE CLASSROOM

Ansible: the
Automation
Framework SHAWN
POWERS

That Thinks
Shawn Powers is the
Associate Editor for
Linux Journal. He’s
also the Gadget Guy

Like a Sysadmin
for LinuxJournal.com,
and he has an
interesting collection
of vintage Garfield
coffee mugs. Don’t
With Ansible, managing 50 servers is a lot like let his silly hairdo
managing one server! fool you, he’s a
pretty ordinary guy
and can be reached
via email at
[email protected].
PREVIOUS Or, swing by the
NEXT #linuxjournal IRC
Kyle Rankin’s
V
V

New Products channel on

Hack and / Freenode.net.

I’VE WRITTEN ABOUT AND TRAINED FOLKS ON

VARIOUS DEVOPS TOOLS THROUGH THE YEARS,
AND ALTHOUGH THEYRE AWESOME ITS OBVIOUS THAT MOST
OF THEM ARE DESIGNED FROM THE MIND OF A DEVELOPER
4HERES NOTHING WRONG WITH THAT BECAUSE APPROACHING
CONFIGURATION MANAGEMENT PROGRAMMATICALLY IS THE
WHOLE POINT 3TILL IT WASNT UNTIL ) STARTED PLAYING WITH
!NSIBLE THAT ) FELT LIKE IT WAS SOMETHING A SYSADMIN

56 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 56 7/19/17 12:54 PM

 THE OPEN-SOURCE CLASSROOM

QUICKLY WOULD APPRECIATE

0ART OF THAT APPRECIATION COMES FROM THE WAY !NSIBLE COMMUNICATES
WITH ITS CLIENT COMPUTERSˆNAMELY VIA 33( !S SYSADMINS YOURE
ALL VERY FAMILIAR WITH CONNECTING TO COMPUTERS VIA 33( SO RIGHT FROM
THE WORD hGOv YOU HAVE A BETTER UNDERSTANDING OF !NSIBLE THAN THE
other alternatives.
7ITH THAT IN MIND )M PLANNING TO SPEND MY NEXT FEW ARTICLES
LEARNING HOW TO TAKE ADVANTAGE OF !NSIBLE )TS A GREAT SYSTEM BUT
WHEN ) WAS FIRST EXPOSED TO IT IT WASNT CLEAR HOW TO START )TS NOT THAT
THE LEARNING CURVE IS STEEP )N FACT IF ANYTHING THE PROBLEM WAS THAT
) DIDNT REALLY HAVE THAT MUCH TO LEARN BEFORE STARTING TO USE !NSIBLE
AND THAT MADE IT CONFUSING &OR EXAMPLE IF YOU DONT HAVE TO INSTALL
AN AGENT PROGRAM !NSIBLE DOESNT HAVE ANY SOFTWARE INSTALLED ON THE
CLIENT COMPUTERS  HOW DO YOU START

Getting to the Starting Line

4HE REASON !NSIBLE WAS SO DIFFICULT FOR ME AT FIRST IS BECAUSE ITS SO FLEXIBLE
WITH HOW TO CONFIGURE THE SERVERCLIENT RELATIONSHIP ) DIDNT KNOW WHAT
) WAS SUPPOSED TO DO 4HE TRUTH IS THAT !NSIBLE DOESNT REALLY CARE HOW
YOU SET UP THE 33( SYSTEM IT WILL UTILIZE WHATEVER CONFIGURATION YOU HAVE
4HERE ARE JUST A COUPLE THINGS TO CONSIDER

 !NSIBLE NEEDS TO CONNECT TO THE CLIENT COMPUTER VIA 33(

 /NCE CONNECTED !NSIBLE NEEDS TO ELEVATE PRIVILEGE SO IT CAN CONFIGURE

the system, install packages and so on.

5NFORTUNATELY THOSE TWO CONSIDERATIONS REALLY OPEN A CAN OF WORMS

Connecting to a remote computer and elevating privilege is a scary thing
TO ALLOW &OR SOME REASON IT FEELS LESS VULNERABLE WHEN YOU SIMPLY INSTALL
AN AGENT ON THE REMOTE COMPUTER AND LET #HEF OR 0UPPET HANDLE PRIVILEGE
ESCALATION )TS NOT THAT !NSIBLE IS ANY LESS SECURE BUT RATHER IT PUTS THE
security decisions in your hands.
.EXT )M GOING TO LIST A BUNCH OF POTENTIAL CONFIGURATIONS ALONG
WITH THE PROS AND CONS OF EACH 4HIS ISNT AN EXHAUSTIVE LIST BUT IT
SHOULD GET YOU THINKING ALONG THE RIGHT LINES FOR WHAT WILL BE IDEAL

57 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 57 7/19/17 12:54 PM

 THE OPEN-SOURCE CLASSROOM

IN YOUR ENVIRONMENT ) ALSO SHOULD NOTE THAT )M NOT GOING TO MENTION
SYSTEMS LIKE 6AGRANT BECAUSE ALTHOUGH 6AGRANT IS WONDERFUL FOR
BUILDING A QUICK INFRASTRUCTURE FOR TESTING AND DEVELOPING ITS SO VERY
DIFFERENT FROM A BUNCH OF SERVERS THAT THE CONSIDERATIONS ARE TOO
dissimilar really to compare.

Some SSH Scenarios

1) SSHing into remote computer as root with password in Ansible config.
) STARTED WITH A TERRIBLE IDEA 4HE hPROSv OF THIS SETUP IS THAT IT
ELIMINATES THE NEED FOR PRIVILEGE ESCALATION AND THERE ARE NO OTHER
USER ACCOUNTS REQUIRED ON THE REMOTE SERVER "UT THE COST FOR SUCH
CONVENIENCE ISNT WORTH IT &IRST MOST SYSTEMS WONT LET YOU 33( IN
AS ROOT WITHOUT CHANGING THE DEFAULT CONFIGURATION 4HOSE DEFAULT
CONFIGURATIONS ARE THERE BECAUSE QUITE FRANKLY ITS JUST A BAD IDEA
to allow the root user to connect remotely. Second, putting a root
PASSWORD IN A PLAIN TEXT CONFIGURATION FILE ON THE !NSIBLE MACHINE IS
MORTIFYING 2EALLY ) MENTIONED THIS POSSIBILITY BECAUSE IT is a possibility,
BUT ITS ONE THAT SHOULD BE AVOIDED 2EMEMBER !NSIBLE ALLOWS YOU TO
CONFIGURE THE CONNECTION YOURSELF AND IT WILL LET YOU DO REALLY DUMB
THINGS 0LEASE DONT
2) SSHing into a remote computer as a regular user, using a password
stored in the Ansible config.
!N ADVANTAGE OF THIS SCENARIO IS THAT IT DOESNT REQUIRE MUCH
CONFIGURATION OF THE CLIENTS -OST USERS ARE ABLE TO 33( IN BY DEFAULT
SO !NSIBLE SHOULD BE ABLE TO USE CREDENTIALS AND LOG IN FINE ) PERSONALLY
DISLIKE THE IDEA OF A PASSWORD BEING STORED IN PLAIN TEXT IN A CONFIGURATION
FILE BUT AT LEAST IT ISNT THE ROOT PASSWORD )F YOU USE THIS METHOD BE SURE
to consider how privilege escalation will take place on the remote server.
) KNOW ) HAVENT TALKED ABOUT ESCALATING PRIVILEGE YET BUT IF YOU HAVE A
PASSWORD IN THE CONFIG FILE THAT SAME PASSWORD LIKELY WILL BE USED TO GAIN
SUDO ACCESS 3O WITH ONE SLIP YOUVE COMPROMISED NOT ONLY THE REMOTE
USERS ACCOUNT BUT ALSO POTENTIALLY THE ENTIRE SYSTEM
3) SSHing into a remote computer as a regular user, authenticating with
a key pair that has an empty passphrase.
4HIS ELIMINATES STORING PASSWORDS IN A CONFIGURATION FILE AT LEAST FOR
THE LOGGING IN PART OF THE PROCESS +EY PAIRS WITHOUT PASSPHRASES ARENT

58 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 58 7/19/17 12:54 PM

 THE OPEN-SOURCE CLASSROOM

IDEAL BUT ITS SOMETHING ) OFTEN DO IN AN ENVIRONMENT LIKE MY HOUSE /N

my internal network, I typically use a key pair without a passphrase to
AUTOMATE MANY THINGS LIKE CRON JOBS THAT REQUIRE AUTHENTICATION 4HIS
ISNT THE MOST SECURE OPTION BECAUSE A COMPROMISED PRIVATE KEY MEANS
UNRESTRICTED ACCESS TO THE REMOTE USERS ACCOUNT BUT ) LIKE IT BETTER THAN A
PASSWORD IN A CONFIG FILE
4) SSHing into a remote computer as a regular user, authenticating with
a key pair that is secured by a passphrase.
4HIS IS A VERY SECURE WAY OF HANDLING REMOTE ACCESS BECAUSE IT
REQUIRES TWO DIFFERENT AUTHENTICATION FACTORS  THE PRIVATE KEY AND
 THE PASSPHRASE TO DECRYPT IT )F YOURE JUST RUNNING !NSIBLE
INTERACTIVELY THIS MIGHT BE THE IDEAL SETUP 7HEN YOU RUN A COMMAND
!NSIBLE SHOULD PROMPT YOU FOR THE PRIVATE KEYS PASSPHRASE AND
THEN ITLL USE THE KEY PAIR TO LOG IN TO THE REMOTE SYSTEM 9ES THE
same could be done by just using a standard password login and not
SPECIFYING THE PASSWORD IN THE CONFIGURATION FILE BUT IF YOURE GOING
to be typing a password on the command line anyway, why not add
THE LAYER OF PROTECTION A KEY PAIR OFFERS
5) SSHing with a passphrase-protected key pair, but using ssh-agent
to “unlock” the private key.
4HIS DOESNT PERFECTLY ANSWER THE QUESTION OF UNATTENDED AUTOMATED
!NSIBLE COMMANDS BUT IT DOES MAKE A FAIRLY SECURE SETUP CONVENIENT AS
WELL 4HE SSH AGENT PROGRAM AUTHENTICATES THE PASSPHRASE ONE TIME AND
THEN USES THAT AUTHENTICATION TO MAKE FUTURE CONNECTIONS 7HEN )M
USING !NSIBLE THIS IS WHAT ) THINK )D LIKE TO BE DOING )F )M COMPLETELY
HONEST ) STILL USUALLY USE KEY PAIRS WITHOUT PASSPHRASES BUT THATS TYPICALLY
BECAUSE )M WORKING ON MY HOME SERVERS NOT SOMETHING PRONE TO ATTACK
4HERE ARE SOME OTHER CONSIDERATIONS TO KEEP IN MIND WHEN CONFIGURING
YOUR 33( ENVIRONMENT 0ERHAPS YOURE ABLE TO RESTRICT THE !NSIBLE USER
WHICH IS OFTEN YOUR LOCAL USER NAME SO IT CAN LOG IN ONLY FROM A SPECIFIC
)0 ADDRESS 0ERHAPS YOUR !NSIBLE SERVER CAN LIVE IN A DIFFERENT SUBNET
BEHIND A STRONG FIREWALL SO ITS PRIVATE KEYS ARE MORE DIFFICULT TO ACCESS
REMOTELY -AYBE THE !NSIBLE SERVER DOESNT HAVE AN 33( SERVER INSTALLED
ON ITSELF SO THERES NO INCOMING ACCESS AT ALL !GAIN ONE OF THE STRENGTHS
OF !NSIBLE IS THAT IT USES THE 33( PROTOCOL FOR COMMUNICATION AND ITS
A PROTOCOL YOUVE ALL HAD YEARS TO TWEAK INTO A SYSTEM THAT WORKS BEST

59 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 59 7/19/17 12:54 PM

 THE OPEN-SOURCE CLASSROOM

IN YOUR ENVIRONMENT )M NOT A BIG FAN OF PROCLAIMING WHAT THE hBEST
PRACTICEv IS BECAUSE IN REALITY THE BEST PRACTICE IS TO CONSIDER YOUR
ENVIRONMENT AND CHOOSE THE SETUP THAT FITS YOUR SITUATION THE BEST

Privilege Escalation
Once your Ansible server connects to its clients via SSH, it needs to be
ABLE TO ESCALATE PRIVILEGE )F YOU CHOSE OPTION  ABOVE YOURE ALREADY
ROOT AND THIS IS A MOOT POINT "UT SINCE NO ONE CHOSE OPTION  RIGHT 
you need to consider how a regular user on the client computer gains
ACCESS !NSIBLE SUPPORTS A WIDE VARIETY OF ESCALATION SYSTEMS BUT IN ,INUX
THE MOST COMMON OPTIONS ARE SUDO AND SU !S WITH 33( THERE ARE A FEW
situations to consider, although there are certainly other options.
1) Escalate privilege with su.
For Red Hat/CentOS users, the instinct might be to use su in order
TO GAIN SYSTEM ACCESS "Y DEFAULT THOSE SYSTEMS CONFIGURE THE ROOT
password during install, and to gain privileged access, you need to
TYPE IT IN 4HE PROBLEM WITH USING SU IS THAT ALTHOUGH IT GIVES YOU
total access to the remote system, it also gives you total access to the
REMOTE SYSTEM 9ES THAT WAS SARCASM !LSO THE SU PROGRAM DOESNT
have the ability to authenticate with key pairs, so the password either
MUST BE INTERACTIVELY TYPED OR STORED IN THE CONFIGURATION FILE !ND
SINCE ITS LITERALLY THE ROOT PASSWORD STORING IT IN THE CONFIG FILE SHOULD
sound like a horrible idea, because it is.
2) Escalate privilege with sudo.
4HIS IS HOW $EBIAN5BUNTU SYSTEMS ARE CONFIGURED ! USER IN THE CORRECT
group has access to sudo a command and execute it with root privileges.
/UT OF THE BOX THIS STILL HAS THE PROBLEM OF PASSWORD STORAGE OR
INTERACTIVE TYPING 3INCE STORING THE USERS PASSWORD IN THE CONFIGURATION
FILE SEEMS A LITTLE LESS HORRIBLE ) GUESS THIS IS A STEP UP FROM USING SU BUT
IT STILL GIVES COMPLETE ACCESS TO A SYSTEM IF THE PASSWORD IS COMPROMISED
!FTER ALL TYPING sudo  su  -­ WILL ALLOW USERS TO BECOME ROOT JUST AS IF
THEY HAD THE ROOT PASSWORD
3) Escalate privilege with sudo and configure NOPASSWD in the
sudoers file.
!GAIN IN MY LOCAL ENVIRONMENT THIS IS WHAT ) DO )TS NOT PERFECT
BECAUSE IT GIVES UNRESTRICTED ROOT ACCESS TO THE USER ACCOUNT AND DOESNT

60 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 60 7/19/17 12:54 PM

 THE OPEN-SOURCE CLASSROOM

REQUIRE ANY PASSWORDS "UT WHEN ) DO THIS AND USE 33( KEY PAIRS WITHOUT
PASSPHRASES IT ALLOWS ME TO AUTOMATE !NSIBLE COMMANDS EASILY )LL NOTE
again, that although it is convenient, it is not a terribly secure idea.
4) Escalate privilege with sudo and configure NOPASSWD on
specific executables.
4HIS IDEA MIGHT BE THE BEST COMPROMISE OF SECURITY AND CONVENIENCE
"ASICALLY IF YOU KNOW WHAT YOU PLAN TO DO WITH !NSIBLE YOU CAN GIVE
./0!337$ PRIVILEGE TO THE REMOTE USER FOR JUST THOSE APPLICATIONS
IT WILL NEED TO USE )T MIGHT GET A LITTLE CONFUSING SINCE !NSIBLE USES
0YTHON FOR LOTS OF THINGS BUT WITH ENOUGH TRIAL AND ERROR YOU SHOULD
BE ABLE TO FIGURE THINGS OUT )T IS MORE WORK BUT DOES ELIMINATE SOME
OF THE GLARING SECURITY HOLES

Implementing Your Plan

/NCE YOU DECIDE HOW YOURE GOING TO HANDLE !NSIBLE AUTHENTICATION
AND PRIVILEGE ESCALATION YOU NEED TO SET IT UP !FTER YOU BECOME
WELL VERSED AT !NSIBLE YOU MIGHT BE ABLE TO USE THE TOOL ITSELF TO
HELP hBOOTSTRAPv NEW CLIENTS BUT AT FIRST ITS IMPORTANT TO CONFIGURE
CLIENTS MANUALLY SO YOU KNOW WHATS HAPPENING )TS FAR BETTER TO
AUTOMATE A PROCESS YOURE FAMILIAR WITH THAN TO START WITH AUTOMATION
FROM THE BEGINNING
)VE WRITTEN ABOUT 33( KEY PAIRS IN THE PAST AND THERE ARE COUNTLESS
ARTICLES ONLINE FOR SETTING IT UP 4HE SHORT VERSION FROM YOUR !NSIBLE
computer, looks something like this:

#  ssh-­keygen  
#  ssh-­copy-­id  -­i  .ssh/id_dsa.pub  [email protected]  
#  ssh  [email protected]

)F YOUVE CHOSEN TO USE NO PASSPHRASE WHEN CREATING YOUR KEY PAIRS

that last step should get you into the remote computer without typing
a password or passphrase.
)N ORDER TO SET UP PRIVILEGE ESCALATION IN SUDO YOULL NEED TO EDIT THE
SUDOERS FILE 9OU SHOULDNT EDIT THE FILE DIRECTLY BUT RATHER USE

#  sudo  visudo

61 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 61 7/19/17 12:54 PM

 THE OPEN-SOURCE CLASSROOM

4HIS WILL OPEN THE SUDOERS FILE AND ALLOW YOU TO MAKE CHANGES SAFELY
IT ERROR CHECKS WHEN YOU SAVE SO YOU DONT ACCIDENTALLY LOCK YOURSELF
OUT WITH A TYPO  4HERE ARE EXAMPLES IN THE FILE SO YOU SHOULD BE ABLE
TO FIGURE OUT HOW TO ASSIGN THE EXACT PRIVILEGES YOU WANT
/NCE ITS ALL CONFIGURED YOU SHOULD TEST IT MANUALLY BEFORE
BRINGING !NSIBLE INTO THE PICTURE 4RY 33(ING TO THE REMOTE CLIENT
AND THEN TRY ESCALATING PRIVILEGE USING WHATEVER METHODS YOUVE
CHOSEN /NCE YOU HAVE CONFIGURED THE WAY YOULL CONNECT ITS TIME
to install Ansible.

Installing Ansible
3INCE THE !NSIBLE PROGRAM GETS INSTALLED ONLY ON THE SINGLE COMPUTER ITS
NOT A BIG CHORE TO GET GOING 2ED (AT5BUNTU SYSTEMS DO PACKAGE INSTALLS
A BIT DIFFERENTLY BUT NEITHER IS DIFFICULT
)N 2ED (AT#ENT/3 FIRST ENABLE THE %0%, REPOSITORY

sudo  yum  install  epel-­release

4HEN INSTALL !NSIBLE

sudo  yum  install  ansible

)N 5BUNTU FIRST ENABLE THE !NSIBLE 00!

sudo  apt-­add-­repository  spa:ansible/ansible  

(press  ENTER  to  access  the  key  and  add  the  repo)

4HEN INSTALL !NSIBLE

sudo  apt-­get  update  

sudo  apt-­get  install  ansible

Configuring Ansible Hosts File

4HE !NSIBLE SYSTEM HAS NO WAY OF KNOWING WHICH CLIENTS YOU WANT IT
TO CONTROL UNLESS YOU GIVE IT A LIST OF COMPUTERS 4HAT LIST IS VERY SIMPLE
and it looks something like this:

62 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 62 7/19/17 12:54 PM

 THE OPEN-SOURCE CLASSROOM

#  file  /etc/ansible/hosts  
 
[webservers]  
blogserver  ansible_host=192.168.1.5  
wikiserver  ansible_host=192.168.1.10  
 
[dbservers]  
mysql_1  ansible_host=192.168.1.22  
pgsql_1  ansible_host=192.168.1.23

4HE BRACKETED SECTIONS ARE SPECIFYING GROUPS )NDIVIDUAL HOSTS CAN BE LISTED
IN MULTIPLE GROUPS AND !NSIBLE CAN REFER EITHER TO INDIVIDUAL HOSTS OR GROUPS
4HIS IS ALSO THE CONFIGURATION FILE WHERE THINGS LIKE PLAIN TEXT PASSWORDS
WOULD BE STORED IF THATS THE SORT OF SETUP YOUVE PLANNED %ACH LINE IN
THE CONFIGURATION FILE CONFIGURES A SINGLE HOST AND YOU CAN ADD MULTIPLE
DECLARATIONS AFTER THE ansible_host  STATEMENT 3OME USEFUL OPTIONS ARE

ansible_ssh_pass  
ansible_become  
ansible_become_method  
ansible_become_user  
ansible_become_pass

The Ansible Vault

I also should note that although the setup is more complex, and not
something you’ll likely do during your first foray into the world of Ansible,
the program does offer a way to encrypt passwords in a vault. Once
you’re familiar with Ansible and you want to put it into production,
storing those passwords in an encrypted Ansible vault is ideal. But in
the spirit of learning to crawl before you walk, I recommend starting in a
non-production environment and using passwordless methods at first.

63 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 63 7/19/17 12:54 PM

 THE OPEN-SOURCE CLASSROOM

Finally, you should test your system to make sure your clients are
CONNECTING 4HE PING TEST WILL MAKE SURE THE !NSIBLE COMPUTER CAN PING
each host:

ansible  -­m  ping  all

!FTER RUNNING YOU SHOULD SEE A MESSAGE FOR EACH DEFINED HOST
showing a ping:  pong IF THE PING WAS SUCCESSFUL 4HIS DOESNT ACTUALLY
TEST AUTHENTICATION JUST THE NETWORK CONNECTIVITY 4RY THIS TO TEST YOUR
authentication:

ansible  -­m  shell  -­a  'uptime'  webservers

9OU SHOULD SEE THE RESULTS OF THE uptime COMMAND FOR EACH HOST IN THE
webservers group.
)N MY NEXT ARTICLE )LL START TO DIG IN TO !NSIBLES ABILITY TO MANAGE
THE REMOTE COMPUTERS )LL LOOK AT VARIOUS MODULES AND HOW YOU CAN
USE THE AD HOC MODE TO ACCOMPLISH IN A FEW KEYSTROKES WHAT WOULD
TAKE A LONG TIME TO HANDLE INDIVIDUALLY ON THE COMMAND LINE )F YOU
DIDNT GET THE RESULTS YOU EXPECTED FROM THE SAMPLE !NSIBLE COMMANDS
above, take this time to make sure authentication is working. Check
out http://docs.ansible.com FOR MORE HELP IF YOU GET STUCK Q

Send comments or feedback via

http://www.linuxjournal.com/contact
or to [email protected].

RETURN TO CONTENTS

64 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 64 7/19/17 12:54 PM

 A S T R AT E G I C C O N F E R E N C E

Developing for the This is your chance to connect

precision agriculture with thought leaders driving the

future of precision agriculture.
market?
W H Y AT T E N D ? WITH A FOCUS ON:

Water + Irrigation
Q Networking
Data
ŅĹĹåÏƋƵĜƋʱčƋåÏĘĹŅĬŅčƼĜĹāƚåĹÏåųŸ
±ÏųŅŸŸ±ƴ±ųĜåƋƼŅüŞĬ±ƋüŅųĵŸţ%åĬåč±ƋåŸ Labor
ƵĜĬĬåŸƋ±ÆĬĜŸĘŸåĹĜŅųěĬåƴåĬÏŅĹƋ±ÏƋŸ±ĹÚ Energy
üŅŸƋåų±ÚĜ±ĬŅčƚåƋʱƋåĹÚƚų埱üƋåųƋĘå Water Management
ÏŅĹüåųåĹÏåűŸåĹÚţ Sensors + Iot
Logistics
Q Conference
Robotics
‰±ĩåĘŅĵå±ÚååŞåųƚĹÚåųŸƋ±ĹÚĜĹčŅüƋĘå
ÏʱĬĬåĹčåŸü±ÏåÚÆƼčųŅƵåųŸØŸåųƴĜÏå Labels
ŞųŅƴĜÚåųŸØ±ĹÚÏŅĬĬå±čƚåŸƵŅųĩĜĹčĜĹ Traceability
±ĹÚ
ŅƋĘåųޱųƋŸŅüƋĘåűƋĜŅűĹÚƵŅųĬÚţ Food Safety
Sustainability
Con- Q Get an “In the Field” Perspective with a
Pre-Conference Tour
a±ƋÏĘĬå±ųĹĜĹčŸ±ĹÚÏŅĹƋ±ÏƋŸčĬå±ĹåÚüųŅĵ
±čě ƋĘåÏŅĹüåųåĹÏåƵĜƋʱųå±ĬěƵŅųĬÚƴĜåƵŅü
Ņű eųĜDŽŅűűŸ±čųĜÏƚĬƋƚų±ĬƋåÏĘĹŅĬŅčƼ±ĹÚƋĘå S ES S IO N TO PIC S INC LU D E:
ŽĹĜƴåųŸĜƋƼŅüeųĜDŽŅűŅÆŅƋĜÏŸ„ƼŸƋåĵţ
T H E STAT E O F T H E I N D U ST RY – P R EC I SION IN
R OW A ND S P EC I A LT Y C R OP P R OD UCTIO N

E X P LO R I N G T H E CO N N ECT I V E T I S S U E B ETWEEN
E X I ST I NG A ND E M E R G I NG T EC HNOLO G IES

140+ COMPANIES
35 LEADING SPEAKERS ®

12 COUNTRIES
185+ ATTENDEES
O c to b e r 1 0 -1 2 , 2 0 17 | P h o e n i x , A Z

#PRECISIONAGVISION
PRECISIONAGVISION.COM

LJ280-Aug2017.indd 65 7/19/17 12:54 PM

 NEW PRODUCTS
NEW PRODUCTS
NEXT
PREVIOUS
Feature: Creating
Shawn Powers’
an Internet Radio

s
s

The Open-Source
Station with Icecast
Classroom
and Liquidsoap

JMR SiloStor NVMe SSD Drives

Compute-intensive
workflows are the
environments in which
the newly developed JMR
SiloStor NVMe family of SSD
drives is designed to show
its colors. Ideal for HPC,
data centers, genome
research, content creation,
CGI/animation, codec
processing and gaming,
among others, the SiloStor drive family comes in three NVMe/PCIe
configurations: single-drive module, x4 PCIe connectivity in 512GB/1TB/2TB
capacities; dual-drive, x8 connectivity in 1TB/2TB/4TB capacities; and
quad-drive module, x8 connectivity, available in 2TB/4TB/8TB capacities.
The dual- and quad-drive cards incorporate a PCIe switch, and the drives can
be striped (on a single card) for additional performance. All SiloStor designs
incorporate active heatsink coolers on the drive modules themselves, maintaining
low operating temperatures even during intensive sequential write operations.
Key performance metrics include <1 mS average access time of <1 mS, 2 million
hours MTBF, 1,200 TBW minimum endurance, 90,000/70,000 IOPS random 4K
read/write speed and 4,000/3,000 MB/sequential read/write speed.
http://jmr.com

66 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 66 7/21/17 8:36 AM

 NEW PRODUCTS

WPS Office 2016 for Linux

0ROMISING THE WORLDS BEST OFFICE EXPERIENCE FOR THE ,INUX COMMUNITY
703 3OFTWARE PRESENTS 703 /FFICE  FOR ,INUX A HIGH PERFORMING YET
CONSIDERABLY MORE AFFORDABLE ALTERNATIVE TO -ICROSOFT /FFICE THAT IS FULLY
COMPATIBLE WITH AND COMPARABLE TO THE CONSTITUENT 0OWER0OINT %XCEL
AND 7ORD APPLICATIONS 4HE 703 /FFICE SUITE WITH MORE THAN  BILLION
INSTALLS ACROSS ALL PLATFORMS IS A COMPLETE OFFICE SUITE INCLUDING 7RITER
0RESENTATION 3PREADSHEETS AND A BUILT IN 0$& READER ,INUX 7INDOWS
!NDROID AND I/3 VERSIONS ARE AVAILABLE 703 /FFICE  FOR ,INUX OFFERS
ENHANCEMENTS FOR THE INTERNATIONAL ,INUX USER COMMUNITY INCLUDING
REMOTE FILE SHARING ADDED SEARCH FUNCTIONALITY UPDATED 703 EXPORT TO 0$&
HYPERLINKS AND IMPROVED )/ OPERATIONS FOR IMPROVED 703 FILE ACCESS SPEED
#OMPATIBILITY WITH -ICROSOFT /FFICE DOCUMENT FORMATS INCLUDES 004 $/#
$/#8 8,3 AND 8,38 4HE ,INUX EDITION OF 703 /FFICE IS COMPATIBLE WITH
&EDORA #ENT/3 /PEN353% 5BUNTU -INT +NOPPIX AND OTHER PLATFORMS
SUPPORTING BOTH  AND  BIT COMPUTING ENVIRONMENTS 4HE LATEST UPDATE
IS MADE POSSIBLE WITH THE SUPPORT OF THE 703 /FFICE ,INUX COMMUNITY
http://www.wps.com and HTTPWPS COMMUNITYORG

67 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 67 7/19/17 12:54 PM

 NEW PRODUCTS

Ocado Technology’s Kubermesh

)NSTEAD OF RELYING ON SERVERS CONCENTRATED IN ONE LARGE DATA CENTER THE
NEW +UBERMESH IS DESIGNED TO SIMPLIFY DATA CENTER ARCHITECTURES FOR SMART
FACTORIES BY ELEGANTLY AND COST EFFECTIVELY LEVERAGING A DISTRIBUTED NETWORK
OF COMPUTING NODES SPREAD ACROSS THE ENTERPRISE $EVELOPED BY /CADO
4ECHNOLOGY A DIVISION OF /CADO THE WORLDS LARGEST ONLINE ONLY SUPERMARKET 
THE +UBERMESH PACKAGE USES CONTAINER BASED TECHNOLOGY AND THE +UBERNETES
SYSTEM TO IMPLEMENT AN ON PREMISES PRIVATE CLOUD ARCHITECTURE IN WHICH
DESKTOP COMPUTERS CAN BE CONFIGURED AS NODES SUPPORTING THE COMPUTE OR
STORAGE FUNCTIONALITY TYPICALLY DELIVERED BY HIGH PERFORMANCE SERVERS IN A
DATA CENTER /CADO 4ECHNOLOGY OBSERVES THAT +UBERMESH BASED NODES ARE
FAULT TOLERANT SECURE FLEXIBLE AND DESIGNED TO PROCESS THE GENEROUS AMOUNTS
OF REAL TIME DATA GENERATED IN SMART FACTORIES "Y DISTRIBUTING DATA CENTER
FUNCTIONALITY INTO A MESH NETWORK OF NODES +UBERMESH ALLEVIATES THE NEED
FOR A DEDICATED DATA CENTER AND COMPLEX NETWORKING INFRASTRUCTURE RESULTING
IN SIGNIFICANT REDUCTIONS IN NOT JUST ENERGY CONSUMPTION BUT ALSO THE CAPITAL
AND SIGNIFICANT OPERATIONAL EXPENDITURES THAT COME WITH MAINTAINING
IN HOUSE HIGH PERFORMANCE SERVERS 7ITH +UBERMESH /CADO 4ECHNOLOGY
HOPES FOR INTERNAL GAINS THROUGH UNLOCKING THE POTENTIAL OF CONTAINER
technology and external gains as the Open Source community deploys
and develops Kubermesh in new and exciting ways.
http://www.ocadotechnology.com

68 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 68 7/19/17 12:54 PM

 NEW PRODUCTS

Nativ Vita
4HE MOTTO hOPEN TO ANYTHINGv UNDERPINS .ATIVS DEVELOPMENT PHILOSOPHY
ON ALL OF ITS AUDIO SOLUTIONS INCLUDING ITS NEW .ATIV 6ITA hTHE WORLDS
FIRST (IGH 2ESOLUTION -USIC 0LAYERv AND TOUCHSCREEN CONTROL CENTER THAT
IS DESIGNED TO FUNCTION AS THE CENTRAL ACCESS POINT FOR ONES ENTIRE MUSIC
COLLECTION 4HIS PHILOSOPHY IS EVIDENT IN .ATIV 6ITAS ,INUX AND OPEN
SOURCE INTERNALS OFFERING ADVANTAGES LIKE SUPPORT FOR VIRTUALLY ANY MUSIC
SERVICEˆEVEN LESSER KNOWN AND REGIONAL SERVICES LIKE *ANGO 2ADIO ++"OX
AND 0ARADISE 2ADIOˆAND EXTENSIBILITY FAR BEYOND PURE AUDIO APPLICATIONS
.ATURALLY .ATIV 6ITA SUPPORTS MAINSTREAM MUSIC SERVICES LIKE !PPLE -USIC
3OUND#LOUD 6EVO 3POTIFY 4)$!, 0ANDORA AND !MAZON -USIC AMONG
OTHERS .ATIV 6ITA CAN STORE UP TO 4" OF MUSIC ON ITS INTERNAL HARD DISK
DRIVES OR 33$S AND CAN ACCESS REMOTE FILES ON A 0# .!3 OR SMARTPHONE
7IRELESS STREAMING TO MULTI ROOM SPEAKER SYSTEMS IS ACHIEVED UTILIZING
POPULAR SOLUTIONS LIKE 3/./3 AND "LUESOUND AND TO HIGH END HEADPHONES
VIA "LUETOOTH APT8 ! HIGH END DIGITAL OUTPUT STAGE WITH MYRIAD OUTPUTS
RANGING FROM !%3%"5 TO 53" !UDIO #LASS  CONNECT THE 6ITA TO AN
AMPLIFIER OR 53" $!# FOR BEST IN CLASS SOUND PERFORMANCE
http://nativsound.com

69 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 69 7/19/17 12:54 PM

 NEW PRODUCTS

Sander van Vugt’s

Linux Foundat ion
Cert ified Engineer
(LFCE) Video
Course (Pearson
IT Certification)
4HE ,INUX &OUNDATION #ERTIFIED %NGINEER
,&#% CERTIFICATION IS FOR ADMINISTRATORS
SEEKING TO INCREASE THEIR BREADTH AND DEPTH OF KNOWLEDGE BEYOND THE
,INUX &OUNDATION #ERTIFIED 3YSTEM !DMINISTRATOR ,&#3 LEVEL 0ROFESSIONALS
AMONG US STRIVING FOR THIS GREATER EXPERTISE HAVE AT THEIR DISPOSAL A
NEW ,&#% ORIENTED EDUCATIONAL RESOURCE 3ANDER VAN 6UGTS ,INUX
&OUNDATION #ERTIFIED %NGINEER ,&#% 6IDEO #OURSE FEATURING TEN HOURS OF
comprehensive video instruction. Containing everything that exam candidates
REQUIRE TO PREPARE FOR AND PASS THE ,&#% EXAM THIS COMPREHENSIVE TRAINING
INCLUDES WHITEBOARD CONCEPT TEACHING TO ILLUSTRATE DIFFICULT CONCEPTS LIVE #,)
INSTRUCTION TO DEMONSTRATE ,INUX IN ACTION SCREENCAST TEACHING HANDS ON
LABS SOLUTION VIDEOS AND PRACTICE EXAM WALK THROUGHS !UTHOR VAN 6UGT WITH
HIS  YEARS OF PRACTICAL ,INUX TEACHING EXPERIENCE COVERS THE ,&#% MATERIAL
IN FIVE MODULES -ANAGING .ETWORKING -ANAGING &ILE 3ERVICES -ANAGING
7EB 3ERVICES -ANAGING -AIL 3ERVICES AND -ANAGING )NFRASTRUCTURE 3ERVICES
0UBLISHER 0EARSON )4 #ERTIFICATION ADDS THAT THE RESOURCE IS ALSO APPROPRIATE
FOR ENGINEERS OR ADMINISTRATORS WHO WANT TO DEVELOP THEIR ,INUX SKILLS OR
WRITE SOFTWARE FOR ,INUX
HTTPWWWINFORMITCOMLIVELESSONS

70 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 70 7/19/17 12:54 PM

 NEW PRODUCTS

Zed A. Shaw’s
Learn P y thon 3
the Hard Way
(Addison-Wesley
Professional)
!UTHOR :ED ! 3HAW MAKES A SIMPLE
promise in his Hard Way SERIES OF
BOOKS FROM PUBLISHER !DDISON 7ESLEY
0ROFESSIONAL h)TLL BE HARD AT FIRST "UT
SOON YOULL JUST GET ITˆAND THAT WILL
FEEL GREAT
v 3HAWS LATEST BOOK IN THE SERIES IS CALLED Learn Python 3 the
Hard Way: A Very Simple Introduction to the Terrifyingly Beautiful World of
Computers and Code. In the book, readers learn Python by working through
 hBRILLIANTLY CRAFTED EXERCISESv IN A PURPOSEFULLY PROSCRIBED MANNER !FTER
READING THE EXERCISE READERS TYPE THE CODE PRECISELYˆWITH NO COPYING AND
PASTING
4HEN READERS FIX THEIR MISTAKES AND WATCH THE PROGRAM RUN 4HE
PROCESS TEACHES ESSENTIALS OF HOW A COMPUTER WORKS WHAT GOOD PROGRAMS
look like, and how to read, write and think about computer code. Shaw
TEACHES EVEN MORE IN  HOURS OF VIDEO WHERE HE SHOWS READERS HOW TO
BREAK FIX AND DEBUG CODEˆLIVE AS HES DOING THE EXERCISES ,ESSONS COVER
TOPICS FROM INSTALLING A COMPLETE 0YTHON ENVIRONMENT TO WORKING WITH
CODE BASIC MATHEMATICS VARIABLES LOOPING AND LOGIC OBJECT ORIENTED
programming, Python packaging, automated testing and much more.
2EADERS BRING THE DISCIPLINE COMMITMENT AND PERSISTENCE TO 3HAWS
FORMULA AND THE OUTPUT WILL BE A 0YTHON PROGRAMMER

HTTPAWPROFESSIONALCOM

71 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 71 7/19/17 12:54 PM

 NEW PRODUCTS

BeBop Sensors, Inc.’s Marcel

Modular Data Gloves
)N THE FABRIC EMBEDDED SENSORS SPACE ALL CONTROLLERS NEED TO BE ACCURATE
AND FAST h)F LATENCY IS MORE THAN n MILLISECONDS YOU ARE OUT OF THE
BANDv CAUTIONS "E"OP 3ENSORS )NC MAKER OF THE NEW -ARCEL -ODULAR
$ATA 'LOVE SOLUTION FOR VIRTUAL AND AUGMENTED REALITY /%-S 5TILIZING
"E"OPS PATENTED FABRIC SENSOR TECHNOLOGY AND DESIGNED FOR ACCURATE
REAL TIME CONTROL AND NAVIGATION IN THESE ENVIRONMENTS THE "E"OP
$ATA 'LOVES ARE AVAILABLE TO /%-S IN   AND  SENSOR VERSIONS
4HEY PROVIDE HAPTICˆTHAT IS KINESTHETICˆFEEDBACK AND SENSE KNUCKLE
AND ABDUCTION MOTION OF THE HUMAN HAND "E"OPS BASIC CONFIGURATION
PROVIDES HIGH SPEED SENSOR PROCESSING AS WELL AS A  OR  DEGREES OF
FREEDOM INERTIAL MEASUREMENT UNIT THAT MEASURES ACCELERATION AND
ANGULAR RATE &AST DETERMINISTIC SENSING PROVIDES SUB FRAME LATENCY
AT (Z FOR REAL TIME CONTROL OF GAMES AND ENVIRONMENTS -EZZANINE
boards can be added to the printed circuit board assembly stack to
ADD FUNCTIONALITY SUCH AS TRANSLATION AND HAPTIC ELECTRONICS ! HAPTIC
audio creation kit is available, enabling content creators to customize
and add to the haptic library.
http://bebopsensors.com

72 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 72 7/19/17 12:54 PM

 NEW PRODUCTS

TeamViewer Linux Host

!T LAST ABANDONING 7).% AND LAUNCHING NATIVE ,INUX SUPPORT 4EAM6IEWER
ANNOUNCED THE AVAILABILITY OF A NEW PREVIEW VERSION OF ITS ,INUX (OST WITH
NATIVE ,INUX SUPPORT 4HE NEW RELEASE OF 4EAM6IEWER A SOLUTION FOR REMOTE
support, remote access and online meetings, addresses additional critical
SYSTEM ADMINISTRATOR REQUIREMENTS INCLUDING SUPPORT OF 7AKE /N ,!.
ASSIGNMENT OF 4EAM6IEWER ACCOUNTS VIA '5) AND ADDITIONAL REGULATION
CAPABILITIES 7AKE /N ,!. SUPPORT GIVERS USERS THE POWER TO WAKE UP
Linux devices that are in standby mode and connected to a power supply.
-EANWHILE ACCOUNT ASSIGNMENT VIA '5) ON THE 4EAM6IEWER CLIENT PERMITS
account owners to share their contacts with each other to support the wider
TEAM AS WELL AS MAINTAIN DEVICES AROUND THE CLOCK  &INALLY A h#ONFIRM
ALLv SETTING LETS USERS ENSURE THAT ALL ACTIONS MUST BE DIRECTLY CONFIRMED FROM
THAT DEVICE WHICH IMPROVES OVERALL SECURITY 4HE 4EAM6IEWER ,INUX (OST
REQUIRES AT LEAST 1T  ,INUX +ERNEL  AND ',)"# 
http://teamviewer.com

Please send information about

releases of Linux-related products
to [email protected]
or New Products c/o Linux Journal,
PO Box 980985, Houston, TX 77098.
Submissions are edited for length
and content.

RETURN TO CONTENTS

73 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 73 7/19/17 12:54 PM

 FEATURE

Creating an
Internet
Radio Station
with Icecast
and Liquidsoap
Ever wanted to stream prerecorded music or a live event, such
as a lecture or concert for an internet audience? With Icecast
and Liquidsoap, you can set up a full-featured, flexible internet
radio station using free software and open standards.

BILL DENGLER

NEXT
PREVIOUS Feature: Linux
V

New Products Filesystem Events

with inotify

74 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 74 7/19/17 12:54 PM

 FEATURE: Creating an Internet Radio
FEATURE
Station with Icecast and Liquidsoap

I
CECAST IS hA STREAMING MEDIA AUDIOVIDEO SERVER THAT CURRENTLY
SUPPORTS /GG 6ORBIS AND 4HEORA  /PUS 7EB- AND -0 STREAMS )T
can be used to create an internet radio station or a privately running
jukebox and many things in between. It is very versatile in that new
FORMATS CAN BE ADDED RELATIVELY EASILY AND SUPPORTS OPEN STANDARDS FOR
COMMUNICATION AND INTERACTIONv
,IQUIDSOAP IS hA POWERFUL AND FLEXIBLE LANGUAGE FOR DESCRIBING YOUR
STREAMS )T OFFERS A RICH COLLECTION OF OPERATORS THAT YOU CAN COMBINE AT
WILL GIVING YOU MORE POWER THAN YOU NEED FOR CREATING OR TRANSFORMING
STREAMS "UT ,IQUIDSOAP IS STILL VERY LIGHT AND EASY TO USE IN THE 5.)8
TRADITION OF SIMPLE STRONG COMPONENTS WORKING TOGETHERv
7HEN COMBINED )CECAST AND ,IQUIDSOAP CAN CREATE A FLEXIBLE FEATURE
RICH INTERNET RADIO STATION )N THIS ARTICLE ) DESCRIBE HOW TO CONFIGURE
)CECAST TO HOST AN INTERNET RADIO STATION 4HEN ) EXPLAIN HOW TO INSTALL AND
CONFIGURE ,IQUIDSOAP TO CONNECT TO )CECAST ADDING RANDOM OR SEQUENTIAL
MUSIC PLAYBACK WITH SMART CROSS FADING PRERECORDED RANDOMLY INSERTED
ANNOUNCEMENTS AND JINGLES A SONG REQUEST SYSTEM AND SUPPORT FOR LIVE
streams, with automated recording and seamless switching between live and
AUTOMATED PROGRAMMING ) ALSO SHOW HOW TO CONFIGURE THE SERVER TO SERVE
YOUR STREAM IN -0 /GG AND /PUS FORMATS FOR MAXIMUM PLAYER COMPATIBILITY
)CECAST 6ORBIS AND RELATED PROJECTS ARE MAINTAINED BY 8IPH/RG
https://www.xiph.org  A NONPROFIT ORGANIZATION THAT DEVELOPS OPEN
MULTIMEDIA STANDARDS AND SOFTWARE 4O ENSURE THAT YOU ARE RUNNING THE
LATEST VERSION OF )CECAST WITH ALL OR MOST FEATURES YOU SHOULD INSTALL
FROM AN OFFICIAL 8IPH/RG REPOSITORY 6ISIT THE LIST OF OFFICIAL REPOSITORIES AT
HTTPSWIKIXIPHORG)CECAST?3ERVER)NSTALLING?LATEST?VERSION?OFFICIAL?8IPH?
REPOSITORIES  AND FOLLOW THE INSTRUCTIONS ON THAT PAGE TO ADD THE )CECAST
REPOSITORY FOR YOUR DISTRIBUTION 4HEN INSTALL USING YOUR SYSTEMS PACKAGE
MANAGER /N $EBIAN BASED SYSTEMS SUCH AS 5BUNTU  YOU MAY BE ASKED
TO hCONFIGURE )CECASTv DURING PACKAGE INSTALLATION SELECT hNOv AS YOU WILL
CONFIGURE THE SERVER MANUALLY IF YOU ARE FOLLOWING ALONG WITH THIS ARTICLE
/PEN THE )CECAST CONFIGURATION FILE USING YOUR PREFERRED TEXT EDITOR /N
$EBIAN BASED SYSTEMS THE FILE IS LOCATED AT ETCICECASTICECASTXML 4HE
LOCATION ON OTHER SYSTEMS MAY DIFFER CHECK YOUR PACKAGES DOCUMENTATION
FOR THE CORRECT PATH 4HE CONFIGURATION FILE IS IN 8-, FORMAT AND IS DIVIDED

75 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 75 7/19/17 12:54 PM

 FEATURE: Creating an Internet Radio
FEATURE
Station with Icecast and Liquidsoap

INTO SEVERAL SECTIONS &IRST ENTER YOUR SERVERS LOCATION AND EMAIL INTO THE
LOCATION AND ADMIN FIELDS RESPECTIVELYˆFOR EXAMPLE

<location>The  Heart  of  Gold</location>  

<admin>[email protected]</admin>

3INCE EACH FORMAT YOULL SET UP IN ,IQUIDSOAP IS A SEPARATE )CECAST

hSOURCEv YOULL QUICKLY EXHAUST THE DEFAULT SOURCE LIMIT OF TWO 3O
change that to ten:

<sources>10</sources>

5NLESS YOU ANTICIPATE LISTENERS CONNECTING FROM SLOW OR LOW BANDWIDTH

ENVIRONMENTS DISABLING )CECASTS BURST ON CONNECT FEATURE WILL
SIGNIFICANTLY DECREASE LATENCY

<burst-­on-­connect>0</burst-­on-­connect>  
               <burst-­size>0</burst-­size>

4HE DEFAULT PASSWORDS hHACKMEv INVITE SECURITY COMPROMISE

#HANGE THEM TO SOMETHING ELSE !LSO ITS PROBABLY A GOOD IDEA TO
CHANGE THE DEFAULT ADMIN USER NAME 4HE FOLLOWING PASSWORDS ARE JUST
EXAMPLES CHANGE THEM FOR YOUR CONFIGURATION BOTH HERE AND WHEN
they are mentioned later in the article:

<source-­password>dontpanic</source-­password>  
       <relay-­password>dontpanic42</relay-­password>  
                       <admin-­user>zaphod</admin-­user>  
                       <admin-­password>2Headsarebetterthanone!</admin-­password>

%NTER YOUR SYSTEMS FULLY QUALIFIED DOMAIN NAME IN THE HOSTNAME FIELD

<hostname>example.com</hostname>

3AVE AND CLOSE THE FILE )F YOU EDITED THE FILE AS ROOT YOULL NEED TO RESET
ITS PERMISSIONS /N $EBIAN BASED SYSTEMS )CECAST RUNS UNDER USER ICECAST

76 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 76 7/19/17 12:54 PM

 FEATURE: Creating an Internet Radio
FEATURE
Station with Icecast and Liquidsoap

Many distributions provide broken

and out-of-date versions of Liquidsoap
in their repositories. For this reason
(along with improved ability to customize
your installation), the Liquidsoap
developers recommend installing it using
the OCaml Package Manager (opam).

AND GROUP ICECAST 4O FIX PERMISSIONS ON A $EBIAN BASED SYSTEM RUN

chown  icecast2:icecast  /etc/icecast2/icecast.xml

/N $EBIAN BASED SYSTEMS )CECASTS SYSTEM SERVICE IS DISABLED BY

DEFAULT /PEN THE FILE ETCDEFAULTICECAST AND SET ENABLED TO TRUE
4HEN SAVE AND CLOSE THE FILE
-OST MODERN ,INUX SYSTEMS USE SYSTEMD FOR SERVICE MANAGEMENT 4O
ENABLE )CECAST ON BOOT AND START IT FOR THIS SESSION RUN THE FOLLOWING
COMMANDS AS ROOT USING SUDO OR SIMILAR 

systemctl  enable  icecast2  

systemctl  start  icecast2

3ERVICE NAMES ON VARIOUS SYSTEMS DIFFER IF THOSE COMMANDS DONT WORK

CHECK YOUR SYSTEMS DOCUMENTATION FOR THE CORRECT SERVICE NAME
-ANY DISTRIBUTIONS PROVIDE BROKEN AND OUT OF DATE VERSIONS OF
,IQUIDSOAP IN THEIR REPOSITORIES &OR THIS REASON ALONG WITH IMPROVED
ABILITY TO CUSTOMIZE YOUR INSTALLATION  THE ,IQUIDSOAP DEVELOPERS
RECOMMEND INSTALLING IT USING THE /#AML 0ACKAGE -ANAGER OPAM  5SE
YOUR DISTROS PACKAGE MANAGER TO INSTALL OPAM )F YOUVE BEEN DOING
EVERYTHING UP TO THIS POINT LOGGED IN AS ROOT YOULL NOW NEED TO CREATE A
NON ROOT USER UNDER WHICH TO INSTALL ,IQUIDSOAP 9OU ALSO NEED TO INSTALL
SUDO AND GIVE THIS NEW USER PERMISSION TO USE IT /N $EBIAN BASED

77 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 77 7/19/17 12:54 PM

 FEATURE: Creating an Internet Radio
FEATURE
Station with Icecast and Liquidsoap

systems, the adduser and gpasswd utilities allow you to create users
AND ADD THEM TO GROUPS RESPECTIVELY /N $EBIAN BASED SYSTEMS RUN THE
FOLLOWING COMMANDS AS ROOT TO ADD A NEW USER AND GRANT IT SUDO ACCESS
FOR OTHER SYSTEMS REFER TO THE DOCUMENTATION  ,ET username represent
THE USER NAME OF THE NEW USER

adduser  username  
gpasswd  -­a  username  sudo

0ERFORMING AS YOUR NON ROOT USER INITIALIZE THE /#AML 0ACKAGE -ANAGER
by running:

opam  init

!NSWER hYESv WHEN ASKED TO MODIFY YOUR PROFILE THIS WILL PLACE
,IQUIDSOAP ON YOUR PATH AND ALLOW IT TO BE EXECUTED WHEN YOU TYPE ITS
NAME 4O APPLY OPAM CHANGES RUN

eval  `opam  config  env`

.EXT INSTALL ,IQUIDSOAPS SYSTEM DEPENDENCIES

opam  install  depext  

opam  depext  taglib  mad  lame  vorbis  cry  ssl  samplerate    
 ´magic  opus  liquidsoap

.OW INSTALL LIQUIDSOAP BY REPLACING depext with install :

opam  install  taglib  mad  lame  vorbis  cry  ssl  samplerate    

 ´magic  opus  liquidsoap

4O SET UP A STARTING POINT FOR THE STATION CONFIGURATION AND ENABLE

,IQUIDSOAP AS A SERVICE THE DEVELOPERS HAVE CREATED LIQUIDSOAP DAEMON A
SET OF SCRIPTS FOR USING ,IQUIDSOAP AS A SYSTEM SERVICE ,IQUIDSOAP DAEMON
USES SYSTEMD FOR SERVICE MANAGEMENT BY DEFAULT THEREFORE IT IS COMPATIBLE
WITH MOST MODERN ,INUX DISTRIBUTIONS 4O SET IT UP INSTALL 'IT USING YOUR

78 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 78 7/19/17 12:54 PM

 FEATURE: Creating an Internet Radio
FEATURE
Station with Icecast and Liquidsoap

SYSTEMS PACKAGE MANAGER THEN RUN THE FOLLOWING AS YOUR NON ROOT USER

git  clone  https://github.com/savonet/liquidsoap-­daemon  

cd  liquidsoap-­daemon  
./daemonize-­liquidsoap.sh

9OU MAY BE PROMPTED TO ENTER YOUR USERS PASSWORD TO AUTHENTICATE

SUDO /NCE THE DMON IS INSTALLED YOULL NOW CREATE A DIRECTORY STRUCTURE
FOR STORING MUSIC JINGLES AND ARCHIVES OF LIVE STREAMS IN YOUR NON ROOT
USERS HOME DIRECTORY 2UN THE FOLLOWING COMMAND

mkdir  -­p  ~/music/music1  ~/music/jingles  ~/archives

.OW OPEN THE FILE MAINLIQ IN THE LIQUIDSOAP DAEMON DIRECTORY !T THIS
POINT THAT FILE JUST CONTAINS

output.dummy(blank())

4HIS LINE SENDS NO AUDIO NOWHERE WHICH IS NOT VERY INTERESTING SO

DELETE THAT LINE AND ADD THE FOLLOWING BASE CONFIGURATION LINES STARTING
WITH  ARE COMMENTS SO THEY ARE IGNORED BY ,IQUIDSOAP  4HIS BASE
CONFIGURATION SETS UP ONE MUSIC PLAYLIST WITH SONGS PLAYED IN RANDOM
order, jingles inserted approximately every seven songs, smart
CROSS FADING SONG REQUESTS AND AUTOMATICALLY RECORDED LIVE STREAMS
music.mp3, music.ogg and music.opus stream stored music and jingles
IN -0 /GG 6ORBIS AND /GG /PUS FORMATS RESPECTIVELY STREAMMP
STREAMOGG AND STREAMOPUS PLAY A LIVE STREAM WHEN AVAILABLE FALLING
back to music when the live stream is down:

#Settings  
set("server.telnet",  true)  
set("server.telnet.port",  1234)  
set("harbor.bind_addr","0.0.0.0")  
#  Music  playlists  
music1  =  playlist("~/music/music1")  
#  Some  jingles  

79 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 79 7/19/17 12:54 PM

 FEATURE: Creating an Internet Radio
FEATURE
Station with Icecast and Liquidsoap

jingles  =  playlist("~/music/jingles")  
#  If  something  goes  wrong,  we'll  play  this  
security  =  single("~/music/default.ogg")  
#  Start  building  the  feed  with  music  
radio  =  random([music1])  
#  Add  the  security,  requests  and  smart  crossfade  
radio  =  fallback(track_sensitive  =  false,    
 ´[smart_crossfade(fallback([request.queue(id="request"),  
´radio])),security])  
#  Now  add  some  jingles  
radio  =  random(weights  =  [1,  7],[jingles,  radio])  #  This  plays    
#  a  jingle  once  every  approximately  seven  songs,  change  7  to    
#  another  number  to  change  this  
#  Add  a  skip  command  for  the  music  stream  
server.register(  
usage="skip",  
description="Skip  the  current  song.",  
"skip",  
fun(_)  -­>  begin  source.skip(radio)  "Done!"  end  
#Add  support  for  live  streams.  
live  =  
audio_to_stereo(input.harbor("live",port=8080,password=  
´"dontpanic1764",buffer=1.0))  #dontpanic1764  is  the    
#  password  used  to  connect  a  live  stream;;  it  can  (and  should)  be  
#  different  from  the  source-­password  in  icecast.xml.  
full  =  fallback(track_sensitive=false,  
[live,radio])  
#  Dump  archives  
file_name  =  '~/archives/%Y-­%m-­%d-­%H:%M:%S$(if  $(title),  
´"-­$(title)","").ogg'  
output.file(%vorbis,file_name,live,fallible=true)  
#  Stream  it  out  
output.icecast(%mp3.vbr,  
host  =  "localhost",  port  =  8000,  
password  =  "dontpanic",  mount  =  "music.mp3",  
name="myStation  Music  Service",  description="This  is  the  myStation    

80 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 80 7/19/17 12:54 PM

 FEATURE: Creating an Internet Radio
FEATURE
Station with Icecast and Liquidsoap

 ´music  stream.  Add  some  information  about  your  station's  automated    

 ´programming.",  
radio)  
output.icecast(%vorbis,  
host  =  "localhost",  port  =  8000,  
password  =  "dontpanic",  mount  =  "music.ogg",  
name="myStation  Music  Service",  description="This  is  the  myStation    
 ´music  stream.  Add  some  information  about  your  station's    
 ´automated  programming.",  
radio)  
output.icecast(%opus(vbr="unconstrained",bitrate=60),  
host  =  "localhost",  port  =  8000,  
password  =  "dontpanic",  mount  =  "music.opus",  
name="myStation  Music  Service",  description="This  is  the  myStation    
 ´music  stream.  Add  some  information  about  your  station's    
 ´automated  programming.",  
radio)  
output.icecast(%mp3.vbr,  
host  =  "localhost",  port  =  8000,  
password  =  "dontpanic",  mount  =  "stream.mp3",  
name="myStation  Main  Stream",  description="The  myStation  main  stream.",  
full)  
output.icecast(%vorbis,    
host="localhost",port=8000,password="dontpanic",  
mount="stream.ogg",  
name="myStation  Main  Stream",  description="The  myStation  main  stream.",  
full)  
output.icecast(%opus(vbr="unconstrained",bitrate=60),    
 ´description="The  myStation  main  stream.",  
host="localhost",port=8000,password="dontpanic",  
mount="stream.opus",  
full)

Multiple Music Playlists

9OU MAY WISH TO SET UP MULTIPLE MUSIC PLAYLISTS PERHAPS WITH DIFFERENT
TYPES OF MUSIC AND CHANGE THE FREQUENCY AT WHICH SONGS FROM EACH

81 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 81 7/19/17 12:54 PM

 FEATURE: Creating an Internet Radio
FEATURE
Station with Icecast and Liquidsoap

PLAYLIST ARE PLAYED 4O DO THIS CREATE DIRECTORIES UNDER MUSIC FOR EACH
PLAYLIST NAMED MUSIC MUSIC AND SO ON 4HEN JUST COPY THE music1
LINE IN THE MUSIC PLAYLISTS SECTION OF MAINLIQ CHANGING THE REFERENCE TO
music1 accordingly.
4O INSERT SONGS RANDOMLY FROM THE NEW PLAYLIST EVERY N SONGS IN
the stream, add a line below radio  =  random([music1]) , where n
REPRESENTS THE APPROXIMATE NUMBER OF SONGS TO PLAY BEFORE INSERTING A
SONG FROM THE NEW PLAYLIST

radio  =  random(weights  =  [1,  n],[music2,  radio])

(ERES AN EXAMPLE WITH THREE MUSIC PLAYLISTS

#  Music  playlists  
music1  =  playlist("~/music/music1")  
music2  =  playlist("~/music/music2")  
music3  =  playlist("~/music/music3")  
...  
radio  =  random([music1])  
radio  =  random(weights  =  [1,  6],[music2,  radio])  
radio  =  random(weights  =  [1,12],[music3,  radio])

File-Based Playlists
)N THE BASE CONFIGURATION ,IQUIDSOAP WILL SEARCH THE DIRECTORY
^MUSICMUSIC RECURSIVELY FOR SONGS TO PLAY (OWEVER YOU ALSO CAN GIVE
,IQUIDSOAP A NEWLINE DELIMITED TEXT FILE OF PATHS TO SONGS EITHER LOCALLY
ON YOUR SYSTEM OR ON THE WEB 4O DO THIS SIMPLY CHANGE THE PATH TO A
DIRECTORY TO A PATH TO YOUR TEXT FILE LIKE THIS

music1  =  playlist("~/music/music1.pls")

Sequential Playback
"Y DEFAULT ,IQUIDSOAP PLAYS TRACKS IN RANDOM ORDER )F YOU WANT TO PLAY TRACKS
SEQUENTIALLY ADD mode="sequential" TO YOUR PLAYLIST DEFINITION LIKE THIS

music1  =  playlist("~/music/music1",mode="sequential")

82 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 82 7/19/17 12:54 PM

 FEATURE: Creating an Internet Radio
FEATURE
Station with Icecast and Liquidsoap

)NSTEAD OF USING random FOR EXAMPLE WHEN ADDING OTHER PLAYLISTS OR

JINGLES  USE rotate :

radio  =  rotate(weights  =  [1,  7],[jingles,  radio])

3EQUENTIAL PLAYBACK IS BEST COMBINED WITH FILE BASED PLAYLISTS AS THEY BOTH
GIVE YOU TOTAL CONTROL OVER THE ORDER IN WHICH TRACKS ARE PLAYED BY ,IQUIDSOAP

Compression and Normalization

)F YOUD LIKE TO ADD A MORE hRADIO LIKEv SOUND TO YOUR AUTOMATED
PROGRAMMING ,IQUIDSOAP SUPPORTS AUTOMATIC COMPRESSION AND
NORMALIZATION 4O COMPRESS AND NORMALIZE THE TRACKS OF A PLAYLIST OR
input.harbor live stream, wrap it in an nrj() operator, like so:

music1  =  nrj(playlist("~/music/music1"))

Talking Over Automated Programming

You can add a mountpoint allowing you to talk over the automated
PROGRAMMING WHICH WILL HAVE ITS VOLUME REDUCED WHILE YOURE CONNECTED
!DD THE FOLLOWING TO YOUR CONFIGURATION ABOVE #Add  support  for  live  
streams  4HE AUTOMATED PROGRAMMING VOLUME WILL BE CHANGED TO 
OF NORMAL WHILE THE MIC IS CONNECTED CHANGE p=0.15 to adjust:

#  Talk  over  stream  using  microphone  mount.  

mic=input.harbor("mic",port=8080,password="dontpanic1764",buffer=1.0)  
radio  =  smooth_add(delay=0.8,  p=0.15,  normal=radio,  special=mic)

Finishing Up
%DIT THE CONFIGURATION AS NECESSARY THEN SAVE AND CLOSE THE FILE
2ECORD A FILE TO ^MUSICDEFAULTOGG THIS FILE WILL BE PLAYED WHEN
,IQUIDSOAP CANNOT FIND OTHER TRACKS TO PLAY 4HE FILE SHOULD TELL LISTENERS
THAT THE STREAM IS DOWN AND GIVE THEM INFORMATION FOR CONTACTING YOU
TO NOTIFY YOU OF THE PROBLEM 0OPULATE THE PLAYLISTS WITH MUSIC THEN
START ,IQUIDSOAP WITH THE FOLLOWING COMMAND

sudo  systemctl  start  liquidsoap

83 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 83 7/19/17 12:54 PM

 FEATURE: Creating an Internet Radio
FEATURE
Station with Icecast and Liquidsoap

%NABLE IT ON BOOT

sudo  systemctl  enable  liquidsoap

/NCE ,IQUIDSOAP IS STARTED VISIT HTTPEXAMPLECOM IN A WEB

BROWSER WHERE EXAMPLECOM IS THE FULLY QUALIFIED DOMAIN NAME OF YOUR
SERVER  )F YOUR SYSTEM IS CONFIGURED PROPERLY MUSICMP MUSICOGG AND
music.opus will appear, playing automated programming. Also, stream.mp3,
stream.ogg and stream.opus will play automated programming unless a
live stream is connected.
)F )CECAST APPEARS BUT NO MOUNTPOINTS ARE LISTED CHECK THE ,IQUIDSOAP
LOGS AT LIQUIDSOAP DAEMONLOGRUNLOG FOR ERRORS )F )CECAST DOESNT LOAD
restart it with systemctl  restart  icecast2 .
4O BROADCAST A LIVE STREAM THROUGH YOUR SERVER YOU WILL NEED A
COMPATIBLE SOURCE CLIENT &OR 7INDOWS ) RECOMMEND !LTACAST
http://www.altacast.com/index.php/downloads  &OR -AC USERS ) SUGGEST
Ladiocast, available in the Mac App Store. For Linux, install DarkIce
THROUGH YOUR SYSTEMS PACKAGE MANAGER /N I/3 ) RECOMMEND I#AST /N
!NDROID ) SUGGEST #OOL -IC )N ALL CASES USE THE FOLLOWING CONFIGURATION

Q (OST YOUR SERVERS FULLY QUALIFIED DOMAIN NAME

Q Port: 8080

Q -OUNT MOUNTPOINT  LIVE OR LIVE FOR A LIVE STREAM )F YOU ENABLED THE
ability to talk over automated programming, replace live with mic to
talk over the music.

Q USERNAME SOURCE SOME CLIENTS DONT PROMPT FOR A USER NAME IN WHICH
CASE SOURCE IS THE IMPLIED DEFAULT 

Q PASSWORD DONTPANIC OR THE PASSWORD YOU SPECIFIED IN THE

INPUTHARBOR CONFIGURATION 

9OU CAN STREAM IN /GG 6ORBIS OR -0 /GG /PUS MAY OR MAY NOT WORK
depending on your source client.

84 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 84 7/19/17 12:54 PM

 FEATURE: Creating an Internet Radio
FEATURE
Station with Icecast and Liquidsoap

In addition to Liquidsoap’s telnet

interface, Icecast also has a web-based
administrative interface that you can
use to view listener statistics, kill
Liquidsoap’s streams or move listeners
among mountpoints.
,IQUIDSOAP OFFERS CONTROL VIA 4#0 OVER TELNET OR SIMILAR  4HE BASE
CONFIGURATION PRESENTED IN THIS ARTICLE ENABLES A SONG REQUEST SYSTEM
AND THE ABILITY TO SKIP TRACKS ON DEMAND "Y DEFAULT THIS INTERFACE
IS AVAILABLE ONLY TO USERS ON THE LOCAL SYSTEM 4HE TELNET PROTOCOL
DOES NOT SUPPORT AUTHENTICATION )F YOU WANT TO MAKE SONG REQUEST
FUNCTIONALITY AVAILABLE TO YOUR USERS YOULL NEED TO WRITE A PROGRAM OR
SCRIPT CUSTOMIZED FOR YOUR STATION THAT INTERFACES WITH ,IQUIDSOAP
#ONNECT TO ,IQUIDSOAP VIA TELNET LIKE SO

telnet  localhost  1234

/NCE CONNECTED YOU CAN REQUEST A SONG WITH THE FOLLOWING WHERE
uri IS AN ABSOLUTE PATH TO AN AUDIO FILE ON YOUR SYSTEM OR A 52, OF AN
AUDIO FILE ON THE INTERNET

request.push  uri

4O SKIP THE CURRENTLY PLAYING SONG AND IMMEDIATELY PLAY THE NEXT
one, simply type skip .
&OR A LIST OF ALL AVAILABLE COMMANDS TYPE help , or type help
FOLLOWED BY THE NAME OF A COMMAND FOR USAGE INFORMATION ON A
particular command.
4O END YOUR SESSION TYPE quit .
)N ADDITION TO ,IQUIDSOAPS TELNET INTERFACE )CECAST ALSO HAS
A WEB BASED ADMINISTRATIVE INTERFACE THAT YOU CAN USE TO VIEW
LISTENER STATISTICS KILL ,IQUIDSOAPS STREAMS OR MOVE LISTENERS AMONG

85 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 85 7/19/17 12:54 PM

 FEATURE: Creating an Internet Radio Station with Icecast and Liquidsoap

MOUNTPOINTS !CCESS IT AT HTTPEXAMPLECOMADMIN WHERE

EXAMPLECOM IS YOUR SERVERS FULLY QUALIFIED DOMAIN NAME  5SE THE
ADMIN USER AND ADMIN PASSWORD YOU SET IN ICECASTXML
!T THIS POINT YOU NOW HAVE A FULLY FUNCTIONAL STREAMING SERVER THAT
SHOULD FIT THE NEEDS OF MOST USERS (OWEVER ,IQUIDSOAP IS EXTREMELY
FLEXIBLE ALLOWING FOR MORE EXOTIC SETUPS FOR SPECIAL USE CASES 2EFER TO THE
,IQUIDSOAP DOCUMENTATION HTTPLIQUIDSOAPFMDOC DEVREFERENCEHTML
FOR INFORMATION ON ADDITIONAL LANGUAGE FEATURES THAT MAY BE USEFUL TO YOUQ

Bill Dengler has been a Linux user and tinkerer since age nine. He was born totally blind due
to a rare genetic condition called Norrie Disease, so he relies on a screen reader to access his
computer. He is currently pursuing an International Baccalaureate diploma. Feel free to send
him questions and comments at [email protected].

Send comments or feedback via

http://www.linuxjournal.com/contact
or to [email protected].

RETURN TO CONTENTS

86 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 86 7/19/17 12:54 PM

 YEAR
WARRANTY

Broad Zero 3-Year

Selection Defects Warranty

Your Source for Supermicro Platform Technology

Talk to a Supermicro Expert! 866.352.1173

LJ280-Aug2017.indd 87 7/19/17 12:54 PM

 FEATURE

Linux
Filesystem
Events with

inotify
Triggering scripts with incron and systemd.
CHARLES FISHER

PREVIOUS
Feature: Creating
NEXT
an Internet Radio
V

Doc Searls’ EOF

Station with Icecast
and Liquidsoap

88 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 88 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

I
t is, at times, important to know when things change in the
,INUX /3 4HE USES TO WHICH SYSTEMS ARE PLACED OFTEN INCLUDE
HIGH PRIORITY DATA THAT MUST BE PROCESSED AS SOON AS IT IS SEEN 4HE
CONVENTIONAL METHOD OF FINDING AND PROCESSING NEW FILE DATA IS TO
POLL FOR IT USUALLY WITH CRON 4HIS IS INEFFICIENT AND IT CAN TAX PERFORMANCE
UNREASONABLY IF TOO MANY POLLING EVENTS ARE FORKED TOO OFTEN
,INUX HAS AN EFFICIENT METHOD FOR ALERTING USER SPACE PROCESSES TO
CHANGES IMPACTING FILES OF INTEREST 4HE INOTIFY ,INUX SYSTEM CALLS WERE
FIRST DISCUSSED HERE IN Linux Journal IN A  ARTICLE BY 2OBERT ,OVE
HTTPWWWLINUXJOURNALCOMARTICLE  WHO PRIMARILY ADDRESSED THE
BEHAVIOR OF THE NEW FEATURES FROM THE PERSPECTIVE OF #
(OWEVER THERE ALSO ARE STABLE SHELL LEVEL UTILITIES AND NEW CLASSES OF
MONITORING DMONS FOR REGISTERING FILESYSTEM WATCHES AND REPORTING EVENTS
,INUX INSTALLATIONS USING SYSTEMD ALSO CAN ACCESS BASIC INOTIFY FUNCTIONALITY
WITH PATH UNITS 4HE INOTIFY INTERFACE DOES HAVE LIMITATIONSˆIT CANT
MONITOR REMOTE NETWORK MOUNTED FILESYSTEMS THAT IS .&3  IT DOES NOT
REPORT THE USERID INVOLVED IN THE EVENT IT DOES NOT WORK WITH PROC OR OTHER
PSEUDO FILESYSTEMS AND MMAP OPERATIONS DO NOT TRIGGER IT AMONG OTHER
CONCERNS %VEN WITH THESE LIMITATIONS IT IS A TREMENDOUSLY USEFUL FEATURE
4HIS ARTICLE COMPLETES THE WORK BEGUN BY ,OVE AND GIVES EVERYONE
who can write a Bourne shell script or set a crontab the ability to react
TO FILESYSTEM CHANGES

The inotifywait Utility

7ORKING UNDER /RACLE ,INUX  OR SIMILAR VERSIONS OF 2ED (AT#ENT/3
3CIENTIFIC ,INUX  THE INOTIFY SHELL TOOLS ARE NOT INSTALLED BY DEFAULT BUT
you can load them with yum:

 #  yum  install  inotify-­tools  

Loaded  plugins:  langpacks,  ulninfo  
ol7_UEKR4                                                                            |  1.2  kB      00:00  
ol7_latest                                                                          |  1.4  kB      00:00  
Resolving  Dependencies  
-­-­>  Running  transaction  check  
-­-­-­>  Package  inotify-­tools.x86_64  0:3.14-­8.el7  will  be  installed  
-­-­>  Finished  Dependency  Resolution  

89 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 89 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

   
Dependencies  Resolved  
   
==============================================================  
Package                  Arch              Version                Repository          Size  
==============================================================  
Installing:  
inotify-­tools      x86_64          3.14-­8.el7          ol7_latest          50  k  
   
Transaction  Summary  
==============================================================  
Install    1  Package  
   
Total  download  size:  50  k  
Installed  size:  111  k  
Is  this  ok  [y/d/N]:  y  
Downloading  packages:  
inotify-­tools-­3.14-­8.el7.x86_64.rpm                              |    50  kB      00:00  
Running  transaction  check  
Running  transaction  test  
Transaction  test  succeeded  
Running  transaction  
Warning:  RPMDB  altered  outside  of  yum.  
   Installing  :  inotify-­tools-­3.14-­8.el7.x86_64                                  1/1  
   Verifying    :  inotify-­tools-­3.14-­8.el7.x86_64                                  1/1  
 
Installed:  
   inotify-­tools.x86_64  0:3.14-­8.el7                                                                                            
 
Complete!

4HE PACKAGE WILL INCLUDE TWO UTILITIES INOTIFYWAIT AND INOTIFYWATCH 

DOCUMENTATION AND A NUMBER OF LIBRARIES 4HE INOTIFYWAIT PROGRAM IS OF
primary interest.
3OME DERIVATIVES OF 2ED (AT  MAY NOT INCLUDE INOTIFY IN THEIR BASE
REPOSITORIES )F YOU FIND IT MISSING YOU CAN OBTAIN IT FROM &EDORAS %0%,

90 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 90 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

REPOSITORY HTTPSFEDORAPROJECTORGWIKI%0%,  EITHER BY DOWNLOADING THE

INOTIFY 20- FOR MANUAL INSTALLATION OR ADDING THE %0%, REPOSITORY TO YUM
!NY USER ON THE SYSTEM WHO CAN LAUNCH A SHELL MAY REGISTER WATCHESˆ
NO SPECIAL PRIVILEGES ARE REQUIRED TO USE THE INTERFACE 4HIS EXAMPLE
watches the /tmp directory:

$  inotifywait  -­m  /tmp  

Setting  up  watches.  
Watches  established.

)F ANOTHER SESSION ON THE SYSTEM PERFORMS A FEW OPERATIONS ON THE FILES

in /tmp:

$  touch  /tmp/hello  
$  cp  /etc/passwd  /tmp  
$  rm  /tmp/passwd  
$  touch  /tmp/goodbye  
$  rm  /tmp/hello  /tmp/goodbye

THOSE CHANGES ARE IMMEDIATELY VISIBLE TO THE USER RUNNING INOTIFYWAIT

/tmp/  CREATE  hello  

/tmp/  OPEN  hello  
/tmp/  ATTRIB  hello  
/tmp/  CLOSE_WRITE,CLOSE  hello  
/tmp/  CREATE  passwd  
/tmp/  OPEN  passwd  
/tmp/  MODIFY  passwd  
/tmp/  CLOSE_WRITE,CLOSE  passwd  
/tmp/  DELETE  passwd  
/tmp/  CREATE  goodbye  
/tmp/  OPEN  goodbye  
/tmp/  ATTRIB  goodbye  
/tmp/  CLOSE_WRITE,CLOSE  goodbye  
/tmp/  DELETE  hello  
/tmp/  DELETE  goodbye

91 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 91 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

! FEW RELEVANT SECTIONS OF THE MANUAL PAGE EXPLAIN WHAT IS HAPPENING

$  man  inotifywait  |  col  -­b  |  sed  -­n  '/diagnostic/,/helpful/p'  

   inotifywait  will  output  diagnostic  information  on  standard  error  and  
   event  information  on  standard  output.  The  event  output  can  be  config-­  
   ured,  but  by  default  it  consists  of  lines  of  the  following  form:  
 
   watched_filename  EVENT_NAMES  event_filename  
 
 
   watched_filename  
       is  the  name  of  the  file  on  which  the  event  occurred.  If  the  
       file  is  a  directory,  a  trailing  slash  is  output.  
 
   EVENT_NAMES  
       are  the  names  of  the  inotify  events  which  occurred,  separated  by  
       commas.  
 
   event_filename  
       is  output  only  when  the  event  occurred  on  a  directory,  and  in  
       this  case  the  name  of  the  file  within  the  directory  which  caused  
       this  event  is  output.  
 
       By  default,  any  special  characters  in  filenames  are  not  escaped  
       in  any  way.  This  can  make  the  output  of  inotifywait  difficult  
       to  parse  in  awk  scripts  or  similar.  The  -­-­csv  and  -­-­format  
       options  will  be  helpful  in  this  case.

)T ALSO IS POSSIBLE TO FILTER THE OUTPUT BY REGISTERING PARTICULAR EVENTS OF

interest with the -­e OPTION THE LIST OF WHICH IS SHOWN HERE

access create move_self

attrib delete moved_to
close_write delete_self moved_from
close_nowrite modify open
close move unmount

92 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 92 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

! COMMON APPLICATION IS TESTING FOR THE ARRIVAL OF NEW FILES 3INCE INOTIFY
MUST BE GIVEN THE NAME OF AN EXISTING FILESYSTEM OBJECT TO WATCH THE
DIRECTORY CONTAINING THE NEW FILES IS PROVIDED ! TRIGGER OF INTEREST IS ALSO
EASY TO PROVIDEˆNEW FILES SHOULD BE COMPLETE AND READY FOR PROCESSING
when the close_write TRIGGER FIRES "ELOW IS AN EXAMPLE SCRIPT TO WATCH
FOR THESE EVENTS

#!/bin/sh  
unset  IFS                                                                  #  default  of  space,  tab  and  nl  
                                                                                   #  Wait  for  filesystem  events  
inotifywait  -­m  -­e  close_write  \  
     /tmp  /var/tmp  /home/oracle/arch-­orcl/  |  
while  read  dir  op  file  
do  [[  "${dir}"  ==  '/tmp/'  &&  "${file}"  ==  *.txt  ]]  &&  
           echo  "Import  job  should  start  on  $file  ($dir  $op)."  
 
     [[  "${dir}"  ==  '/var/tmp/'  &&  "${file}"  ==  CLOSE_WEEK*.txt  ]]  &&  
           echo  Weekly  backup  is  ready.  
 
     [[  "${dir}"  ==  '/home/oracle/arch-­orcl/'  &&  "${file}"  ==  *.ARC  ]]  &&  
           su  -­  oracle  -­c  'ORACLE_SID=orcl  ~oracle/bin/log_shipper'  &  
 
     [[  "${dir}"  ==  '/tmp/'  &&  "${file}"  ==  SHUT  ]]  &&  break  
 
     ((step+=1))  
done  
 
echo  We  processed  $step  events.

4HERE ARE A FEW PROBLEMS WITH THE SCRIPT AS PRESENTEDˆOF ALL THE
AVAILABLE SHELLS ON ,INUX ONLY KSH THAT IS THE !44 +ORN SHELL WILL
REPORT THE hSTEPv VARIABLE CORRECTLY AT THE END OF THE SCRIPT !LL THE OTHER
shells will report this variable as null.
4HE REASON FOR THIS BEHAVIOR CAN BE FOUND IN A BRIEF EXPLANATION ON
THE MANUAL PAGE FOR "ASH h%ACH COMMAND IN A PIPELINE IS EXECUTED AS A
SEPARATE PROCESS IE IN A SUBSHELL v 4HE -IR"3$ CLONE OF THE +ORN SHELL

93 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 93 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

has a slightly longer explanation:

#  man  mksh  |  col  -­b  |  sed  -­n  '/The  parts/,/do  so/p'  

   The  parts  of  a  pipeline,  like  below,  are  executed  in  subshells.  Thus,  
   variable  assignments  inside  them  fail.  Use  co-­processes  instead.  
 
   foo  |  bar  |  read  baz                    #  will  not  change  $baz  
   foo  |  bar  |&  read  -­p  baz            #  will,  however,  do  so

!ND THE PDKSH DOCUMENTATION IN /RACLE ,INUX  FROM WHICH -IR"3$

MKSH EMERGED HAS SEVERAL MORE MENTIONS OF THE SUBJECT

General  features  of  at&t  ksh88  that  are  not  (yet)  in  pdksh:  
   -­  the  last  command  of  a  pipeline  is  not  run  in  the  parent  shell  
   -­  `echo  foo  |  read  bar;;  echo  $bar'  prints  foo  in  at&t  ksh,  nothing  
       in  pdksh  (ie,  the  read  is  done  in  a  separate  process  in  pdksh).  
   -­  in  pdksh,  if  the  last  command  of  a  pipeline  is  a  shell  builtin,  it    
       is  not  executed  in  the  parent  shell,  so  "echo  a  b  |  read  foo  bar"    
       does  not  set  foo  and  bar  in  the  parent  shell  (at&t  ksh  will).  
       This  may  get  fixed  in  the  future,  but  it  may  take  a  while.  
 
$  man  pdksh  |  col  -­b  |  sed  -­n  '/BTW,  the/,/aware/p'  
   BTW,  the  most  frequently  reported  bug  is  
       echo  hi  |  read  a;;  echo  $a      #  Does  not  print  hi  
   I'm  aware  of  this  and  there  is  no  need  to  report  it.

4HIS BEHAVIOR IS EASY ENOUGH TO DEMONSTRATEˆRUNNING THE SCRIPT ABOVE

WITH THE DEFAULT BASH SHELL AND PROVIDING A SEQUENCE OF EXAMPLE EVENTS

$  cp  /etc/passwd  /tmp/newdata.txt  

$  cp  /etc/group  /var/tmp/CLOSE_WEEK20170407.txt  
$  cp  /etc/passwd  /tmp/SHUT

GIVES THE FOLLOWING SCRIPT OUTPUT

#  ./inotify.sh    

94 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 94 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

Setting  up  watches.  

Watches  established.  
Import  job  should  start  on  newdata.txt  (/tmp/  CLOSE_WRITE,CLOSE).  
Weekly  backup  is  ready.  
We  processed  events.

%XAMINING THE PROCESS LIST WHILE THE SCRIPT IS RUNNING YOUgLL ALSO SEE TWO
SHELLS ONE FORKED FOR THE CONTROL STRUCTURE

$  function  pps  {  typeset  a  IFS=\|  ;;  ps  ax  |  while  read  a  

do  case  $a  in  *$1*|+([!0-­9]))  echo  $a;;;;  esac;;  done  }  
 
 
$  pps  inot  
   PID  TTY            STAT      TIME  COMMAND  
 3394  pts/1        S+          0:00  /bin/sh  ./inotify.sh  
 3395  pts/1        S+          0:00  inotifywait  -­m  -­e  close_write  /tmp  /var/tmp  
 3396  pts/1        S+          0:00  /bin/sh  ./inotify.sh

!S IT WAS MANIPULATED IN A SUBSHELL THE hSTEPv VARIABLE ABOVE

WAS NULL WHEN CONTROL FLOW REACHED THE ECHO 3WITCHING THIS FROM
#/bin/sh to #/bin/ksh93 will correct the problem, and only one shell
process will be seen:

#  ./inotify.ksh93  
Setting  up  watches.  
Watches  established.  
Import  job  should  start  on  newdata.txt  (/tmp/  CLOSE_WRITE,CLOSE).  
Weekly  backup  is  ready.  
We  processed  2  events.  
 
 
$  pps  inot  
   PID  TTY            STAT      TIME  COMMAND  
 3583  pts/1        S+          0:00  /bin/ksh93  ./inotify.sh  
 3584  pts/1        S+          0:00  inotifywait  -­m  -­e  close_write  /tmp  /var/tmp

95 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 95 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

!LTHOUGH KSH BEHAVES PROPERLY AND IN GENERAL HANDLES SCRIPTS FAR

MORE GRACEFULLY THAN ALL OF THE OTHER ,INUX SHELLS IT IS RATHER LARGE

$  ll  /bin/[bkm]+([aksh93])  /etc/alternatives/ksh  

-­rwxr-­xr-­x.  1  root  root    960456  Dec    6  11:11  /bin/bash  
lrwxrwxrwx.  1  root  root            21  Apr    3  21:01  /bin/ksh  -­>  
                                                                                             /etc/alternatives/ksh  
-­rwxr-­xr-­x.  1  root  root  1518944  Aug  31    2016  /bin/ksh93  
-­rwxr-­xr-­x.  1  root  root    296208  May    3    2014  /bin/mksh  
lrwxrwxrwx.  1  root  root            10  Apr    3  21:01  /etc/alternatives/ksh  -­>  
                                                                                                       /bin/ksh93

4HE MKSH BINARY IS THE SMALLEST OF THE "OURNE IMPLEMENTATIONS

ABOVE SOME OF THESE SHELLS MAY BE MISSING ON YOUR SYSTEM BUT YOU
CAN INSTALL THEM WITH YUM  &OR A LONG TERM MONITORING PROCESS MKSH
IS LIKELY THE BEST CHOICE FOR REDUCING BOTH PROCESSING AND MEMORY
FOOTPRINT AND IT DOES NOT LAUNCH MULTIPLE COPIES OF ITSELF WHEN IDLE
assuming that a coprocess is used. Converting the script to use a Korn
COPROCESS THAT IS FRIENDLY TO MKSH IS NOT DIFFICULT

#!/bin/mksh  
unset  IFS                                                            #  default  of  space,  tab  and  nl  
                                                                             #  Wait  for  filesystem  events  
inotifywait  -­m  -­e  close_write  \  
     /tmp/  /var/tmp/  /home/oracle/arch-­orcl/  \  
     2>/dev/null  |&                                            #  Launch  as  Korn  coprocess  
 
while  read  -­p  dir  op  file                            #  Read  from  Korn  coprocess  
do  [[  "${dir}"  ==  '/tmp/'  &&  "${file}"  ==  *.txt  ]]  &&  
           print  "Import  job  should  start  on  $file  ($dir  $op)."  
 
     [[  "${dir}"  ==  '/var/tmp/'  &&  "${file}"  ==  CLOSE_WEEK*.txt  ]]  &&  
           print  Weekly  backup  is  ready.  
 
     [[  "${dir}"  ==  '/home/oracle/arch-­orcl/'  &&  "${file}"  ==  *.ARC  ]]  &&  
           su  -­  oracle  -­c  'ORACLE_SID=orcl  ~oracle/bin/log_shipper'  &  

96 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 96 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

 
     [[  "${dir}"  ==  '/tmp/'  &&  "${file}"  ==  SHUT  ]]  &&  break  
 
     ((step+=1))  
done  
 
echo  We  processed  $step  events.

.OTE THAT 4HE +ORN AND "OLSKY REFERENCE ON THE +ORN

SHELL OUTLINES THE FOLLOWING REQUIREMENTS IN A PROGRAM
OPERATING AS A COPROCESS https://www.amazon.com/
.EW +ORN3HELL #OMMAND 0ROGRAMMING ,ANGUAGEDP 

Caution: 4HE CO PROCESS MUST

Q Send each output message to standard output.

Q (AVE A .EWLINE AT THE END OF EACH MESSAGE

Q Flush its standard output whenever it writes a message.

An fflush(NULL) IS FOUND IN THE MAIN PROCESSING LOOP OF THE

INOTIFYWAIT SOURCE AND THESE REQUIREMENTS APPEAR TO BE MET
4HE MKSH VERSION OF THE SCRIPT IS THE MOST REASONABLE COMPROMISE
FOR EFFICIENT USE AND CORRECT BEHAVIOR AND ) HAVE EXPLAINED IT AT SOME
LENGTH HERE TO SAVE READERS TROUBLE AND FRUSTRATIONˆIT IS IMPORTANT TO
AVOID CONTROL STRUCTURES EXECUTING IN SUBSHELLS IN MOST OF THE "ORNE
FAMILY (OWEVER HOPEFULLY ALL OF THESE ERSATZ SHELLS SOMEDAY FIX THIS
BASIC FLAW AND IMPLEMENT THE +ORN BEHAVIOR CORRECTLY

A Practical Application—Oracle Log Shipping

/RACLE DATABASES THAT ARE CONFIGURED FOR HOT BACKUPS PRODUCE A
STREAM OF hARCHIVED REDO LOG FILESv THAT ARE USED FOR DATABASE
RECOVERY 4HESE ARE THE MOST CRITICAL BACKUP FILES THAT ARE PRODUCED
in an Oracle database.
4HESE FILES ARE NUMBERED SEQUENTIALLY AND ARE WRITTEN TO A LOG

97 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 97 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

DIRECTORY CONFIGURED BY THE $"! !N INOTIFYWATCH CAN TRIGGER ACTIVITIES

to compress, encrypt and/or distribute the archived logs to backup
AND DISASTER RECOVERY SERVERS FOR SAFEKEEPING 9OU CAN CONFIGURE
/RACLE 2-!. TO DO MOST OF THESE FUNCTIONS BUT THE /3 TOOLS ARE MORE
CAPABLE FLEXIBLE AND SIMPLER TO USE
4HERE ARE A NUMBER OF IMPORTANT DESIGN PARAMETERS FOR A SCRIPT
handling archived logs:

Q ! hCRITICAL SECTIONv MUST BE ESTABLISHED THAT ALLOWS ONLY A SINGLE

PROCESS TO MANIPULATE THE ARCHIVED LOG FILES AT A TIME /RACLE WILL
SOMETIMES WRITE BURSTS OF LOG FILES AND INOTIFY MIGHT CAUSE THE
HANDLER SCRIPT TO BE SPAWNED REPEATEDLY IN A SHORT AMOUNT OF TIME
/NLY ONE INSTANCE OF THE HANDLER SCRIPT CAN BE ALLOWED TO RUNˆANY
OTHERS SPAWNED DURING THE HANDLERS LIFETIME MUST IMMEDIATELY
EXIT 4HIS WILL BE ACHIEVED WITH A TEXTBOOK APPLICATION OF THE FLOCK
PROGRAM FROM THE UTIL LINUX PACKAGE

Q 4HE OPTIMUM COMPRESSION AVAILABLE FOR PRODUCTION APPLICATIONS

APPEARS TO BE LZIP http://www.nongnu.org/lzip  4HE AUTHOR CLAIMS
THAT THE INTEGRITY OF HIS ARCHIVE FORMAT IS SUPERIOR TO MANY MORE
well known utilities, both in compression ability and also structural
INTEGRITY HTTPWWWNONGNUORGLZIPXZ?INADEQUATEHTML  4HE
LZIP BINARY IS NOT IN THE STANDARD REPOSITORY FOR /RACLE ,INUXˆIT IS
AVAILABLE IN %0%, AND IS EASILY COMPILED FROM SOURCE

Q .OTE THAT  :IP HTTPWWW ZIPORG USES THE SAME ,:-!

ALGORITHM AS LZIP AND IT ALSO WILL PERFORM !%3 ENCRYPTION ON THE
DATA AFTER COMPRESSION %NCRYPTION IS A DESIRABLE FEATURE AS IT WILL
EXEMPT A BUSINESS FROM BREACH DISCLOSURE LAWS http://www.ncsl.org/
RESEARCHTELECOMMUNICATIONS AND INFORMATION TECHNOLOGYSECURITY
BREACH NOTIFICATION LAWSASPX IN MOST 53 STATES IF THE BACKUPS ARE
LOST OR STOLEN AND THEY CONTAIN h0ROTECTED 0ERSONAL )NFORMATIONv
00)  SUCH AS BIRTHDAYS OR 3OCIAL 3ECURITY .UMBERS 4HE AUTHOR OF
LZIP DOES HAVE HARSH THINGS TO SAY REGARDING THE QUALITY OF  :IP
ARCHIVES USING ,:-! AND THE openssl  enc program can be
USED TO APPLY !%3 ENCRYPTION AFTER COMPRESSION TO LZIP ARCHIVES

98 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 98 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

OR ANY OTHER TYPE OF FILE AS ) DISCUSSED IN A PREVIOUS ARTICLE

HTTPWWWLINUXJOURNALCOMCONTENTFLAT FILE ENCRYPTION OPENSSL
AND GPG  )M FOREGOING FILE ENCRYPTION IN THE SCRIPT BELOW AND
USING LZIP FOR CLARITY

Q 4HE CURRENT LOG NUMBER WILL BE RECORDED IN A DOT FILE IN THE

/RACLE USERS HOME DIRECTORY )F A LOG IS SKIPPED FOR SOME REASON
A RARE OCCURRENCE FOR AN /RACLE DATABASE  LOG SHIPPING WILL STOP
! MISSING LOG REQUIRES AN IMMEDIATE AND FULL DATABASE BACKUP
EITHER COLD OR HOT ˆSUCCESSFUL RECOVERIES OF /RACLE DATABASES
cannot skip logs.

Q 4HE scp program will be used to copy the log to a remote server,
AND IT SHOULD BE CALLED REPEATEDLY UNTIL IT RETURNS SUCCESSFULLY

Q )M CALLING THE GENUINE  +ORN SHELL FOR THIS ACTIVITY AS IT IS THE
MOST CAPABLE SCRIPTING SHELL AND ) DONT WANT ANY SURPRISES

Given these design parameters, this is an implementation:

#  cat  ~oracle/archutils/process_logs  
 
#!/bin/ksh93  
 
set  -­euo  pipefail  
IFS=$'\n\t'    #  http://redsymbol.net/articles/unofficial-­bash-­strict-­mode/  
 
(  
 flock  -­n  9  ||  exit  1                    #  Critical  section-­allow  only  one  process.  
 
 ARCHDIR=~oracle/arch-­${ORACLE_SID}  
 
 APREFIX=${ORACLE_SID}_1_  
 
 ASUFFIX=.ARC  
 

99 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 99 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

 CURLOG=$(<~oracle/.curlog-­$ORACLE_SID)  
 
 File="${ARCHDIR}/${APREFIX}${CURLOG}${ASUFFIX}"  
 
 [[  !  -­f  "$File"  ]]  &&  exit  
 
 while  [[  -­f  "$File"  ]]  
 do  ((NEXTCURLOG=CURLOG+1))  
 
       NextFile="${ARCHDIR}/${APREFIX}${NEXTCURLOG}${ASUFFIX}"    
 
       [[  !  -­f  "$NextFile"  ]]  &&  sleep  60    #  Ensure  ARCH  has  finished  
 
       nice  /usr/local/bin/lzip  -­9q  "$File"  
 
       until  scp  "${File}.lz"  "yourcompany.com:~oracle/arch-­$ORACLE_SID"  
       do  sleep  5  
       done  
 
       CURLOG=$NEXTCURLOG  
 
       File="$NextFile"  
 done  
 
 echo  $CURLOG  >  ~oracle/.curlog-­$ORACLE_SID  
 
)  9>~oracle/.processing_logs-­$ORACLE_SID

4HE ABOVE SCRIPT CAN BE EXECUTED MANUALLY FOR TESTING EVEN WHILE
THE INOTIFY HANDLER IS RUNNING AS THE FLOCK PROTECTS IT
A standby server, or a DataGuard server in primitive standby
MODE CAN APPLY THE ARCHIVED LOGS AT REGULAR INTERVALS 4HE SCRIPT
BELOW FORCES A  HOUR DELAY IN LOG APPLICATION FOR THE RECOVERY
OF DROPPED OR DAMAGED OBJECTS SO INOTIFY CANNOT BE EASILY USED
IN THIS CASEˆCRON IS A MORE REASONABLE APPROACH FOR DELAYED FILE
PROCESSING AND A RUN EVERY  MINUTES WILL KEEP THE STANDBY AT THE

100 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 100 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

desired recovery point:

#  cat  ~oracle/archutils/delay-­lock.sh  
 
#!/bin/ksh93  
 
(  
 flock  -­n  9  ||  exit  1                            #  Critical  section-­only  one  process.  
 
 WINDOW=43200                                            #  12  hours  
 
 LOG_DEST=~oracle/arch-­$ORACLE_SID  
 
 OLDLOG_DEST=$LOG_DEST-­applied  
 
 function  fage  {  print  $((  $(date  +%s)  -­  $(stat  -­c  %Y  "$1")  ))  
   }  #  File  age  in  seconds  -­  Requires  GNU  extended  date  &  stat  
 
 cd  $LOG_DEST  
 
 of=$(ls  -­t  |  tail  -­1)                          #  Oldest  file  in  directory  
 
 [[  -­z  "$of"  ||  $(fage  "$of")  -­lt  $WINDOW  ]]  &&  exit  
 
 for  x  in  $(ls  -­rt)                                        #  Order  by  ascending  file  mtime  
 do  if  [[  $(fage  "$x")  -­ge  $WINDOW  ]]  
       then  y=$(basename  $x  .lz)                    #  lzip  compression  is  optional  
 
                 [[  "$y"  !=  "$x"  ]]  &&  /usr/local/bin/lzip  -­dkq  "$x"  
 
                 $ORACLE_HOME/bin/sqlplus  '/  as  sysdba'  >  /dev/null  2>&1  <<-­EOF  
    recover  standby  database;;  
    $LOG_DEST/$y  
    cancel  
    quit  
    EOF  

101 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 101 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

 
                 [[  "$y"  !=  "$x"  ]]  &&  rm  "$y"  
 
                 mv  "$x"  $OLDLOG_DEST  
       fi  
 
 done  
)  9>  ~oracle/.recovering-­$ORACLE_SID

)VE COVERED THESE SPECIFIC EXAMPLES HERE BECAUSE THEY INTRODUCE

tools to control concurrency, which is a common issue when using
INOTIFY AND THEY ADVANCE A FEW FEATURES THAT INCREASE RELIABILITY AND
MINIMIZE STORAGE REQUIREMENTS (OPEFULLY ENTHUSIASTIC READERS WILL
introduce many improvements to these approaches.

The incron System

,UKAS *ELINEK IS THE AUTHOR OF THE INCRON PACKAGE THAT ALLOWS USERS TO SPECIFY TABLES
OF INOTIFY EVENTS THAT ARE EXECUTED BY THE MASTER INCROND PROCESS $ESPITE THE
REFERENCE TO hCRONv THE PACKAGE DOES NOT SCHEDULE EVENTS AT REGULAR INTERVALSˆ
IT IS A TOOL FOR FILESYSTEM EVENTS AND THE CRON REFERENCE IS SLIGHTLY MISLEADING
4HE INCRON PACKAGE IS AVAILABLE FROM %0%, )F YOU HAVE INSTALLED THE
repository, you can load it with yum:

#  yum  install  incron  

Loaded  plugins:  langpacks,  ulninfo  
Resolving  Dependencies  
-­-­>  Running  transaction  check  
-­-­-­>  Package  incron.x86_64  0:0.5.10-­8.el7  will  be  installed  
-­-­>  Finished  Dependency  Resolution  
 
Dependencies  Resolved  
 
=================================================================  
 Package              Arch              Version                      Repository        Size  
=================================================================  
Installing:  

102 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 102 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

 incron                x86_64          0.5.10-­8.el7            epel                    92  k  

 
Transaction  Summary  
==================================================================  
Install    1  Package  
 
Total  download  size:  92  k  
Installed  size:  249  k  
Is  this  ok  [y/d/N]:  y  
Downloading  packages:  
incron-­0.5.10-­8.el7.x86_64.rpm                                            |    92  kB      00:01            
Running  transaction  check  
Running  transaction  test  
Transaction  test  succeeded  
Running  transaction  
   Installing  :  incron-­0.5.10-­8.el7.x86_64                                                    1/1    
   Verifying    :  incron-­0.5.10-­8.el7.x86_64                                                    1/1    
 
Installed:  
   incron.x86_64  0:0.5.10-­8.el7                                                                                                      
 
Complete!

On a systemd distribution with the appropriate service units, you can

START AND ENABLE INCRON AT BOOT WITH THE FOLLOWING COMMANDS

#  systemctl  start  incrond  

#  systemctl  enable  incrond  
Created  symlink  from  
     /etc/systemd/system/multi-­user.target.wants/incrond.service  
to  /usr/lib/systemd/system/incrond.service.

)N THE DEFAULT CONFIGURATION ANY USER CAN ESTABLISH INCRON SCHEDULES

4HE INCRONTAB FORMAT USES THREE FIELDS

<path>  <mask>  <command>

103 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 103 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

Below is an example entry that was set with the -­e option:

$  incrontab  -­e                #vi  session  follows  

 
$  incrontab  -­l  
/tmp/  IN_ALL_EVENTS  /home/luser/myincron.sh  $@  $%  $#

You can record a simple script and mark it with execute permission:

$  cat  myincron.sh  
#!/bin/sh  
 
echo  -­e  "path:  $1  op:  $2  \t  file:  $3"  >>  ~/op  
 
$  chmod  755  myincron.sh

4HEN IF YOU REPEAT THE ORIGINAL TMP FILE MANIPULATIONS AT THE START OF
THIS ARTICLE THE SCRIPT WILL RECORD THE FOLLOWING OUTPUT

$  cat  ~/op  
 
path:  /tmp/  op:  IN_ATTRIB              file:  hello  
path:  /tmp/  op:  IN_CREATE              file:  hello  
path:  /tmp/  op:  IN_OPEN                  file:  hello  
path:  /tmp/  op:  IN_CLOSE_WRITE    file:  hello  
path:  /tmp/  op:  IN_OPEN                  file:  passwd  
path:  /tmp/  op:  IN_CLOSE_WRITE    file:  passwd  
path:  /tmp/  op:  IN_MODIFY              file:  passwd  
path:  /tmp/  op:  IN_CREATE              file:  passwd  
path:  /tmp/  op:  IN_DELETE              file:  passwd  
path:  /tmp/  op:  IN_CREATE              file:  goodbye  
path:  /tmp/  op:  IN_ATTRIB              file:  goodbye  
path:  /tmp/  op:  IN_OPEN                  file:  goodbye  
path:  /tmp/  op:  IN_CLOSE_WRITE    file:  goodbye  
path:  /tmp/  op:  IN_DELETE              file:  hello  
path:  /tmp/  op:  IN_DELETE              file:  goodbye

104 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 104 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

7HILE THE IN_CLOSE_WRITE event on a directory object is

USUALLY OF GREATEST INTEREST MOST OF THE STANDARD INOTIFY
EVENTS ARE AVAILABLE WITHIN INCRON WHICH ALSO OFFERS SEVERAL
UNIQUE AMALGAMS

$  man  5  incrontab  |  col  -­b  |  sed  -­n  '/EVENT  SYMBOLS/,/child  process/p'  

 
EVENT  SYMBOLS  
 
These  basic  event  mask  symbols  are  defined:  
 
IN_ACCESS                    File  was  accessed  (read)  (*)  
IN_ATTRIB                    Metadata  changed  (permissions,  timestamps,  extended  
                                     attributes,  etc.)  (*)  
IN_CLOSE_WRITE          File  opened  for  writing  was  closed  (*)  
IN_CLOSE_NOWRITE      File  not  opened  for  writing  was  closed  (*)  
IN_CREATE                    File/directory  created  in  watched  directory  (*)  
IN_DELETE                    File/directory  deleted  from  watched  directory  (*)  
IN_DELETE_SELF          Watched  file/directory  was  itself  deleted  
IN_MODIFY                    File  was  modified  (*)  
IN_MOVE_SELF              Watched  file/directory  was  itself  moved  
IN_MOVED_FROM            File  moved  out  of  watched  directory  (*)  
IN_MOVED_TO                File  moved  into  watched  directory  (*)  
IN_OPEN                        File  was  opened  (*)  
 
When  monitoring  a  directory,  the  events  marked  with  an  asterisk  (*)  
above  can  occur  for  files  in  the  directory,  in  which  case  the  name  
field  in  the  returned  event  data  identifies  the  name  of  the  file  within  
the  directory.  
 
The  IN_ALL_EVENTS  symbol  is  defined  as  a  bit  mask  of  all  of  the  above  
events.  Two  additional  convenience  symbols  are  IN_MOVE,  which  is  a  com-­  
bination  of  IN_MOVED_FROM  and  IN_MOVED_TO,  and  IN_CLOSE,  which  combines  
IN_CLOSE_WRITE  and  IN_CLOSE_NOWRITE.  
 
The  following  further  symbols  can  be  specified  in  the  mask:  

105 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 105 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

 
IN_DONT_FOLLOW        Don't  dereference  pathname  if  it  is  a  symbolic  link  
IN_ONESHOT        Monitor  pathname  for  only  one  event  
IN_ONLYDIR        Only  watch  pathname  if  it  is  a  directory  
 
Additionally,  there  is  a  symbol  which  doesn't  appear  in  the  inotify  sym-­  
bol  set.  It  is  IN_NO_LOOP.  This  symbol  disables  monitoring  events  until  
the  current  one  is  completely  handled  (until  its  child  process  exits).

4HE INCRON SYSTEM LIKELY PRESENTS THE MOST COMPREHENSIVE INTERFACE

TO INOTIFY OF ALL THE TOOLS RESEARCHED AND LISTED HERE !DDITIONAL
CONFIGURATION OPTIONS CAN BE SET IN ETCINCRONCONF TO TWEAK INCRONS
BEHAVIOR FOR THOSE THAT REQUIRE A NON STANDARD CONFIGURATION

Path Units under systemd

7HEN YOUR ,INUX INSTALLATION IS RUNNING SYSTEMD AS 0)$  LIMITED
INOTIFY FUNCTIONALITY IS AVAILABLE THROUGH hPATH UNITSv AS IS DISCUSSED
in a lighthearted article by Paul Brown at OCS-Mag
HTTPWWWOCSMAGCOMMONITORING FILE ACCESS FOR DUMMIES 
4HE RELEVANT MANUAL PAGE HAS USEFUL INFORMATION ON THE SUBJECT

$  man  systemd.path  |  col  -­b  |  sed  -­n  '/Internally,/,/systems./p'  

 
Internally,  path  units  use  the  inotify(7)  API  to  monitor  file  systems.  
Due  to  that,  it  suffers  by  the  same  limitations  as  inotify,  and  for  
example  cannot  be  used  to  monitor  files  or  directories  changed  by  other  
machines  on  remote  NFS  file  systems.

Note that when a systemd path unit spawns a shell script, the
$HOME AND TILDE  ~ OPERATOR FOR THE OWNERS HOME DIRECTORY MAY
NOT BE DEFINED 5SING THE TILDE OPERATOR TO REFERENCE ANOTHER
USERS HOME DIRECTORY FOR EXAMPLE ^NOBODY DOES WORK EVEN
WHEN APPLIED TO THE SELF SAME USER RUNNING THE SCRIPT 4HE
/RACLE SCRIPT ABOVE WAS EXPLICIT AND DID NOT REFERENCE ^ WITHOUT
SPECIFYING THE TARGET USER SO )M USING IT AS AN EXAMPLE HERE
5SING INOTIFY TRIGGERS WITH SYSTEMD PATH UNITS REQUIRES TWO FILES

106 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 106 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

4HE FIRST FILE SPECIFIES THE FILESYSTEM LOCATION OF INTEREST

$  cat  /etc/systemd/system/oralog.path  
 
[Unit]  
Description=Oracle  Archivelog  Monitoring  
Documentation=http://docs.yourserver.com  
 
[Path]  
PathChanged=/home/oracle/arch-­orcl/  
 
[Install]  
WantedBy=multi-­user.target

4HE PathChanged parameter above roughly corresponds to the

close-­write EVENT USED IN MY PREVIOUS DIRECT INOTIFY CALLS 4HE FULL
COLLECTION OF INOTIFY EVENTS IS NOT CURRENTLY SUPPORTED BY SYSTEMDˆIT
is limited to PathExists , PathChanged and PathModified , which are
described in man  systemd.path .
4HE SECOND FILE IS A SERVICE UNIT DESCRIBING A PROGRAM TO BE EXECUTED )T
MUST HAVE THE SAME NAME BUT A DIFFERENT EXTENSION AS THE PATH UNIT

$  cat  /etc/systemd/system/oralog.service  
 
[Unit]  
Description=Oracle  Archivelog  Monitoring  
Documentation=http://docs.yourserver.com  
 
[Service]  
Type=oneshot  
Environment=ORACLE_SID=orcl  
ExecStart=/bin/sh  -­c  '/root/process_logs  >>  /tmp/plog.txt  2>&1'

4HE oneshot parameter above alerts systemd that the program that it
FORKS IS EXPECTED TO EXIT AND SHOULD NOT BE RESPAWNED AUTOMATICALLYˆTHE

107 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 107 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

RESTARTS ARE LIMITED TO TRIGGERS FROM THE PATH UNIT 4HE ABOVE SERVICE
CONFIGURATION WILL PROVIDE THE BEST OPTIONS FOR LOGGINGˆDIVERT THEM TO
DEVNULL IF THEY ARE NOT NEEDED
5SE systemctl  start ON THE PATH UNIT TO BEGIN MONITORINGˆA COMMON
error is using it on the service unit, which will directly run the handler only
ONCE %NABLE THE PATH UNIT IF THE MONITORING SHOULD SURVIVE A REBOOT
!LTHOUGH THIS LIMITED FUNCTIONALITY MAY BE ENOUGH FOR SOME CASUAL USES
OF INOTIFY IT IS A SHAME THAT THE FULL FUNCTIONALITY OF INOTIFYWAIT AND INCRON
are not represented here. Perhaps it will come in time.

Conclusion
!LTHOUGH THE INOTIFY TOOLS ARE POWERFUL THEY DO HAVE LIMITATIONS 4O REPEAT
THEM INOTIFY CANNOT MONITOR REMOTE .&3 FILESYSTEMS IT CANNOT REPORT THE
USERID INVOLVED IN A TRIGGERING EVENT IT DOES NOT WORK WITH PROC OR OTHER
PSEUDO FILESYSTEMS MMAP OPERATIONS DO NOT TRIGGER IT AND THE INOTIFY
QUEUE CAN OVERFLOW RESULTING IN LOST EVENTS AMONG OTHER CONCERNS
%VEN WITH THESE WEAKNESSES THE EFFICIENCY OF INOTIFY IS SUPERIOR TO MOST
OTHER APPROACHES FOR IMMEDIATE NOTIFICATIONS OF FILESYSTEM ACTIVITY )T ALSO
IS QUITE FLEXIBLE AND ALTHOUGH THE CLOSE WRITE DIRECTORY TRIGGER SHOULD
SUFFICE FOR MOST USAGE IT HAS AMPLE TOOLS FOR COVERING SPECIAL USE CASES
)N ANY EVENT IT IS PRODUCTIVE TO REPLACE POLLING ACTIVITY WITH INOTIFY
watches, and system administrators should be liberal in educating the
user community that the classic crontab is not an appropriate place to
CHECK FOR NEW FILES 2ECALCITRANT USERS SHOULD BE CONFINED TO 5LTRIX ON
A 6!8 UNTIL THEY DEVELOP SUFFICIENT APPRECIATION FOR MODERN TOOLS AND
APPROACHES WHICH SHOULD RESULT IN MORE EFFICIENT ,INUX SYSTEMS AND
happier administrators. Q

Charles Fisher has an electrical engineering degree from the University of Iowa and works as a
systems and database administrator for a Fortune 500 mining and manufacturing corporation.
He has previously published both journal articles and technical manuals on Linux for UnixWorld
and other McGraw-Hill publications.

108 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 108 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

Archiving /etc/passwd
Tracking changes to the password file involves many different types of
inotify triggering events. The vipw utility commonly will make changes
to a temporary file, then clobber the original with it. This can be seen
when the inode number changes:

#  ll  -­i  /etc/passwd  

199720973  -­rw-­r-­-­r-­-­  1  root  root  3928  Jul    7  12:24  /etc/passwd  
   
#  vipw  
[  make  changes  ]  
You  are  using  shadow  passwords  on  this  system.  
Would  you  like  to  edit  /etc/shadow  now  [y/n]?  n  
   
#  ll  -­i  /etc/passwd  
203784208  -­rw-­r-­-­r-­-­  1  root  root  3956  Jul    7  12:24  /etc/passwd

The destruction and replacement of /etc/passwd even occurs with

setuid binaries called by unprivileged users:

$  ll  -­i  /etc/passwd  

203784196  -­rw-­r-­-­r-­-­  1  root  root  3928  Jun  29  14:55  /etc/passwd  
   
$  chsh  
Changing  shell  for  fishecj.  
Password:  
New  shell  [/bin/bash]:  /bin/csh  
Shell  changed.  
   
$  ll  -­i  /etc/passwd  
199720970  -­rw-­r-­-­r-­-­  1  root  root  3927  Jul    7  12:23  /etc/passwd

For this reason, all inotify triggering events should be considered when

109 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 109 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

tracking this file. If there is concern with an inotify queue

overflow (in which events are lost), then the OPEN, ACCESS and
CLOSE_NOWRITE,CLOSE triggers likely can be immediately ignored.

All other inotify events on /etc/passwd might run the following

script to version the changes into an RCS archive and mail them
to an administrator:

#!/bin/sh  
   
#  This  script  tracks  changes  to  the  /etc/passwd  file  from  inotify.  
#  Uses  RCS  for  archiving.  Watch  for  UID  zero.  
   
[email protected]  
   
TPDIR=~/track_passwd  
   
cd  $TPDIR  
   
if  diff  -­q  /etc/passwd  $TPDIR/passwd  
then  exit                                                                                  #  they  are  the  same  
else  sleep  5                                                                            #  let  passwd  settle  
         diff  /etc/passwd  $TPDIR/passwd  2>&1  |                #  they  are  DIFFERENT  
         mail  -­s  "/etc/passwd  changes  $(hostname  -­s)"  "$PWMAILS"  
         cp  -­f  /etc/passwd  $TPDIR                                          #  copy  for  checkin  
   
#        "SCCS,  the  source  motel!  Programs  check  in  and  never  check  out!"  
#          -­-­  Ken  Thompson  
   
         rcs  -­q  -­l  passwd                                                        #  lock  the  archive  
         ci  -­q  -­m_  passwd                                                        #  check  in  new  ver  
         co  -­q  passwd                                                                #  drop  the  new  copy  
fi  >  /dev/null  2>&1

110 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 110 7/19/17 12:54 PM

 FEATURE: Linux Filesystem Events with inotify

Here is an example email from the script for the above chfn operation:

-­-­-­-­-­Original  Message-­-­-­-­-­  
From:  root  [mailto:[email protected]]    
Sent:  Thursday,  July  06,  2017  2:35  PM  
To:  Fisher,  Charles  J.  <[email protected]>;;  
Subject:  /etc/passwd  changes  myhost  
   
57c57  
<  fishecj:x:123:456:Fisher,  Charles  J.:/home/fishecj:/bin/bash  
-­-­-­  
>  fishecj:x:123:456:Fisher,  Charles  J.:/home/fishecj:/bin/csh

Further processing on the third column of /etc/passwd might detect

UID zero (a root user) or other important user classes for emergency
action. This might include a rollback of the file from RCS to /etc and/or
SMS messages to security contacts.

Send comments or feedback via

http://www.linuxjournal.com/contact
or to [email protected].

RETURN TO CONTENTS

111 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 111 7/19/17 12:54 PM

 FREE DOWNLOADS

A Field Guide to the World

of Modern Data Stores
4HERE ARE MANY TYPES OF DATABASES AND DATA ANALYSIS TOOLS TO CHOOSE FROM WHEN
building your application. Should you use a relational database? How about a
KEY VALUE STORE -AYBE A DOCUMENT DATABASE )S A GRAPH DATABASE THE RIGHT FIT
7HAT ABOUT POLYGLOT PERSISTENCE AND THE NEED FOR ADVANCED ANALYTICS

)F YOU FEEL A BIT OVERWHELMED DONT WORRY 4HIS GUIDE LAYS OUT THE VARIOUS
DATABASE OPTIONS AND ANALYTIC SOLUTIONS AVAILABLE TO MEET YOUR APPS UNIQUE NEEDS

9OULL SEE HOW DATA CAN MOVE ACROSS DATABASES AND DEVELOPMENT LANGUAGES SO YOU CAN WORK IN YOUR FAVORITE
ENVIRONMENT WITHOUT THE FRICTION AND PRODUCTIVITY LOSS OF THE PAST

Sponsor: IBM

> https://geekguide.linuxjournal.com/content/field-guide-world-modern-data-stores

Why NoSQL? Your database options in the new

non-relational world
4HE CONTINUAL INCREASE IN WEB MOBILE AND )O4 APPLICATIONS ALONGSIDE EMERGING TRENDS SHIFTING ONLINE
CONSUMER BEHAVIOR AND NEW CLASSES OF DATA IS CAUSING DEVELOPERS TO REEVALUATE HOW THEIR DATA IS
STORED AND MANAGED 4ODAYS APPLICATIONS REQUIRE A DATABASE THAT IS CAPABLE OF PROVIDING A SCALABLE
FLEXIBLE SOLUTION TO EFFICIENTLY AND SAFELY MANAGE THE MASSIVE FLOW OF DATA TO AND FROM A GLOBAL USER BASE

$EVELOPERS AND )4 ALIKE ARE FINDING IT DIFFICULT AND SOMETIMES EVEN IMPOSSIBLE TO QUICKLY INCORPORATE ALL OF THIS DATA INTO
THE RELATIONAL MODEL WHILE DYNAMICALLY SCALING TO MAINTAIN THE PERFORMANCE LEVELS USERS DEMAND 4HIS IS CAUSING MANY TO
LOOK AT .O31, DATABASES FOR THE FLEXIBILITY THEY OFFER AND IS A BIG REASON WHY THE GLOBAL .O31, MARKET IS FORECASTED TO
NEARLY DOUBLE AND REACH 53$ BILLION IN 

Sponsor: IBM

> https://geekguide.linuxjournal.com/content/why-nosql-your-database-options-new-non-relational-world

Estimating CPU Per Query With Weighted Linear Regression

9OUR DATABASE SERVER IS SUDDENLY USING A LOT OF #05 RESOURCES 1UICK WHAT CAUSED IT 4HIS IS A FAMILIAR
QUESTION FOR ENGINEERS OF ALL PERSUASIONS !ND ITgS OFTEN IMPOSSIBLE TO ANSWER

4HERE ARE GOOD REASONS WHY ITgS HARD TO FIGURE OUT WHAT CONSUMES RESOURCES LIKE #05 )/ AND MEMORY IN
A COMPLEX PIECE OF SOFTWARE SUCH AS A DATABASE 4HE FIRST PROBLEM IS THAT MOST DATABASE SERVER SOFTWARE
DOESNgT OFFER ANY WAY TO MEASURE OR INSPECT THAT TYPE OF PERFORMANCE DATA 4HE DATABASE SERVER ISNgT
OBSERVABLE 4HIS PROBLEM ARISES IN TURN FROM THE COMPLEXITY OF THE DATABASE SERVER SOFTWARE AND THE WAY
IT DOES ITS WORK WHICH ACTUALLY PRECLUDES MEASURING RESOURCE CONSUMPTION ACCURATELY

Author: Baron Schwartz

3PONSOR 6IVID#ORTEX

> https://geekguide.linuxjournal.com/content/estimating-cpu-query-weighted-linear-regression

112 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 112 7/19/17 12:54 PM

 FREE DOWNLOADS

Database Performance Monitoring Buyer’s Guide

-ORE AND MORE COMPANIES HAVE BEGUN TO RECOGNIZE DATABASE PERFORMANCE MANAGEMENT AS
A VITAL NEED $ESPITE ITS WIDESPREAD IMPORTANCE GOOD DATABASE PERFORMANCE MANAGEMENT
REQUIRES SPECIALIZED EXPERTISE WITH CUSTOM APPROACHES YET ALL TOO OFTEN ORGANIZATIONS RELY ON
ONE SIZE FITS ALL SOLUTIONS THAT THEORETICALLY CHECK THE BOX BUT IN PRACTICE DO LITTLE OR NOTHING
TO HELP THEM FIND OR PREVENT DATABASE RELATED OUTAGES AND PERFORMANCE PROBLEMS

4HIS BUYERgS GUIDE IS DESIGNED TO HELP YOU UNDERSTAND WHAT DATABASE MANAGEMENT REALLY
REQUIRES SO YOUR INVESTMENTS IN A SOLUTION PROVIDE THE GREATEST POSSIBLE ULTIMATE VALUE

3PONSOR 6IVID#ORTEX

> https://geekguide.linuxjournal.com/content/database-performance-monitoring-buyer%E2%80%99s-guide

The Essential Guide To Queueing Theory

7HETHER YOURE AN ENTREPRENEUR ENGINEER OR MANAGER LEARNING ABOUT QUEUEING THEORY IS A
GREAT WAY TO BE MORE EFFECTIVE 1UEUEING THEORY IS FUNDAMENTAL TO GETTING GOOD RETURN ON
YOUR EFFORTS 4HATS BECAUSE THE RESULTS YOUR SYSTEMS AND TEAMS PRODUCE ARE HEAVILY INFLUENCED
by how much waiting takes place, and waiting is waste. Minimizing this waste is extremely
IMPORTANT )TS ONE OF THE BIGGEST LEVERS YOU WILL FIND FOR IMPROVING THE COST AND PERFORMANCE
OF YOUR TEAMS AND SYSTEMS

Author: Baron Schwartz

3PONSOR 6IVID#ORTEX

> https://geekguide.linuxjournal.com/content/essential-guide-queueing-theory

Sampling a Stream of Events

With a Probabilistic Sketch
Stream processing is a hot topic today. As modern Big Data processing systems have evolved,
STREAM PROCESSING HAS BECOME RECOGNIZED AS A FIRST CLASS CITIZEN IN THE TOOLBOX 4HATS BECAUSE
WHEN YOU TAKE AWAY THE HOW OF "IG $ATA AND LOOK AT THE UNDERLYING GOALS AND END RESULTS
DERIVING REAL TIME INSIGHTS FROM HUGE HIGH VELOCITY HIGH VARIETY STREAMS OF DATA IS A FUNDA
MENTAL CORE USE CASE 4HIS EXPLAINS THE EXPLOSIVE POPULARITY OF SYSTEMS SUCH AS !PACHE +AFKA
!PACHE 3PARK !PACHE 3AMZA !PACHE 3TORM AND !PACHE !PEXˆTO NAME JUST A FEW

Author: Baron Schwartz

3PONSOR 6IVID#ORTEX

> https://geekguide.linuxjournal.com/content/sampling-stream-events-probabilistic-sketch

113 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 113 7/19/17 12:54 PM

 EOF

The Actually
Distributed
Web
DOC SEARLS

Doc Searls is Senior

Editor of Linux Journal.
Maybe now we can make what we should have He is also a fellow with
the Berkman Center for
made with the web in the first place. Internet and Society
at Harvard University
and the Center for
Information Technology
and Society at
PREVIOUS UC Santa Barbara.
V

Feature: Linux Filesystem Events with inotify

I
thought my mind was through getting blown until I
HEARD IN MID *UNE  THAT "RAVE https://brave.com
RAISED  MILLION IN LESS THAN  SECONDS
HTTPSTECHCRUNCHCOMBRAVE ICO  MILLION
 SECONDS BRENDAN EICH  THOUGH AN )#/ )NITIAL #OIN
/FFERING HTTPWWWINVESTOPEDIACOMTERMSIINITIAL COIN
OFFERING ICOASP  ) DID KNOW )#/S WERE HOT STUFF ) ALSO
KNEW "RAVES )#/ WAS ABOUT TO HAPPEN BECAUSE "RENDAN
%ICH HTTPSENWIKIPEDIAORGWIKI"RENDAN?%ICH  THE
COMPANY #%/ SAID SO OVER BREAKFAST TWO DAYS EARLIER 3O
MY SEAT BELT WAS FASTENED BUT THE ACCELERATION OF THE )#/
STILL LEFT MY MENTAL ASS ON THE PAVEMENT TWO COUNTIES BACK
3INCE THEN )VE HYPER FOCUSED ON CRYPTOCURRENCIES
https://en.wikipedia.org/wiki/Cryptocurrency  TOKENS

114 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 114 7/19/17 12:54 PM

 EOF

Figure 1. Crypto Currency Market Capitalizations (http://coinmarketcap.com/charts)

HTTPTOKENFACTORYIOSMART BETA  DISTRIBUTED LEDGERS

https://en.wikipedia.org/wiki/Distributed_ledger  )#/S AND THE REST OF
IT FOR TWO REASONS /NE IS THAT THERE IS A CRAZE GOING ON 3EE &IGURE 
4HE OTHER IS THAT THE INVESTMENT HERE INCLUDES A MEASURE OF FAITH THAT
WE CAN ONCE AGAIN IMAGINE FULL AGENCY FOR INDIVIDUALS AS DISTRIBUTED
peers on the internet, and that many positive personal, social, economic,
POLITICAL AND OTHER TRANSFORMATIONS WILL ARISE FROM THAT AGENCY
0HIL 7INDLEY WHO NOW CHAIRS THE 3OVRIN &OUNDATION https://sovrin.org 
TOLD ME YESTERDAY THAT THIS IS THE THIRD TECH REVOLUTION OF HIS LIFETIME
h4HE FIRST WAS THE 0# AND THE SECOND WAS THE )NTERNET 4HIS IS THE
THIRDv HE SAID )M INCLINED TO AGREE SIMPLY BECAUSE SO MANY OF US
ARE SEEING A WIDE OPEN FUTURE WHERE BEFORE THERE WAS JUST A WALL OF
silos. I lamented that wall here in Linux Journal, way back in September
 HTTPWWWLINUXJOURNALCOMCONTENTWAY RANCH 

!S ENTITIES ON THE 7EB WE HAVE DEVOLVED #LIENT SERVER HAS BECOME

CALF COW 4HE CLIENTˆTHATS YOUˆIS THE CALF AND THE 7EB SITE IS THE COW
7HAT YOU GET FROM THE COW IS MILK AND COOKIES 4HE MILK IS WHAT YOU GO
TO THE SITE FOR 4HE COOKIES ARE WHAT THE SITE GIVES TO YOU MOSTLY FOR ITS

115 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 115 7/19/17 12:54 PM

 EOF

OWN BUSINESS PURPOSES CHIEF AMONG WHICH IS TRACKING YOU LIKE AN ANIMAL
4HERE ARE PERHAPS A BILLION OR MORE SERVER COWS NOW EACH WITH ITS OWN
hBRANDv AS MARKETERS AND CATTLE OWNERS LIKE TO SAY 

4HIS IS NOT WHAT THE .ETS FOUNDERS HAD IN MIND .OR WAS IT WHAT 4IM
"ERNERS ,EE MEANT FOR HIS 7ORLD 7IDE 7EB OF HYPERTEXT DOCUMENTS TO
BECOME "UT ITS WHAT WEVE GOT AND ITS GETTING WORSE

3O ) WANT TO SHARE WHAT )M THINKING ABOUT THIS WHOLE NEW THING WHICH
HAS NO ONE LABEL  IN FAITH THAT WE MIGHT BRING A ,INUX ISH SENSIBILITY TO IT
) AM ALSO ENCOURAGED THAT THE ,INUX &OUNDATION IS ALREADY AHEAD OF
THE CURVE WITH THE (YPERLEDGER 0ROJECT https://www.hyperledger.org 
hAN OPEN SOURCE COLLABORATIVE EFFORT CREATED TO ADVANCE CROSS INDUSTRY
BLOCKCHAIN TECHNOLOGIESv 4HOSE INDUSTRIES ALREADY INCLUDE hLEADERS
IN FINANCE BANKING )NTERNET OF 4HINGS SUPPLY CHAINS MANUFACTURING
AND 4ECHNOLOGYv
4HE ASPIRATIONS FOR NEW CURRENCIES TOKENS DISTRIBUTED LEDGERS AND
PROGRAMMING ENVIRONMENTS IN THIS EMERGING MEGA SPACE ARE ALSO IN SOME
WAYS SIMILAR TO THOSE OF ,INUX EARLY ON 2EMEMBER ,INUS TALK ABOUT
hWORLD DOMINATIONv HTTPCATBORGESRWRITINGSWORLD DOMINATIONWORLD
DOMINATION HTML TWO DECADES BEFORE IT CAME TRUE )TS LIKE THAT
without the Linus.
"OTH THE INTERNET AND ,INUX WERE EASY CALLS IN THE EARLY S EVEN
IF RELATIVELY FEW PEOPLE CALLED THEM /N THE NETWORK SIDE CLOSED
hONLINE SERVICESv SUCH AS !/, AND #OMPUSERVE WERE THEIR OWN BEST
ARGUMENT FOR A NETWORK OF NETWORKS THAT SUPPORTED EVERYBODY AND
FAVORED NOBODY 3O DID THE CLOSED ISOLATED AND DOOMED NETWORKS
inside every large enterprise. On the operating system side, BSD was
ALREADY PROVING ITSELF AS AN OPEN ALTERNATIVE TO COUNTLESS WARRING AND
CLOSED 5.)8ES AND WAS BUSY FORKING INTO THREE DIFFERENT BRANCHES
HELPING OPEN THE WAY FOR ,INUX 
.OW THE ONE CLEAR THING IS THAT THE INTERNETS ORIGINAL PROMISE OF
SUPPORTING EVERYBODY AND FAVORING NOBODY IS STILL UNDER FULFILLED
MEANING THE OPPORTUNITIES ARE STILL VAST REGARDLESS OF HOW MUCH OF
LIFE ON THE NET IS LIVED INSIDE THE FEUDAL CASTLES OF WHAT IN %UROPE THEY
call GAFA: Google, Amazon, Facebook and Apple.

116 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 116 7/19/17 12:54 PM

 EOF

)N HIS BLOG POST ABOUT 0ROTOCOL ,ABS IN -AY  https://www.usv.com/

BLOGPROTOCOL LABS  "RAD "URNHAM WROTE hALL OF US AT 5NION 3QUARE
6ENTURES https://www.usv.com BELIEVE IN THE DECENTRALIZED EMERGENT
PERMISSIONLESS INNOVATION THAT WAS SO CENTRAL TO THE VITALITY OF THE EARLY
INTERNETv (E CONTINUED

4HE KEY TO MITIGATING THE MARKET POWER OF THE WEB GIANTS IS OPEN PROTOCOLS
FURTHER UP THE STACK HTTPWWWUSVCOMBLOGFAT PROTOCOLS  )F AN OPEN
PUBLIC COMMUNICATIONS NETWORK THE )NTERNET UNLOCKED THE DISTRIBUTION
bottlenecks that characterized the media industry, an open public data
LAYER IS THE KEY TO UNLEASHING ANOTHER WAVE OF INNOVATION )T IS THE MISSION
OF 0ROTOCOL ,ABS TO COORDINATE THE EFFORTS OF A LARGE AND PASSIONATE
COMMUNITY OF OPEN SOURCE CONTRIBUTORS TO CREATE THESE PROTOCOLS

It is an audacious mission. As you move higher in the stack the

COMPLEXITY OF THE PROTOCOLS IS EXPONENTIALLY GREATER ,UCKILY THEY ARE
NOT STARTING FROM SCRATCH *UAN "ENET THE FOUNDER OF 0ROTOCOL ,ABS
IS THE CREATOR OF )0&3 THE )NTERPLANETARY &ILE 3YSTEM HTTPSIPFSIO 
an increasingly popular protocol that allows content on the web to be
ADDRESSED DIRECTLY INSTEAD OF BY REFERENCE TO A FILE LOCATED ON A SPECIFIC
SERVER 4HIS SUBTLE BUT PROFOUND CHANGE MEANS THAT A PROVABLY UNIQUE
PIECE OF CONTENT IS NO LONGER TIED TO A SPECIFIC SERVER BUT CAN EXIST
anywhere there is a little surplus storage capacity on the web. Protocol
Labs and everyone else working on open protocols today has another
ADVANTAGE THAT WAS NOT AVAILABLE TO THE CREATORS OF THE ORIGINAL )NTERNET
PROTOCOLS 4HEY HAVE BLOCKCHAINS

Blockchain based crypto tokens have been described as the native

BUSINESS MODEL OF OPEN SOURCE SOFTWARE http://continuations.com/
POSTCRYPTO TOKENS AND THE COMING AGE OF PROTOCOL  4HEY
HAVE THE PROMISE OF BEING ABLE TO FUND THE CRITICAL SHARED INFRASTRUCTURE
OF THE INFORMATION ECONOMY IN A WAY THAT EQUITY CAN NOT 0ROTOCOLS ARE
MORE VALUABLE WHEN THEY ARE OPEN AND SHARED BROADLY "UT EQUITY IS MOST
VALUABLE IF A COMPANY CAN EXTRACT MONOPOLY PROFITS FROM A RESOURCE THEY
EXCLUSIVELY CONTROL 7HEN A PROTOCOL INCORPORATES AN INCENTIVE IN THE FORM
OF A CRYPTO TOKEN IT CAN RESOLVE THIS INHERENT CONTRADICTION

117 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 117 7/19/17 12:54 PM

 EOF

4O START IMAGINING WHERE THIS GOES IT MAY HELP TO REVISIT

-ARSHALL -C,UHANS FRAMEWORK FOR UNDERSTANDING THE EFFECTS OF
a new technology in the world, which I wrote about in my May
 %/& HTTPWWWLINUXJOURNALCOMCONTENTWILL ANYTHING MAKE
LINUX OBSOLETE  %VERY NEW MEDIUM READ TECHNOLOGY HAS FOUR
SETS OF EFFECTS HE SAID WHICH CAN BE BEST DISCOVERED IN ANSWERS
TO FOUR QUESTIONS

Q 7HAT DOES THE MEDIUM ENHANCE

Q 7HAT DOES THE MEDIUM MAKE OBSOLETE

Q 7HAT DOES THE MEDIUM RETRIEVE THAT HAD BEEN OBSOLESCED EARLIER

Q 7HAT DOES THE MEDIUM REVERSE OR FLIP INTO WHEN PUSHED TO EXTREMES

(E ALSO PROVIDED A GRAPHICAL FRAME FOR THE ANSWERS &IGURE  

Figure 2. Marshall
McLuhan’s Framework

118 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 118 7/19/17 12:54 PM

 EOF

3O LETS DROP cryptocurrencies IN THE MIDDLE OF THAT -Y FIRST Hmm SAYS THEYˆ

Q %NHANCE EXCHANGE

Q Retrieve the bazaar.

Q /BSOLESCE FIAT CURRENCY

Q 2EVERSE INTO A "ABEL OF MUTUALLY UNINTELLIGIBLE CURRENCIES

7HILE distributed ledgersˆ

Q %NHANCE PEER TO PEER

Q Retrieve individual agency.

Q /BSOLESCE PLATFORM DOMINANCE

Q 2EVERSE INTO ONE TO ONE

2EMEMBER THAT THIS IS A HEURISTIC EXERCISE POSED AS QUESTIONS TO

ENCOURAGE MANY DIFFERENT ANSWERS 7E NEED TO ASK THESE KINDS OF
QUESTIONS AND KEEP REVISING OUR ANSWERS UNTIL THE WHOLE THING GETS REAL
I am sure the one real thing we already know is that protocols are
CAUSES AND PLATFORMS ARE EFFECTS 3INCE PLATFORMS TEND TO BE THE MOST
OBVIOUS EFFECTS THEY ALSO DISTRACT US FROM THE BOUNDLESS OTHER THINGS
protocols cause.
,ETS LOOK AT )0&3 FOR EXAMPLE 4HE 7HY PAGE HTTPSIPFSIOWHY AT THE
)0&3 SITE EXPLAINS HOW IT DOES FOR THE WEB WHAT (440 CANNOT OR HAS NOT 

Q h(440 IS INEFFICIENT AND INEXPENSIVEWITH VIDEO DELIVERY

A 00 APPROACH COULD SAVE  IN BANDWIDTH COSTSv
HTTPMATHOREGONSTATEEDU^KOVCHEGYWEBPAPERSPP VDNPDF 

Q h(UMANITYS HISTORY IS DELETED DAILY)0&3 PROVIDES HISTORIC VERSIONING LIKE

GIT AND MAKES IT SIMPLE TO SET UP RESILIENT NETWORKS FOR MIRRORING OF DATAv

119 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 119 7/19/17 12:54 PM

 EOF

Q h4HE WEBS CENTRALIZATION LIMITS OPPORTUNITY)0&3 REMAINS TRUE TO THE

ORIGINAL VISION OF THE OPEN AND FLAT WEB BUT DELIVERS THE TECHNOLOGY
WHICH MAKES THAT VISION A REALITYv

Q h/UR APPS ARE ADDICTED TO THE BACKBONE)0&3 POWERS THE CREATION OF

diversely resilient networks which enable persistent availability with or
WITHOUT )NTERNET BACKBONE CONNECTIVITYv

!ND HERES THE LIST FOR (OW

%ACH FILE AND ALL OF THE BLOCKS WITHIN IT ARE GIVEN A UNIQUE FINGERPRINT CALLED
a cryptographic hash. IPFS removes duplications across the network and
TRACKS VERSION HISTORY FOR EVERY FILE %ACH NETWORK NODE STORES ONLY CONTENT IT
IS INTERESTED IN AND SOME INDEXING INFORMATION THAT HELPS FIGURE OUT WHO IS
STORING WHAT 7HEN LOOKING UP FILES YOURE ASKING THE NETWORK TO FIND NODES
STORING THE CONTENT BEHIND A UNIQUE HASH %VERY FILE CAN BE FOUND BY HUMAN
READABLE NAMES USING A DECENTRALIZED NAMING SYSTEM CALLED )0.3 !ND OF
COURSE THERES A WHITE PAPER BY *UAN"ENET HTTPSGITHUBCOMIPFSPAPERS
RAWMASTERIPFS CAPPFSIPFS PP FILE SYSTEMPDF 
)TS TIME TO RE DISTRIBUTE THE 7EB
and countless other things built in the Send comments or feedback via
STACK ABOVE THE INTERNETS ORIGINAL AND http://www.linuxjournal.com/contact
still transcendent protocols. IPFS is one or to [email protected].
WAY 4HERE ARE MANY OTHERS
,ETS MAKE THEM HAPPENQ
RETURN TO CONTENTS

ADVERTISER INDEX
Thank you as always for supporting our advertisers by buying their products!
ADVERTISER URL PAGE #
ATTENTION ADVERTISERS
!LL 4HINGS /PEN HTTPWWW!LL4HINGS/PENORG 
The Linux Journal brand’s following has grown
$RUPALIZEME HTTPDRUPALIZEME 
(0# 7ALLSTREET HTTPWWWFLAGGMGMTCOMHPC  to a monthly readership nearly one million strong.
)NTER$RONE HTTPWWW)NTER$RONECOM 
Encompassing the magazine, Web site, newsletters
0EER  (OSTING HTTPGOPEERCOMLINUX 
3ILICON -ECHANICS HTTPWWWSILICONMECHANICSCOM  and much more, Linux Journal offers the ideal con-
353% HTTPSUSECOMSTORAGE  tent environment to help you reach your marketing
6ISION HTTPPRECISIONAGVISIONCOM 
7I34%- HTTPWWWWOMENINSTEMCONFERENCECOM  objectives. For more information, please visit
7OMEN )N ,INUX 3UMMIT HTTPWOMENINLINUXCOM  http://www.linuxjournal.com/advertising.

120 | August 2017 | http://www.linuxjournal.com

LJ280-Aug2017.indd 120 7/19/17 12:54 PM

 Instant Access to Premium
Online Drupal Training
Instant access to hundreds of hours of Drupal
training with new videos added every week!

Learn from industry experts with real world

H[SHULHQFHEXLOGLQJKLJKSURȴOHVLWHV

Learn on the go wherever you are with apps

for iOS, Android & Roku

We also offer group accounts. Give your

whole team access at a discounted rate!

Learn about our latest video releases and

RIIHUVȴUVWE\IROORZLQJXVRQ)DFHERRNDQG
7ZLWWHU#GUXSDOL]HPH

Go to http://drupalize.me and
get Drupalized today!

LJ280-Aug2017.indd 121 7/19/17 12:54 PM

 Where every interaction matters.

break down
your innovation barriers
power your business to its full potential
When you’re presented with new opportunities, you want to focus on turning
them into successes, not whether your IT solution can support them.

Peer 1 Hosting powers your business with our wholly owned FastFiber NetworkTM,

solutions that are secure, scalable, and customized for your business.

Unsurpassed performance and reliability help build your business foundation to

be rock-solid, ready for high growth, and deliver the fast user experience your
customers expect.

Want more on cloud?

Call: 844.855.6655 | go.peer1.com/linux | Vew Cloud Webinar:

Public and Private Cloud | Managed Hosting | Dedicated Hosting | Colocation

LJ280-Aug2017.indd 122 7/19/17 12:54 PM