SIG-NOC Meeting - Stuttgart

04/08/2015 Icinga - Open Source Monitoring

WWW.ICINGA.ORG
Me
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Michael Friedrich
• @dnsmichi, 31, Austrian
• Application Developer @NETWAYS
• Icinga responsibilities
• Core 1.x & 2.x Developer
• Release Manager
• On the team since 2009

#icinga
Agenda
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Icinga Project
• Icinga 1.x
• Icinga 2
• Icinga Web 2
• Demo
• Conclusion & Vision

#icinga
THE ICINGA PROJECT
The Icinga Project – Introduction
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

Open Source Enterprise Monitoring

Icinga is a scalable and extensible monitoring system

which checks the availability of your resources, notifies
users of outages and provides extensive BI data.

#icinga
The Icinga Project – About us
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Originally forked from Nagios in April 2009

• Independent, redesigned version Icinga 2 since 2014
• Different teams with autonomic responsibilities

#icinga
The Icinga Project - Icinga Exchange
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

#icinga
The Icinga Project - Icinga Exchange - GitHub
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

#icinga
The Icinga Project – Tools & Platform
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

Icinga Quality, Testing and Community Support

Website and Open Source Ticketing System

Icinga 2 Icinga Core 1.x Icinga Doc Icinga Reports

C++-based source with C-based source based on based on Jasper

multiple components MySQL, PostgreSQL, Oracle Markdown Reports

IDO Livestatus Cluster … IDOUTILS

Icinga Web 2
Based on PHP / responsive design
3rd Party Tools
Icinga Web 1.x
based on PHP using ExtJS, Agavi MVC

#icinga
ICINGA 1
Icinga 1 - Overview
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Original fork from Nagios 3.2.0

• Improved Core, intergrated IDOUtils DB backend
• Enhanced Classic UI & new Icinga Web 1.x
• Stable version: 1.13.1 (5.4.2015)
• 38 releases until today
• We’ll keep it in maintenance!

#icinga
Icinga 1 – Changes 2014/2015
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Core
– Usability, security & performance fixes
– Recurring downtimes script
• Classic UI
– Enhanced filters for modified by attributes
– Support for compressed logfiles
• Icinga Web 1
– Rewritten session handling
– Custom variables in grids

#icinga
ICINGA 2 - INTRODUCTION
Icinga 2 – Use cases
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

Monitors

Host Switches Services

OS Storage
Host Hardware

Apps Databases Network

#icinga
Icinga 2 – Use cases
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

Alerts
Mail

SMS

Voice

Ticket

#icinga
Icinga 2 – Use cases
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

Reports

#icinga
Icinga 2 - Overview
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Stable Version: 2.3.3 (26.3.2015)

• Next major version: 2.4.0 (Q4 2015)
• Supports all major distributions
• Puppet, Chef and Ansible support
• Packages and Vagrant Box available now!

https://dev.icinga.org/projects/i2/roadmap

#icinga
Icinga 2 - Overview
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Powerful CLI
• MySQL & PostgreSQL support
• New configuration format
• Includes a extensive template library
• Compatible with plugins, addons, UIs

#icinga
Icinga 2 – Vagrant
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

#icinga
Icinga 2 - Architecture
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

Perfdata Graphite Livestatus

Notify Compat

Checker Gelf IDO

#icinga
Icinga 2 - Enable Livestatus
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

Livestatus

#icinga
ICINGA 2 - CLUSTER
Icinga 2 - Cluster Overview
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

Cluster

CA Zones API

#icinga
Icinga 2 - Cluster Architecture
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• API communication using SSL

• Bidirectional connections supported
• Zone support for logic splits in the configuration
• Hash-based load distribution
• “binlog” like retention for monitoring events and messages
• Distributed components throughout the cluster
• Automatic redistribution of checks
• Integrated health check

#osmc
Icinga 2 - Cluster Distribution
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

IDO Graphite

Perfdata GELF

Open
TSDB
Checker

Statusdat

Livestatus Checker

#icinga
Icinga 2 - Cluster Zones
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

IDO Graphite

Perfdata IDO

Checker

Checker

Host
Statusdat

Host Host

Livestatus Checker

#icinga
Icinga 2 – Resource optimization
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

30.000 Services
3000 800
700
Cycles in millions

2500
600

Megabyte
2000
500
1500 400
300
1000
200
500
100
0 0
CPU-Cycles Memory

#icinga
ICINGA 2 - CONFIGURATION
Icinga 2 - Configuration
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• New configuration format

• Won’t miss the old config syntax!
• It is really time for change
• You will love it!

#icinga
Icinga 2 – Configuration: Syntax Highlighting
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Use syntax highlighting!

# cd /usr/share/doc/icinga2-common*/syntax

# PREFIX=~/.vim
# mkdir –p $PREFIX/{syntax,ftdetect}
# cp vim/syntax/icinga2.vim $PREFIX/syntax/
# cp vim ftdetect/icinga2.vim $PREFIX/ftdetect/

# vim /etc/icinga2/icinga2.conf

#icinga
Icinga 2 – Configuration: Console
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Use ‘icinga2 console’ to test values, functions, etc!

• https://www.icinga.org/resources/live-console/
• Install rlwrap 
# yum install rlwrap

# rlwrap icinga2 console

#icinga
Icinga 2 – Configuration: Value Types
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Get to know all the value types!

#icinga
Icinga 2 – Configuration: Constants
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• PluginDir, MysqlPassword, SnmpCommunity, …

#icinga
Icinga 2 – Configuration: Templates
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Templates for generic attributes

#icinga
Icinga 2 – Configuration: Custom Attributes
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Numbers, Strings, Booleans, Arrays, Dictionaries

• Filters, Match Conditions, additional information

#icinga
Icinga 2 – Configuration: Commands
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Optional/conditional arguments, functions, default values

#icinga
Icinga 2 – Hosts and Services
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Checkable objects and attributes

#icinga
Icinga 2 – Configuration: Apply Services
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Match on (custom) attributes, names (assign)

• Exclude objects (ignore)

#icinga
Icinga 2 – Configuration: Apply For
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• For loops over arrays/dictionaries

#icinga
Icinga 2 – Configuration: Notifications
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Dynamic notification rules

#icinga
Icinga 2 – Configuration: Dependencies
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Host-on-Service, Service-on-Host, Classic

Icinga 2

HOST
router
Reachable?
SERVICE
router-uplink

HOST
website-server

#icinga
Icinga 2 – Configuration: Conditionals
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• If-then-else

#icinga
Icinga 2 – Configuration: Loops
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• While, for and break/continue

#icinga
Icinga 2 – Configuration: Type methods & Object Accessors
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Array#sort, Dictionary#contains, String#find, …

• http://docs.icinga.org/icinga2/latest/doc/module/icinga2/chapter/language-reference#language-reference
• http://docs.icinga.org/icinga2/latest/doc/module/icinga2/chapter/library-reference#library-reference

• Object accessors
– get_host(), get_service(), etc
– Access config and runtime attributes
– e.g. get_host(NodeName).state

#icinga
Icinga 2 – Configuration: Functions
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Use pre-defined or your own

– regex(), match(), …
• Custom attributes, command, argument values

#icinga
Icinga 2 – Configuration – Another cool thing
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

#icinga
Icinga 2 – Configuration – Time dependent thresholds
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Time dependent thresholds

#icinga
ICINGA 2 - MIGRATION
Icinga 2 - Migration
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Start with a fresh server

• Inventory of plugins & configuration
• Define your configuration strategy with Icinga 2
• Create the best Command and Apply definitions only once
• Only add hosts (and users) in the future
• More migration hints on the docs!

http://docs.icinga.org/icinga2/latest/doc/module/icinga2/chapter/migration#migration

#icinga
Icinga 2 – Migration: Assign service to hosts in Icinga 1.x
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

define service{
host_name linux1,linux2,linux3,...,linux9
service_description ssh-check
other service directives ...
}

#icinga
Icinga 2 – Migration: Apply service to hosts in Icinga 2
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

apply Service "ssh" {

import "generic-service”
check_command = "ssh”
assign where host.address && host.vars.os == "Linux”
}

#icinga
Icinga 2 – Migration – Assign members to Hostgroup in Icinga 1.x
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

define hostgroup{
hostgroup_name linux-servers
alias Linux Servers
members linux1,linux2,linux3
}

#osmc
Icinga 2 – Migration – Assign members to Hostgroup in Icinga 1.x
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

object Host “mysql-server1" {

address = "10.0.0.1“
check_command = "hostalive“
}

object HostGroup "mysql-server" {

display_name = "MySQL Server“
assign where match("*mysql*", host.name)
}

#osmc
ICINGA WEB 2
Icinga Web 2 - Motivation: Classic UI
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Not extendable
• Lacks real user management
– Authorization based on contacts - not enough for enterprise
needs
• Not usable on mobile devices
• Same machine as core
• Heavy I/O

#icinga
Icinga Web 2 – Motivation: Icinga Web 1.x
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Complicated XML configuration

• Cached configuration
• Framework overhead
• Not useable on mobile devices
• Slow in big environments
• Complicated API

#icinga
Icinga Web 2: Key features
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Lightweight like Classic UI & flexible as Icinga Web 1.x

• Easy to extend and embed into other projects
• Support for multiple authentication providers
– Internal DB
– LDAP
– HTTP Auth
• Support for IDO database & Livestatus
• Supports multiple backends simultaneously, as failover backend
• Responsive actions (tablet, mobile, etc)
• FAST!

#icinga
Icinga Web 2 - Overview
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Beta Version: 2.0.0-beta3 (13.3.2015)

• Release Candidate: 2.0.0-rc1 (late May 2015)
• Final version: 2.0.0 (scheduled for Q2 2015)
• Supports all major distributions
• Apache & Nginx support
• Packages and Vagrant Box available now!

https://dev.icinga.org/projects/icingaweb2/roadmap

#icinga
Icinga Web 2 - Foundation
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

Framework / Icinga PHP2Library

Web

Backends Authentication Setup Wizard

#icinga
Icinga Web 2 - Modules
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

Business
Graphite PNP
Process

Monitoring Docs

Web 2

#icinga
Icinga Web 2 - Dashboard
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

#icinga
Icinga Web 2 – Detail View
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

#icinga
Icinga Web 2 – Event Grid
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

#icinga
Icinga Web 2 – Service Grid
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

#icinga
Icinga Web 2: CLI
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Uses Icinga Web 2 framework & backends

#icinga
Icinga Web 2: Responsiveness
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Your dashboard
• Workstation
• Tablet
• Mobile
• …

#icinga
Icinga Web 2 – Modules: PNP
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

#icinga
Icinga Web 2 – Modules: Business Processes
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

#icinga
Icinga Web 2 – Modules: Ticketsystems
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

#icinga
Icinga Web 2 – Modules: More, more, more
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Currently in development
– PNP (Graphing)
– Business Process
– Generic TTS (Ticket System Intergration)
• Coming
– Graphite (Graphing)
– Logstash as History Detail
– …

#icinga
Icinga Web 2 – Modules: Prototypes
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Ask Mr. Gelf 

#icinga
Icinga 2 - Demo
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

DEMO

#icinga
Icinga 2 – Demo
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

Get VirtualBox >= 4.2.16 && Vagrant >= 1.6.5

$ git clone --recursive https://github.com/Icinga/icinga-vagrant.git

$ cd icinga-vagrant
$ cd icinga2x-cluster

$ vagrant up

$ vagrant ssh icinga2a

$ vagrant ssh icinga2b

$> sudo –i

http://192.168.33.10 & http://192.168.33.20

#icinga
THE COMMUNITY
The Community - Users
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

You?
#icinga
The Community - Icinga Camps 2015
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

Kuala
Antwerp
Lumpur
2015
2015

April
June13th
9th

Portland 2015

October 10th

#icinga
CONCLUSION
Conclusion
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• Download Icinga 2
– Use packages
– Play with Vagrant and VirtualBox
• Rethink you configuration strategy
– Follow extensive migration docs, it is time for a change!
• Install Icinga Web 2 and play with it
• Give us feedback
– dev.icinga.org
– IRC
– Twitter

#icinga
VISION
Vision
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

• No enterprise edition
• Enterprise integration

#icinga
Questions & Answers
EXCHANGE. DEV. WIKI. BLOG. DOC. | WWW.ICINGA.ORG

Web www.icinga.org

Git git.icinga.org

Development dev.icinga.org

Wiki wiki.icinga.org Get heard

Support support.icinga.org
Get involved
Twitter twitter.com/icinga

Facebook facebook.com/icinga Get support

#icinga