ELK
ELASTIC SEARCH,
LOGSTASH,
KIBANA
& BEATS
What is Elastic stack?
What is Elastic stack?
Installing ElasticSearch
● Lunch the ubuntu 16 Server with 4gb ram,EIP.
● #apt-get update
● #apt-get install openjdk-8-jre-headless
# wget
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.3.deb
#dpkg -i elasticsearch-5.6.3.deb
● Change the vi /etc/elasticsearch/elasticsearch.yml
cluster.name : globo-clustering
Node.name : public DNS of Elasticsreach (in production only
private DNS)
Network.host : private ip of elasticsreach / public ip
● Increase the memory map count by ‚
# sysctl -w vm.max_map_count=262144
● Restart services
# service elasticsearch start
● Test by executing
http://<ipadress>:9200
SG allow All traffic
By default elasticsearch runs on port 9200
Install Logstash:
● Lunch instant of Ubuntu 16.04 4gb ram
● #apt-get update
● Install java #apt-get install openjdk-8-jre-headless
● #wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo
apt-key add -
● #echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" |
sudo tee -a /etc/apt/sources.list.d/elasticsearch-5.x.list
● Execute #apt-get update && apt-get install logstash
● Logstash is stored in
● # cd /usr/share/logstash and move to this directory using cd
● Now execute this command #bin/logstash -e "input { stdin {} } output {
stdout {} }" enter some value.
Logtash:
# service logstash status
#cd /etc/logstash/conf.d
#vi beats.conf
input {
beats {
port => "5044"
}
}
output {
elasticsearch {
hosts => [ "54.255.170.251:9200" ]
# service logstash start
#cd /usr/share/logstash
#bin/logstash -f /etc/logstash/conf.d/
Kibana
● Lunch instance of Ubuntu 16.04 2gb ram
● #apt-get update
#wget -qO - https://packages.elastic.co/GPG-KEY-elasticsearch | sudo apt-key
add -
● # echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo
tee -a /etc/apt/sources.list.d/elasticsearch-5.x.list
● #apt-get update
● #apt-get install kibana
#vi /etc/kibana/kibana.yml
server.host: private <ipaddress> of kibina
server.name: private <hostname> of kibina
elasticsearch.url: <elasticsearchurl> of electric search of public ip
#service kibana start
http://kinbapublicip:5601
Beats
Install
winlogbeat on
windows
● https://artifacts.elastic.co/downloads/beats/winlogbeat/winlogbeat-5.6.3-wi
ndows-x86_64.zip
● https://artifacts.elastic.co/downloads/beats/winlogbeat/winlogbeat-6.3.2-wi
ndows-x86.zip
● Extract
● Rename to winlogbeat, then copy paste in c://program files
● open vi winlogbeat.yml
tags: ["ap-southeast-1"]
fields:
globo_environment: production
Comment the elasticsearch
#----------------------------- Logstash output --------------------------------
● From powershell install winlogbeat template by using following command
● Cd /program files/winlogbeat
● Invoke-WebRequest -Method Put -InFile winlogbeat.template.json -Uri
http://54.255.170.251:9200/_template/winlogbeat?pretty -ContentType
application/json
● From Powershell install winlogbeat service using following command
\install-service-winlogbeat.ps1
● Start service using start-service winlogbeat
Service and restart the Winlogbeat on windows
service kibana start
http://54.169.238.188:5601
winlogbeat-*
timestap@
Create
Descover
Visualization
save
Install
Metricbeat on windows
https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-6.2.4-w
indows-x86_64.zip
Extract
Rename to metricbeat, then copy paste in c://program files
metricbeats.yml
tags: ["ap-southeast-1"]
fields:
globo_environment: production
Comment the elasticsearch
#----------------------------- Logstash output --------------------------------
output.logstash:
# The Logstash hosts
hosts: ["13.229.50.243:5044"]
Winpcap download install
Ps > C:/progaramfiles/metricsbeats> Invoke-WebRequest -Method Put
-InFile metricbeat.template.json -Uri
http://54.255.170.251:9200/_template/metricbeat?pretty -ContentType
application/json
Install-service-metricbeat.ps1
Services Start the metricbeats
Install
Filebeat on redhat:
Lunch redhat instance
#curl -L -O
https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.6.4-x86_64
.rpm
#sudo rpm -vi filebeat-5.6.4-x86_64.rpm
#cd /etc/filebeat/
Vi filebeat.yml
tags: ["ap-southeast-1"]
fields:
globo_environment: production
#----------------------------- Logstash output --------------------------------
output.logstash:
# The Logstash hosts
hosts: ["13.229.50.243:5044"]
curl -H 'Content-Type: application/json' -XPUT
'http://3.12.147.174:9200/_template/filebeat'
-d@/etc/filebeat/filebeat.template.json
sudo /etc/init.d/filebeat start
Open kibana and create pattern
