ELK in Security Analytics

ELK in Security Analytics

• 1.
  ELK in Securityanalytics -- Lionel Faleiro Lionel Faleiro [ @sandmaxprime ]
• 2.
  About Me • Trainerand Security Analyst at Institute of Information Technology / Network Intelligence India • 4+ years experience in IT • Conducted Trainings at multiple corporates • Part of the DFIR Team at NII • Key domains – Security Analytics, Malware Analysis, Log Analysis, Intrusion Response Lionel Faleiro [ @sandmaxprime ]
• 3.
  Lionel Faleiro [@sandmaxprime ]
• 4.
  What Big Data.. •IS: • Store large volumes of data • Enables us to run queries on the data set • IS NOT: • Hadoop, Hive, Pig, Yarn – these are technologies • Does not automatically give you analytical results Lionel Faleiro [ @sandmaxprime ]
• 5.
  Why use BigData in Security? • User-behaviour analytics • Fraud Detection • Log correlation from additional sources • Forensic analysis on large volumes of data Lionel Faleiro [ @sandmaxprime ]
• 6.
  Known SIEM Issues • Unableto ingest a lot of log sources • Cost of storage is high • Requires more compute power • Licensing issues • Monitoring on each endpoint is problematic • Current monitoring is static in nature • Too many alerts Lionel Faleiro [ @sandmaxprime ]
• 7.
  SIEM + ELK= SOC 2.0 • SIEM Functions • Alerts for standard IT issues • Rules based correlations • Standard reporting/queries • ELK Functions • Visualize Logs for anomalies • Ingest logs from multiple sources with large volume • Implement Threat-Hunting strategy • Custom search and querying Lionel Faleiro [ @sandmaxprime ]
• 8.
  This is not ELK.. LionelFaleiro [ @sandmaxprime ]
• 9.
  What is ELK? •E is a NoSQL databased that is based on the Lucene search engine • Stores data in an unstructured way • Cannot use SQL to query it. • L is a log pipeline tool that accepts inputs, executes transformations and outputs the data into various targets • K is a visualization layer Lionel Faleiro [ @sandmaxprime ]
• 10.
  ELK Overview • Beats •Log shippers – Windows events, system status, network traffic • Elasticsearch • Data storage, search engine • Logstash • Log management component. Ingest, Process, Output • Kibana • - Create visualizations and dashboards Lionel Faleiro [ @sandmaxprime ]
• 11.
  ELK Architecture Lionel Faleiro[ @sandmaxprime ]
• 12.
  Elasticsearch • Based onApache Lucene • Open-source search engine library • Created by Shay Banon • Extends Lucene to store, index and search • JSON over HTTP Lionel Faleiro [ @sandmaxprime ]
• 13.
  Logstash • Integrated Logmanagement framework • Log collection • Centralization • Parsing • Storage • Written in Jruby • Runs in JVM • Multiple input mechanism • TCP/UDP • Files • Sysog Lionel Faleiro [ @sandmaxprime ]
• 14.
  Logstash: Conf • Input{} • Filter {} • Output {} Lionel Faleiro [ @sandmaxprime ]
• 15.
  Lionel Faleiro [@sandmaxprime ]
• 16.
  Kibana • Visualization platform •Tight integration with Elasticsearch Lionel Faleiro [ @sandmaxprime ]
• 17.
  Beats • Filebeat • Metricbeat •Packetbeat • Winlogbeat • Hearbeat Lionel Faleiro [ @sandmaxprime ]
• 18.
  Filebeat • A lightweightway to forward and centralize logs and files Lionel Faleiro [ @sandmaxprime ]
• 19.
  Metricbeat Lionel Faleiro [@sandmaxprime ]
• 20.
  Packetbeat • Packetbeat isa lightweight network packet analyzer that sends data to Logstash or Elasticsearch • It supports many application layer protocols, from database to key-value stores to HTTP and low-level protocols Lionel Faleiro [ @sandmaxprime ]
• 21.
  Lionel Faleiro [@sandmaxprime ]
• 22.
  Winlogbeat • Winlogbeat livestreams Windows event logs to Elasticsearch and Logstash in a lightweight way • Read from any windows event log channel Lionel Faleiro [ @sandmaxprime ]
• 23.
  Lionel Faleiro [@sandmaxprime ]
• 24.
  Heartbeat • Monitor servicesfor their availability with active probing • Heartbeat pings via ICMP, TCP, and HTTP, and also has support for TLS, authentication and proxies. Lionel Faleiro [ @sandmaxprime ]
• 25.
  Use Cases • Nginx/Apache •Sysmon Integration • Forensics Imaging Lionel Faleiro [ @sandmaxprime ]