OSQUERY

The Mastermind Notes

Definition
Osquery is an open-source agent that converts the operating system
into a relational database. It allows us to ask questions from the
tables using SQL queries, like returning the list of running processes,
a user account created on the host, and the process of communicating
with certain suspicious domains.

The SQL language implemented in Osquery is not an entire SQL language

that you might be accustomed to, but rather it's a superset of
SQLite.

Realistically all your queries will start with a SELECT statement. This
makes sense because, with Osquery, you are only querying
information on an endpoint. You won't be updating or deleting any
information/data on the endpoint.

In Osquery, all commands are prefixed with [.].

Installation
Step 1 – Installing osquery on Your Server
To begin using osquery, you must first install it on your server.
Although you can compile it from source, the simpler method involves
adding its official Ubuntu repository since it’s not available through
the standard Ubuntu repositories.
1. Add the GPG Key:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys

1484120AC4E9F8A1A577AEEE97A80C63C9D8B80B

2. Add the osquery Repository:

sudo add-apt-repository "deb [arch=amd64] https://osquery-

packages.s3.amazonaws.com/xenial xenial main"
3. Update Your Package List:

sudo apt-get update

4. Install osquery:

sudo apt-get install osquery

By default, osquery doesn't provide immediate functionality out of the

box. Whether you're using the interactive shell ( osqueryi ) or the
background daemon ( osqueryd ), you'll need to configure it using flags
or a config file.

To view available options for the daemon, run:

osqueryd --help

This displays a variety of flags and parameters for managing

configurations, such as:

--flagfile PATH – Load additional flags from a file

--config_path VALUE – Path to the JSON configuration file
--daemonize – Run osquery as a daemon
--config_tls_endpoint – TLS endpoint for pulling configuration
files
--aws_region – Specify AWS region if using cloud integration

To explore interactive shell-specific flags, use:

osqueryi --help

You can launch the interactive shell with:

osqueryi --verbose
You'll then see output showing osquery initialization and any missing
configurations:

I0105 01:52:54.987584 4761 init.cpp:364] osquery initialized

[version=2.1.2]
...
I0105 01:52:54.988209 4761 init.cpp:606] Error reading config:
config file does not exist: /etc/osquery/osquery.conf
...
osquery>

These messages indicate that certain features aren't yet operational.

For instance, querying YARA tables ( select * from yara; ) may return
nothing, or querying the system logs ( select time, severity, message
from syslog; ) might show warnings like:

Table syslog is event-based but events are disabled

Please see the table documentation: https://osquery.io/docs/#syslog

This means additional configuration is needed to fully enable osquery's

features.
When you're done exploring the shell, exit by typing:

.exit

Enabling osquery to Access System Logs (Useful

for Local Security Monitoring)
To let osquery query system logs, you need to modify your system's
logging configuration. On Ubuntu 16.04, this involves updating the
rsyslog configuration.
1. Open the rsyslog config file:

sudo nano /etc/rsyslog.conf

2. Add a template and pipe action at the bottom of the file: These
lines define how log messages are formatted and direct them into a
named pipe used by osquery:

template(
name="OsqueryCsvFormat"
type="string"
string="%timestamp:::date-
rfc3339,csv%,%hostname:::csv%,%syslogseverity:::csv%,%syslogfacility-
text:::csv%,%syslogtag:::csv%,%msg:::csv%\n"
)
*.* action(type="ompipe" Pipe="/var/osquery/syslog_pipe"
template="OsqueryCsvFormat")

3. Save and exit the editor.

4. Restart the rsyslog service to apply changes:

sudo systemctl restart rsyslog

Next, we'll configure osquery to run with default options and schedule
basic queries.

Creating a Config File

To simplify the execution of osqueryi and ensure consistent options,
create a configuration file at /etc/osquery/osquery.conf . This file also
allows you to schedule queries and enable built-in query packs.

Structure of the Configuration File:

1. Options: Settings for daemon and shell behavior

2. Schedule: Timed queries to collect specific data
3. Packs: Predefined sets of queries included with osquery
4. Decorators: Metadata prepended to scheduled query results

1. Create and open the config file:

 sudo nano /etc/osquery/osquery.conf

2. Paste the following JSON configuration:

{
"options": {
"config_plugin": "filesystem",
"logger_plugin": "filesystem",
"logger_path": "/var/log/osquery",
"disable_logging": "false",
"log_result_events": "true",
"schedule_splay_percent": "10",
"pidfile": "/var/osquery/osquery.pidfile",
"events_expiry": "3600",
"database_path": "/var/osquery/osquery.db",
"verbose": "false",
"worker_threads": "2",
"enable_monitor": "true",
"disable_events": "false",
"disable_audit": "false",
"audit_allow_config": "true",
"host_identifier": "hostname",
"enable_syslog": "true",
"audit_allow_sockets": "true",
"schedule_default_interval": "3600"
},
"schedule": {
"crontab": {
"query": "SELECT * FROM crontab;",
"interval": 300
},
"system_profile": {
"query": "SELECT * FROM osquery_schedule;"
},
"system_info": {
"query": "SELECT hostname, cpu_brand, physical_memory FROM
system_info;",
"interval": 3600
}
},
 "decorators": {
"load": [
"SELECT uuid AS host_uuid FROM system_info;",
"SELECT user AS username FROM logged_in_users ORDER BY time DESC
LIMIT 1;"
]
},
"packs": {
"osquery-monitoring": "/usr/share/osquery/packs/osquery-
monitoring.conf",
"incident-response": "/usr/share/osquery/packs/incident-
response.conf",
"it-compliance": "/usr/share/osquery/packs/it-compliance.conf",
"vuln-management": "/usr/share/osquery/packs/vuln-
management.conf"
}
}

3. Save and close the file.

4. Validate your configuration:

sudo osqueryctl config-check

You should see a success message, confirming the configuration is

valid. If there are issues, the output will include file and line number
details for troubleshooting.

With this in place, osquery is ready to run with custom settings,

scheduled queries, and system logging support. Next, you can begin
configuring specialized packs like file integrity monitoring.

Setting Up the osquery File Integrity Monitoring

Pack
Monitoring changes to critical file paths is essential for system
security. To enable this, osquery allows you to define a custom file
integrity monitoring (FIM) pack.
1. Create the FIM pack file:
 sudo nano /usr/share/osquery/packs/fim.conf

2. Add the following configuration to monitor /home, /etc, and /tmp

directories:

{
"queries": {
"file_events": {
"query": "select * from file_events;",
"removed": false,
"interval": 300
}
},
"file_paths": {
"homes": [
"/root/.ssh/%%",
"/home/%/.ssh/%%"
],
"etc": [
"/etc/%%"
],
"home": [
"/home/%%"
],
"tmp": [
"/tmp/%%"
]
}
}

3. Save and close the file.

4. Update the packs section in **/etc/osquery/osquery.conf** if not
already added: Ensure the fim pack is listed like so:

"packs": {
"fim": "/usr/share/osquery/packs/fim.conf",
"osquery-monitoring": "/usr/share/osquery/packs/osquery-
monitoring.conf",
"incident-response": "/usr/share/osquery/packs/incident-
 response.conf",
"it-compliance": "/usr/share/osquery/packs/it-compliance.conf",
"vuln-management": "/usr/share/osquery/packs/vuln-management.conf"
}

5. Validate the configuration again:

sudo osqueryctl config-check

Once validated, osquery is now set up to monitor file integrity in real-

time. Next, you can begin using osqueryi to run interactive queries
against your system.

Using osqueryi to Perform Ad-hoc Security

Checks
The interactive shell osqueryi enables you to perform security queries
directly on your system without needing to start the osquery daemon.
You can still apply the full configuration settings by referencing the
config file.
1. Launch osqueryi with config file and verbose mode:

sudo osqueryi --config_path /etc/osquery/osquery.conf --verbose

Note: When using the configuration file, run osqueryi as root to ensure
full access.
2. Check current logged-in users:

select * from logged_in_users;

Review the host field to validate all IPs are familiar.

3. View historical login data:

select * from last;

Helps track past login activity.
4. Check if IPTables firewall is active:

select * from iptables;

If no output is shown, the firewall isn't configured.

5. Narrow firewall check by key fields:

select chain, policy, src_ip, dst_ip from iptables;

This helps highlight unexpected IP sources.

6. Review scheduled cron jobs:

select command, path from crontab;

Unfamiliar or suspicious commands may indicate malware.

7. Detect setuid-enabled binaries:

select * from suid_bin;

Run this periodically and compare outputs to detect unauthorized

binaries.
8. List loaded kernel modules with live status:

select name, used_by, status from kernel_modules where status="Live";

Monitor changes over time for anomaly detection.

9. Identify listening ports:

select * from listening_ports;

Compare results with expected services (e.g., SSH on port 22).

10. Track recent file system activity:

select target_path, action, uid from file_events;

Useful for auditing user changes or spotting unauthorized
modifications.
11. Schema and table exploration:

.schema <table_name>
.tables

Use these to explore available tables and their structures.

Many additional queries are packaged in the built-in packs and can be
used both interactively or scheduled via osqueryd . In the next section,
you'll learn how to launch and manage the osquery daemon to run
these queries on schedule.

Investigation methodology
When it comes to investigating an endpoint with Qsquery, we follow
the below order

List the tables in the current osquery installed on the end point
we highlight the table which contains the aspect of windows we
want to investigate. Example is investigating processes so we
would be looking the table that contains information about
processes
We search for the table
Display the schema and columns
Start the investigation by displaying and querying data using
SELECT statement.

Entering the interactive mode

osqueryi

Display the help menu

 .help

Displaying version and other values for the current

settings

.show

change output mode

.mode [mode]
modes:
list
pretty
line
column
csv

Querying the version of os query installed on a

windows endpoint

.table osquery_info

Checking the available tables

.tables

Displaying tables containing specific aspect of

windows

This is useful if you want to list and search the tables associated with
processes, users,.etc

.tables user
.tables process
 .tables network

Displaying and querying the columns of a table

.schema table_name

Display information about running processes

Assuming the table that contains details about processes is processes

SELECT * FROM processes;

We can also query specific columns from the table processes

Example [1]

SELECT pid, name, path FROM processes;

Example [2]

SELECT pid, name, path FROM processes WHERE name='lsass.exe';

Display number of running processes

The count(*) can be used to display the count or number of entries

for any table. Below is an example for the processes table

SELECT count(*) from processes;

Join the results of two tables. It requires on

common column
OSquery can also be used to join two tables based on a column that is
shared by both tables
 SELECT pid, name, path FROM osquery_info JOIN processes USING (pid);

Investigate services such as windows defender

select name,description from services where name like "WinD%";

Investigate windows defender logs

osquery> select eventid,datetime from win_event_log_data where source

= "Applications and Services Logs-Microsoft-Windows-Windows
Defender/Operational" and eventid like '1116' ;

Investigate sysmon logs

select eventid from win_event_log_data where source="Microsoft-

Windows-Sysmon/Operational" order by datetime limit 1;

Investigate Browsers

Investigating Chrome Extensions

First, we list the schema (table description) of
the chrome_extensions table.

.schema chrome_extensions

Based on the output, lets say we are interested in identifier and

name from the table;

SELECT identifier as id, name FROM chrome_extensions;

Security Packs
osquery provides a variety of built-in security packs that group related
queries into categories aligned with common security operations.
These packs are defined in your configuration and offer value across
compliance, threat detection, and vulnerability management.

Security Compliance Pack

These queries help organizations stay aligned with standards such as

GDPR and PCI DSS by monitoring critical system configurations.
Example:

"disk_encryption": {
"query": "select * from disk_encryption;",
"interval": "86400",
"version": "1.4.5",
"platform": "posix",
"description": "Retrieves the current disk encryption status for
the target system.",
"value": "Identifies a system potentially vulnerable to disk
cloning."
},
"chrome_extensions": {
"query": "select * from users join chrome_extensions using (uid);",
"interval": "86400",
"version": "1.4.5",
"description": "Retrieves the list of extensions for Chrome in the
target system.",
"value": "General security posture."
}

Vulnerability Management Pack

This pack helps detect outdated or malicious software installations and

supports proactive patching and asset monitoring.
Example:

"backdoored_python_packages": {
"query": "select name as package_name, version as package_version,
path as package_path from python_packages where package_name =
'acqusition' or package_name = 'apidev-coop' or package_name = 'bzip'
or package_name = 'crypt' or package_name = 'django-server' or
 package_name = 'pwd' or package_name = 'setup-tools' or package_name =
'telnet' or package_name = 'urlib3' or package_name = 'urllib';",
"interval": "86400",
"platform": "posix",
"version": "1.4.5",
"description": "Watches for the backdoored Python packages
installed on system.",
"value": "Assures no known-malicious Python packages are present."
}

Incident Response & Intrusion Detection Pack

These queries are useful for detecting suspicious behavior, active

compromises, or persistent access methods.
Examples:

"logged_in_users": {
"query": "select liu.*, p.name, p.cmdline, p.cwd, p.root from
logged_in_users liu, processes p where liu.pid = p.pid;",
"interval": "3600",
"platform": "posix",
"version": "1.4.5",
"description": "Retrieves the list of all the currently logged in
users in the target system.",
"value": "Detects unauthorized logins and accounts involved in an
incident."
},
"open_sockets": {
"query": "select distinct pid, family, protocol, local_address,
local_port, remote_address, remote_port, path from
process_open_sockets where path <> '' or remote_address <> '';",
"interval": "86400",
"platform": "posix",
"version": "1.4.5",
"description": "Retrieves all the open sockets per process in the
target system.",
"value": "Helps detect malware via suspicious socket connections."
}
These packs make osquery a powerful, modular tool for security
teams looking to automate system monitoring, compliance checks, and
threat hunting.

Logs
When osqueryd runs scheduled or pack-based queries, it writes
results to log files in JSON format located at:

1. Differential Logs

Path: /var/log/osquery/osqueryd.results.log
These logs track changes between the current and previous runs of a
query. Each entry indicates what data was added or removed.
Example:

{
"action": "added",
"columns": {
"name": "osqueryd",
"path": "/usr/local/bin/osqueryd",
"pid": "97830"
},
"name": "processes",
"hostname": "hostname.local",
"calendarTime": "Tue Sep 30 17:37:30 2014",
"unixTime": "1412123850",
"epoch": "314159265",
"counter": "1"
}
{
"action": "removed",
"columns": {
"name": "osqueryd",
"path": "/usr/local/bin/osqueryd",
"pid": "97650"
},
"name": "processes",
"hostname": "hostname.local",
 "calendarTime": "Tue Sep 30 17:37:30 2014",
"unixTime": "1412123850",
"epoch": "314159265",
"counter": "1"
}

Use differential logs to detect changes like new processes, users, or

modifications to system state.

2. Snapshot Logs

Path: /var/log/osquery/osqueryd.snapshots.log
These logs provide the full results of a query at a specific moment,
regardless of whether there were changes.
Example:

{
"action": "snapshot",
"snapshot": [
{
"parent": "0",
"path": "/sbin/launchd",
"pid": "1"
},
{
"parent": "1",
"path": "/usr/sbin/syslogd",
"pid": "51"
},
{
"parent": "1",
"path": "/usr/libexec/UserEventAgent",
"pid": "52"
},
{
"parent": "1",
"path": "/usr/libexec/kextd",
"pid": "54"
}
],
 "name": "process_snapshot",
"hostIdentifier": "hostname.local",
"calendarTime": "Mon May 2 22:27:32 2016 UTC",
"unixTime": "1462228052",
"epoch": "314159265",
"counter": "1"
}

Snapshot logs are useful for full-state audits and system inventory
without focusing on differences.

Both types of logs support security auditing, compliance validation,

and incident response through structured, machine-readable output.

Forensics
Identifying Timestomping and Deleted File Artifacts (NTFS Forensics)
Attackers often attempt to erase or manipulate evidence to avoid
detection. One common anti-forensics technique is timestomping – the
act of modifying file timestamps to mislead forensic investigators.
Using osquery with NTFS forensic extensions, you can detect
anomalies that may indicate timestomping.

NTFS Structure Basics

NTFS stores file metadata in the Master File Table (MFT). Each file
record contains:

$STANDARD_INFORMATION ($SI): Timestamps related to file content.

$FILE_NAME ($FN): Timestamps related to file name and path.

Each file has both sets of timestamps, and discrepancies between

them are red flags.

Example 1: Timestamp Inconsistency

Check for mismatches between $FN and $SI timestamps:

 SELECT path, fn_btime, btime
FROM ntfs_file_data
WHERE device="\\.\PhysicalDrive0" AND partition=3 AND
directory="/Users/mmyers/Desktop/test_dir" AND fn_btime != btime;

Check for files whose creation time is later than their last modified
time:

SELECT filename, path

FROM ntfs_file_data
WHERE device="\\.\PhysicalDrive0" AND partition=2 AND
path="/Users/Garret/Downloads" AND fn_btime > ctime OR btime > ctime;

Such inconsistencies strongly suggest timestamp manipulation.

Example 2: Missing Timestamp Precision

Many timestomping tools use second-level precision, unlike natural

OS-generated nanosecond precision.

Authentic NTFS timestamp:

131683876627452045 = precise and likely real

Forged timestamp via system utility:

131683876620000000 = rounded to nearest second, likely faked

osquery displays NTFS timestamps in raw 64-bit integers, which

allows analysts to detect these discrepancies.

Example 3: Artifacts of Deleted Files

Even after a file is deleted, NTFS often retains a copy of its metadata
in the directory index ($FN attributes marked as inactive). This slack
space may include:

Filename
 Timestamps
Partial metadata

Querying the unused entries:

SELECT parent_path, filename, slack

FROM ntfs_indx_data
WHERE parent_path="/Users/mmyers/Desktop/test_dir" AND slack != 0;

This helps reconstruct file history even after deletion. Be aware that
not all deleted files will leave slack artifacts, and renamed files will
also generate new entries, which requires deeper correlation.

Threat Detection & Response

osquery is a powerful tool for real-time investigation, threat hunting,
and anomaly detection. Below are several tactical use cases that
illustrate how security teams can leverage osquery for detecting
malicious activity.

Detecting Deleted or Modified Binaries for Running

Processes

Attackers often run malicious binaries and then delete the original file
to evade detection. Use this query to identify such processes:

SELECT name, path, pid FROM processes WHERE on_disk = 0;

This helps flag processes whose backing executable has been removed
or tampered with.

Detecting Indicators of Compromise (IOCs)

osquery allows you to search for known IOCs on disk or in memory.

These sample queries, contributed by Facebook, target known
persistence methods used by the Hacking Team OSX backdoor:
Check for suspicious device files:
 SELECT * FROM file WHERE path = '/dev/ptmx0';

Check suspicious app identifiers or packages:

SELECT * FROM apps

WHERE bundle_identifier = 'com.ht.RCSMac'
OR bundle_identifier LIKE 'com.yourcompany.%'
OR bundle_package_type LIKE 'OSAX';

Check for persistence via LaunchDaemons and launch agents:

SELECT * FROM launchd

WHERE label = 'com.ht.RCSMac'
OR label LIKE 'com.yourcompany.%'
OR name IN ('com.apple.loginStoreagent.plist',
'com.apple.mdworker.plist', 'com.apple.UIServerLogin.plist');

These types of IOCs can be adapted to track other known malware or

persistence patterns.

Detecting New or Unexpected Kernel Modules

Kernel modules can be used to load rootkits. Run the following query
regularly and compare against a known-good baseline:

SELECT name FROM kernel_modules;

Any new or unexpected module names could indicate compromise and

should be investigated.

Malware Analysis
ools like Sysmon and osquery offer exceptional visibility into endpoint
behavior, logging diverse system events that you can forward to a
SIEM or threat analysis platform.

Emotet Case Study: Understanding Infection Chain

 “Notes Cataloge/Cyber Security Study Notes/Blue Team/Osquery/1.png” could
not be found.

This guide walks through analyzing Emotet, a widely spread banking

Trojan. Delivered via phishing and malicious Office macros, Emotet
typically executes PowerShell scripts to fetch and install its payload.

Step 1: Capture PowerShell Execution

Run a Windows sample with osquery installed. Ensure Script Block
Logging is enabled. Then query the powershell_events table to extract
the base64-encoded command and its decoded script block.

SELECT time, script_text FROM powershell_events;

You'll clearly see the PowerShell activity responsible for dropping

malware.
Step 2: Catch Network Activity by Path
Track outbound network connections made during the malware
download by joining process_open_sockets and processes :

SELECT processes.name,
process_open_sockets.remote_address,
process_open_sockets.remote_port
FROM process_open_sockets
LEFT JOIN processes ON process_open_sockets.pid = processes.pid
WHERE process_open_sockets.remote_port != 0
AND processes.name != '';

This lets you link network behavior to specific processes

Step 3: Detect Disk Writes
Identify newly written files, especially in user directories,using:

SELECT path, size

FROM file
WHERE path LIKE 'C:\\Users\\%%'
 AND mtime > (SELECT local_time FROM time) - 100
AND filename != '.';

This reveals the Emotet dropper (e.g., squarectx.exe ) as it's written

to disk.
Step 4: Identify Running Malware
List currently running processes together with user context:

SELECT processes.pid, users.username, processes.path

FROM processes
LEFT JOIN users ON users.uid = processes.uid
WHERE processes.path != '';

This helps confirm active malware execution

Step 5: Trace Command & Control Communications
Reusing the socket query, you can detect if the malware is contacting
external C2 servers, critical intel for incident response.

Persistence Techniques Detection

Monitoring Registry & Startup Entries

In a case study using Shrug ransomware (.NET-based), we detect
persistence mechanisms by querying:

Newly created files in temp directories:

SELECT ... FROM file WHERE path LIKE 'C:\Users\%%' AND mtime > ...;

New registry entries under CurrentVersion\Run pointing to the

payload.
Startup items:

SELECT source, name, path FROM startup_items;

These detect auto-run entries like shrug.exe created by the malware.

Additionally:
 SELECT path, name, type, data FROM registry WHERE key LIKE
'HKEY_USERS\%\Shrug';

This reveals registry values like installation dates, victim IDs, and
cryptographic keys.
Monitoring Network Presence
Track outbound communication to suspect domains using socket
queries, useful for detecting C2 connections.
Spotting Encryption Activity via File Modification Time
Identify modified files with non-native extensions (e.g. .SHRUG ) by
comparing mtime across a user directory scan.
Detect New Tasks & Services
Use osquery tables to uncover stealthy persistence:

SELECT name, action, path, enabled, next_run_time

FROM scheduled_tasks;

SELECT name, display_name, start_type, path, user_account

FROM services;

These queries identify newly added scheduled tasks or malicious

services registered by malware.

OSX/Dummy on macOS
Preparing the Environment

Modify /etc/security/audit_control to include pc flags for

recording process events.
Restart the system to apply changes.

Enabling Event Logging

Enable disable_events=false and disable_audit=false via flag file

( /etc/osquery/osquery.flag ) or by passing flags to osqueryi or
osqueryd .
Executing the Malware & Capturing Activity
Run OSX/Dummy in the test VM. osquery will record system events,
which you can view using:

SELECT * FROM process_events;

Set display mode to line within osqueryi ( .mode line ) to get easy-
to-parse event logs. You can then export this data and reconstruct
process trees manually (using PID and parent PID data).

Observed Malware Behavior

From analysis of process_events , osquery reveals:

Creation of /tmp/script.sh
Use of chown and chmod to modify permissions
Moving the script to /var/root via mv
Execution of script via launchctl
Involved child processes such as Python reverse shell connection

osquery also captures file, network, and DNS events through the
corresponding tables ( file_events , socket_events ,
dns_lookup_events ).

Insights & Benefits

osquery enables real-time visibility into execution and

persistence of malware.
You can reconstruct what malicious code did—file drops,
ownership changes, persistence techniques—using only the SQL
output.
Unlike sandboxes, osquery can run on actual production
endpoints, including remote or offline devices, offering broader
reach for detection.

Comparison Table: Sandboxing vs. osquery

Capability Sandboxing osquery
File system activity Full visibility file_events table
Network activity Full visibility socket_events ,
dns_lookup_events
Process execution Full runtime process_events
context
Runtime code API-level behavior Not available directly
analysis

While sandboxing offers granular API tracing, osquery provides

scalable, endpoint‑based event logging, often sufficient for forensic
and security investigation.

Detecting Malicious Downloads

Technologies used: osquery · Rsyslog · Kafka · Docker · Python3 ·
VirusTotal · Slack alerting

A proof‑of‑concept pipeline designed to detect malicious file

downloads:

osquery monitors file system events, capturing created files.

A Rsyslog client on macOS sends osquery logs to a central
Rsyslog server.
That server forwards logs to Kafka.
A Dockerized Python3 consumer reads the Kafka topic, extracts
file hashes.
These hashes go to VirusTotal for a verdict.
If VirusTotal flags the file as malicious, the pipeline sends an
alert to Slack.

Objectives

Detect malicious downloads using osquery + VirusTotal

Provide osquery configuration examples
Demonstrate Python + Kafka integration
 Show Docker deployment of Rsyslog and Kafka components
Use VirusTotal for file reputation checks.

Prerequisites & Scope

This is a proof-of-concept, not production-ready.

Assumes familiarity with osquery, Kafka, Rsyslog, and Docker
operations.

Component Overview

Component Role
osquery Provides SQL‑style access to live system data (e.g. file events
via file_events )
Rsyslog High-performance log handler that accepts input, processes,
and forwards to Kafka
Kafka Event streaming platform handling high volumes of log data
VirusTotal Scans submitted file hashes across dozens of antivirus engines
via API

System Flow

1. osquery tracks new file creations.

2. Logs are sent from the macOS endpoint to a Rsyslog server.
3. Rsyslog forwards these logs to a Kafka topic.
4. A Dockerized Python app consumes Kafka messages, parses file
hash.
5. That hash is sent to VirusTotal for analysis.
6. If the score passes a configurable threshold, the Python app
sends a Slack alert.

Implementation Steps

1. Get a VirusTotal API Key

Register at VirusTotal, go to Settings > API Key, and save your
personal API key.
 2. Deploy Kafka and rsyslog with Docker:

git clone https://github.com/holdmybeer-osquery/virustotal

cd virustotal/docker
sudo vim /etc/hosts

Add:

127.0.0.1 kafka.local rsyslog.local

Run:

docker-compose up -d

3. Setup osquery and rsyslog on macOS

brew install osquery

brew install rsyslog
cp osquery/osquery.conf /usr/local/etc/osquery/osquery.conf
cp osquery/flags.conf /usr/local/etc/osquery/flags.conf
sudo cp rsyslog/rsyslog-client.conf /usr/local/etc/rsyslog.conf
brew services start rsyslog

4. Test Kafka Ingestion

Kafka Consumer Test:

from kafka import KafkaConsumer

consumer = KafkaConsumer('osquery',
bootstrap_servers='localhost:9092')
for msg in consumer:
print(msg)

5. Deploy Python Processing Service

Edit the configuration:

cp docker/config.yml.sample docker/config.yml
 vim docker/config.yml

Populate:

virustotal:
api_key: "YOUR_API_KEY"
kafka:
server: kafka:9092
topic: osquery
slack:
webhook_url: "YOUR_SLACK_WEBHOOK_URL"
threshold: 10

Run:

docker-compose -f docker-compose-app.yml up -d

6. Simulate File Downloads

Download test files:

wget http://www.eicar.org/download/eicar.com

Check for VirusTotal alert in Slack.

Notes & Improvements

Expand osquery file path coverage.

Lower osquery polling interval.
Add file carving logic for deeper inspection.

Using Osquery in The Cloud

Alibaba Cloud

Installing osquery on Your Server

To begin using osquery, you must first install it on your server.
Although you can compile it from source, the simpler method involves
adding its official Ubuntu repository since it’s not available through
the standard Ubuntu repositories.

If you're using Alibaba Cloud ECS with an Ubuntu server:

export OSQUERY_KEY=1484120AC4E9F8A1A577AEEE97A80C63C9D8B80B
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys
$OSQUERY_KEY
sudo add-apt-repository 'deb [arch=amd64] https://pkg.osquery.io/deb
deb main'
sudo apt-get update
sudo apt-get install osquery
sudo service osqueryd restart

Querying the System with osqueryi on Alibaba Cloud

You can use osqueryi , the interactive shell, to run SQL-like queries on
system state:

osqueryi

Example query to list active processes:

select pid, name, path, cmdline, system_time from processes limit 5;

Example output:

+------+-----------------+------+-------------------------------------
---+-------------+
| pid | name | path | cmdline
| system_time |
+------+-----------------+------+-------------------------------------
---+-------------+
| 1 | systemd | | /sbin/init splash
| 3810 |
| 10 | rcu_sched | |
| 8470 |
| 1025 | rabbitmq-server | | /bin/sh
/usr/lib/rabbitmq/bin/rabbitmq | 0 |
 | 1051 | redis-server | | /usr/bin/redis-server
127.0.0.1:6379 | 9020 |
| 106 | kthrotld | |
| 0 |
+------+-----------------+------+-------------------------------------
---+-------------+

Step 7 Configuring osqueryd to Run in the Background

To run background queries, configure the following files:
1. /etc/osquery/osquery.conf This JSON file defines:

options : Behavior of osqueryd

schedule : Queries to run at defined intervals
packs : Paths to additional query packs
file_paths : Directories to watch for file changes

Example:

{
"options": {
"events_max": 100000,
"enable_monitor": true,
"host_identifier": "uuid"
},
"schedule": {
"deb_packages": {
"query": "select * from deb_packages;",
"interval": 86400,
"snapshot": true,
"platform": "linux",
"description": "Fetches all deb packages installed in system"
}
},
"packs": {
"incident-response": "/usr/share/osquery/packs/incident-
response.conf",
"it-compliance": "/usr/share/osquery/packs/it-compliance.conf",
"osx-attacks": "/usr/share/osquery/packs/osx-attacks.conf",
"vuln-management": "/usr/share/osquery/packs/vuln-
 management.conf",
"hardware-monitoring": "/usr/share/osquery/packs/hardware-
monitoring.conf",
"ossec-rootkit": "/usr/share/osquery/packs/ossec-rootkit.conf"
},
"file_paths": {
"configurations": [
"/etc/%%",
"/bin/%%"
]
}
}

2. /etc/osquery/osquery.flags Contains startup flags:

--config_plugin=filesystem
--logger_plugin=filesystem
--logger_path=/var/log/osquery
--host_identifier=uuid
--utc=true

You can generate these configs yourself or copy defaults:

cp /usr/share/osquery/osquery.example.conf /etc/osquery/osquery.conf

Then edit the pack list as needed. Restart osquery:

sudo service osqueryd restart

With these configurations, osquery is ready to run scheduled queries

and monitor key system areas in the background. Continue monitoring
with osqueryi or log-based reviews from /var/log/osquery .

Done !
Check out other cheat sheets and study notes using the below link
https://shop.motasem-notes.net