Correlation Analysis – Lab Solutions Guide

Overview
Welcome to the Splunk Education lab environment. These lab exercises will test your knowledge of using
Splunk commands to analyze and correlate events.

Scenario
You will use data from the international video game company, Buttercup Games. A list of source types is
provided below.

NOTE: This is a lab environment driven by data generators with obvious limitations. This is not a
production environment. Screenshots approximate what you should see, not the exact output.

Index Type Sourcetype Interesting Fields

web Online sales access_combined action, bytes, categoryId, clientip, itemId,
JSESSIONID, price, productId, product_name,
referer, referer_domain, sale_price, status,
user, useragent

network Web security cisco_wsa_squid action, cs_method, cs_mime_type, cs_url,

appliance data cs_username, sc_bytes, sc_http_status,
sc_result_code, severity, src_ip, status, url,
usage, x_mcafee_virus_name, x_wbrs_score,
x_webcat_code_abbr

©2023 Splunk Inc. All rights reserved. Correlation Analysis 25 September 2023 1
Common Commands and Functions
These commands and statistical functions are commonly used in searches but may not have been explicitly
discussed in the course. Please use this table for quick reference. Click on the hyperlinked SPL (Search
Processing Language) to be taken to the Search Manual for that command or function.
SPL Type Description Example

Sorts results in Sort the first 100 src_ip values in descending order
descending or ascending
sort command
order by a specified field.
| sort 100 -src_ip
Can limit results to a
specific number.
Return events with a count value greater than 30
Filters search results
where command
using eval-expressions.
| where count > 30
Rename SESSIONID to 'The session ID'
Renames one or
rename command
more fields.
| rename SESSIONID as "The session ID"

Remove the host field from the results

Keeps (+) or removes (-)
fields command
fields from search
results.
| fields - host

Calculate the total sales, i.e. the sum of price values.

Calculates aggregate
stats command
statistics over the
results set.
| stats sum(price)

Concatenate first_name and last_name values with a

Calculates an expression space to create a field called "full_name"
eval command and puts the resulting
value into a new or
existing field.
| eval full_name=first_name." ".last_name

Output vendorCountry, vendor, and sales values to a

table command Returns a table. table
| table vendorCountry, vendor, sales

Returns the sum of the

Calculate the sum of the bytes field
statistical values of a field. Can be
sum() function used with stats,
timechart, and chart
| stats sum(bytes)
commands.

Returns the number of Count all events as "events" and count all events that
occurrences of all events contain a value for action as "action"
count or statistical
or a specific field. Can
count() function
be used with stats, | stats count as events,
timechart, and chart count(action) as action
commands.

Refer to the Search Reference Manual for a full list of commands and functions.
©2023 Splunk Inc. All rights reserved. Correlation Analysis 25 September 2023 2
Lab Exercise 1 – Calculate Co-Occurrence Between Fields
Description
Configure the lab environment user account. Then, use the transaction command to correlate events.

Steps
Log into Splunk and change the account name and time zone.

Set up your lab environment to fit your time zone. This also allows the
instructor to track your progress and assist you if necessary.
Log into your Splunk lab environment using the username and
password provided to you.
You may see a pop-up window welcoming you to the lab environment.
You can click Continue to Tour but this is not required. Click Skip to
dismiss the window.
Click on the username you logged in with (at the top of the screen) and
then choose Account Settings from the drop-down menu.
After you complete step 6,
In the Full name box, enter your first and last name.
you will see your name in
Click Save. the web interface.
Reload your browser to reflect the recent changes to the interface.
(This area of the web interface will be referred to as user name.)

NOTE: Sometimes there can be delays in executing an action like saving in the UI or returning results
of a search. If you are experiencing a delay, please allow the UI a few minutes to execute
your action.

Navigate to user name > Preferences.

Choose your local time zone from the Time zone drop-down menu.
Click Apply.
(Optional) Navigate to user name > Preferences > SPL Editor > Search auto-format and click on the
toggle to activate auto-formatting. Then click Apply. When the pipe character is used in search, the SPL
Editor will automatically begin the pipe on a new line.

Search auto-format disabled (default)

Search auto-format enabled

©2023 Splunk Inc. All rights reserved. Correlation Analysis 25 September 2023 3
Scenario: Sales wants a report of all purchase events correlated with a unique JSESSIONID over the
last 60 minutes. The report should include information about the time of the event, the
actions performed during the session, and the client IP.

Correlate events based on JSESSIONID that involve a value for action. Then, filter results to
show only events that involved a purchase.

Search for all events in the online store (index=web sourcetype=access_combined) during the last 60
minutes.
index=web sourcetype=access_combined
Display a table that shows the _time, clientip, JSESSIONID, and action fields. Note that the actions are
listed in reverse chronological order (most to least recent.) (Hint: Use the table command.)
index=web sourcetype=access_combined
| table _time, clientip, JSESSIONID, action

Modify your search to only include events with a value in the action field.
index=web sourcetype=access_combined action=*
| table _time, clientip, JSESSIONID, action

Remove the table command and all the arguments being passed to it. Use the transaction command to
create groups of transactions based on the JSESSIONID field.
index=web sourcetype=access_combined action=*
| transaction JSESSIONID

©2023 Splunk Inc. All rights reserved. Correlation Analysis 25 September 2023 4
 Modify your search to display the transactions in a table.
index=web sourcetype=access_combined action=*
| transaction JSESSIONID
| table JSESSIONID, clientip, action

NOTE: By default, the values in the action column are ordered alphabetically, ignoring duplicates.

View only transactions that contain at least one purchase event. Use the search command to find
transactions containing a purchase.

NOTE: The search command must be downstream from the transaction command.
index=web sourcetype=access_combined action=*
| transaction JSESSIONID
| table JSESSIONID, clientip, action
| search action=purchase

©2023 Splunk Inc. All rights reserved. Correlation Analysis 25 September 2023 5
 Save your search as a report with the name L1S1.
a. Click Save As > Report
b. For Title, enter L1S1.
c. Save.
d. You can View your report or exit out of the Your Report Has Been Created window by clicking
the X in the upper-right corner.
e. You can access your saved reports using the Reports tab in the application bar.

Your recently saved L1S1 report will be visible in the Reports tab.

©2023 Splunk Inc. All rights reserved. Correlation Analysis 25 September 2023 6
Scenario: Sales needs a report of online store transactions that lasted longer than one minute and
involved the purchase action.

Edit the previous search so that the duration field is available to manipulate. Then, use this
field to filter results to only show events longer than 1 minute.

If not already displayed, run your L1S1 search again.

a. Set the search mode to Verbose Mode, which will re-execute your search.
b. Click the Events tab. Notice the new fields generated by the transaction command: duration
and eventcount.
Modify your search to add the duration and eventcount fields to your table after the clientip field. Run
your search in Smart Mode.
index=web sourcetype=access_combined action=*
| transaction JSESSIONID
| table JSESSIONID, clientip, duration, eventcount, action
| search action=purchase

Pipe results to the following eval command.

| eval durationMinutes=round(duration/60,1)

The eval command creates a new field called durationMinutes and populates this field with the value of
duration divided by 60 rounded to 1 decimal place.
index=web sourcetype=access_combined action=*
| transaction JSESSIONID
| table JSESSIONID, clientip, duration, eventcount, action
| search action=purchase
| eval durationMinutes=round(duration/60,1)

©2023 Splunk Inc. All rights reserved. Correlation Analysis 25 September 2023 7
 Modify your search to find data where the durationMinutes is greater than one minute. Adjust the table
to display only JSESSIONID, clientip, action, durationMinutes, and eventcount, in that order. (Hint
Refer to the Common Commands and Functions table at the beginning of this document to find a
command that filters search results.)
index=web sourcetype=access_combined action=*
| transaction JSESSIONID
| search action=purchase
| eval durationMinutes=round(duration/60,1)
| table JSESSIONID, clientip, action, durationMinutes, eventcount
| where durationMinutes > 1

Save your search as report, L1S2.

Scenario: Sales wants a report of all events correlated with a unique clientip over the last 60 minutes
that began with the addtocart action and ended with the purchase action.

Use the transaction command with the startswith and endswith options to group events by
clientip that started with action=addtocart and ended with action=purchase.

Search for all events from the online store (index=web sourcetype=access_combined) in the last 60
minutes and correlate the events based on clientip.
index=web sourcetype=access_combined
| transaction clientip
Use the startswith and endswith options of the transaction command to display transactions that
begin with an addtocart action and end with a purchase action.
index=web sourcetype=access_combined
| transaction clientip startswith=action=addtocart endswith=action=purchase
Display clientip, JSESSIONID, product_name, action, duration, eventcount, and price in a table.

©2023 Splunk Inc. All rights reserved. Correlation Analysis 25 September 2023 8
 index=web sourcetype=access_combined
| transaction clientip startswith=action=addtocart endswith=action=purchase
| table clientip, JSESSIONID, product_name, action, duration, eventcount, price

Save your search as report, L1S3.

CHALLENGE Exercise: Report the most common HTTP status errors that occurred during the last 30
days on the online sales web servers and the internal web appliance within a proximity of 5 minutes or
less. Only include days with more than 5 of these frequent errors.

Search HTTP status error events (status>399) from the online sales web servers (index=web
sourcetype=access_combined) and the web appliance (index=network
sourcetype=cisco_wsa_squid) during the last 30 days. For best performance, use the fields
command to limit extracted fields to only sourcetype and status. (Hint: See the Common Commands
and Functions table for information on how to use fields.)
(index=network sourcetype=cisco_wsa_squid) OR
(index=web sourcetype=access_combined) status>399
| fields sourcetype, status
Create transactions based on status field values and limit the span to 5 minutes.

NOTE: If you do not see results, increase the maxspan value.

(index=network sourcetype=cisco_wsa_squid) OR
(index=web sourcetype=access_combined) status>399
| fields sourcetype, status
| transaction status maxspan=5m
Limit the results to only transactions that contain at least one event from each sourcetype.
(index=network sourcetype=cisco_wsa_squid) OR
(index=web sourcetype=access_combined) status>399
| fields sourcetype, status
| transaction status maxspan=5m
| search sourcetype=access_combined AND sourcetype=cisco_wsa_squid
Use timechart to count events by status.
(index=network sourcetype=cisco_wsa_squid) OR
(index=web sourcetype=access_combined) status>399
| fields sourcetype, status
| transaction status maxspan=5m
| search sourcetype=access_combined AND sourcetype=cisco_wsa_squid
| timechart count by status

©2023 Splunk Inc. All rights reserved. Correlation Analysis 25 September 2023 9
 Discard rows that have fewer than 5 errors for all status values. (Hint: Use the addtotals command
without additional arguments.)
(index=network sourcetype=cisco_wsa_squid) OR
(index=web sourcetype=access_combined) status>399
| fields sourcetype, status
| transaction status maxspan=5m
| search sourcetype=access_combined AND sourcetype=cisco_wsa_squid
| timechart count by status
| addtotals
| search Total>4

Remove the Total column and display the data as a Line chart.
(index=network sourcetype=cisco_wsa_squid) OR
(index=web sourcetype=access_combined) status>399
| fields sourcetype, status
| transaction status maxspan=5m
| search sourcetype=access_combined AND sourcetype=cisco_wsa_squid
| timechart count by status
| addtotals
| search Total>4
| fields - Total

©2023 Splunk Inc. All rights reserved. Correlation Analysis 25 September 2023 10
 Save your search as report, L1X.
Optionally, for this line chart, set Multi-series Mode to Yes. Observe the change in how the lines are
represented. (Hint: It's one of the Format options on the General tab.)

©2023 Splunk Inc. All rights reserved. Correlation Analysis 25 September 2023 11
Lab Exercise 2 – Analyze Multiple Data Sources
Description
Use the append command to analyze dissimilar data sources into one search.

Steps
Scenario: The Sales department would like to see a list of sales by productid for the last hour as
well as the previous hour.

Use the append command to create a search that displays results from two different time
ranges. Then, align results using the first function.

Search for successful purchase events in the online store that involve a value for productId (index=web
sourcetype=access* productId=* action=purchase status=200) over the last 24 hours from the
previous hour. (Hint: Include the following time modifiers in your basic search: earliest=-1h@h
latest=@h.)
index=web sourcetype=access* productId=* action=purchase status=200 earliest=-1h@h
latest=@h
Pipe results to the following stats command.

| stats sum(price) as lastHourSales by productId

The stats command calculates the sum of price values for each productId. The values are listed under
a column called lastHourSales.
index=web sourcetype=access* productId=* action=purchase status=200 earliest=-1h@h
latest=@h
| stats sum(price) as lastHourSales by productId
Use the append command to add an additional search of the previous hour. This search will look similar to
the first search with the following differences:
• The time modifiers should capture the previous hour: earliest=-2h@h latest=-1h@h
• The results of the calculation performed by stats should be named "previousHourSales".
index=web sourcetype=access* productId=* action=purchase status=200 earliest=-1h@h
latest=@h
| stats sum(price) as lastHourSales by productId
| append
[search index=web sourcetype=access* productId=* action=purchase status=200
earliest=-2h@h latest=-1h@h
| stats sum(price) as previousHourSales by productId]

©2023 Splunk Inc. All rights reserved. Correlation Analysis 25 September 2023 12
 Select Visualization, then select the Line Chart.

Your results should look misaligned and less meaningful. Use the first function to overlay the two
searches into one clean line chart.
index=web sourcetype=access* productId=* action=purchase status=200 earliest=-1h@h
latest=@h
| stats sum(price) as lastHourSales by productId
| append
[search index=web sourcetype=access* productId=* action=purchase status=200
earliest=-2h@h latest=-1h@h
| stats sum(price) as previousHourSales by productId]
| stats first(*) as * by productId
| fillnull

Save your results as a report named L2S1.

©2023 Splunk Inc. All rights reserved. Correlation Analysis 25 September 2023 13