KNOWLEDGE GUIDE – ADVANCED USERS
FortiSIEM
Integrating with MISP
Importing Threat Intelligence IOC from MISP into the
FortiSIEM CMDB.
US ATP Team
March 2020
Introduction
Purpose and Scope
FortiSIEM is advanced Security Information Event Management system which incorporates an event database (proprietary NoSQL
or elastic search database) with a CMDB postgreSQL database. Both databases are utilized in terms of analytics
(search/reporting/dashboarding) and event correlation, with the CMDB categorizing nearly 200,000 unique multi-vendor events
into functional correlation categories.
MISP (Malware Information Sharing Project) is an Open Source Threat Intelligence Platform and a community-driven project
- www.misp-project.org. What MISP provides amongst many things is an IOC and indicators database allowing users to store
technical and non-technical information about malware samples, incidents, attackers and intelligence.
The purpose of this FortiSIEM integration is to currently query MISP and produce a list of indicators to populate into the FortiSIEM
CMDB threat related containers.
Disclaimer
When mentioned in this document and this document only, the following terms and definitions will apply:
• This document and provided script are provided as is, and may not be 100% accurate use caution.
• The script is designated as an internal/PoV usage script and hence is designed without error checking or resource
restraints and hence should only be used on a test/PoV system only.
• TAC is not expected to support this script.
• Usage of this script is at your own risk.
• Any questions should be posted via FUSE or FNDN or directly to [email protected]
2
ADVANCED KNOWLEDGE GUIDE – Integrating with MISP
Usage
Installing the MISP script
In addition to this document a file name fsmMISPIntegration.zip should also have also been provided.
Create a new directory for this program under the /root directory for example MISP on your FortiSIEM test environment and then
copy this file via WinSCP/Cyberduck or favourite secure copy program to this directory, where it should be unzipped and then
executed.
For example, this process can be obtained via the commands below.
mkdir MISP
cd MISP
unzip fsmMISPIntegration.zip
chmod +x fsmMISP.sh
Before Running the MISP script for the first time.
The fsmMISP script has some pre-requites that need populating before running the script. Use Vi or similar to define your MISP API
Key and Server IP details along with the IP Address of your FortiSIEM Supervisor node.
#Enter your MISP API Key and Server Details
MISP_APIKEY="<enter here>"
MISP_SERVER="<enter here>"
#Enter your FSM Server IP
FSM_IP="<enter here>"
Understanding the MISP script components.
The fsmMISP script provides some query JSON files in the same directory it is extracted. These files are related to the indicator
types than can be imported into FortiSIEM, and each provides a custom query to extract this data from your MISP platform.
For example looking at the file misp_url_query.json :
{
"request": {
"type": "url",
"category": "Network activity",
"last": "1d",
"enforceWarninglist": "True"
}
}
The type in this case is url for the last 1 day. These files are customizable, for example the time can be set to 5d or 12h or 30m etc.
Each file has a separate query for the following data:
File Name Description
misp_domain_query.json DNS Domains
misp_hashes_query.json MD5, SHA1 and SHA256 Hashes
misp_ip_query.json Source and Destination IP
misp_md5_query.json Only MD5 Hashes
misp_sha1_query.json Only SHA1 Hashes
misp_sha256_query.json Only SHA256 Hashes
misp_url_query.json URL IOC
3
ADVANCED KNOWLEDGE GUIDE – Integrating with MISP
MISP Script Output
The fsmMISP script uses the MISP API to make one or more queries and grab the resultant IOC data as JSON, which is then
formatted and relevant fields are converted to CSV format for import into FortiSIEM.
An example of the converted output for URL is as below:
#URL,Malware Type,Description,Last Seen
"http://soheylistore.ir:80:/modules/mod_feed/feed.php","OSINT - Carbon Paper:
Peering into Turla’s second stage backdoor","C&C server addresses (hacked websites
used as 1st level of proxies","2017-03-30 02:54PM"
Preparing your FortiSIEM for MISP IOC Data
The fsmMISP script is expected to be run locally on your FortiSIEM Supervisor and the CSV data written to a local folder that the
FortiSIEM can reach for import (ie: itself). In production this could run on a different machine, with a 3 party URL used for
rd
accessing the data.
Prepare the Supervisor
From an SSH session, create a new directory on the Supervisor as follows:
mkdir /var/www/html/ioc
This is the location that the fsmMISP script will dump its CSV files ready for import.
Prepare the FortiSIEM GUI
Within the FortiSIEM Resources Tab under the various Threat Intel categories, create a folder for the MISP data.
For example, the following folders could be created: (names are not important)
Parent Folder Custom Folder
Malware URLs MISP URL
Malware IPs MISP IP
Malware Domains MISP Domain
Malware Hash MISP MD5 etc..
Running the MISP script
Execute the MISP script from the extracted directory via one of the two methods below.
./fsmMISP.sh <options>
Or
./fsmMISP.sh
The <options> above are one or more comma separated entries as per the table below.
Once the script completes the output (CSV files) are created under the /var/www/html/ioc folder.
4
ADVANCED KNOWLEDGE GUIDE – Integrating with MISP
Option Description
url MISP URLS returned via misp_url_query.json
md5 MISP MD5 returned via misp_md5_query.json
sha1 MISP SHA1 returned via misp_sha1_query.json
sha256 MISP SHA256 returned via misp_sha256_query.json
hashes MISP MD5,SHA1 & SHA256 hashes returned via
misp_hashes_query.json
domain MISP DOMAINS returned via misp_domain_query.json
ip MISP Source and Destination IP returned via
misp_ip_query.json
NULL (ie: no option specified) ALL MISP Queries above are performed
Examples
# Collect MISP md5, ip and urls
./fsmMISP.sh md5,ip,url
# Collect MISP sha1 hashes only
./fsmMISP.sh sha1
The script can be scheduled to run every so often via a cron job on the FortiSIEM backend.
Scheduling FortiSIEM to Populate the CMDB with IOC Data
Once the CSV files are ready, they can be imported (and scheduled if required) via the GUI under the Resources Tab.
Malware IP
Navigate and then select to the custom MISP IP folder previously created, then select More -> Update.
Choose the option to Update via API and then click the pencil option and populate the URL as http://127.0.0.1/ioc/misp_ip.csv
Notice here the Data Update options for Full or Incremental updates.
For the Data Mapping for the CSV file, define the following:
5
ADVANCED KNOWLEDGE GUIDE – Integrating with MISP
Then click Save.
Then click the plus icon against schedule and set a time / recurrence pattern as necessary and then click Save and Done when
completed.
Once the schedule is met, then the MISP IPs should be imported as below:
Malware Domain
For the MISP Domains, select the custom folder created earlier and again set to update via API, and this time use the following
URL and Mappings and schedule as necessary.
URL: http://127.0.0.1/ioc/misp_domain.csv
6
ADVANCED KNOWLEDGE GUIDE – Integrating with MISP
Once the schedule is met, then the MISP Domains should be imported as below:
Malware Hashes
For the MISP MD5, SHA1,SHA256 or the aggregated Hashes option (all formats together), select the custom folder or folders
created earlier and again set to update via API, and this time use the following URL and Mappings and schedule as necessary.
Obviously, you only need to define the entries that you need.
URL for MD5: http://127.0.0.1/ioc/misp_md5.csv
URL for SHA1: http://127.0.0.1/ioc/misp_sha1.csv
URL for SHA256: http://127.0.0.1/ioc/misp_sha256.csv
URL for All Hashes: http://127.0.0.1/ioc/misp_hashes.csv
Mappings are the same for each type, and MD5 is shown below as an example:
Once the schedule is met, the MISP MD5 should be imported as below:
7
ADVANCED KNOWLEDGE GUIDE – Integrating with MISP
Malware URL
For the MISP URLs, select the custom folder created earlier and again set to update via API, and this time use the following URL
and Mappings and schedule as necessary.
URL: http://127.0.0.1/ioc/misp_url.csv
Once the schedule is met, then the MISP URLs should be imported as below:
MISP script logging
Two logging options have been provided.
A) misp_log.txt log file in the current directory the script is run.
This records the results of the last time the script was run.
FSM MISP Last Run at Mar 13 20.45.56 2020
Process MD5
FSM : 30841 records ready for upload
B) Syslog to FortiSIEM
The script will send syslog messages to the IP address of your FortiSIEM.
A parser (misp_parser.xml) is in the directory where the script was extracted, which can be deployed. (The parser test
message is within the parser).
Results will look like the following:
8
ADVANCED KNOWLEDGE GUIDE – Integrating with MISP
