Benni0 App for MISP (TA_misp)

The main purpose of this Splunk App is the import of attributes/IOCs from MISP into a Splunk index.
In order to use these IOCs for detection either as lookup or in Splunk Enterprise Security, the App provides some reports to generate IOC lookup-tables.
These lookup-tables are compatible with the Threat Intelligence Framework of Splunk Enterprise Security.

You can read the latest version of this readme on Github.

Configuration

Before the App can be used, the MISP instance must be configured. The configuration can be found under App Settings -> Configuration.

Accounts

At least one instance must be configured.

In App Settings -> MISP App Settings a default instance can be set (maybe a browser refresh is necessary if the instance is recently configured). This instance is used per default for all custom commands and for the alert action if no instance is specified.

Importing IOCs into Splunk

The App provides two modular inputs for importing MISP attributes/IOCs and MISP events.
Both inputs are support pulling the MISP data in batches, which should avoid HTTP request limits or memory limits.
Additional these inputs uses checkpoints, so the import process starts where the last execution has stopped.
To restart the input from the beginning, you have to manually clean the checkpoint using splunk clean inputdata <input_name> as documented here: Set-up-checkpointing.
The inputs can be configured in the App UI under App Settings -> Inputs.

MISP Event Input

The MISP Event Input imports MISP event data. The Events are imported regarding their publish timestamp, so it may be not possible to import unpublished events.

MISP Indicator Input

The MISP Indicator Input imports MISP indicators. The Indicators are imported regarding their events publish timestamp, so it may be not possible to import indicators from unpublished events.

[!NOTE]

Before indicators are ingested, the events are queried by publish timestamp and for each events all attributes are pulled (filters may apply). If one Event, from which the attributes are already ingested, is published another time, all attributes will also be ingested another time, regardless of their attribute timestamps.

Setup Examples

To have an up to date IOC list, it is possible to setup the indicator input using a small import interval, like 5 minutes (300 seconds). Then schedule the provided reports also by frequent schedule, maybe 10 minutes. This will keep your IOC lookup-tables up 2 date.

By default the provided reports use a linear decaying score starting by 100 and decaying over 180 days (100 for hashes). This calculation can be modified direct in the report or also in the misp_decaying_scores.csv lookup-table for specific tags or organisations. It is also possible to set a static value. Due to this implementation the weight is calculated each time when the report is scheduled. IOCs where the score is zero or lower, are ignored.

[!IMPORTANT]

To perform efficient weight calculation based on specific tags, the tag field in Splunk must not be a multi value field. This means Expand Tags must be activated, which will create an event for each tag. This will result in a significant increase of Splunk events, but it is necessary to avoid mvexpand which has a limit of events.

False Positives and Critical IOCs

It is possible to handle false positives by tagging them in MISP and set the weight for this tag to zero using misp_decaying_scores.csv lookup-table. In this case it is important, that these Attributes stay in Splunk continuously, which also may apply for critical IOCs. This can be achieved by adding another indicator input, which filters the tags for false positives and critical iocs. The input should be scheduled once a day, shouldn't use continuous importing and should override the attribute timestamps. With this configuration these indicators are ingested each days with the actual timestamp and are active as long the tag exists.

Commands

Search Attributes

Search for MISP attributes on a MISP instance using the MISP API.

Queries a list of MISP attributes and provides filter and data normaization features. It is possible to filter tags, events, values, timestamps, to_ids etc. and to normalize the output using normalize_fields (enabled by default).

Syntax

which makes app maintenance much easier.

| mispsearchattributes (misp_instance=<string>)? (limit=<int>)? (normalize_fields=(t|f))? (publish_date=<YYYY-MM-DD>)?
| mispsearchattributes (published=(t|f))? (include_context=(t|f))? (value=<string>)?

For all supported parameters see search_attributes_command.py

Search Events

Search for MISP events on a MISP instance using the MISP API.

Searches for all MISP events which include the given ioc value.
You may specify the misp instance with "misp_instance" parameter, otherwise the configured default_instance is used.

Syntax

| mispsearchevents (misp_instance=<str>)? (ioc=<str>)?

For all supported parameters see search_events_command.py

Alerts

Add Sighting

Adds a sighting to MISP Attribute by its value.

Lookups / Weight calculation

The App provides reports for lookuptable generation by category:

• MISP_TI_Domain_IOCs -> MISP_TI_Domain_IOCs.csv
• MISP_TI_URL_IOCs -> MISP_TI_URL_IOCs.csv
• MISP_TI_Email_IOCs -> MISP_TI_Email_IOCs.csv
• MISP_TI_HASH_IOCs -> MISP_TI_HASH_IOCs.csv
• MISP_TI_IP_IOCs -> MISP_TI_IP_IOCs.csv

These reports generates lookuptables which have the required fields for the Splunk Threat Intelligence Framework. The weight is calculated using a linear decaying function $100 * (1-\tfrac{age(days)}{DecayLifetime(days)}^\tfrac{1}{DecaySpeed(default:1)})$ , which is linear decreasing over the lifetime. If the decaying behavior should be changed, the reports must be changed. The weight can also be affected by the misp_decaying_scores.csv lookuptable. In this table it is possible to specify static weights or decaying configurations (dynamic) for tags (Expand Tags must be enabled in input) or creator organizations (id). Type 'static' uses the weight column and type 'dynamic' calculates a score based on DecayLifetime and DecaySpeed.

For more information about the Splunk Enterprise Threat Intelligence Framework see:

• Using threat intelligence in Splunk Enterprise Security
• Threat Intelligence framework in Splunk ES

Build

The app is developed using the Splunk Add-On UCC Framework. To build it the following commands can be used:

pip install splunk-add-on-ucc-framework splunk-packaging-toolkit
ucc-gen build
slim package output/TA_misp

Thanks to CIRCL

Many thanks to CIRCL for maintaining MISP, providing it for free, merge most of my pull requests and for the permission to use their logo for this app.

Thanks to Splunk

Many thanks to Splunk for developing the Splunk Add-On UCC Framework, which makes app maintenance much easier.