IPURL IOC Ingestion
Overview
Details
Am working to fix the issue related to it not working on a Cluster environment.
This Add-on collects IPs, URLs and Domains from well known Open-source websites that can be used by Threat Intelligence analysts or Cyber Security Centres for better correlations of their use cases or searches. It is needed by any security team that do not use MISP and need to retrieve open source IPs, URLs and Domains.
The Add-on downloads IPs, Domains, URLs and Phishing Domains from Proof Point IP blocklist, Abuse CNC blocklist, URLHAUS, OpenPhish, DigitalSide Threat-Intel repo FQDN domains, Mitchell Krogza Github phishing domain lists and Romain Marcoux Github phishing domain lists..
All these lists are cleaned and placed into a CSV file that can be used for correlation after the user has created a new input after the installation.
NOTE: This add-on works properly only on a Single Instance of Splunk Cloud and Splunk Enterprise. It does not work on a Cluster environment.
Am working to fix this issue.
This Add-on does not add any logs to Splunk license because it does not ingest the logs into any index.
The logs are filtered by it and properly formatted into csv files that are named on the add-on.
It is built for security teams that do not have MISP installed but need to ingest IOCs that can be used for use case correlation or for threat hunting. It does not have all the IPs, and URLs but it has some that can help any security team from reputable open source websites such as Abuse.ch and Proofpoint firewall block IPs
Release Notes
Version 1.0.6
- Added more IOCs sources from reputable sites such as OpenPhish urls, DigitalSide Threat-Intel repo FQDN domains, Mitchell Krogza Github phishing domain lists and Romain Marcoux Github phishing domain lists.
- Removed the default clicked action. Users will need to check the button to start collecting IOCs when creating the input.
Version 1.0.4
- Removed Cisco Talos IP Blocklist since it is a little bit difficult to download using the basic python script
- Made the add-on compliant with Splunk Cloud 9.
Version 1.0.2
New feature for version 1.0.2
-
Added a search named ioc-opensourceIP that combines all the IPs into one CSV file and removes duplicate IPs from the lists. This search runs once daily and replaces the old csv file with a new one every time it runs.
-
In order for the search which is located on the IPURL IOC Ingestion app to run, it needs to be enabled.
Note: When it runs the first time, it produces this error "The lookup table 'ioc-opensourceIP.csv' requires a .csv or KV store lookup definition".
That is what happens in Splunk when the outputlookup command is used in a search and needs to create a new file that was not previously created. Refresh to see the results and since the new csv file is created. It should run smoothly.
Version 1.0.0
This Addon downloads IPs and URL from Proofpoint IP blocklist, Cisco Talos Snort blocklist, Abuse CNC blocklist and URLHAUS. All these lists are cleaned and placed into a CSV file that can be used for correlation.
A search will be needed by the security team to combine these IPs into one CSV file to remove duplicates from all these sources on version 1.0.0.
Note: In order for the Add-on to start downloading new IOCs, a new input will need to be created after the installation.
As regarding Splunk Cloud, this Add-on should be installed on Splunk Cloud IDM but the new input should be created on the Splunk Cloud Search Head in order for the CSV files to be properly used in the correlation.
As regarding Splunk Enterprise, this add-on can be installed on either the Indexer or Search head depending on where the correlation needs to be done.