icon/x Created with Sketch.

Splunk Cookie Policy

We use our own and third-party cookies to provide you with a great online experience. We also use these cookies to improve our products and services, support our marketing campaigns, and advertise to you on our website and other websites. Some cookies may continue to collect information after you have left our website. Learn more (including how to update your settings) here.
Accept Cookie Policy

We are working on something new...

A Fresh New Splunkbase
We are designing a New Splunkbase to improve search and discoverability of apps. Check out our new and improved features like Categories and Collections. New Splunkbase is currently in preview mode, as it is under active development. We welcome you to navigate New Splunkbase and give us feedback.

Accept License Agreements

This app is provided by a third party and your right to use the app is in accordance with the license provided by that third-party licensor. Splunk is not responsible for any third-party apps and does not provide any warranty or support. If you have any questions, complaints or claims with respect to this app, please contact the licensor directly.

Thank You

Downloading Datalake2Splunk
SHA256 checksum (datalake2splunk_102.tgz) d6526caf59cf86a3ff50d2fc429616de7c0bfd2ab3852f4fcbbf5e352b5faa44 SHA256 checksum (datalake2splunk_101.tgz) 860d5e3c07d9e8191d1bb77e68331295d7b730c89dd5e81c277eff20daf8518c SHA256 checksum (datalake2splunk_100.tgz) b718b32c24e5d2482ae8d9769fb55eb17afd564703e5bd6b2ef3b77d6c6cba71
To install your download
For instructions specific to your download, click the Details tab after closing this window.

Flag As Inappropriate

splunk

Datalake2Splunk

Splunk Cloud
Overview
Details
This app provides the ability to ingest Orange Datalake threat intel given an api token.
It replaces former app Datalake Connect (https://splunkbase.splunk.com/app/5463) to be able to pull data using a query hash obtained from Datalake thus having all filters available.

Datalake2Splunk allows you to get Datalake data into Splunk for Orange Cyberdefense Datalake customers.
Datalake2splunk modular input builds a bulk-search using defined parameters and contacts Datalake API. It transforms the received JSON into a csv used as a lookup or can index Data into a custom index.
The IOC are available by using the lookups. Threat matches can be found by correlating with events log. Enterprise Security Threat Intelligence automates the threat matches job.

Prerequisites

  • This app has been developed and tested with Python 3.7.17 so the minimal Splunk version needed is 9.1.
  • Internet access.
  • Bulk-search account on Datalake.

General Usage

Configuration

  • Account: you can whoose an name for the account and enter your datalake account name, PASSWORD field must contain your Datalake API token generated in your profile/ Long Term Tokens
  • If needed configure Proxy and provide your proxy configuration in order to allow internet access to Datalake2Splunk

Input

o Name: a name for your input
o Interval: time in seconds between 2 gets. The process can take a while depending on the data volume, so we do not recommend under 1800.
o Index: If there is a need to index or if it is Splunk Cloud Classic Experience choose the right index otherwise whatever index you choose, no data will be indexed from this input. This is just a splunk requirement.
o Query Hash: enter your datalake query hash obtained via the URL /gui/search?query_hash=<>
o Token: choose which account (created in configuration) will be used to get Datalake information.
You’ll need to create one input per type and can have different Threat_Scores or Last Updated parameters for each Type. Each input will lead to the creation of one distinct lookup file named after ioc type.
On Splunk Cloud Classic experience, the data inputs are managed on the Input Data Manager which is not the Search Head. It means the standard data collection method described in the part “1. How it works” can’t work. An alternative mechanism has been built.
o Storage Type: choose if the IOCs should be indexed. Required for Splunk Cloud Classic Experience

Splunk Cloud Classic Experience

IOCs are first downloaded by the IDM and then sent to the Indexers to be indexed. When indexed, saved searches have to be enabled on the Search Head to aggregate the IOCs in lookups. Depending on what IOC needs to be used in lookups, the following search can be enabled:
- Datalake - Build Datalake <ioc_type> lookup
In addition, the macro datalake-index has be to configured with the index name where the IOCs are stored.

Use Datalake Intelligence

Lookups containing your regularly downloaded data are located under $SPLUNK_HOME/etc/apps/TA-datalake2splunk/lookups and named:
datalake_<ioc_type>.csv
All Lookup Definitions have been previously created and included in TA-Datalake2splunk (transforms.conf) with read permission for other apps.

with Enterprise Security

From Enterprise Security main page click Configure -> Data Enrichment -> Threat Intelligence Management -> Sources
Click New and local to create a new intelligence entry and fill as follow:
- Name and Description: whatever name you choose
- Type: threatlist by default, (you can put “malware” type for datalake_file)
- URL is local path to lookup definition so lookup://datalake_<type>
- Weight is a risk_score multiplier. 1 to have no impact on assets and identities.
- Interval (in seconds) no need to have a smaller interval than what’s configured in the input. This control the frequency at which ES will update Intelligence KV Stores.
- Use Maximum age to control when splunk will delete the data from this lookup in Threat_Intelligence KV Stores. If you have very frequent pull, maximum age should remain low.
- Keep “Is Threat Intelligence” checked

This must be done for each type accepted by ES: ip, ip_range, domain, fqdn, url, file, email, ssl, regkey.
Next step is to check if ES has ingested the data.
There is a KV store for each type.
ip_intel for domain and ip, email_intel for email, http_intel for url, file_intel for file, certificate_intel for ssl and registry_intel for RegKey.

Enrich

A lot of data is stacked in “description” field for this is the only way to add value to each line and will be parsed during Matching searches in order to add value to Threat Activity data later. You can use macro datalake_parser(description) to extract the data.
If you want to fully exploit collected data, you might want to modify the on-boarded Threat Activity search.
From Enterprise Security main page
Filter Correlationsearch in DA-ESS-Threat_Intelligence App and look for “Threat Activity Detected”
Edit the Search field
At the end add:

| `datalake_parser(threat_description)`

Without ES

Intelligence data is searchable by any app.
You can call data thanks to the lookup_definitions. Such as

|inputlookup datalake_domain
|inputlookup datalake_url 

Now you’ll need to correlate your events with your logs. Either by using lookup command or inputlookup and check for a match.
You can easily get detailed data and correlate by using macro datalake_parser

Reports and Dashboards

Datalake IOC Stats offers insight on the content of the lookups such as most frequent threat types or sources. This is the base work to have a macro overview over IOC collection and a Splunk developer can edit those dashboards.

Dashboards are in development

Release Notes

Version 1.0.2
Oct. 28, 2024

Corrected savedsearches.conf
Added Readme.txt

Version 1.0.1
Oct. 28, 2024

Macros, Savedsearches and Dashboard added
Props for Datalake indexing
Lookup definitions for automatic lookups

Vetting changes for Splunk Cloud

Version 1.0.0
Oct. 22, 2024

Subscribe Share

Are you a developer?

As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps from Splunk, our partners and our community. Find an app for most any data source and user need, or simply create your own with help from our developer portal.

Follow Us:
Splunk, Splunk>,Turn Data Into Doing, Data-to-Everything, and D2E are trademarks or registered trademarks of Splunk LLC in the United States and other countries. All other brand names,product names,or trademarks belong to their respective owners.