Data Onboarding Checklist
Overview
Details
"Garbage in, garbage out" is an important concept to keep in mind when working with data in Splunk. This phrase emphasizes that the quality of the input data has a direct impact on the accuracy and usefulness of the results obtained from Splunk. If low-quality, wrong-parsed or incomplete data is used as input, it can result in inaccurate or unreliable output. The Data Onboarding Checklist App provides required steps and assists you during Get-Data-In-Process.
This is a beta release. Send your feedback, corrections and suggestions to splunk@compek.net
Onboarding Checklist helps to onboard sourcetypes using established best practices, avoid common pitfails and validate data.
"Garbage in, garbage out" is an important concept to keep in mind when working with data in Splunk. This phrase emphasizes that the quality of the input data has a direct impact on the accuracy and usefulness of the results obtained from Splunk. If low-quality, wrong-parsed or incomplete data is used as input, it can result in inaccurate or unreliable output. The Onboarding Checklist provides required steps and assists you during Get-Data-In-Process.
Go through these steps from top to bottom. Click on each step to perform validation and read additional information.
| Check | Expected Result |
| Source: Timestamp and Timezone | Timestamp and Timezone are correct, there are no "future" events. |
| Indexer: Timestamp and Timezone | Timestamp and Timezone are correct, there are no "future" events. |
| Logging delay | There are no significat logging delays. |
| Indexer: Timestamp Recognition | Timestamp parsed correctly, there are no "defaulting to previous" . |
| Index is explicitly defined | Correct index is used. Nothing in "main" or "lastchance". |
| Sourcetype is explicitly defined | Correct sourcetype. |
| Host extraction | Make sure host extracted or set correctly. |
| Integrity | All events reach Splunk, no events are lost. |
| Integrity (network interruptions) | Short network interruptions shouldn't lead to a loss of events. |
| Secure Transfer | TLS, certificate validation, mTLS |
| Truncation | Long events aren't truncated. |
| Multiline for single-line-events | There are no multiline events for single-line sourcetypes. |
| Linebreaking of multiline events | Multiline events are splitted correctly. |
| Duplicates | There are no duplicate events. |
| Field Extraction | Events parsed correctly, fields are extracted. |
| Setting location | All settings are placed inside of the respective App/TA. There are no sourcetype related configuration settings in system/local or in unrelated apps/TAs. |
| Magic 8 | All of the "Magic 8" configurations are explicitly defined. |
Some tips:
- Start with validation and fixing timestamp issues, before continuing with other steps.
- Unless all timestamp issues are fixed, set the time picker's latest time in the future, for example +1d - this allows to include events with the wrong timestamp.
- Most queries require access to the _internal index.
Release Notes
Version 0.0.7
added integrity testing scripts and SPL searches
Version 0.0.6
improved timestamp error detection, minor fixes
Version 0.0.5
first public release, consider it beta