This Splunk app provides custom source types and views into the status of Puppet installations that are configured to send reports, facts and metrics with the splunk_hec, puppet_metrics_collector and pe_status_check Puppet modules.
You can take action on this data by connecting a Splunk installation to the PE Orchestration service via the Puppet Alert Orchestrator add-on for Splunk.
Once the application has been installed follow the steps below to configure the Puppet Report Viewer:
Create an Splunk HEC token for the app:
Settings > Data Input.HTTP Event Collector token with a name of your choice.indexer acknowledgement is not enabled.main indexmain.
After configuring splunk_hec, puppet_metrics_collector, and pe_status_check the Overview tab will start showing data from Puppet reports, while the PE Metrics tab will start displaying graphs related to a number of useful Puppet metrics and results of status checks.
This app includes the following custom source types:
Data from puppetlabs-splunk_hec module:
puppet:factspuppet:summaryData from puppetlabs-puppet_metrics_collector module:
puppet:metricsData from puppetlabs-pe_event_forwarding module:
puppet:activities_classifierpuppet:activities_code_managerpuppet:activities_consolepuppet:activities_rbacpuppet:activitypuppet:events_summarypuppet:jobsData from Puppet Alert Orchestrator add-on for Splunk:
puppet:actionpuppet:boltpuppet:detailedData from Puppet infrastructure nodes via Splunk Forwarder:
puppet:service_logspuppet:access_logsNote: Access and Service logs for the following Puppet services are currently supported.
All dashboard views support using custom indexes for storing event data. The searches assume each sourcetype can be stored in it's own index. There is one top level macro (puppet_index) which defaults to "".
As such, you can specify different HEC tokens for Summary Reports, Facts, and Metrics in splunk_hec. If you configure multiple HEC tokens to utilize different indexes, change the index value in the dashboards to reflect this.
Create five indexes named puppet_data, puppet_summary_data, puppet_facts_data, puppet_metrics_data, and puppet_detailed_data.
Create five HEC tokens:
puppet_data with sourcetype of automatic and the index puppet_data.puppet_summary with sourcetype of puppet:summary and the index puppet_summary_data.puppet_facts with sourcetype of puppet:facts and the index of puppet_facts_data.puppet_metrics with sourcetype of puppet:metrics and the index of puppet_metrics_data.puppet_detailed with sourcetype of puppet:detailed and the index of puppet_detailed_data.Configure the splunk_hec module with the following token parameters:
splunk_hec::token with the puppet_data token value.splunk_hec::token_summary with the puppet_summary token value.splunk_hec::token_facts with the puppet_facts token value.splunk_hec::token_metrics with the puppet_metrics token value.Update the search macros to use the new values:
Settings > Knowledge.Search Macros.puppet_index and change the definition to index=puppet_data, click save.puppet_summary_index and change the definition to index=puppet_summary_data, click save.puppet_facts_index and change the definition to index=puppet_facts_data, click save.puppet_metrics_index and change the definition to index=puppet_metrics_data, click save.puppet_detailed_index and change the definition to index=puppet_detailed_data, click save.Upon reloading the Overview tab in the Puppet Report Viewer app, and you should begin seeing data. Alternatively, perform the following search:
`puppet_all_index` sourcetype=puppet:*
Top API Calls
`puppet_logs_index` sourcetype=puppet:access_logs source=*puppetserver-access.log
| rename request as Endpoint
| chart avg(elapsed_time) as "Time (ms)" by Endpoint
| sort - "Time (ms)"
Longest Compile Time
`puppet_logs_index` sourcetype=puppet:access_logs source=*puppetserver-access.log method=POST request=/puppet/v3/catalog*
| chart max(elapsed_time) as "Time (ms)" by client
| sort - "Time (ms)"
Longest Query Time
`puppet_logs_index` sourcetype=puppet:access_logs source=*puppetdb-access.log request=/pdb/query/*
| chart max(elapsed_time) as "Time (ms)" by client
| sort - "Time (ms)"
Sync Issues
`puppet_logs_index` sourcetype=puppet:service_logs service=sync log_level=WARN OR ERROR
| chart values(message) as Message by log_level
If the Puppet Report Viewer does not appear to show any data after you have followed the configuration steps for both this app and the splunk_hec module; first check that data is being successfully sent to the Splunk server by following the troubleshooting and verification steps in the splunk_hec documentation.
If events in the puppet:detailed source type is not showing up in search, it means that the "Generate a Detailed Report" Alert is not configured properly with the Puppet Alert Orchestrator add-on. If this Alert is enabled, and the aforementioned add-on is configured, you can view the logs with the following Splunk search:
index=_internal sourcetype=splunkd component=sendmodalert (action="puppet_run_task_investigate" OR action="puppet_run_task" OR action="puppet_run_task_act" OR action="puppet_generate_detailed_report")
There is also a view into the Alert Actions logs under the Actions tab which will show these searches as well.
If there are no error messages, verify that the configured HEC works can be used to submit a report manually:
$ curl -k -H "Authorization: Splunk <yourHECtoken>" https://localhost:8088/services/collector/event -d '{"sourcetype": "puppet:detailed", "event": "exampletest"}'
The endpoint will respond with either a success or an error message. Follow steps in the HTTP Event Collector documentation to resolve issues with the HEC endpoint.
If there are other error messages in the logs related to PE RBAC tokens from the Puppet side, run the following command to query the PuppetDB API:
curl -k 'https://<your.puppetdb.server>:8081/pdb/query/v4/nodes' -H "X-Authentication: <token contents>"
Fixes:
New Features:
puppet:access_logspuppet:service_logsFixes:
Breaking Changes:
New Features:
by host for comparison view.puppetlabs-pe_status_check module.Fixes:
New Features:
Fixes:
New Features:
puppet:activities_console, and puppet:activities_code_manager source types added.Fixes:
AUTO_KV_JSON setting to allow for the default value (true); JSON parsing now occurs at search-time.New Features:
3.0.3:
New Features:
- puppet:jobs, puppet:activities_rbac, and puppet:activities_classifier sourcetypes added.
Fixes:
- Some of the panels in the Overview dashboard still contain the "X" button in the upper right to close the pop-up panel when you click on the primary panel. Some of the other panels lost the X.
Changed the drilldown to set/unset the token that shows the drilldown panel on click. Effect of the change is that clicking on the panel with the drilldown hidden shows the drilldown. Clicking on the panel with the drilldown showing hides the drilldown.
Also removed the remaining "X" buttons.
Standardize some visual formatting - moved "units" to "captions" on the images to "hosts, seconds, etc" shows up underneath the reported numbers, rather than next to them, removed odd height settings.
New Features:
- puppet:events_summary and puppet:activity sourcetypes added.
Fixes:
- The searches that are used in the Overview tab to display resources under the "# of Resources" element and associated table used to display the number of reports. They have been corrected to display the number of resources.
This is a Viewer only release Puppet Report Viewer, it does not contain any custom alert actions and upgrading to this will require also installing the new Puppet Alert Actions App.
As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps from Splunk, our partners and our community. Find an app for most any data source and user need, or simply create your own with help from our developer portal.