Outcold Solutions provide solutions for monitoring Kubernetes, OpenShift and Docker clusters in Splunk Enterprise and Splunk Cloud. We offer Splunk applications, which give you insights across all containers environments. We are helping businesses to reduce complexity related to logging and monitoring by providing easy-to-use and deploy solutions for Linux and Windows containers. We deliver applications to help developers monitor their applications and operators to keep their clusters healthy. With the power of Splunk Enterprise and Splunk Cloud, we offer a unique solution to help you keep all the metrics and logs in one place, allowing you to quickly address complex questions on container performance and cluster health.
We provide solutions for monitoring Kubernetes, OpenShift and Docker clusters in Splunk Enterprise and Splunk Cloud. With 10 minutes setup, you will get a monitoring solution, that includes log aggregation, performance and system metrics, metrics from the control plane and application metrics, a dashboard for reviewing network activity, and alerts to notify you about cluster or application performance issues.
All our solutions are powered by the Collectord, a container-native software built by Outcold Solutions that provides capabilities for discovering, transforming and forwarding logs, collecting system metrics, collecting metrics from the control plane of the orchestration frameworks and forwarding network activity. Collectord provides flexible and powerful tools for transforming logs. With our software you can hide sensitive information from the loglines before forwarding them. With Collectord you can reduce the licensing costs associated with logging aggregation by choosing which data you want to forward from the log streams. Collectord forwards container logs, host logs and can discover logs written by the containerized applications.
See detailed metrics from containers and processes, including performance metrics, utilization metrics and security insights. Forward application-specific metrics, exported in Prometheus format. Use prebuilt Splunk dashboards for a comprehensive overview.
Aggregate logs from containers, applications, and servers. Use flexible mappings to filter logs enriched with container metadata, correlate logs with metrics, and leverage Splunk capabilities for analyzing logs. Use Collectord to transform logs before they reach Splunk, remove sensitive information, remove PII data to help keep your logs GDPR compliant. With Collectord you can reduce licensing and storage costs by choosing which loglines you want to forward.
Diagnose cluster issues by looking at historical events, monitoring allocations, and regulating cluster capacity. Leverage pre-built alerts for monitoring the health of the clusters out of the box.
Define access to the data by clusters, namespaces and even pods or containers. Review network activities, happening inside your cluster, and outside connections. Verify containers running with elevated security permissions. Use audit logs for monitoring changes in deployments.
Use one tool to collect and forward logs and metrics required by developers for reviewing performance and health of their applications. With the annotations developers can define how they want to see the data in log aggregation tool, specify multiline log patterns, removing terminal escape codes, override types, sources and indexes.
Supports collectorforopenshift version 5.24.x and below (see https://www.outcoldsolutions.com for latest configuration)
- New alert: Cluster Warning: Node Condition
- Bug fix: Workload dashboard might show a warning for the Events table telling about using wildcard in the middle of the string
Collectord updates:
- Upgrade golang to 1.24.2.
- Upgrade SQLite to 3.48.0.
- Added ability to hide process command line arguments
- Prometheus metrics can be forwarded to Splunk Metrics Index.
- Allow configuring TLS Version and show TLS version used for outputs using the collectord verify command
- To make watchImplementation=2 compatible with previous version, attach the Kind and apiVersion to objects forwarded from the list calls.
- Collectord verify command shows logging driver configured for Docker Daemon
- Added ability to parse unixtimestamp in the application logs with format @unixtimestamp
...
Supports collectorforopenshift version 5.23.x and below (see https://www.outcoldsolutions.com for latest configuration)
- To better support installations with large number of nodes and containers, default behavior for most of the dashboards
is to require pressing a Submit button after selecting filters.
- Overview Dashboard - new table with Not Ready Containers.
- Pod Dashboard - include container statuses table.
- Audit Dashboard - include user agent, and update compatibility with latest audit formats.
- Audit Dashboards - small performance improvement for the new installations.
- Host dashboard - show node conditions table.
- Host dashboard - show only external eht* interfaces in network stats.
Collectord updates:
- Implement new and improved watch mechanism for Kubernetes resources to handle large clusters.
- Change the default pipe join configuration to have max size of 1MB instead of 100KB.
- Allow ...
Collectord updates:
Supports collectorforopenshift version 5.22.x and below (see https://www.outcoldsolutions.com for latest configuration)
- Workload dashboard - add Pod OwnerKind and OwnerName, PriorityClass, and Pod Requests/Limits
- Address too many data points in host and workload dashboard in network graphs
- Additional CPU Metrics: CPU IOWait, Steal and Idle in Top Hosts dashboards.
- Showing CPU IOWait in Host dashboard.
- Alert Container CPU Throttled - exclude container with low CPU usage.
- New dashboard Review->Disk Stats for the host.
- Exclude virtual ethernet interfaces from host dashboard.
- Support memory limits and requests expressed in milli-bytes.
Collectord updates:
- Allow disabling IP address Lookup in net_socket_table input.
- Better handling of zombie processes in proc_stats input.
- Allow configuring user Splunk outputs using CDR SplunkOutput.
- Allow blacklisting labels from forwarded metadata.
...
Supports collectorforopenshift version 5.21.x and below (see https://www.outcoldsolutions.com for latest configuration)
- Compatibility updates for the version 5.21 of Collectord
- New Dashboard: Review -> CPU (Throttled, Limits, Requests)
- Alert update: High number of GRPC errors
- Alert update: Container CPU Throttled
- Network tables update: show UDP connections for Host, Workloads, Containers, and Pods
- Network Connection Dashboard: allows filtering by namespaces
- Show maximum and average number of Pods per cluster in Clusters (Allocations and usage) dashboard
- Update Cluster Resource Quota and Resource Quota dashboards to support comparing milli-cores and cores
Collectord updates:
- Support for global replace configurations for Collectord, allowing to sanitize data before forwarding to Splunk
- Support journald as logging driver for container logs
...
Supports collectorforopenshift version 5.20.x and below (see https://www.outcoldsolutions.com for latest configuration)
- New dashboard: Review - Cluster Resource Quotas
- Show Pod conditions on the Pod dashboard
- Bug fix: Pods dashboard filters out pods not on the host network.
- Compatibility updates for the version 5.20 of Collectord
Collectord updates:
- Multi-architecture images for amd64 and arm64
- Allow sending logs to multiple Splunk HEC endpoints simultaneously
- New annotation collectord.io/volume.{N}-logs-onvolumedatabase to keep acknowledgement information about forwarded logs on the volume
- Allow including placeholder templates in the annotation collectord.io/volume.{N}-logs-glob
- Support for new outputs (ElasticSearch and OpenSearch)
- Collectord produces diag file without performance data, if flag --include-performance-profiles is not set
...
Supports collectorforopenshift version 5.19.x and below (see https://www.outcoldsolutions.com for latest configuration)
- Create scheduler dashboard (and move those metrics from controller dashboard)
- Update dashboards for latest changes in the metric names for API Server, Controller and Scheduler
- Update Kubelet dashboard to support various container runtimes
- Audit (users and projects) dashboard: show access to non-projects resources
- Logs dashboard: show container and pod as separate filters
- New alert for Collectord alarms for node diagnostics (reboot required, and entropy)
- Bug fix: hosts dashboard does not filter events by host
Collectord updates:
- Splunk output supports maximumMessageLength to truncate messages exceeding this size
- Splunk output supports requireExplicitIndex to ignore all events that don't have explicit index defined
...
Collectord updates:
- Support cgroupv2
- New ability to specify the message field name for the logs extraction with annotations extractionMessageField
- Collectord improves grace period for expired licenses allowing to bootstrap new nodes for 14 days
- Support of journald database written with systemd library 247+
- Upgrade go runtime to 1.17.9
- Bug fix: cleanup the diag, exclude the real license key
- Bug fix: collectord reports high CPU usage for just started containers or hosts
- Bug fix: update pods/container labels when user updates them (prior restart was required)
- Bug fix: set now as a date for container logs with corrupted log files instead of 0 timestamp
- Bug fix: include the values of whitelists and blacklists in diag
...
5.17.370 - 2021-10-20
- Show milicores/cores CPU usage instead of percents
- New dashboard: Review - Resource Quotas
- Review - Projects: filter by project name
- Review - Clusters: filter by node label
- Review - Clusters: include max and avg usage
- Bug fix: storage dashboard might not render in some Splunk versions
- Bug fix: Projects dashboard shows only one namespace label
Collectord updates:
- Upgrade to Go 1.17.2
- Support query in Prometheus URLs for metrics
- Collectord now reports source and source type for the events with incorrect index
- Support for licensing server
- Allow to specify multiple values for blacklist and whitelist for host logs
- Support for CPU-based licenses
- Bug fix: Collectord clogs the output with WARN messages for stopped containers running with Containerd
...
Requires collectorforopenshift version 5.16.363 or above (see https://www.outcoldsolutions.com for latest configuration)
Collectord updates:
Collectord updates:
5.16.360 - 2021-03-16
Collectord updates:
Collectord updates:
\0 symbol from the labels values in the prometheus metricsRequires collectorforopenshift version 5.15.300 or above (see https://www.outcoldsolutions.com for latest configuration)
Collectord updates:
Collectord updates:
Requires collectorforopenshift version 5.12.271 or above (see https://www.outcoldsolutions.com for latest configuration)
Collectord updates:
- Bug fix: when event pattern is used for joining multi-line events, the error can not be showed if raised by the input in pipeline.
- Bug fix: reduce warnings failed to get the new event in pipeline - submitted
- Stability improvements
Collectord updates:
Requires collectorforopenshift version 5.11.260 or above (see https://www.outcoldsolutions.com for latest configuration)
Collectord updates:
Collectord updates:
Requires collectorforopenshift version 5.9.240 or above (see https://www.outcoldsolutions.com for latest configuration)
Collectord updates:
Requires collectorforopenshift version 5.8.230 or above (see https://www.outcoldsolutions.com for latest configuration)
Collectord updates:
Requires collectorforopenshift version 5.7.220 or above (see https://www.outcoldsolutions.com for latest configuration)
Collectord updates:
Requires collectorforopenshift version 5.6.212 or above (see https://www.outcoldsolutions.com for latest configuration)
Read more https://www.outcoldsolutions.com/docs/monitoring-openshift/release-history/
Requires collectorforopenshift version 5.5.202 or above (see https://www.outcoldsolutions.com for latest configuration)
Collectord updates:
- Fixed: Interval 0 in Prometheus input can crash the collectord.
- Fixed: When both glob and match are set for the application logs, the glob pattern can block the match pattern from
finding the files in the volume.
Requires collectorforopenshift version 5.4.201 or above (see https://www.outcoldsolutions.com for latest configuration)
Collectord updates:
- Fixed: Better handling rotated files (less open fd)
- Fixed: Events input can hang in the err loop.
Requires collectorforkubernetes version 5.4 or above (see https://www.outcoldsolutions.com for latest configuration)
Collectord updates:
- New: Attach EC2 metadata fields
- New: Basic Auth for Proxy (License Server and Splunk)
- Fixed: Collectord verifies reports CRI-O as unsupported runtime.
- Fixed: Rare crash on Prometheus metrics definition.
- Fixed: Better handling of acknowledgment database corruption.
- Fixed: When handling incorrect indexes, collectord can send index with an empty string, that Splunk recognize as an incorrect index
Requires collectorforopenshift version 5.3 or above (see https://www.outcoldsolutions.com for latest configuration)
https://www.outcoldsolutions.com/docs/monitoring-openshift/release-history/
5.2.180 - 2018-10-28
- Fixed: lookup with alerts causing very often replication activities on SHC
5.2.179 - 2018-10-17
- Fixed: changed search time for few alerts that cause false positives with indexing lag on large installations
Requires collectorforopenshift version 5.2 or above (see https://www.outcoldsolutions.com for latest configuration)
For details https://www.outcoldsolutions.com/docs/monitoring-openshift/release-history/
5.2.179 - 2018-10-17
- Fixed: changed search time for few alerts that cause false positives with indexing lag on large installations
Requires collectorforopenshift version 5.2 or above (see https://www.outcoldsolutions.com for latest configuration)
For details https://www.outcoldsolutions.com/docs/monitoring-openshift/release-history/
Requires collectorforopenshift version 5.2 or above (see https://www.outcoldsolutions.com for latest configuration)
For details https://www.outcoldsolutions.com/docs/monitoring-openshift/release-history/
https://www.outcoldsolutions.com/docs/monitoring-openshift/release-history/
Highlights:
- Application logs
- Annotations for fields extraction, hiding sensitive information, time extraction, redirecting to /dev/null, stripping terminal colors and more
For more details:
https://www.outcoldsolutions.com/docs/monitoring-openshift/release-history/
3.0.23 - bug fixes release
3.0.22
New overview, security and capacity dashboards. Workload aggregation dashboard.
A lot of of bug fixes and performance improvements.
Relese Notes: https://www.outcoldsolutions.com/docs/monitoring-openshift/release-history/#30-2018-02-07
Upgrade instructions: https://www.outcoldsolutions.com/docs/monitoring-openshift/upgrade-2-to-3/
Requires collectorfor openshift version 3.0 or above (see https://www.outcoldsolutions.com for latest configuration)
New overview, security and capacity dashboards. Workload aggregation dashboard.
A lot of of bug fixes and performance improvements.
Relese Notes: https://www.outcoldsolutions.com/docs/monitoring-openshift/release-history/#30-2018-02-07
Upgrade instructions: https://www.outcoldsolutions.com/docs/monitoring-openshift/upgrade-2-to-3/
Requires collectorfor openshift version 3.0 or above (see https://www.outcoldsolutions.com for latest configuration)
Requires collectorforopenshift version 2.1.62.171219 or above
Monitoring Openshift 2.1
Requires collectorforopenshift version 2.1.62.171219 or above
Monitoring Openshift 2.1
Requires collectorforopenshift version 2.1.62.171219 or above
As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps from Splunk, our partners and our community. Find an app for most any data source and user need, or simply create your own with help from our developer portal.