How Our Customers Recovered 50% Of Their Splunk Cost

Introduction

A customer of ours once challenged the log reduction volumes we were able to achieve using Cribl (Basic Statistics). So we came up with the idea of verifying the results using their Splunk dashboards.

To get Splunk power and satisfy finance people, security professionals always struggle to come up with the ideal balance that fits their aimed logs’ coverage within the approved budget.  In addition to the low SNR (signal to noise ratio) that is a common nature of SIEMS systems with more that 60% of ingested logs being redundant or just noise with no added security value.

But what if we can enhance the SNR of our SIEM? Removing redundancy and noise from SIEM logs will only bring good results.

Financial Gains

• Less Cost: Significantly reduce what you are paying for your Splunk license
• Simpler Infrastructure: Minimize the infrastructure needed for running Splunk
• Enhanced Storage: Optimize your storage costs, no matter what your data environment looks like

Operational Gains

• Faster security detection: Reduce your log volume without compromising your log quality (we demonstrate this here)
• Better observability: Improve log visibility without breaking your budget on Splunk costs, thanks to Cribl Logstream
• Less effort for parsing and normalising logs: free up your indexers from the complex parsing operations by sending simplified logs formats with Cribl

Why Cribl:

Cribl enables observability by giving you the power to make choices that best serve your business, without the negative tradeoffs. As your goals evolve, you have the flexibility to make new choices including new tools and destinations.

We at Leo CybSec believe in the value Cribl brings to our customers, and through this blog, we will verify Cribl’s capability to reduce Splunk licensing while maintaining logs fidelity for various noisy log sources that cause headaches and sleepless nights for Splunk admins.

Windows Security And Sysmon Logs

What did we do? 

We sent the same copy of logs via two routes: 

• Splunk universal forwarders ⇒ Splunk Indexers 
• Splunk universal forwarders ⇒ Cribl ⇒ Splunk Indexers

Then we used Splunk searches and Dashboards to prove: 

• The License Reduction for logs going through Cribl 
• Better performing Splunk searches 

In our Lab we have built the below architecture and components:

The Splunk Architecture

A multi-site Splunk deployment with 4 Indexers and Index Discovery enabled

Log Sources

A single Windows server sending:

• Windows Logs (Security) 
• Sysmon Logs

The Windows server has two output options enabled that at any point, it will be sending logs to an indexer and a Cribl worker.

The Cribl Architecture

A Cribl cluster of one master and two workers

Enabled source for Splunk TCP 9997 logs from the windows Server

A Cribl Route to deal with Windows Logs

Cribl Pipeline to apply the processing on the Windows/Sysmon logs using Cribl Packs

Load Balanced Cribl Destination to the Splunk Cluster

Logs’ volume reduction reported by Cribl basic statistics (we will prove this with Splunk)

Logs Flow

Splunk UF ⇒ Cribl Source (Splunk TCP 9997) ⇒ Cribl Route (Windows) ⇒ Logs processed with Cribl Pipeline (WindowsXMLEvents)  ⇒ Cribl destination (Splunk Load Balanced ) ⇒ Splunk indexer

Findings

Logs are coming load balanced (Cribl acting as a Splunk universal forwarder sending logs to all indexers and both sites)

Running the same search on both indexes within the same timeframe.   

Searching on Cribl-routed logs

Same Search on non Cribl-routed logs

Splunk calculated license for each index

Achieved License Reduction for Windows / Sysmon Logs

Conclusion

• Splunk Dashboards verified the License reduction of Windows / Sysmon logs using Cribl 
• Sending Logs via Cribl is a straightforward process with minimum effort but great outcome

What other log sources do you want to see reduced with Cribl? 

What other destinations do you want to reduce the logs ingestion volume to?

Please visit our website www.leocybsec.com or drop us an email at info@leocybsec.com  and will be more than happy to help and answer any questions.