To be effective, the SOC must be optimised with EDR and NDR, but few have deployed the latter, preventing them from achieving a SOC Visibility Triad. Ian Tinney looks at the obstacles involved and how you can overcome them using data distillation.
Too many Security Operations Centres (SOCs) are set up to fail. The Ponemon Institute recently found only 50% of organisations think their SOC is effective, and that’s because they’re often not optimised in terms of the data they’re collecting. For a SOC, less is more – the last thing the security team needs is a mountain of events that they need to trawl through and investigate. Threats are becoming more numerous and more sophisticated, making it more important than ever to keep on top of security.
According to the Verizon 2021 Data Breach Investigations Report (DBIR), the situation has worsened during the pandemic, with breaches increasing by a third over the past year. Phishing has risen 11% and ransomware 6%, while 80% of breaches weren’t even detected internally – they were discovered by external parties – which suggests the SOC isn’t delivering.
Some manufacturers have added a security analysis feature to their NDR products, but these only tend to work well with specific technologies, usually aligned with the manufacturer’s own product set.
Since the NDR data will inevitably have to be sent to a SIEM, it doesn’t always make sense to pay for the analysis twice; once by an expensive NDR product and then again by the SIEM.
To help solve this and other issues, Corelight created the concept of Open NDR. The principle of Open NDR is that the NDR does not need to do analysis that the SIEM will do and instead should focus on analysing the network traffic to produce rich, contextual metadata that the SIEM can use for its analysis. Furthermore, Open NDR does not care which SIEM you use, allowing you the freedom to change SIEM or your MSSP at a later date.
In the case of Corelight, the metadata it produces is 1% of the volume of data it analyses. But whilst this represents a considerable reduction in data going to the SIEM and whilst all of this data is useful, not all SIEMs will use all of the NDR data. This means there will always be some redundancy, leading to inefficiency in the data flow from the NDR.
But what if you could further filter the data to ensure ONLY the type of data your specific SIEM needs could get through? Using an Event Stream Processor (ESP) to supplement the data analysis of the NDR, you could surgically remove the data that the SIEM never uses. 4Data calls this data distillation. This data distillation could save between 50% and 70% of the volume of data sent by the NDR to the SIEM, depending on which protocols the organisation is analysing. This further significant reduction makes it cost-effective to use NDR, allowing you to complete the SOC Visibility Triad.
This doesn’t mean you have to compromise on the data you’re collecting. You could still route a full-fidelity copy of the data to cheaper object storage while sending the reduced copy of data to the expensive SIEM. That way, if you need to do a deep dive to conduct an investigation at a later date, the full-fidelity version of the data is still there and can be routed to the SIEM for analysis using the ESP’s ‘Replay’ feature.
The data processing costs saved through data distillation are substantial. That spend can then be put to better use, such as by further expanding the SOC Visibility Triad through the deployment of more Open NDR sensors. Or you could simply use the money saved to reduce the TCO of your SIEM and/or storage.
Data distillation in practice
NDR is arguably the most useful technology in the Triad, but it produces a lot of data, which, once you send to a SIEM, can make achieving the Triad prohibitively expensive. We have developed a solution using marketing-leading components to address the elephant in the room – NDR analysis.
Using the Corelight Open NDR solution in concert with the Cribl LogStream ESP, we have devised a SIEM-agnostic solution that makes NDR analysis cost-effective by reducing data by 50-70%. Data generated by Corelight sensors is passed through a series of pipelines within Cribl LogStream which understand the data structures used in leading SIEMs, enabling us to remove the data the SIEM will never use. This means NDR analysis, and therefore the SOC Visibility Triad, becomes accessible and achievable for any organisation of any size.
To find out more about our data distillation solution and how it works, please contact us for a one-to-one demo or to find out more about the solution, download our datasheet.