To be effective, the SOC must be optimised with EDR and NDR, but few have deployed the latter, preventing them from achieving a SOC Visibility Triad. Ian Tinney looks at the obstacles involved and how you can overcome them using data distillation.
Too many Security Operations Centres (SOCs) are set up to fail. The Ponemon Institute recently found only 50% of organisations think their SOC is effective, and that’s because they’re often not optimised in terms of the data they’re collecting. For a SOC, less is more – the last thing the security team needs is a mountain of events that they need to trawl through and investigate. Threats are becoming more numerous and more sophisticated, making it more important than ever to keep on top of security.
According to the Verizon 2021 Data Breach Investigations Report (DBIR), the situation has worsened during the pandemic, with breaches increasing by a third over the past year. Phishing has risen 11% and ransomware 6%, while 80% of breaches weren’t even detected internally – they were discovered by external parties – which suggests the SOC isn’t delivering.
Bolstering the SIEM
To monitor network security, mid to large organisations either employ security analysts (expensive and in short supply) or invest in SIEM. A SIEM is only as effective as the data it receives, and traditionally these have relied upon data from log files, which leave significant gaps in the security landscape. Even with the addition of User and Entity Behaviour Analytics (UEBA), some things will inevitably go undetected due to the lack of coverage of the whole estate.
To fill these gaps, Gartner recommends combining SIEM/UEBA with the newer EDR (Endpoint Detection and Response) and NDR (Network Detection and Response) technologies to form what is known as the SOC Visibility Triad.
EDR focuses on monitoring and preventing endpoint attacks. NDR creates full visibility across the network in real-time.
While malicious attackers can subvert EDR, NDR sees everything because everything needs to be communicated across the network and it is impossible to rewrite all network traffic. So, NDR does a lot more than simply fill in the gaps left by EDR and is an essential part of the modern SOC
But whilst many organisations already have an EDR tool, the SMEs in particular often don’t have an NDR solution. This is usually because of the costs associated with analysing the enormous amount of data that NDR generates.
NDRs typically capture and analyse data from 10 or 100Gbps networks, but this results in enormous flows of telemetry data that are expensive for the SIEM to ingest.
Open NDR
Some manufacturers have added a security analysis feature to their NDR products, but these only tend to work well with specific technologies, usually aligned with the manufacturer’s own product set.
Since the NDR data will inevitably have to be sent to a SIEM, it doesn’t always make sense to pay for the analysis twice; once by an expensive NDR product and then again by the SIEM.
To help solve this and other issues, Corelight created the concept of Open NDR. The principle of Open NDR is that the NDR does not need to do analysis that the SIEM will do and instead should focus on analysing the network traffic to produce rich, contextual metadata that the SIEM can use for its analysis. Furthermore, Open NDR does not care which SIEM you use, allowing you the freedom to change SIEM or your MSSP at a later date.
In the case of Corelight, the metadata it produces is 1% of the volume of data it analyses. But whilst this represents a considerable reduction in data going to the SIEM and whilst all of this data is useful, not all SIEMs will use all of the NDR data. This means there will always be some redundancy, leading to inefficiency in the data flow from the NDR.
Data Distillation
But what if you could further filter the data to ensure ONLY the type of data your specific SIEM needs could get through? Using an Event Stream Processor (ESP) to supplement the data analysis of the NDR, you could surgically remove the data that the SIEM never uses. 4Data calls this data distillation. This data distillation could save between 50% and 70% of the volume of data sent by the NDR to the SIEM, depending on which protocols the organisation is analysing. This further significant reduction makes it cost-effective to use NDR, allowing you to complete the SOC Visibility Triad.
This doesn’t mean you have to compromise on the data you’re collecting. You could still route a full-fidelity copy of the data to cheaper object storage while sending the reduced copy of data to the expensive SIEM. That way, if you need to do a deep dive to conduct an investigation at a later date, the full-fidelity version of the data is still there and can be routed to the SIEM for analysis using the ESP’s ‘Replay’ feature.
The data processing costs saved through data distillation are substantial. That spend can then be put to better use, such as by further expanding the SOC Visibility Triad through the deployment of more Open NDR sensors. Or you could simply use the money saved to reduce the TCO of your SIEM and/or storage.
Data distillation in practice
NDR is arguably the most useful technology in the Triad, but it produces a lot of data, which, once you send to a SIEM, can make achieving the Triad prohibitively expensive. We have developed a solution using marketing-leading components to address the elephant in the room – NDR analysis.
Using the Corelight Open NDR solution in concert with the Cribl LogStream ESP, we have devised a SIEM-agnostic solution that makes NDR analysis cost-effective by reducing data by 50-70%. Data generated by Corelight sensors is passed through a series of pipelines within Cribl LogStream which understand the data structures used in leading SIEMs, enabling us to remove the data the SIEM will never use. This means NDR analysis, and therefore the SOC Visibility Triad, becomes accessible and achievable for any organisation of any size.
To find out more about our data distillation solution and how it works, please contact us for a one-to-one demo or to find out more about the solution, download our datasheet.