CardinalOps 2022 Report on the State of SIEM Detection Risk analyzed aggregated and anonymized data from production security information and event management (SIEM) instances to understand security operations center (SOC) preparedness to detect the latest adversary techniques in MITRE ATT&CK, the industry-standard catalog of common adversary behaviors based on real-world observations. 


This is important because detecting malicious activity early in the intrusion lifecycle is a key factor in preventing material impact on the organization. The analysis shows that actual detection coverage remains far below what most organizations expect and that many organizations are unaware of the gap between their assumed theoretical security and the defenses they have in place.


The data set for this analysis spanned diverse SIEM solutions — including Splunk, Microsoft Sentinel, and IBM QRadar — encompassing more than 14,000 log sources, thousands of detection rules, and hundreds of log source types, spanning diverse industry verticals including financial services, manufacturing, telecommunications, and MSSP/MDR service providers. Using MITRE ATT&CK as the baseline, CardinalOps found that, on average:


  • Enterprise SIEMs contain detections for fewer than 5 of the top 14 ATT&CK techniques employed by adversaries in the wild.
  • SIEMs are missing detections for 80% of the complete list of 190+ ATT&CK techniques.
  • 15% of SIEM rules are broken and will never fire, primarily due to fields that are not extracted correctly or log sources that are not sending the required data.
  • 75% of organizations that forward identity logs such as Active Directory and Okta to their SIEM, do not use them — which is concerning because identity monitoring is one of the most critical data sources for strengthening zero trust.
  • 75% of out-of-the-box detection content provided by SIEM vendors is disabled due to noisiness and customization challenges experienced by detection engineering teams.




SIEM-Detections-for-Zero-Trust-—-CardinalOps.jpeg

Image courtesy of CardinalOps



These major gaps in detection coverage can be attributed to a number of challenges faced by SOCs and their detection engineering teams. At the top of the list is constant change in the threat landscape, organizational attack surfaces, and business priorities, combined with an exponential increase in complexity resulting from an ever-increasing number of log source types and telemetry from diverse data sources (endpoint, identity, cloud, etc.). Difficulty in recruiting and retaining skilled security personnel is also a major factor. And many enterprises are still relying on manual and error-prone processes for developing new detections, which makes it difficult for engineering teams to scale effectively and reduce their backlogs.


“Organizations need to become more intentional about detection in their SOCs. What should we detect? Do we have use cases for those scenarios? Do they actually work? Do they help my SOC analysts effectively triage and respond?” said Dr. Anton Chuvakin, Head of Security Solution Strategy, Google Cloud. “Detection use cases are the core of security monitoring activities. Having structured and repeatable processes is essential for prioritization, aligning monitoring efforts to security strategy, and maximizing the value obtained from your security monitoring tools.”


To help organizations address their detection challenges, the 2022 CardinalOps report includes a series of best practices to help SOC teams measure and increase the robustness of their detection coverage, so they can continuously improve their detection posture over time.


You can download the full report here.