Straight questions don’t always elicit straight answers, so the security leaders grappling to keep their security metrics programs on track can at least take some crumbs of comfort from that.

That’s because generating security metrics is hard; much harder than it looks and much harder than it should be. Large enterprises especially have highly complex, ever changing infrastructure, people, processes and applications. Their teams are overloaded with the manual work required to produce up-to-date security metrics for the wide range of stakeholders who need them – spending up to one-third of their total time producing metrics reports, according to our recent research.

But security metrics are also extremely important – they lie at the center of any effective security program.

Why metrics are so important

Ultimately, security metrics are the objective measurements that answer key questions about how well the organization reduces or mitigates risk, which is the primary purpose of employing controls. Executed well, these metrics give enterprises a stronger security posture by ensuring a control failure doesn’t become a security incident.

Security metrics are in high demand. Why?

• More accountability to stakeholders. Infosec is a top boardroom concern among business executives who need to manage risk and know everything is being done to prevent data breaches and avoid falling foul of regulations. Likewise, shareholders, staff, partners, customers and – of course – regulators also have a need to understand security to varying extents
• An unbeatable aid to decision making. Security metrics are critical to short and long-term decision making. From spotting process failures and gaps in controls coverage, to substantiating strategic technology investments.
• Performance management. If you can’t measure it, you can’t manage it – right? Especially when it comes to quantitative data. The security team has KPIs just like everyone else, and metrics are essential to tracking their performance and demonstrating their impact in a domain where no news is good news.

Why CISOs are constrained from delivering metrics at scale

The size of the security metrics challenge should not be underestimated, and CISOs are finding it difficult.

We know that the range of threat scenarios creates an enhanced need for security metrics from within organizations. We also know that external regulators – simply through the execution of their responsibilities – are placing additional stress onto already overstretched resources.

Regulators want metrics as proof of compliance with their legislated standards. Such requests are coming at increased frequency and urgency. For example, the Singapore financial regulator’s Cyber Hygiene Notice requires firms under its jurisdiction to give a continuous, 360-degree view of every asset across multiple controls.

Regulations like SHIELD, CCPA and GDPR require organizations to protect systems holding sensitive data such as PII. To demonstrate compliance, organizations have to identify assets that host PII and ensure the right controls are deployed and performing as expected. Organizations with immature security metrics programs would struggle to address requests like these. Make no mistake, this is a very big deal – regulators have been showing their teeth in the last 12 months, dishing out record multimillion-dollar fines to the likes of Equifax, British Airways and Marriott.

The principal issue is the manually intensive nature of producing and disseminating security metrics. In our survey of 400 CISOs, 60% admitted to using spreadsheets like Microsoft Excel to calculate them. In other words, methods that can be time consuming and prone to human error. Organizations have adopted them because data about control effectiveness are siloed in each individual tool, which typically don't integrate with one another. Without platforms that can unify the data for reporting, teams spend their time switching between tools and updating their own dashboards.

Why producing good security metrics is so difficult

There are several key challenges at play that impact the ability of security teams to produce good security metrics:

• Time-consuming processes and request overload. Our research looked in detail at exactly how much time it took to satisfy requests, and how frequently requests come in. In short, it’s a daily occurrence and stakeholder groups like regulators, IT teams, the board, auditors and even customers are putting requests in almost twice a month or more.
• Lack of trust in the accuracy of data. As stated above, the overwhelmingly manual nature of data collection, collation, analysis and presentation makes security metrics prone to human error and potentially out-of-date before they arrive. Another related problem is the prevalence of qualitative, subjective, questionnaire-type assessments, which are even further from the robust, quantitative process security teams really need to support their work.
• Lack of flexibility and business context. It hasn’t been easy for CISOs to construct their largely manual security metrics program from the ground up.

Context is crucial in gaining value from security metrics, but the manual heavy lifting required would make determining context all but prohibitive. So, rather than a principally manual approach, a superior alternative would be a continuous and automated metrics program with little to no manual intervention. This would allow CISOs to identify cyber risks by surfacing the context that emerges across multiple metrics and prioritizing these according to business impact.

Much of this is achieved in the Continuous Controls Monitoring (CCM) approach to risk management, covered in more detail below.

The five stages of security metrics maturity

Conversations and assessments with many security, compliance and risk professionals at large financial services institutions and other highly regulated industries – and backed up with our own research – lead me to conclude that security leaders are struggling to get the resources or direction to mature their metrics programs.

Improving your metrics program not only relieves the manual burdens of time, inaccuracy and lack of trust in data, but also highlights problem areas. Prioritizing these areas and focusing your resources to address them improves your overall security and compliance posture.

We can define the five stages of maturity as follows:

• Basic: Subjective, manual, point-in-time data collection and dissemination that relies on questionnaires, spreadsheets and consultancies.
• Elementary: Still manual, relying on snapshots from data sampling. Exposed to tooling and quality problems.
• Intermediate: Features basic automation, data ingestion and storage for simple correlation. But also risks assumption and data problems.
• Upper Intermediate: Automatic, continuous controls monitoring, providing 360-degree view of IT assets, automated inventory, knowledge graph and business perspectives in a multi-framework approach.
• Advanced: Predictive CCM, which is automatic and continuous. This approach is able to conduct future extrapolation, predict control failures and take automated action.

Our research found around one-third of CISOs would score their own program as “basic”, “elementary” or “intermediate”. That is likely to be a very conservative figure, with the majority of organizations – if properly assessed – likely to fall into these categorizations.

How to achieve a mature security metrics program

Many CISOs have turned to recognized frameworks (e.g. NIST Cyber Security Framework, CIS Critical Security Controls) to set the boundaries of their security metrics programs.

What’s needed is a platform without boundaries; a single voice that gives straight answers to searching questions. One that puts an end to compromises, manual struggles and having to say, “it’s complicated.”

Getting there will require either a substantial and wide-ranging internal development project or investing in platforms such as Continuous Controls Monitoring.

Gartner defines CCM as: “a set of technologies that automates the assessment of operational controls’ effectiveness and the identification of exceptions.” Also, that it is “runtime and transaction-level monitoring and is most useful for operational controls.”

With CCM, data is cleaned, normalized, aggregated, de-duplicated and correlated as part of the entity resolution process. By unifying disparate data, it identifies previously unknown or unmanaged assets and control coverage gaps.

It also enables self-service access to current and historical data so that any metrics request can be fulfilled accurately, efficiently and in a timely fashion.

CCM is proven to save time and resource costs by automating security and compliance monitoring and controls assurance. At its heart, CCM is a single source of truth organizations can use to address all manner of security metrics requirements, providing the ability to prioritize risk aligned to critical business operations, demonstrate compliance, present an up-to-date view of security posture, and restore trust among all stakeholders in the accuracy of data.