Chief information security officers (CISOs) and their teams must have timely access to accurate and meaningful application security (AppSec) data to do their jobs effectively. This visibility is essential for reporting on the organization’s overall risk posture to the executive leadership and the Board of directors. It’s necessary to identify where the most significant AppSec risks lie, what to prioritize for remediation and to provide crucial forensic insight in the event of a breach. And it’s also the cornerstone to achieve DevSecOps.

You can’t achieve DevSecOps unless everyone responsible for developing the product works from the same data set and can truly understand application security risks. CISOs, Chief Product Security Officers (CPSOs) and their teams must be able to communicate around AppSec risk with business unit (BU) leaders and product owners who, more and more, are being held accountable for the security of their products. And all these teams must collaborate with development leaders and DevOps teams, who work to highly rapid release cycles, and can no longer afford to be sidelined by security problems within the applications they are developing.

Specifically, security, product and development leaders must have visibility into issues such as:

• What is the state of our AppSec program? Where are the gaps, the most significant risks to the business, what should we prioritize for remediation?
• Do we have 100% static application security testing (SAST), software composition analysis (SCA) and dynamic application security testing (DAST) scanning coverage for all our mission-critical applications in development? If not, do we have a plan to get there? 
• What’s our progress in detecting and remediating vulnerabilities month over month? What are those vulnerabilities? Do they even matter? Are they systemic across teams?
• What are our top riskiest applications? Is there a problem with a specific application or DevOps team? Why is this happening? What’s the best way to address it? 
• Are we in compliance with regulations and with our own policies and service-level agreements (SLAs)? Can we track that?

But the current state of application security across most organizations means that answering these questions and gaining AppSec visibility is no easy task. In many organizations we work with, application security is becoming more decentralized, with DevOps teams now handling at least some AppSec scanning, often using the tools they select themselves. Moreover, we’re finding that AppSec scanning is still relatively immature and inconsistent in terms of coverage and the types of applications scanned, while the tools used are generating an unwieldy amount of disparate data. As a result, organizations are struggling to handle all their AppSec data and make sense of all it all, much less answer these questions accurately, in a timely fashion, and in an easily consumable format appropriate for the many audiences that need this insight.

A Quick Fix vs. a Long-term Solution

Some security teams attempt to address this problem by centralizing all their existing vulnerability data in a business intelligence (BI) tool. It’s relatively quick and easy to do, cheap and probably adequate if all needed is a dashboard to showcase compliance with the AppSec program or a monthly overview report for the leadership team.

While such a BI dashboard may address some of the CISOs immediate needs, it cannot provide a long-term foundation for risk reporting that is holistic, strategic, scalable or drives practical improvements in application security across the organization.

• Accuracy: To produce an AppSec risk dashboard, BI tools need to ingest data from multiple tools – each with its own formatting, scoring and prioritization. Standard BI tools do not normalize data from various sources into a common risk framework or aggregate, correlate and compress related issues to remove noise and create an even playing field from which you can gain a clear – and accurate –picture of AppSec risk. Thus, for example, 100 instances of cross-site scripting in the same application component may be blown out of proportion, even though it’s only a single linked vulnerability. Attempting to undertake this normalization through custom scripting is a heavy lift requiring expertise and expertise with significant time on their hands.

• Scope: The structure of the modern enterprise, together with the shift to a more decentralized approach to application security and the demand for a more agile development process, will likely raise many questions – and conflicts - around ownership and inclusion in these reports. The most optimistic outcome is that the reports will be provided - siloed - for each business unit, which has its benefits but will not provide a comprehensive view of enterprise-wide risk, which is critical for the CISO, executive leadership team and the Board.

• Management & Maintenance: AppSec is not static – it constantly changes together with the evolution of the company, the products it develops, its infrastructure, processes and tools. To provide an up-to-date view of AppSec risk, the BI tool’s data model must be managed and maintained in real-time, in line with any changes across the organization. So, when a DevOps team starts using a new scanning tool or starts working on a new mission-critical application, adding this information to the BI tool must be quick and easy. 

• Actionable: BI-generated reports are just that, reports. They are not designed to drive triage and remediation efforts through workflows, automation, or self-service capabilities, which are a critical part of an App Sec program. 

Build The Right Foundation for Comprehensive AppSec Risk Reporting – 8 Questions to Ask

Before attempting to build a solution for AppSec visibility internally, consider some of these questions: 

1. What is the primary use for these reports: audits, corporate risk assessment, compliance (regulatory, internal), vulnerability management, patch management? Will the content and level of detail be tailored to each of the use cases?  
2. Who are the requestors and consumers of the reports? How will the reports be delivered?
3. Which business units and/or application teams will this reporting include? What are the criteria for inclusion?       
4. Do you know all the different types of reports needed? What are the required outputs? Who will define them?  
5. How many applications does your company have, and how many different application security scanning tools are being used across the organization? 
6. Will the reports cover all these applications and tools? Will reports be available on the individual components of the applications in addition to the aggregate business application?       
7. Is there a specific format for the scan data? Are APIs being utilized, and how automated is the ingestion process?   
8. Who is sponsoring, staffing and funding the internal reporting effort? 
9. Has funding and staffing been allocated for ongoing maintenance and enhancements of the reports beyond the initial project?  
10. Will the reports foster a shared responsibility for AppSec and help drive remediation of any security issues? Will they map to the various stages of the software development life cycle (SDLC) with enough detail? Is sufficient guidance being provided to developers to identify, prioritize and remediate vulnerabilities? Can the reports compare outputs from different AppSec tools? Can reports highlight bad coding practices within or across Development teams to identify training and development opportunities?   

As you go on the journey to DevSecOps, make sure you have the right solution that can deliver the level of visibility into AppSec risk that the CISO requires, together with the critical reports needed to drive shared responsibility, accountability and effective AppSec remediation throughout your organization.