Cyberattacks are on the rise, posing the perpetual threat of devastating organizations’ corporate networks and applications. These risks have only grown in recent days as more employees than ever are working remotely and connecting to their networks and applications through a VPN. Manual security pentests have quickly become vital parts of companies’ defense strategies. The decision to implement a formal testing program can help ensure continuous security coverage and eliminate vulnerabilities for both your organization and any customer that uses your software.

Defining a Pentest

Right now, there are three primary manual security testing options available to organizations: traditional penetration testing in which organizations pay a security service provider to test a specific asset or set of assets using a clearly defined methodology, bug bounties that are open-ended programs where any security professional or hacker can search for vulnerabilities in an application or asset and get paid based on the perceived ‘quality’ of each finding, i.e., the level of risk it poses, and Pentest-as-a-Service (PtaaS), an emerging delivery model that uses a platform-driven solution and a freelance talent pool to manually search for vulnerabilities.

The addition of Software-as-a-Service (SaaS) platform technology to traditional pentest consulting models drives workflow efficiencies by connecting pentesters with specific skill sets and experience levels directly with engineering teams to address identified vulnerabilities. Because of this and because the model is a relatively new option for organizations to implement, the guide below focuses specifically on PtaaS and how to make security testing effective, easy and seamless.

What is a Pentest Program?

A program is a clearly defined series of pentests designed to systematically identify and remediate vulnerabilities in one or more assets or asset groups. A program typically follows an annual, renewable cycle, with testing completed periodically throughout the duration—for example, on a weekly, monthly, quarterly, bi-annual, or annual basis. By planning pentest programs annually, security leaders can ensure full coverage of assets and identify the depth of coverage needed for each asset.

These programs should be structured to examine the most critical assets, or those that are most frequently deployed, on a regular basis. This ensures:

• Continual testing coverage for critical and frequently updated assets.
• A much higher probability that high-risk vulnerabilities will be quickly identified and remediated.

NOTE -It’s important to realize pentest programs DON’T fulfill the same function as DAST/SAST solutions OR bug bounty programs. Consistent pentesting of assets by experts with domain-specific skills is necessary to identify vulnerabilities that require business context not available from bug bounty programs or automated scanners.

Anatomy of a Pentest Program

• Plan. Identify the asset(s) that need to be tested, the cadence of testing and time frames.
• Scope. Determine testing depth and resources.
• Test. Conduct thorough automated and manual testing with domain-specific experts.
• Remediation. Partner security teams with engineering to resolve or mitigate identified vulnerabilities.
• Retest. Deploy testers to ensure fixes/mitigations have addressed each vulnerability. Repeat. Use small, frequent tests to ensure continual coverage.

Repeated testing and retesting are vital because IT infrastructure is never static. New assets, patches and lines of code are added all the time, and new vulnerabilities are inevitably introduced over time. As a result, if the most recent pentest of an asset was six months ago, there’s a very high chance it has new vulnerabilities that haven’t been identified.

Using a programmatic approach to pentests allows organizations to continuously evaluate the security of digital assets. This ensures that newly introduced vulnerabilities are consistently identified, which in turn leads to a reduction in cyber risk and higher overall level of security.

However, the benefits of a pentest program don’t stop there. Maintaining a pentest program also gives organizations the opportunity to build an ongoing relationship with their testers. Over time, testers gain a much deeper understanding of the assets they are working with, helping them to identify deeper issues and vulnerabilities that would otherwise be missed.

Finally, a pentest program can also help organizations identify systemic issues that arise across multiple assets. These issues often indicate the need for targeted developer training or implementation of new technologies. Through this insight, pentest programs help organizations improve their overall security posture by addressing problems at their root cause.

How to Build a Pentest Program

Building your first pentest program can be daunting. However, with careful planning, any organization can get a pentest program up and running within a short timeframe by following five simple steps.

Step 1: Set program objective(s)

Before you enter into a pentest program, determine what it needs to achieve. For instance, do you need to achieve compliance with one or more frameworks? Do you have specific assets that need to be tested and hardened? Your objectives will inform the structure of your pentest program, so be sure to involve all stakeholders in this step.

Common objectives include:

• Ensure testing of all assets required for compliance (e.g. PCI-DSS).
• Ensure all assets with sensitive data are tested regularly.
• Ensure business critical assets are tested regularly.
• Ensure all major critical releases are tested.

Step 2: Identify which assets are most critical and/or at the highest risk of cyberattack.

It may be that not all assets are equally important to your organization. Your pentest program should aim to provide maximum coverage for your most critical assets while testing less critical assets more infrequently.

Step 3: Determine how often assets are changed or updated.

A pentest program should provide continuous security coverage for in-scope assets. For that to be possible, pentests should be scheduled each time a significant codebase change is rolled out.

Step 4: Check whether any major infrastructure changes are planned for the program period.

Significant changes to infrastructure, such as a change in cloud providers or a move from on-premise to the cloud, warrant a higher-than-average frequency and depth of pentesting. When scheduling pentests, make sure you have a full picture of the organization’s intended digital strategy during the program period. Equally, plan for some level of ad hoc testing to account for any surprises along the way.

Step 5: Plan the logistics of your pentest program.

The precise content of a pentest can vary significantly depending on the asset being tested and your priorities. For instance, if you want to hunt down business logic flaws or privilege escalation vulnerabilities, it makes sense to provide testers with login credentials for your asset. If you want to know what an external threat actor could do without credentials, an uncredentialed test may be appropriate. Equally, you may decide to include both credentialed and uncredentialed approaches in a single pentest.

Step 6: Define the scope and depth of your program.

Once planning is complete, it’s time to decide which assets will be included in your pentest program.

A word of caution: a common mistake that security buyers make is including too many assets in a pentest and not allowing pentesters enough time to thoroughly test for vulnerabilities.

If you aren’t sure, ask your pentest provider for guidance to ensure your assets can be fully tested in the time allocated.

Black Box vs. Grey Box Pentesting

One of the most common pentesting conundrums is whether to use a ‘black box’ or ‘grey box’ approach.

Black box pentesting is where testers have no prior knowledge of the assets being tested and aren’t provided with login credentials. It requires pentesters to find flaws and vulnerabilities without any knowledge of data flow, design, architecture, or roles/permissions.

Grey box pentesting equips testers with some level of information about an asset’s function, business logic and permissions, and may also provide them with login credentials. This is intended to make the pentesting process more efficient, faster and more productive. Note that grey box pentesting is not the same as white box pentesting, where testers usually have access to source code and conduct targeted testing based on what is found in the code.

Which is better? Some argue that black box pentests are more realistic, as real-world threat actors won’t have access to credentials or guidance, but this is not always viable. Time is often a deciding factor. The time allocated to pentesting is always limited, whereas threat actors have as much time as they need to find weaknesses in an asset. If testers are forced to spend time learning about the asset and reverse engineering its functionality, that diminishes the time they can spend searching for vulnerabilities. As a result, the chance that testers will find high severity vulnerabilities, such as business logic flaws, are much higher with a grey box pentest where they’re not going in cold.

To illustrate this, here’s a simple example. A B2B asset is designed to provide a business service. There may be support documentation, but it’s unlikely to be publicly available. B2B assets typically have complex workflows and permissions, which may be unclear to a first-time user. If pentesters are forced to develop their own understanding of how the asset works, that will significantly reduce the time they can spend searching for vulnerabilities, and there is a chance they will overlook something and miss a serious vulnerability.

Once again, if real threat actors were to target the same asset, they would not suffer the same disadvantages because they have the freedom to spend as long as needed to find a weakness. So while there are benefits to black box testing, grey box pentesting is a better option for most organizations.

NOTE The ‘black box’ vs ‘grey box’ debate is NOT only about credentials. In many cases, the most important advantage a pentester can gain from grey box testing is a clearer understanding of an asset’s business logic, permissions and intended function.

Who to Involve in Your Pentest Program

Just like any aspect of security, no pentest program can succeed without support from other areas of the business. Two stakeholders in particular need to be closely engaged with:

1. Engineering/Product

Finding vulnerabilities is half the battle. The other half is closing them promptly and successfully.

For this to happen, vulnerability results from your pentest program need to be routed directly into standard development systems and workflows. In other words, your pentest program should support the secure development lifecycle (SDLC).

Engage with engineering and/or product leads as early as possible and seek their input throughout the planning stage of your pentest program. Developer time will be needed to resolve identified vulnerabilities, and it’s best if this can be scheduled in advance to ensure sufficient capacity.

2. Leadership

Leadership buy-in is critical for any security initiative. Pentest programs work best when they are run consistently over time, so you’ll want to be sure that you have budgetary support beyond the current year.

Fortunately, in recent years security has inched closer to the boardroom. In 2020, security is firmly established as a critical business function and most boards demand oversight in some capacity. As a result, gaining approval for security initiatives that have genuine business value is much easier than it once was. Providing regular reporting to leadership executives is an excellent way to evidence the ROI of your pentest program. By highlighting the number and quality of vulnerabilities identified, as well as their potential impact on operations, you’ll be demonstrating the business value of your program.

However, when seeking buy in, it’s critical to communicate in a way that executives understand and are interested in. To achieve this, you must have an understanding of what the organization as a whole is trying to achieve and how your pentest program can support those objectives. If you can produce metrics that clearly tie your program to the overall success of the organization, gaining leadership buy-in will be much more achievable.

Incrementally Improving Security Outcomes

Perhaps the single greatest benefit of a pentest program over ad hoc testing is the ability to learn from each pentest and improve. Each time you conduct a pentest, you’ll have an opportunity to learn more about how to run an ideal pentest program for your organization.

If you take the opportunity to learn from each engagement and implement changes, the quality of your findings will improve exponentially over time.

Key Takeaways

• Pentesting is distinct from both legacy penetration testing and bug bounties. It provides the benefits of both approaches while compensating for their deficiencies.
• Pentests are smaller, more frequent engagements than legacy penetration tests. And they are conducted by highly experienced security professionals with domain-specific expertise.
• A pentest program is a series of pentests designed to systematically identify and remediate vulnerabilities in one or more assets or asset groups.
• In general, ‘grey box’ pentesting produces more and higher-quality findings than ‘black box’. This is especially true for complex B2B assets where business logic and permissions aren’t obvious, and/or when the time allocated is not sufficient for testers to conduct a full reconnaissance exercise.
• Engaging with key stakeholders is critical to the success of a pentest program. In particular, engineering leads should be actively involved, while executives should be given regular updates.
• The top benefit of a pentest program over ad hoc is the ability to constantly improve. Asking for feedback from your pentesters and identifying your own internal learning points will help you drastically improve the quantity and quality of findings over time.