Whether it’s done to meet compliance requirements or just as a general best practice, most organizations are now testing their own networks for security weaknesses, and if they’re not, they should be. The many different types of tests can be confusing for the uninitiated; we will take a look at the common types along with their strengths and weaknesses here.

Network Vulnerability Scans

Network vulnerability scans are the most basic type of test, relying on a scanning tool that looks for known common vulnerabilities and reports them with severity ratings. Quick, cost-efficient and safe, these scans are great for making sure that system patches are up to date and security configurations settings are locked down. When run regularly they can provide an early warning if a system is missing patches or misconfigured.

Many organizations only scan their network from the Internet. While Internet-facing vulnerabilities may seem like the easiest for an attacker to exploit, scanning only from this perspective leaves an organization blind to the vulnerabilities that an attacker would leverage to move between systems once they gained a foothold inside the network. In order to identify these internal vulnerabilities, scans should also be performed from behind the firewall.

Even internal network scans leave blind spots; by default the scanners can only check services that listen for network communications. However, many of today’s attacks are the result of phishing campaigns that target Web browsers, PDF viewers, and other “client” software that is not visible to a network scan. These attackers usually go on to exploit other local operating system vulnerabilities to get administrator privileges. To solve this problem, scanning tools can be configured with authentication credentials that enable them to log in to their targets during internal scans and check local software as well. This approach is strongly recommended as it will give the most complete view of the patch and configuration status of an organization.

One additional flaw with network vulnerability scanners is that they are only as good as their vulnerability signatures, which are based on databases of known vulnerabilities. As a result, these scanners will not identify any flaw that hasn’t been publicly reported yet, including vulnerabilities in obscure or custom applications. This is an issue as attackers regularly leverage vulnerabilities in custom applications in order to gain access to the data they contain or the underlying network.

Application Vulnerability Scans

Unlike network scanners, application scanners are designed specifically to look for previously undocumented vulnerabilities in custom applications. Rather than checking for a list of known vulnerability signatures, they will exercise all of an application’s functionality in order to find common types of flaws. It’s important to know that because of the amount of data they send to an application, these scanners must be used carefully. There are many horror stories about scanners dumping garbage data into a database or triggering thousands of emails.

As advanced as application scanners are, there are still vulnerabilities that they miss. Some vulnerabilities are too subtle for an automated scanner to detect but will be obvious to a human with a little intuition. As with network scans, a clean report is a good start but doesn’t necessarily mean there are no problems.

Penetration Testing

With any kind of scanning, it is vitally important not to simply concentrate on fixing the “high” or “critical” severity vulnerabilities reported by these scans. The fact is that real-world breaches are rarely the result of a single critical network vulnerability; real attackers may chain together a few low to medium severity network vulnerabilities or combine them with “local” vulnerabilities that aren’t visible from the network.

Penetration tests build on the concept of network and application vulnerability scanning by adding skilled hackers that can simulate real-world attacks against network services, applications, or both simultaneously. The testers will try to combine and exploit vulnerabilities found by the scanners as well as look for the types of vulnerabilities that scanners miss. While this is more time consuming and expensive than scanning tools alone, it provides a much more realistic assessment of how much effort would be required to breach an organization.

Although many organizations try to cut costs by limiting penetration testing to “critical” systems, the value of a penetration test is only as good as its scope. Real attackers have no qualms about exploiting other unrelated systems to get to the ones with sensitive data, so testers should have the freedom to expand their scope as needed.

As important as it is, no testing is without risk. Although testers will take great care to avoid causing damage, they are still attempting to exploit flaws in software and there will always be a chance of unintended consequences. Organizations should remember that the results of a penetration test are highly dependent on the skill of the team performing it. I’ve also seen many cases where an organization contracted a new penetration testing firm and found severe vulnerabilities that a previous firm missed. For these reasons, it’s worth paying a little extra for a team with a solid reputation.