Penetration Tests Are Not Vulnerability Assessments

I often, as well as many of my peers and colleagues, have to explain to a client the difference between a vulnerability assessment and a penetration test.  There are a lot of security consulting firms out there that show up to a client’s worksite, run a vulnerability scanner, validate some findings, copy and paste the scanner’s results and call their “penetration test” complete, collect their check, and move on to the next victim…err I mean client.

This may sound pretty harsh, but companies can’t afford sugar coatings.  The threats are real and your assets are mission critical.  I feel like a broken record having to say that, but how many people still use the same password for everything.  Doing an annual vulnerability assessment and checking a box on a compliance worksheet will not protect your company from serious threats.  These threats could potentially take down your company by exposing sensitive information to your competitors or worse the public.  Not only that, but a vulnerability assessment done this week might be out of date next week, since new exploits and security patches for software running on a network may be released next week.

If you’re hired to do something, do it right the first time and provide your client with the value that not only they deserve but paid for.  In all fairness though, the blame doesn’t only fall on the shoulders of the company providing the “penetration test”/ vulnerability assessment it also falls on the company that hired them.  A lot of companies cannot distinguish the difference between the two.

Any wannabe hacker/script kiddie can fire up a Nessus scanner, web application scanner, and decipher the results.  This is not an article bashing the use of the Nessus Vulnerability Scanner; Nessus is a great vulnerability scanner. Here at MassiveLabs this is typically part of our information gathering on a target, but we don’t stop there and we also don’t start there.  Moving forward let’s use the 80/20 rule.  Penetration tests are 80 percent manual, if not more, and 20 percent automated.  We may use tools that help automate some of the processes such as brute forcing a password or port scanning to discover live hosts, open ports, and running services.

The biggest difference where a vulnerability assessment differs from a penetration test is exploitation and a human.  Vulnerability scans are automated by following a set of steps predetermined in a script which may have the ability to exploit that one vulnerability.  A human has the ability to discover potential holes in your security by thinking outside the box, daisy chaining multiple vulnerabilities, testing business logic of an application, analyzing the data collected by the vulnerability scanner, and combining all that with their own observations to get to their end result.

The easiest way I explain the difference to my clients is with a simple scenario.  Let’s say for the sake of argument your vulnerability scanner discovers an open file share that anyone can read and write to.  Great, we have a finding and the vulnerability scanner will provide a remediation along the lines of locking down permissions.  But let’s take that same file share and have a human analyze the results.  What if in the file share there was a file that contained all the passwords for every device and server on the network? Now the human can take the file and start using those passwords across the network to gain further access and potentially discover a plethora of vulnerabilities three layers deeper than the vulnerability scanner could ever reach.

MassiveLabs is not trying to set a new trend, but simply telling you the right way to perform a penetration test.  According to the PCI DSS Council in section 11.3 they define the following: “vulnerability assessment simply identifies and reports noted vulnerabilities, whereas a penetration test attempts to exploit the vulnerabilities to determine whether unauthorized access or other malicious activity is possible.” To provide one more source of evidence on the subject, the EC Council says that, “Penetration testing simulates methods that intruders use to gain unauthorized access to an organization’s networked system and then compromise them.  Penetration testers may use proprietary and/or open source tools to test known technical vulnerabilities in networked systems.  Apart from automated techniques, penetration testing involves manual techniques for conducting targeted testing on specific systems…”

Vulnerability assessments certainly have their place in the security onion, but hopefully we have clarified the difference between a penetration test and a vulnerability assessment.  Our penetration testers here at MassiveLabs will actually exploit vulnerabilities in systems, add user accounts, compromise machines, and potentially gain control of the entire network.  Most companies will stop when they have gained “root” access or domain admin control of a network, but we also provide post exploitation such as looking for and exfiltrating sensitive information to test data loss prevention as well as patch management within the network.  Throughout the engagement our consultants will generate a list of findings that we will document and provide remediation recommendations customized for your environment.  Some of these findings may be found in a vulnerability assessment, but most will be findings the scanners cannot find through their automated scripts.

Now that you know the difference between a vulnerability assessment and a penetration test we encourage you to set up a network vulnerability scanner in your environment. Your network vulnerability scanner should run continually throughout the year with the ability to alert staff of any critical or severe vulnerabilities.  We also recommend you hire a security consulting firm that knows the difference between the two and can perform a true penetration test annually, at a bare-minimum, on your entire network.