Penetration Testing

The Penetration Testing practice involves standard outside-in testing of the sort carried out by security specialists. Penetration testing focuses on vulnerabilities in the final configuration and provides direct feeds to defect management and mitigation.

Penetration Testing Level 1

[PT1.1: 111] Use external penetration testers to find problems.

External penetration testers are used to demonstrate that the organization’s code needs help. Breaking a high-profile application to provide unmistakable evidence that the organization isn’t somehow immune to the problem often gets the right attention. Over time, the focus of penetration testing moves from trying to determine if the code is broken in some areas to a sanity check done before shipping. External penetration testers that bring a new set of experiences and skills to the problem are the most useful.

[PT1.2: 98] Feed results to the defect management and mitigation system.

Penetration testing results are fed back to development through established defect management or mitigation channels, with development responding via a defect management and release process. Emailing the results to various people doesn’t generate useful results. Properly done, this exercise demonstrates the organization’s ability to improve the state of security, and many firms are emphasizing the critical importance of not just identifying but actually fixing security problems. One way to ensure attention is to add a security flag to the bug-tracking and defect management system. The organization might leverage developer workflow or social tooling (e.g., Slack, JIRA) to communicate change requests, but those requests are still tracked explicitly as part of a vulnerability management process. 

[PT1.3: 88] Use penetration testing tools internally.

The organization creates an internal penetration testing capability that uses tools. This capability (e.g., group, team) can be part of the SSG or part of a specialized team elsewhere in the organization, with the tools complementing manual efforts to improve the efficiency and repeatability of the testing process. Tools used usually include off-the-shelf products built specifically for application penetration testing, network penetration tools that specifically understand the application layer, container and cloud configuration testing tools, and custom scripts. Free-time or crisis-driven efforts aren’t the same as an internal capability.

Penetration Testing Level 2

[PT2.2: 33] Penetration testers use all available information.

Penetration testers, whether internal or external, use source code, design documents, architecture analysis results, misuse and abuse cases, code review results, and cloud environment and other deployment configuration to do deeper analysis and find more interesting problems. To effectively do their job, penetration testers often need everything created throughout the SSDL, so an SSDL that creates no useful artifacts about the code will make this effort harder. Having access to the artifacts is not the same as using them.

[PT2.3: 34] Schedule periodic penetration tests for application coverage.

The SSG collaborates in periodically testing all applications in its purview according to an established schedule, which could be tied to a calendar or a release cycle. High-profile applications should get a penetration test at least once a year, even if new releases haven’t yet moved into production. This testing serves as a sanity check and helps ensure that yesterday’s software isn’t vulnerable to today’s attacks. The testing can also help maintain the security of software configurations and environments, especially containers and components in the cloud. One important aspect of periodic testing is to make sure that the problems identified are actually fixed and don’t creep back into the build. New automation created for CI/CD deserves penetration testing as well.

Penetration Testing Level 3

[PT3.1: 23] Use external penetration testers to perform deep-dive analysis.

The organization uses external penetration testers to do deep-dive analysis for critical projects and to introduce fresh thinking into the SSG. These testers should be domain experts and specialists who keep the organization up to speed with the latest version of the attacker’s perspective and have a track record for breaking the type of software being tested. Skilled penetration testers will always break a system, but the question is whether they demonstrate new kinds of thinking about attacks that can be useful when designing, implementing, and hardening new systems. Creating new types of attacks from threat intelligence and abuse cases typically requires extended timelines, which is essential when it comes to new technologies, and prevents checklist-driven approaches that look only for known types of problems.

[PT3.2: 12] Customize penetration testing tools.

The SSG collaborates in either creating penetration testing tools or adapting publicly available ones to more efficiently and comprehensively attack the organization’s software. Tools will improve the efficiency of the penetration testing process without sacrificing the depth of problems that the SSG can identify. Automation can be particularly valuable in organizations using agile methodologies because it helps teams go faster. Tools that can be tailored are always preferable to generic tools. Success here is often dependent upon both the depth of tests and their scope.