Penetration Testing

The Penetration Testing practice involves standard outside-in testing of the sort carried out by security specialists. Penetration testing focuses on vulnerabilities in the final configuration and provides direct feeds to defect management and mitigation.

Penetration Testing Level 1

[PT1.1: 114] Use external penetration testers to find problems.

External penetration testers are used to demonstrate that the organization’s software needs help. Finding critical vulnerabilities in high-profile applications provides the evidence that executives often require. Over time, the focus of penetration testing moves from trying to determine if the code is broken in some areas to a sanity check done before shipping or on a periodic basis. External penetration testers who bring a new set of experiences and skills to the problem are the most useful.

[PT1.2: 102] Feed results to the defect management and mitigation system.

All penetration testing results are fed back to engineering through established defect management or mitigation channels, with development and operations responding via a defect management and release process. In addition to application vulnerabilities, also track results from testing other software such as containers and infrastructure configuration. Properly done, this exercise demonstrates the organization’s ability to improve the state of security and emphasizes the importance of not just identifying but actually fixing security problems. One way to ensure attention is to add a security flag to the bug-tracking and defect management system. The organization might leverage developer workflow or social tooling (e.g., JIRA, Slack) to communicate change requests, but these requests are still tracked explicitly as part of a vulnerability management process.

[PT1.3: 88] Use penetration testing tools internally.

The organization creates an internal penetration testing capability that uses tools as part of an established process. Execution can be part of the SSG or part of a specialized team elsewhere in the organization, with the tools complementing manual efforts to improve the efficiency and repeatability of the testing process. The tools used will usually include off-the-shelf products built specifically for application penetration testing, network penetration tools that specifically understand the application layer, container and cloud configuration testing tools, and custom scripts. Free-time or crisis- driven efforts aren’t the same as an internal capability.

Penetration Testing Level 2

[PT2.2: 38] Penetration testers use all available information.

Penetration testers, whether internal or external, routinely make use of all artifacts created throughout the SSDL to do more comprehensive analysis and find more problems. Example artifacts include source code, design documents, architecture analysis results, misuse and abuse cases, code review results, and cloud environment and other deployment configurations. Focusing on high-risk applications is a good way to start. An SSDL that creates no useful artifacts about the code will make this effort harder. Having access to the artifacts is not the same as using them.

[PT2.3: 45] Schedule periodic penetration tests for application coverage.

All applications are tested periodically, which could be tied to a calendar or a release cycle. High-risk applications could get a penetration test at least once per year, for example, even if there have not been substantive code changes—other applications might receive different kinds of security testing on a similar schedule. Any security testing performed must focus on discovering vulnerabilities, not just checking a process or compliance box. This testing serves as a sanity check and helps ensure that yesterday’s software isn’t vulnerable to today’s attacks. The testing can also help maintain the security of software configurations and environments, especially for containers and components in the cloud. One important aspect of periodic security testing across the portfolio is to make sure that the problems identified are actually fixed. Software that isn’t an application, such as automation created for CI/CD, infrastructure-as- code, and so on, deserves some security testing as well.

Penetration Testing Level 3

[PT3.1: 26] Use external penetration testers to perform deep-dive analysis.

The SSG uses external penetration testers to do a deep-dive analysis on critical software systems or technologies and to introduce fresh thinking. One way to do this is to simulate persistent attackers using goal-oriented red team exercises. These testers are domain experts and specialists who keep the organization up to speed with the latest version of the attacker’s perspective and have a track record for breaking the type of software being tested. When attacking the organization’s software, these testers demonstrate a creative approach that provides useful knowledge to the people designing, implementing, and hardening new systems. Creating new types of attacks from threat intelligence and abuse cases typically requires extended timelines, which is essential when it comes to new technologies, and prevents checklist-driven approaches that look only for known types of problems.

[PT3.2: 15] Customize penetration testing tools.

Build a capability to create penetration testing tools or adapt publicly available ones to attack the organization’s software more efficiently and comprehensively. Creating penetration testing tools requires a deep understanding of attacks (see [AM2.1], [AM3.1]) and technology stacks (see [AM2.2]). Customizing existing tools goes beyond configuration changes and extends tool functionality to find new issues. Tools will improve the efficiency of the penetration testing process without sacrificing the depth of problems that the SSG can identify. Automation can be particularly valuable in organizations using Agile methodologies because it helps teams go faster. Tools that can be tailored are always preferable to generic tools. Success here is often dependent upon both the depth and scope of tests enabled through customized tools.