Code Review

The Code Review practice includes use of code review tools, development of tailored rules, customized profiles for tool use by different roles (for example, developers versus auditors), manual analysis, and tracking/measuring results.

Code Review Level 1

[CR1.2: 80] Perform opportunistic code review.

The SSG ensures code review for high-risk applications is performed in an opportunistic fashion, such as by following up a design review with a code review looking for security issues in not only source code and dependencies but also deployment artifact configuration (e.g., containers) and automation metadata (e.g., infrastructure-as-code). This informal targeting often evolves into a systematic approach. Code review could involve the use of specific tools and services, or it might be manual, but it has to be part of a proactive process. When new technologies pop up, new approaches to code review might become necessary.

[CR1.4: 102] Use automated tools.

Static analysis incorporated into the code review process makes the review more efficient and consistent. Automation won’t replace human judgement, but it does bring definition to the review process and security expertise to reviewers who typically aren’t security experts. Note that a specific tool might not cover an entire portfolio, especially when new languages are involved, so additional local effort might be useful. Some organizations might progress to automating tool use by instrumenting static analysis into source code management workflows (e.g., pull requests) and delivery pipeline workflows (build, package, and deploy) to make the review more efficient, consistent, and in line with release cadence. Whether use of automated tools is to review a portion of the source code incrementally, such as a developer committing new code or small changes, or to conduct full-program analysis by scanning the entire codebase, this service should be explicitly connected to a larger SSDL defect management process applied during software development, not just used to “check the security box” on the path to deployment.

[CR1.5: 49] Make code review mandatory for all projects.

Code review is mandatory for all projects under the SSG’s purview, with a lack of code review or unacceptable results stopping a release, slowing it down, or causing it to be recalled. While all projects must undergo code review, the process might be different for different kinds of projects. The review for low-risk projects might rely more heavily on automation, for example, whereas high-risk projects might have no upper bound on the amount of time spent by reviewers. Having a minimum acceptable standard forces projects that don’t pass to be fixed and reevaluated. A code review tool with nearly all the rules turned off (so it can run at CI/CD automation speeds, for example) won’t provide sufficient defect coverage. Similarly, peer code review or tools focused on quality and style won’t provide useful security results.

[CR1.6: 32] Use centralized reporting to close the knowledge loop.

The bugs found during code review are tracked in a centralized repository that makes it possible to do both summary and trend reporting for the organization. The code review information can be incorporated into a CISO-level dashboard that might include feeds from other parts of the security organization (e.g., penetration tests, security testing, black-box testing, and white-box testing). Given the historical code review data, the SSG can also use the reports to demonstrate progress and drive the training curriculum (see [SM2.5 Identify metrics and use them to drive budgets]). Individual bugs make excellent training examples. Some organizations have moved toward analyzing this data and using the results to drive automation.

[CR1.7: 51] Assign tool mentors.

Mentors are available to show developers how to get the most out of code review tools. If the SSG has the most skill with the tools, it could use office hours or other outreach to help developers establish the right configuration or get started on interpreting results. Alternatively, someone from the SSG might work with a development team for the duration of the first review they perform. Centralized use of a tool can be distributed into the development organization or toolchains over time through the use of tool mentors, but providing installation instructions and URLs to centralized tools isn’t the same as mentoring. Increasingly, mentorship extends to tools associated with deployment artifacts (e.g., container security) and infrastructure (e.g., cloud configuration). In many organizations, satellite members (e.g., champions) take on the tool mentorship role. 

Code Review Level 2

[CR2.6: 25] Use automated tools with tailored rules.

Custom rules are created and used to help uncover security defects specific to the organization’s coding standards or the framework-based or cloud-provided middleware it uses. The same group that provides tool mentoring (see [CR1.7 Assign tool mentors]) will likely spearhead this customization. Tailored rules can be explicitly tied to proper usage of technology stacks in a positive sense and avoidance of errors commonly encountered in a firm’s codebase in a negative sense. To reduce the workload for everyone, many organizations also create rules to remove repeated false positives and turn off checks that aren’t relevant.

[CR2.7: 17] Use a top N bugs list (real data preferred).

The SSG maintains a living list of the most important kinds of bugs that it wants to eliminate from the organization’s code and uses it to drive change. Many organizations start with a generic list pulled from public sources, but lists such as the OWASP Top 10 rarely reflect an organization’s bug priorities. The list’s value comes from being specific to the organization, built from real data gathered from code review, testing, software composition analysis, and actual incidents, and prioritized for prevention efforts. Simply sorting the day’s bug data by number of occurrences won’t produce a satisfactory list because these data change so often. To increase interest, the SSG can periodically publish a “most wanted” report after updating the list. One potential pitfall with a top N list is that it tends to only include known problems. Of course, just building the list won’t accomplish anything; everyone has to use it to fix bugs.

Code Review Level 3

[CR3.2: 9] Build a capability to combine assessment results.

Combine assessment results so that multiple analysis techniques feed into one reporting and remediation process. In addition to code review, analysis techniques might include dynamic analysis, software composition analysis, container scanning, cloud services monitoring, and so on. The SSG might write scripts to gather data automatically and combine the results into a format that can be consumed by a single downstream review and reporting solution. The tricky part of this activity is normalizing vulnerability information from disparate sources that use conflicting terminology. In some cases, using a standardized taxonomy (e.g., a CWE-like approach) can help with normalization. Combining multiple sources helps drive better-informed risk mitigation decisions.

[CR3.3: 4] Create capability to eradicate bugs.

When a new kind of bug is discovered in the firm’s software, the SSG ensures rules are created to find it and helps use these rules to identify all occurrences of the new bug throughout the codebases and runtime environments. It becomes possible to eradicate the bug type entirely without waiting for every project to reach the code review portion of its lifecycle. A firm with only a handful of software applications built on a single technology stack will have an easier time with this activity than firms with many large applications built on a diverse set of technology stacks. A new development framework or library, rules in RASP or a next-generation firewall, or using cloud configuration tools to provide guardrails can often help in eradication efforts.

[CR3.4: 1] Automate malicious code detection.

Automated code review is used to identify dangerous code written by malicious in-house developers or outsource providers. Examples of malicious code that could be targeted include backdoors, logic bombs, time bombs, nefarious communication channels, obfuscated program logic, and dynamic code injection. Although out-of-the-box automation might identify some generic malicious-looking constructs, custom rules for the static analysis tools used to codify acceptable and unacceptable code patterns in the organization’s codebase will quickly become a necessity. Manual code review for malicious code is a good start but insufficient to complete this activity at scale. While not all backdoors or similar code were meant to be malicious when they were written (e.g., a developer’s feature to bypass authentication during testing), such things have a tendency to stay in deployed code and should be treated as malicious code until proven otherwise.

[CR3.5: 0] Enforce coding standards.

The enforced portions of an organization’s secure coding standards often start out as a simple list of banned functions, with a violation of these standards being sufficient grounds for rejecting a piece of code. Other useful coding standard topics might include proper use of cloud APIs, use of approved cryptography, memory sanitization, and many others. Code review against standards must be objective: it shouldn’t become a debate about whether the noncompliant code is exploitable. In some cases, coding standards are specific to language constructs and enforced with tools (e.g., codified into SAST rules). In other cases, published coding standards are specific to technology stacks and enforced during the code review process or using automation. Standards can be positive (“do it this way”) or negative (“do not use this API”), but they must be enforced to be useful.