Sorry, not available in this language yet
FAQ
Here you will find frequently asked questions about BSIMM. For the full and unexpurgated model, download the BSIMM report here.
Who needs to worry about software security?
Who's actually responsible for software security inside these companies?
Yikes, 125 activities sounds like a lot—why so many?
Why are some activities highlighted in the BSIMM scorecard?
I'm more of a visual person. What does this study look like graphically?
Is everybody in the study equally good at software security?
What is a software security group? Do I have to have one?
If BSIMM is so important, how has the world gotten along without it for so long?
I get it, but my boss doesn't. How do software security initiatives get off the ground?
What is BSIMM?
BSIMM (pronounced “bee simm”) is short for Building Security In Maturity Model. The BSIMM report is a study of real-world software security initiatives (SSIs) organized so that you can determine where you stand with your software security initiative and how to evolve your efforts over time.
Why software security?
Software security is about building software to be secure even when it’s under attack. As we’ve learned from years of reviewing network security breaches, protecting software is much easier if the software is built with security in mind. Furthermore, security is a property and not a thing, so software security—being resistant to attack—involves much more than simply adding security features like encryption or passwords to software.
Who needs to worry about software security?
Organizations that develop and depend on software to do business (and everybody does these days) need software that won’t leak millions of identity records, call election results into question, incur huge legal liabilities, or allow secrets to fall into the wrong hands. The only way to make software trustworthy is to build security in. In short, everyone who relies on software needs BSIMM.
What makes BSIMM so special?
We built BSIMM entirely from the observations we made by studying real software security initiatives. BSIMM doesn’t tell you what you should do; instead, it tells you what everyone else is actually doing. This “observe and report” approach to software security science stands in sharp contrast to prescriptive approaches based on personal experience.
Whom did you study?
There are 130 firms included in BSIMM13. On average, they had practiced software security for five years at the time of their current assessment. All 130 firms agree that the success of their initiatives hinges on having an internal group devoted to software security—a software security group (SSG). The average size of an SSG is 25.7 people (the smallest is 1, the largest is 892, and the median is 8.0). Often there is a satellite group of others (developers, architects, and people in the organization directly engaged in and promoting software security), and that group on average consists of 112 people (the median is 40). The average number of developers in participating organizations is 2,146 (the smallest is 25, the largest is 100,000, and the median is 800), yielding an average ratio of SSG to development of 5.11%.
All told, the BSIMM report describes the work of 11,850 SSG members helping about 410,000 developers do good security work on about 145,000 applications.
Who’s actually responsible for software security inside these companies?
The executives in charge of the software security initiatives we studied have a variety of titles. Examples include
- AVP of corporate security
- Director of AppSec
- Director of security assurance
- VP of InfoSec
- VP of application risk management
- Senior director of shared services
- Manager of software security engineering
- Senior director of enterprise software
- Senior director of infrastructure and security
- Senior director of global InfoSec innovation
- Global head of security testing
- System manager of systems development
- Security architect
- Executive director of product operations
- Manager of InfoSec operations
- Information systems manager
- Embedded systems cybersecurity leader
We observe a fairly wide variety in terms of where the SSG is situated in an organization. In BSIMM-V, we saw CISOs as the nearest executive in 21 of 67 firms, which grew in BSIMM6 to 31 of 78, and again in BSIMM7 with 52 of 95. Since then, the percentage has remained relatively flat even as the BSIMM community has grown. Figure 1 shows the variety of executives that oversee software security efforts.
Figure 1: Nearest executive to the SSG
What’s in BSIMM?
BSIMM’s primary organizing feature is its software security framework. That framework comprises four domains—governance, intelligence, SSDL touchpoints, deployment—that include 12 practices.
- Governance: Strategy and metrics, compliance and policy, training
- Intelligence: Attack models, security features and design, standards and requirements
- SSDL touchpoints: Architecture analysis, code review, security testing
- Deployment: Penetration testing, software environment, configuration management and vulnerability management
Each practice includes related activities, for a total of 125 activities observed in BSIMM13. During the study, we kept track of how many times each activity was observed. (To interpret individual activities, download a copy of the BSIMM report, which describes each of the 125 activities in detail.)
For each activity, we give a description and one or more real examples to illustrate how organizations make it happen. The examples are never the only way to conduct a given activity, but we think they’re helpful for understanding software security reality.
Figure 2: BSIMM13 Example Firm Scorecard
Yikes, 125 security activities sound like a lot—why so many?
Don’t worry—BSIMM is an observational model, which means that when we see an activity being conducted in multiple participant organizations, we add it to the model. The model is cumulative, and no organization carries out all activities. Over the years, we’ve found a surprising amount of common ground between the financial services organizations, independent software vendors (ISVs), and IoT companies we studied, but their initiatives are by no means identical, and every organization is at least a little bit different. You wouldn’t implement a direct copy of a friend’s financial plan, so you shouldn’t expect to lift someone else’s software security initiative either. Use BSIMM as a source of ideas and general guidance—as a trail guide rather than as a cookbook.
Why are some activities highlighted in the BSIMM scorecard?
The 12 activities listed in Figure 3 are those we observed most often in each practice.
Figure 3: Most common activity by practice
I’m more of a visual person. What does this look like graphically?
To give you some idea of the analysis capabilities provided by BSIMM, here are three spider charts showing the average maturity level of a number of organizations for the 12 practices. Figure 4 shows data from all BSIMM firms (AllFirms). Figure 5 shows data from technology firms plotted against nontechnology firms.
Figure 4: AllFirms data
Figure 5: Technology firms vs. nontechnology firms
Three verticals in BSIMM operate in highly regulated industries: financial services, healthcare, and insurance. In our long experience with BSIMM, we’ve seen large financial services firms react to regulatory pressures by starting SSIs earlier than insurance and healthcare firms. However, for the first time, the SSG average ages for financial services and insurance firms are now the same, 5.2 years, compared to 4.5 years in healthcare firms. Despite the narrowing of this age difference, financial services firms still display higher maturity. This likely reflects a longer history of software security activity in the financial services vertical, coupled with an influx of younger financial services firms that have comparatively new but relatively mature SSGs.
Although organizations in the healthcare vertical include some mature outliers, the data for these three regulated verticals shows that it lags the others in most practices, but is ahead in architecture analysis, as shown in Figure 6.
Compared to financial services firms, the insurance vertical is ahead in security testing but close or lagging in other practices. The biggest differences between the insurance and financial services verticals are in compliance and policy, security features and design, penetration testing, and configuration management and vulnerability management, where the financial services vertical leads insurance.
Figure 6: Highly regulated industries
Is everybody in the study equally good at software security?
No. By computing a score for each firm in the study, we can also look at relative maturity and average maturity for one firm compared to the others. For BSIMM13, the range of BSIMM scores was between 41 to 50, vs. 31 to 40 in BSIMM12.
As evident in Figure 7, in general as firms mature and continue to use BSIMM as a measurement tool over time (e.g., round 2, round 3+), they tend to have higher scores.
Figure 7: BSIMM13 Score Distribution
Is BSIMM a standard?
BSIMM is not a standard like ISO 27001 or the official rules of table tennis. Instead, BSIMM describes the set of activities practiced by the most successful software security initiatives in the world. In that sense, it is a de facto standard because it’s what organizations actually do. You could say we discovered it rather than invented it.
What should I do with BSIMM?
If you don’t have a software security initiative, you need one. And you can use BSIMM to get started: It can help you figure out how many people you’ll need in your software security group, what those people should do first, and what kinds of things they’ll probably be thinking about in a few years. If you already have a software security initiative, you can use BSIMM to learn where you stand and make plans for the future.
How much does BSIMM cost?
BSIMM is free: We released it under a Creative Commons license. This means it’s as “open” as any other model, and you can take it and use it as inspiration for your own internal documents, or use our published data to make a model of your own. If you do those things, you’re required to tell people where the material came from. In other words, point back to BSIMM. If you need a little help, contact us.
What is a software security group? Do I have to have one?
All BSIMM participants have an internal group devoted to software security—the software security group (SSG). We’ve never observed an organization carrying out the activities in BSIMM successfully without an SSG. We noted an average ratio of SSG to development of 3.01% across the 130 organizations we studied. For organizations with 800 developers or fewer, the largest ratio observed was 51.4% and the smallest was 5.11%. For organizations with more than 800 developers, the largest ratio observed was 14.9% and the smallest was 1.03%. To remind you of the particulars in terms of actual bodies, SSG size on average among the 130 firms is 25.7 people (the smallest is 1, the largest is 892, and the median is 8).
If BSIMM is so important, how has the world gotten along without it for so long?
For many software makers (including ISVs, banks, health care firms, governments, and others), software security has been a twenty-first-century concern at best, and an executive-level concern for 10 years at most. The collective “we” are just now reaching the point where we’ve accumulated enough experience to compare notes and talk about what works at a macro level. Secure programming, penetration testing, and the like have been topics for a while now, but the best methods for organizing software security initiatives have taken longer to emerge. BSIMM captures those activities into an observational model that is free and open for everyone to use.
I get it, but my boss doesn’t. How do software security initiatives get off the ground?
Over the years of BSIMM, we’ve seen the same need to justify software security as we saw back in the days before IT security became mainstream. Back then, some executives simply didn’t get why firewalls were necessary, or how intrusion detection helped prevent a small issue from becoming a big one, or how simply teaching people how to think about security could actually change corporate culture. In those early days, even managers who understood the problem intellectually were sometimes sorely tempted to see just how long the firm could wait before it became their turn to be victimized.
In the BSIMM data pool, we’ve seen software security groups get their charter and funding under the following broad sets of circumstances:
- Executive management reacted to ongoing events and decreed that they would make secure software and fund the means to do it (e.g., Bill Gates’ 2002 security memo at Microsoft, which directed that security be a primary consideration).
- An established upper management group responsible for some form of compliance determined—possibly after costly failures—that software security was a necessary expense in the firm’s governance processes.
- The IT security crew proved that a breach was not their fault but rather a result of a security vulnerability in the firm’s applications.
- An influential software security entrepreneur (e.g., in the CIO, CTO, legal, or governance group) worked the system to get the ball rolling and then parlayed early successes into funding for an actual program.
- A development group organically developed a variety of “Sec” roles as part of their journey into DevOps (e.g., CloudSec, BuildSec, ContainerSec), and an executive standardized these efforts into a program to push knowledge and process into all development groups.
It seems that the hard part these days isn’t necessarily selling upper management on the problem but convincing them that you’re the right person to lead the solution, and that you actually have a plan. If you’re responsible for software security—even if it’s just in the sense that you’re the person who would be fired after a major software security failure—and you can’t get resources for a program that will address the issues, quit now. Life is just too short for that kind of nonsense, and there are plenty of employers willing to make real use of your abilities.
What are a few key themes highlighted by the latest BSIMM study?
Shift everywhere
Starting more than 15 years ago, the “shift left” trend drove firms to focus on moving security testing earlier in the development process. Now, “shift everywhere” extends the trend into making security testing continuous throughout the SDLC. A shift everywhere approach is useful for more than just testing for vulnerabilities in a timely fashion; it also facilitates automating governance checks and measuring risk in all phases of the SDLC.
Software supply chain risk management
Increased media attention on supply chain instability and critical vulnerabilities discovered in third-party libraries has increased executive attention on software risk that doesn’t originate in a firm’s own SDLC. Efforts to manage software supply chain risk are focusing on tracking and securing software that is integrated into software built in-house and ensuring that software suppliers follow best practices.
Security integration into developer toolchains
Developers and software vendors continue to make progress in integrating security options into CI/CD pipelines and toolchains. These integrations provide faster and tighter processes that reduce friction, improve coverage, and make the shift everywhere concept a reality.
Expanding software security beyond applications and products
Application security teams have had to follow whenever software security has expanded, including infrastructure-as-code, containers, cloud platforms, and more. The key to keeping up with new security domains is redefining how the SSI is positioned within the firm. It’s no longer enough to have a hands-off, test-and-enforce approach to security. Instead, SSI leaders need to become thought leaders, influencers, mentors, and enablers of their peers to ensure that infrastructure is built securely.
