Here you will find frequently asked questions about the BSIMM. For the full and unexpurgated model, download the BSIMM document here.
Here you will find frequently asked questions about the BSIMM. For the full and unexpurgated model, download the BSIMM document here.
Who needs to worry about software security?
What makes the BSIMM so special?
Who's actually responsible for software security inside these companies?
Yikes, 121 activities sounds like a lot—why so many?
Why are some activities highlighted in the BSIMM scorecard?
I'm more of a visual person. What does this study look like graphically?
Is everybody in the study equally good at software security?
What should I do with the BSIMM?
What is a software security group? Do I have to have one?
If the BSIMM is so important, how has the world gotten along without it for so long?
I get it, but my boss doesn't. How do software security initiatives get off the ground?
What are a few key themes highlighted by the latest BSIMM study?
BSIMM (pronounced “bee simm”) is short for Building Security In Maturity Model. The BSIMM is a study of real-world software security initiatives organized so that you can determine where you stand with your software security initiative and how to evolve your efforts over time.
Software security is about building software to be secure even when it’s under attack. As we’ve learned from years of reviewing network security breaches, protecting software is much easier if the software is built with security in mind. Furthermore, security is a property and not a thing, so software security—being resistant to attack—involves much more than simply adding security features like encryption or passwords to software.
Organizations that develop and depend on software to do business (and everybody does these days) need software that won’t leak millions of identity records, call election results into question, incur huge legal liabilities, or allow secrets to fall into the wrong hands. The only way to make software trustworthy is to build security in. In short, everyone who relies on software needs the BSIMM.
We built the BSIMM entirely from observations we made by studying real software security initiatives. The BSIMM does not tell you what you should do; instead, it tells you what everyone else is actually doing. This “observe and report” approach to software security science stands in sharp contrast to prescriptive approaches based on personal experience.
There are 130 firms included in BSIMM11. On average, they had practiced software security for 4.3 years at the time of their current assessment (with values ranging from less than a year to 15 years as of July 2020). All 130 firms agree that the success of their initiatives hinges on having an internal group devoted to software security—a software security group (SSG). The average size of an SSG is 13.9 people (the smallest is 1, the largest is 160, and the median is 7.0). Often there is a satellite group of others (developers, architects, and people in the organization directly engaged in and promoting software security), and that group on average consists of 51.2 people (the smallest is 0, the largest is 1,500, and the median is 0). The average number of developers in participating organizations is 3,770.5 (the smallest is 5, the largest is 100,000, and the median is 1,000), yielding an average ratio of SSG to development of 2.01% (the median is 0.63%).
All told, the BSIMM describes the work of 1,801 SSG members working with a satellite of 6,656 people to secure the software—nearly 176,269 applications—developed by 490,167 developers.
The executives in charge of the software security initiatives we studied have a variety of titles. Examples include:
We observe a fairly wide variety in terms of where the SSG is situated in an organization. In particular, 67 of the 130 participating firms have SSGs that are run by a CISO or report to a CISO as their nearest senior executive. And 21 of the firms report to a CTO as their closest senior executive; 6 report to a CIO, 8 to a CSO, 4 to a COO, 2 to a CRO, and 1 to a CAO. There are 20 SSGs that report through some type of technology or product organization.
The BSIMM’s primary organizing feature is its software security framework. That framework comprises four domains—governance, intelligence, SSDL touchpoints, deployment—that include 12 practices:
Each practice includes related activities, for a total of 121 activities observed in BSIMM11. During the study, we kept track of how many times each activity was observed. The table below shows the resulting data. (To interpret individual activities, download a copy of the BSIMM, which describes each of the 121 activities in detail.)
For each activity, we give a description and one or more real examples to illustrate how organizations make it happen. The examples are never the only way to conduct a given activity, but we think they’re helpful for understanding software security reality.
Don’t worry—the BSIMM is an observational model, which means that when we see an activity being conducted in multiple participant organizations, we add it to the model. The model is cumulative, and no organization carries out all activities. Over the years, we’ve found a surprising amount of common ground between the financial services organizations, independent software vendors (ISVs), and IoT companies we studied, but their initiatives are by no means identical, and every organization is at least a little bit different. You wouldn’t implement a direct copy of a friend’s financial plan, so you shouldn’t expect to lift someone else’s software security initiative either. Use the BSIMM as a source of ideas and general guidance—as a trail guide rather than as a cookbook.
The 12 highlighted activities are those we observed most often in each practice.
Activity | Description |
[SM1.4] | Implement lifecycle governance |
[CP1.2] | Identify PII obligations |
[T1.1] | Conduct awareness training |
[AM1.2] | Create a data classification scheme and inventory |
[SFD1.1] | Integrate and deliver security features |
[SR1.3] | Translate compliance constraints to requirements |
[AA1.1] | Perform security feature review |
[CR1.4] | Use automated tools along with manual review |
[ST1.1] | Ensure QA performs edge/boundary value condition testing |
[PT1.1] | Use external penetration testers to find problems |
[SE1.2] | Ensure host and network security basics are in place |
[CMVM1.1] | Create or interface with incident response |
To give you some idea of the analysis capabilities provided by the BSIMM, here are three spider charts showing average maturity level over some number of organizations for the 12 practices. The first chart shows data from all BSIMM firms (which we call AllFirms). The second chart shows data from a sample firm plotted against AllFirms.
Three verticals in the BSIMM operate in highly regulated industries: insurance, health care, and financial services. In our experience, large financial services firms reacted first to the regulatory changes of the 1990s and early 2000s and started their SSIs much earlier than insurance and health care firms. Even as the number of financial services firms doubled over the past five years (adding a large influx into the BSIMM data pool of newly started initiatives), the average age of financial services SSG at assessment time remains 4.9 years, versus 3.8 years for insurance and 3.7 years for health care.
Time spent maturing their collective SSIs shows up clearly in the side-by-side comparison. Although the insurance vertical includes some mature outliers, the data for these three regulated verticals show insurance generally lags behind in software security. We see a starker contrast in health care, with virtually no outliers.
No. By computing a score for each firm in the study, we can also take a look at relative maturity and average maturity for one firm against the others. The majority of BSIMM11 participants have a score in the 16 to 45 range, with an average SSG age of 2.5 to 4.2 years.
We’re pleased that the BSIMM continues to grow year after year. The overall dataset in BSIMM11 is nearly 38 times the size it was for the original publication. Note that once we exceeded a sample size of 30 firms, we began to apply statistical analysis, yielding statistically significant results.
The BSIMM is not a standard like ISO 27001 or the official rules of table tennis. Instead, the BSIMM describes the set of activities practiced by the most successful software security initiatives in the world. In that sense, it is a de facto standard because it’s what organizations actually do. You could say we discovered it rather than invented it.
If you don’t have a software security initiative, you need one. And you can use the BSIMM to get started: It can help you figure out how many people you’ll need in your software security group, what those people should do first, and what kinds of things they’ll probably be thinking about in a few years. If you already have a software security initiative, you can use the BSIMM to learn where you stand and make plans for the future.
The BSIMM is free: We released it under a Creative Commons license. This means it’s as “open” as any other model, and you can take it and use it as inspiration for your own internal documents, or use our published data to make a model of your own. If you do those things, you’re required to tell people where the material came from. In other words, point back to the BSIMM. If you need a little help, contact us.
All BSIMM participants have an internal group devoted to software security—the software security group (SSG). We’ve never observed an organization carrying out the activities in the BSIMM successfully without an SSG. We noted an average ratio of SSG to development of 2.01% across the 130 organizations we studied. That means one SSG member for every 50 developers when we average the ratios for each participating firm. For organizations with 500 developers or fewer, the largest ratio observed was 51.4% and the smallest was 0.4%. For organizations with more than 500 developers, the largest ratio observed was 3.0% and the smallest was 0.1%. To remind you of the particulars in terms of actual bodies, SSG size on average among the 130 firms is 13.9 people (the smallest is 1, the largest is 160, and the median is 7.0).
For many software makers (including ISVs, banks, health care firms, governments, and others), software security has been a twenty-first-century concern at best, and an executive-level concern for 10 years at most. The collective “we” are just now reaching the point where we’ve accumulated enough experience to compare notes and talk about what works at a macro level. Secure programming, penetration testing, and the like have been topics for a while now, but the best methods for organizing software security initiatives have taken longer to emerge. The BSIMM captures those activities into an observational model that is free and open for everyone to use.
Over the years of the BSIMM, we’ve seen the same need to justify software security as we saw back in the days before IT security became mainstream. Back then, some executives simply didn’t get why firewalls were necessary, or how intrusion detection helped prevent a small issue from becoming a big one, or how simply teaching people how to think about security could actually change corporate culture. In those early days, even managers who understood the problem intellectually were sometimes sorely tempted to see just how long the firm could wait before it became their turn to be victimized.
In the BSIMM data pool, we’ve seen software security groups get their charter and funding under the following broad sets of circumstances:
It seems that the hard part these days isn’t necessarily selling upper management on the problem but convincing them that you’re the right person to lead the solution, and that you actually have a plan. If you’re responsible for software security—even if it’s just in the sense that you’re the person who would be fired after a major software security failure—and you can’t get resources for a program that will address the issues, quit now. Life is just too short for that kind of nonsense, and there are plenty of employers willing to make real use of your abilities.
“Shift left” is becoming “shift everywhere”
BSIMM data shows that “shift left” is becoming “shift everywhere,” and there is a growing preference for continuous, event-based security telemetry throughout a value stream rather than a single, point-in-time analysis.
Engineering-led vs. SSG-led software security efforts
Increased reliance on engineering-led activities is a theme throughout BSIMM11, where resiliency is a primary goal for software rather than simply security or quality.
DevSecOps
BSIMM data shows an increased adoption of DevSecOps principles across organizations, incorporating security activities throughout the CI/CD pipeline and focusing on rapid deployment of secure and resilient software.
What leading brands are doing to address appsec
BSIMM11 describes activity trends among the 130 firms in the data pool and notes activities observed across emerging, maturing, and optimizing level SSIs. BSIMM data shows that high-maturity initiatives carry out numerous activities in all 12 of the practices described by the model.