Software Security

Being Explicit About Security Weaknesses

Robert A. Martin
MITRE Corporation

Software acquirers want assurance that the products they are obtaining are reviewed for known types of security weaknesses.
Acquisition groups in large government and private organizations are beginning to use such reviews as part of future con-
tracts, but the tools and services for performing them are new and, until recently, there was no nomenclature, taxonomy, or
standard to define their capabilities and coverage. A standard dictionary of software security weaknesses has been created by

M
the community to serve as a unifying language of discourse and as a measuring stick for comparing tools and services.
ore and more organizations want that are found in today’s products. As an cussed above and the result was called the
assurance that the software products alternate approach, under sponsorship of Preliminary List of Vulnerability Examples
they acquire and develop are free of DHS NCSD, and as part of MITRE’s par- for Researchers (PLOVER) [6]. PLOVER
known types of security weaknesses. ticipation in the DHS-sponsored NIST was a document that listed more than 1,500
High-quality tools and services for finding SAMATE effort, MITRE investigated the diverse, real-world examples of vulnerabil-
security weaknesses in code are new. The possibility of leveraging the Common ities, identified by their CVE name. The
question of which tool/service is appro- Vulnerabilities and Exposures (CVE) ini- vulnerabilities are organized within a
priate/better for a particular job is hard to tiative’s experience in analyzing more than detailed conceptual framework that enu-
answer given the lack of structure and 20,000 real-world vulnerabilities reported merates the 300 individual types of weak-
definition in the software product assess- and discussed by industry and academia. nesses that cause the vulnerabilities. The
ment industry. As part of the creation of the CVE list weaknesses were simply grouped within 28
There are several ongoing efforts to [4] that is used as the source of vulnera- higher-level categories with a large number
begin to resolve some of these shortcom- bilities for the National Vulnerability of real-world vulnerability examples for
ings, including the Department of Database [5], MITRE’s CVE initiative each type of weakness. PLOVER repre-
Homeland Security (DHS) National Cyber during the last six years has developed a sents the first cut of a truly bottom-up
Security Division (NCSD)-sponsored preliminary classification and categoriza- effort to take real-world observed,
Software Assurance Metrics and Tool tion of vulnerabilities, attacks, faults, and exploitable vulnerabilities that do exist in
Evaluation (SAMATE) project [1] led by other concepts that can be used to help code, to abstract them and group them into
the National Institute of Standards and define this arena. However, the original common classes representing more general
Technology (NIST), the Object Manage- groupings used in the development of potential weaknesses that could lead to
ment Group (OMG) Software Assurance CVE, while sufficient for that task, were exploitable vulnerabilities in code, and
(SwA) Special Interest Group (SIG) [2], too rough to be used to identify and cate- then, finally to organize them in an appro-
and the Department of Defense (DoD)- gorize the functionality found within the priate relative structure so as to make them
sponsored Code Assessment Methodol- offerings of the code security assessment accessible and useful to a diverse set of
ogy Project, which is part of the industry. For example, in order to support audiences for a diverse set of purposes.
Protection of Vital Data effort [3] con- the development of CVE content, it is
ducted by Concurrent Technologies sufficient to separate the reported vulner- Creating a Community Effort
Corporation, among others. While these abilities in products into working cate- As part of the DoD/DHS SwA working
efforts are well placed, timely in their gories such as weak/bad authentication, groups and the NIST SAMATE project,
objectives, and will surely yield high value buffer overflow, cryptographic error, MITRE fostered the creation of a commu-
in the end, they require a common descrip- denial of service, directory traversal, nity of partners from industry, academia,
tion of the underlying security weaknesses information leak, or cross-site scripting. and government to develop, review, use,
that can lead to exploitable vulnerabilities For assessing code, however, this granular- and support a common weaknesses dictio-
in the software that they are targeted to ity of classification groupings was too nary that can be used by those looking for
resolve. Without such a common descrip- large and indefinite. Of the categories list- weaknesses in code, design, or architecture
tion, these efforts, as well as the DoD’s ed, for example, cross-site scripting actual- as well as those teaching and training soft-
own software and systems assurance ly has eight different types of issues that ware developers about the code, design, or
efforts, cannot move forward in a mean- need to be addressed or looked for when architecture weaknesses that they should
ingful fashion or be aligned and integrated assessing code; buffer overflow covered avoid due to the security problems they can
with each other to provide the needed 10 different code constructs to look for. have on applications, systems, and net-
answers to secure our networked systems. So, to support use in code assessment, works. The effort is called the Common
additional fidelity and succinctness was Weakness Enumeration (CWE) initiative.
A Different Approach needed as well as additional details and The work from PLOVER became the
Past attempts at developing this kind of descriptive information for each of the dif- major source of content for draft one of
effort have been limited by a very narrow ferent categories such as effects, behaviors, the CWE dictionary.
technical domain focus or have largely and the implementation details. The pre- An important element of the CWE
focused on high-level theories, tax- liminary classification and categorization initiative is to be transparent to all on what
onomies, or schemes that do not reach the work used in the development of CVE was we are doing, how we are doing it, and
level of detail or variety of security issues revised to address the types of issues dis- what we are using to develop the CWE

4 CROSSTALK The Journal of Defense Software Engineering March 2007

 Being Explicit About Security Weaknesses

CWE
Dictionary

• call and count the same

• enable metrics

Figure 1: The CWE Effort’s Context and Community

dictionary. We believe this transparency is the CVE List [4].
important during the initial creation of the • The Comprehensive, Lightweight While the third draft of CWE continued
Covering What Tools Find
CWE dictionary so that all of the partici- Application Security Process (CLASP) expanding the descriptions and improving
pants in the CWE community are com- from Secure Software, which yielded the consistency and linkages, subsequent
fortable with the end result and will not be more than 90 weakness concepts [8]. drafts will incorporate the specific details
hesitant about incorporating CWE into • The issues contained in Fortify’s Seven and descriptions of the 16 organizations
what they do. Figure 1 shows the overall Pernicious Kingdoms papers, which con- that have agreed to contribute their intel-
CWE context and community involve- tributed more than 110 weakness con- lectual property to the CWE initiative.
• ment ofLLC.
AppSIC, the effort. We believe the trans- cepts [9].
• MITRE Corporation Under non-disclosure agreements with
parency should also be available to partic- Working• from these collections as well MITRE, which allow the merged collec- Table 1: T
ipants and users that will visit after the ini- as those contained in the 13 other publicly tion of(NSA)
their individual contributions to be
• Aspect Security NIST
tial CWE dictionary is available on the available information sources listed on the publicly shared in the CWE List,
• Cenzic, Inc. • National Security Agency Weakness
• CWEfor
Center WebEducation
site [7]; all of the
and publicly avail- CWE
Research in Web• site sources Carolina
North page, we developed Application Security Consortium, Cenzec,
State University
able sourceAssurance
content is beingand on the the first •draftOMG
hostedSecurity/ of the CWE list, which Core Security, Coverity, Fortify,
Information Com
site for anyone to review or use for their entailed almost 500 separate weaknesses. Interoperability Clearinghouse, Klocwork,
own research and analysis. Response Team/ It took approximately six months to move Ounce Labs, Parasoft, proServices
Purdue University • Open Web Application Security Project
Currently, more than 41 organizations, from what we created in PLOVER to the Corporation, Secure Software, Security
• Computer Emergency (OWASP)
shown in Table
Coordination 1 (see page
Center 6), are partici- first draft•of Oracle
(CERT/CC) CWE. TheCorporation
CWE content is Innovation Inc., SofCheck, SPI Dynamics,
• patingInc.
Cigital, in the creation and population of captured in an XML document
• Ounce Labs, Inc. and fol- Veracode, and Watchfire are all contribut-
theScan
CWE dictionary. lows the CWE schema. Two months later, ing their knowledge and experience to
we updated CWE to draft 2 with the building out the CWE dictionary. Draft 4
• Code Labs • Palamida
incorporation of changes that included is the first draft version to include details
• CoreKick-Starting
Security Technologies • Parasoft Corporation
a Dictionary
• To continue
Coverity, Inc.the creation of the CWE dic- cleaning up • the names of items, Corporation
proServices reworking from this set of information sources.
• tionary,
Fortify we broughtInc.
Software, together as much pub- the structure, and filling Software,
• Secure in the descriptive
Inc. Draft 5 of CWE encompasses more
lic content as possible using the following details for many more of the items. The than 600 nodes with specific details and
three primary sources: first change to the CWE schema came
• Security University examples of weaknesses for many of the
• IBM • Security Innovation, Inc.
• The PLOVER collection [6] that iden- about with the addition of language and entries. Figure 2 shows the transition from
• Interoperability Clearing House
• JamestifiedMadisonmore than 300 weakness types platform ties
University for weaknessesDesigns,
• Semantic and the addi-Inc.
PLOVER to CWE drafts 1-5 and the con-
• Johns created
Hopkins by determining
University the root issues tion of specific CWE identifications
Applied • SofCheck, Inc. for tent structure changes that occurred dur-
Physics behind 1,500 of the vulnerabilities in each weakness.
Laboratory • SPI Dynamics, Inc. ing the revisions. While the initial transi-
• KDMMarch 2007
Analytics • Unisys www.stsc.hill.af.mil 5
• Kestrel Technology • VERACODE
Software Security

support the SAMATE and OMG SwA

SIG efforts that are developing software
metrics,1:software security tool metrics, the
• AppSIC, LLC. • MITRE Corporation
Table The Common
software security tool survey, the method-
• Aspect Security • NIST
• National Security Agency (NSA)
ology Community
for validating software security tool
• Cenzic, Inc. Weakness Enumeration
• Center for Education and Research in • North Carolina State University
claims, and reference datasets for testing.
Information Assurance and Security/ • OMG

For example, a major aspect of the

Purdue University • Open Web Application Security Project

SAMATE project is the development and

• Computer Emergency Response Team/ (OWASP)

open sharing of test applications that have

Coordination Center (CERT/CC) • Oracle Corporation

been salted with known weaknesses so

• Cigital, Inc. • Ounce Labs, Inc.
• Code Scan Labs • Palamida
that those wishing to see how effective a
• Core Security Technologies • Parasoft Corporation
particular tool or technique is in finding
• Coverity, Inc. • proServices Corporation

that type of weakness will have test mate-

• Fortify Software, Inc. • Secure Software, Inc.

rials readily available. These test sets are

• IBM • Security Innovation, Inc.

referred to as the SAMATE test reference

• Interoperability Clearing House • Security University

datasets (TRDs). NIST has chosen to

• James Madison University • Semantic Designs, Inc.
• Johns Hopkins University Applied • SofCheck, Inc.
organize the SAMATE TRDs by CWE
Physics Laboratory • SPI Dynamics, Inc.
weakness type and will also include vary-
• KDM Analytics • Unisys

ing levels of complexity, as appropriate to

• Kestrel Technology • VERACODE

each type of weakness, so that tools that

• Klocwork, Inc. • Watchfire Corporation

are more or less effective in finding com-

• Microsoft Corporation • Web Application Security Consortium

plex examples of a particular CWE weak-

• Massachusets Institute of (WASC)
Technology Lincoln Labs • Whitehat Security, Inc.
ness can be identified. Correct constructs
Table 1: The CWE Community that are closely aligned to the CWEs but
tion from PLOVER to CWE took six several of drafts of CWE (draft 6 in are correct implementations will also be
months, each subsequent updated draft February, 2007 and draft 7 in May, 2007), included in the TRDs to help identify the
has occurred on a bimonthly basis. which will be available for open community false-positive effectiveness of the tools.
In addition to the sources supplying spe- comments and refinement as CWE moves Adding complexity descriptions to the
cific knowledge from tools or analysts, we forward. A major part of the future work will CWE schema will allow SAMATE and
are also leveraging the work, ideas, and con- be refining and defining the required attrib- CWE to continue to support each other.
tributions of researchers at Carnegie utes of CWE elements into a more formal The OMG’s SwA SIG, which is using
Mellon’s CERT/CC, IBM, KDM Analytics, schema defining the metadata structure nec- CWEs as one type of software issue that
Kestrel Technology, MIT’s Lincoln Labs, essary to support the various uses of CWE tools will need to be able to locate within
North Carolina State University, Oracle, the dictionary. Figure 3 shows a sample of the the eventual OMG SwA technology
OWASP, Security Institute, Unisys, the descriptive content of an entry from CWE approach, needs more formal descriptions
WASC, Whitehat Security, and any other draft 5. This example is for the Double Free of the weaknesses in CWE to allow their
interested parties that wish to contribute. weakness, CWE identification (ID) 415. technological approaches to apply. OMG’s
The merging and combining of the con- However, the CWE schema will also planned approach for this is the use of
tributed materials is being incorporated into be driven by the need to align with and their Semantics of Business Vocabulary
and Rules (SBVR) language to articulate
Figure 2: From PLOVER to CWE, Drafts 1-5 formal language expressions of the differ-
ent CWEs. The CWE schema will have to
be enhanced to allow SBVR expressions
of each CWE to be included. The CWE
PLOVER CWE CWE CWE CWE CWE
draft 1 draft 2 draft 3 draft 4 draft 5

will house the official version of the

SBVR expression of that CWE.
PLOVER CWE CWE CWE CWE

The CWE dictionary content is already

+ draft 1 draft 2 draft 3 draft 4

provided in several formats and will have

publicly + + + +
available name definition 50 new 45 new

additional formats and views added into its

vulnerability clean up expansion items added, items added,

contents as the initiative proceeds.

taxonomy and of 150 definitions definitions
content description items, expansion of expansion of

Currently one of the ways for viewing

expansion language/ 100 items, 100 items

CWE is through the CWE content page

platform ties name changes,
added and and structure

that contains an expanding/ contracting

CWE-IDs adjusted for

hierarchical taxonometric view while another

assigned new content

is through an alphabetic dictionary. The

end items in the hierarchical view are
hyperlinked to their respective dictionary
entries. Graphical depictions of CWE
content, as well as the contributing
sources, are also available. Finally, the
XML and XML Schema Definition (XSD)
for CWE are provided for those who wish
® CERT is registered in the U.S. Patent and Trademark
Office by Carnegie Mellon University.
Aug 2005 Mar 2006 May 2006 Jul 2006 Sep 2006 Dec 2006

6 CROSSTALK The Journal of Defense Software Engineering March 2007

 Being Explicit About Security Weaknesses

to do their own analysis/review with other opportunities where other security-related and come up with verification methods
tools. Dot notation representations, a stan- efforts and CWE can leverage each other to for validating the effectiveness of tools
dard method for encoding graphical plots the benefit of both. Examples of the syner- to identify the presence of CWEs in
of information, will be added in the future. gies that are possible include the following: code. The effectiveness of these test
Finally, a process to acknowledge capa- • Mapping of CWEs to CVEs that would approaches could be explored with the
bilities that incorporate CWEs has been help bridge the gap between the potential goal of identifying a method or meth-
established. This CWE Compatibility and sources of vulnerabilities and examples ods that are effective and economical
CWE Effectiveness program is similar to of their observed instances providing to apply to the validation process.
the certification and branding program concrete information for better under- • Establishing a bi-directional alignment
used by the CVE effort but has two distinct standing the CWEs and providing some between the common weaknesses enu-
parts, compatibility and effectiveness. The validation of the CWEs themselves. meration and the SAMATE metrics
basic stages of the compatibility program • Creating a validation framework for effort.
are a formalized process for capability tool/service vendor claims, whether • Using the SAMATE software security
owners to publicly declare their use of used by the purchasers themselves or tool and services survey effort to
CWEs and a public documentation of how through a third-party validation ser- leverage this common weaknesses dic-
their capability fulfills the requirements for vice, would be able to heavily leverage tionary as part of the capability frame-
finding those CWEs. The effectiveness the common weaknesses dictionary as work to effectively and unambiguously
program, which only applies to assessment its basis of analysis. To support this, describe various tools and services in a
capabilities, consists of a public declaration the community would need to define consistent apples-to-apples fashion.
about which CWEs a capability covers and the mechanisms used to exploit the • Mapping between the CWEs and the
collection of publicly available test results, various CWEs for the purposes of common attack pattern enumeration
showing how effective the capability is in helping to clarify the CWE groupings and characterization effort that would
finding those CWEs. Figure 3: Entry for CWE-ID 415, Double Free Weakness
Additional Impact and Double Free

Transition Opportunities CWE ID 415

Tied to CWE
The establishment of the CWE effort is
Description Calling free() twice on the same memory address can lead to a buffer overflow.

yielding consequences of the following

Likelihood of Exploit Low to Medium

three types: direct impact and value, align-

Common Consequences Access control: Doubly freeing memory may result in a write-whatwhere condition,

ment with and support of other existing

allowing an attacker to execute arbitrary code.

efforts, and enablement of new follow-on

Potential Mitigations Implementation: Ensure that each allocation is freed only once. After freeing a chunk,

efforts to provide value that is not cur-

set the pointer to NULL to ensure the pointer cannot be freed again. In complicated
error conditions, be sure that clean-up routines respect the state of allocation properly.

rently being pursued.

If the language is object oriented, ensure that object destructors delete each chunk of
memory only once.

The direct impacts include the follow-

ing:
Demonstrative Example 1: The following code shows a simple example of a double free vulnerability.
Examples

• Providing a common language of dis-

course for discussing, finding, and
char* ptr = (char*)malloc (SIZE);
...

dealing with the causes of software

security vulnerabilities as they are man-
ifested in code, design, or architecture.
free(buf2R1);

• Allowing purchasers to compare, eval-

free(buf1R2);
}

uate, and select software security tools

and services that are most appropriate
Observed Examples CAN-2004-0642 - Double-free resultant from certain error conditions.
CAN-2004-0772 - Double-free resultant from certain error conditions.

to their needs – including having some

CAN-2005-1689 - Double-free resultant from certain error conditions.

level of assurance of the assortment

CAN-2003-0545 - Double-free from invalid ASN.1 encoding.
CAN-2003-1048 - Double-free from malformed GIF.

of CWEs that a given tool would find.

CAN-2005-0891 - Double-free from malformed GIF.

Software purchasers will be able to

CVE-2002-0059 - Double-free from malformed compressed data.

compare coverage of tool and service

Context Notes This is usually resultant from another Weakness, such as an unhandled error or race

offerings against the list of CWEs and

condition between threads. It could also be primary to Weaknesses such as buffer
overflows.

the programming languages that are

used in the software they are acquiring.
Also a Consequence.

• Enabling the verification of coverage

When a program calls free() twice with the same argument, the program's memory
management data structures become corrupted. This corruption can cause the

claims made by software security tool

program to crash or, in some circumstances, cause two later calls to malloc() to return

vendors and service providers (this is

the same pointer. If malloc() returns the same value twice and the program later
gives the attacker control over the data that is written into this doubly-allocated

supported through CWE metadata

memory, the program becomes vulnerable to a buffer overflow attack.

and alignment with the SAMATE ref-

Node Relationships Child Of - Resource Management Errors (399)

erence dataset).
Peer - Use After Free (416)

• Enabling government and industry to

Peer - Write-what-where condition (123)
Parent Of - Signal handler race condition (364)

leverage this standardization in their Source Taxonomies PLOVER - DFREE - Double-Free Vulnerability

acquisition contractual terms and con-

7 Pernicious Kingdoms - Double Free

ditions.
CLASP - Doubly freeing memory

There will also be a variety of alignment

Applicable Platforms C
C++

March 2007 www.stsc.hill.af.mil 7

Software Security

provide the users of these resources 2. OMG. Object Management Group.

COMING EVENTS the ability to quickly identify the par- Jan. 2007 <http://swa.omg.org>.
ticular weaknesses that are targeted by 3. Concurrent Technologies Corpora-
various types of attacks and to better tion. The Code Assessment Method-
understand the context of individual ology Project. Jan. 2007 <www.ctc.com>.
April 3-5
weaknesses through understanding 4. MITRE Corporation. The Common
SAS Expo 2007
Sea-Air-Space how they would typically be targeted Vulnerabilities and Exposures (CVE)
Washington D.C. for exploitation. In combination, these Initiative. Jan. 2007 <http://cve.
two resources offer significantly higher mitre.org>.
value than either does on its own. 5. NIST. National Vulnerability Database.
www.sasexpo.org/2007

• Bi-directional mapping between CWEs Jan. 2007 <http://nvd.nist.gov>.

and coding rules, such as those 6. MITRE Corporation. The Preliminary
April 4-5
EIG 2007 deployed as part of the DHS NCSD List of Vulnerability Examples for
Excellence in Government 2007 BuildSecurityIn Web site [10], would be Researchers. Dec. 2006 <http://cve.
used by tools and in manual code mitre.org/docs/plover/>.
inspections to identify common weak- 7. MITRE Corporation. The Common
Washington D.C.

nesses in software. Weakness Enumeration Initiative

www2.govexec.com/EIG2007
• Incorporating CWE into the DHS <http://cwe.mitre.org>.
April 4-6 NCSD SwA common body of knowl- 8. Viega, J. The CLASP Application
edge, hosted on the BuildSecurityIn Security Process. Secure Software, Inc.,
Web site. 2005 <www.securesoftware.com>.
ICCSA 2007

• Leveraging of the OMG technologies to 9. McGraw, G., B. Chess, and K. Tsipen-

5th International Conference on
articulate formal, machine-parsable def- yuk. “Seven Pernicious Kingdoms: A
Computer Science and Applications
San Diego, CA initions of the CWEs to support analy- Taxonomy of Software Security Err-
www.conferencehome.com/iccsa.htm sis of applications within the OMG ors.” NIST Workshop on Software Se-
standards-based tools and models. curity Assurance Tools, Techniques, and
Finally, there are two follow-on opportu- Metrics, Long Beach, CA, Nov. 2005.
nities that are currently not being pursued 10. DHS NCSD. BuildSecurityIn. Dec.
April 22-26
but could provide significant added value 2006 <http://buildsecurityin.us-cert.
2 Annual Functional Sizing Summit
nd

Vancouver, BC, Canada to the software security industry: gov>.

www.ifpug.org/conferences • Expansion of the coding rules catalog
on the DHS BuildSecurityIn Web site to
include full mapping against the CWEs
/annual.htm About the Author
for all relevant technical domains. Robert A. Martin is a
• Identification and definition of specif- principal engineer in
April 22-26
ic domains (language, platform, func-
MITRE’s Information
North American CACS
tionality, etc.) and relevant protection
and Computing Technol-
North American Computer, Audit,
profiles based on coverage of CWEs.
These domains and profiles could pro- ogies Division. For the
Control and Security Conference
Grapevine, TX
vide a valuable tool to security testing past seven years, his
www.isaca.org
strategy and planning efforts. efforts have been focused on the inter-
play of risk management, cyber security
Conclusion standards, critical infrastructure protec-
This work is already helping to shape and
April 23-26
tion, and the use of software-based tech-
mature the code security assessment nologies and services. Martin is a mem-
NMDAS 2007
industry, and it promises to dramatically ber of the Association for Computing
The 2007 Nano Materials for Defense
Application Symposium accelerate the use and utility of automa- Machinery, Armed Forces Communica-
tion-based assessment capabilities for
tions and Electronics Association,
San Diego, CA
organizations and the software systems
they acquire, develop, and use.◆ Institute of Electrical and Electronics
www.usasymposium.com
Engineers (IEEE), and the IEEE
June 18-21 Computer Society. He has a bachelor’s
degree and a master’s degree in electrical
Acknowledgments
2007 Systems and Software The work contained in this paper was fund-
ed by DHS NCSD and is based on the engineering from Rensselaer Polytechnic
efforts of a large number of individuals, but
Technology Conference
Institute, and a master’s of business
special thanks is made for the contributions degree from Babson College.
of Steve Christey, Janis Kenderdine, Conor
Harris, David Harris, and Sean Barnum. MITRE Corporation
Tampa Bay, FL
202 Burlington RD
References
1. NIST. “The Software Assurance Met-
www.sstc-online.org Bedford, MA 01730-1420
rics and Tool Evaluation (SAMATE)
COMING EVENTS: Please submit conferences, seminars, Phone: (781) 271-3001
Project.” Jan. 2007 <http://samate.
symposiums, etc. that are of interest to our readers at
Fax: (781) 271-8500
nist.gov>.
least 90 days before registration. E-mail announce-
ments to [email protected]. E-mail: [email protected]

8 CROSSTALK The Journal of Defense Software Engineering March 2007