Blakely Et Al 2022 - Exploring The Information Con - 202508221804 - 36066
Blakely Et Al 2022 - Exploring The Information Con - 202508221804 - 36066
A R T I C L E I N F O A B S T R A C T
Keywords: A number of institutions make reports available regarding the types, impacts, or origins of
COSO cybersecurity breaches. The information content of cyber breach reports is examined in light of
Computer Security Principle 15 of the 2017 Committee on Sponsoring Organizations Enterprise Risk Management
Computer Crime
(COSO ERM) information security control framework to understand the degree to which cyber
Risk analysis
Security management
breach reports reflect the established COSO internal control framework. This study utilizes the
Incident COSO ERM internal control framework to examine whether current cyber breach reports contain
Breach information that may influence a firm’s ability to assess substantial change within its industry due
to external forces (COSO ERM Principle 15). As such, this study focuses on data breaches, a
special type of cyber incident, which may result in the loss of confidential information. Cyber
decision makers rely on this type of information to calibrate information security programs to
ensure coverage of relevant threats and the efficient use of available funds. These reports may be
used for the purposes of cybersecurity risk assessment and strategic planning. We compare,
contrast, and analyzie the reports to identify their utility in such contexts. We also provide an
overview of the current cybersecurity reporting environment and suggest revisions to US national
cyber policy with the intent of increasing the benefit to reporters and consumers of the data.
This study is focused on education as to the current structure of breach reporting based upon
our review and synthesis of publicly-available breach reports.
In this study, we review nine (9) reports that meet four (4) criteria. We relate these criteria to
the framework provided by COSO ERM Principle 15 by analyzing and placing the criteria into a
taxonomy developed for this purpose. We analyze the degree to which the reports are comple
mentary, reflect potential improvements of internal controls, and provide recommendations for
ways in which these types of reports might be used by practitioners, while highlighting potential
limitations. Our findings indicate that the sample reports contain little information that may be
incorporated to improve the risk profile of an entity. We provide recommendations to improve the
information content and timeliness of breach reports.
* Corresponding author.
E-mail addresses: [email protected] (B. Blakely), [email protected] (J. Kurtenbach), [email protected] (L. Nowak).
https://doi.org/10.1016/j.accinf.2022.100568
1. The information content of cyber breach reports using an internal control framework
1.1. Introduction
Information Security, as a profession, has matured a great deal since the early 2000s. System or network administrators focused on
reactionary defenses to attacks that now seem more analogous of a prank than a federal crime. Research into the detection and
prevention of attacks was wide-ranging, although cybersecurity was seldom discussed outside the information technology department.
Since approximately 2000, defenders and attackers have matured. Information communications technology (ICT) networks have
become increasingly critical to modern life and have faced constant pressure to be managed in a risk-conscious and economical way
(Hughes et al, 2017). Nation-states, organized crime syndicates, activist groups, and even terrorist organizations fund attackers.
Vulnerabilities have moved outside the cyber world as humans are targeted to gain access to information assets.
COSO was organized in 1985 to study factors that lead to fraudulent financial reporting. The COSO framework for evaluating
internal controls was published in 1992 (Committee on Sponsoring Organizations of the Treadway Commission, 2013). ERM was
added as a focus area in 2004, and updated in 2017. The American Institute of Certified Public Accountants (AICPA) has issued several
Statements of Auditing Standards (SAS) that address the internal control objectives of prevention, detection, and correction to
minimize the impact of fraudulent financial reporting. With the growth of cyber attacks, the AICPA has also focused on the service
organization controls (SOC) that provide a level of assurance for the internal controls of third party, cloud, computer storage and data
management software.
Cyber risks are considered alongside any other material consideration in the COSO ERM framework. However, the ability of in
formation security professionals to give quantitative justifications for tactical and strategic decisions, in parity with their financial and
legal counterparts, face two primary challenges. First, while some organizations might be large or mature enough to have detailed
internal metrics and reporting on incidents and breaches, this is not an universal expectation. Second, the investment of time and
money required to build such a reporting infrastructure might be beyond the reach of smaller organizations or those with less expertise
in incident and breach analysis.
Of interest in this study is the degree to which breach reports contain information about breaches that reflect internal controls that
may be utilized, or strengthened, to thwart future breaches. Based upon the COSO ERM internal control framework, we evaluate
published reports for information that reflects failures of internal controls that may benefit practitioners in reducing the risk profile of
the entities. We explore current US cyber-breach-reporting policies and utilize the COSO ERM internal control framework to examine
and support recommendations for standardization of public policy to better capture cross-industry reporting.
Throughout this study, we use the term breach and incident in a specific way. There are many similar definitions for these terms from
different frameworks and regulations. We utilize the definitions promulgated by the US National Initiative for Cybersecurity Careers
and Studies Cybersecurity and Infrastructure Security Agency that are based on NIST and ISO standards United States National
Initiative for Cybersecurity Careers and Studies Cybersecurity and Infrastructure Security Agency (NICCS CISA), 2022:
• An incident is an occurrence that actually or potentially results in adverse consequences to (adverse effects on) (poses a threat to)
an information system or the information that the system processes, stores, or transmits and that may require a response action to
mitigate the consequences.
• A data breach is the unauthorized movement or disclosure of sensitive information to a party, usually outside the organization, that
is not authorized to have or see the information.
There currently exist disparate requirements for reporting breaches affecting US corporations and governmental entities. This study
attempts to categorize breach reporting by the information content vis-à-vis common internal controls from the COSO ERM frame
work. Breach reports are typically focused on personally identifiable information (PII). Individual industries such as healthcare (where
HIPAA pertains) or financial services (the domain of the Gramm-Leach-Bliley Act), might have additional breach-notification re
quirements. The Biden administration recently issued an executive order with stringent requirements on breach reporting applicable to
federal contractors (United States, 2021). The United States Federal Depositors Insurance Corporation (FDIC) recently tightened
reporting requirements for member banks (United States FDIC, 2021). Additionally, due to the lack of standardized requirements,
companies may have to juggle multiple contractual provisions demanded by customers regarding security controls and reporting
requirements. The Wall Street Journal notes that industry groups urge lawmakers to streamline cyber breach reporting as companies
raise concerns about the burden of compliance with overly broad definitions of breaches and multiplying regulatory requirements
(Rundle, 2021).
1.2. Outline
In Section II, we provide a review of the literature as pertains to the standard-setting process that has existed in the areas of in
formation technology and draw comparisons from the standard-setting process in other industries and countries to provide a
framework for improvement in the US breach-reporting architecture. Ancillary to our primary focus on the information content of
cyber breach reports, we provide a twofold demonstration that supports the need for such a framework for more consistent and
rigorous breach reporting: (1) an analysis of the information that current information security decision-makers typically reference, and
(2) an explanation of the challenges and limitations in using these data for quantitative risk assessment and forecasting models.
Section III contains the research methodology and describes the process by which we analyzed, categorized, and combined data
2
B. Blakely et al. International Journal of Accounting Information Systems 46 (2022) 100568
from the included reports. Section IV provides a discussion of our observations and details the results of this analysis, explores lim
itations that we encountered in compiling the data, and makes recommendations for improved reporting. Section V contains the
conclusions and recommendations for future research.
Information security professionals are information technology professionals by trade and, therefore, may not have the type of
training or experience that those who manage other types of enterprise risk management, e.g., legal or accounting disciplines. Current
cyber risk management methodologies may fail to support risk and reward trade-off decisions, which may be skewed by over
confidence of key decision-makers, and suffer from a lack of quality data (Fenz et al., 2014; Ruan, 2017). Thus, breaches still happen, in
part, due to intuitive judgments where data-driven decisions would have been appropriate (Young, Beebe and Chang, 2012; Julisch,
2013; March and Shapira, 2016).
Some of these issues may be addressed by focusing IT risk assessments on frameworks such as Control Objectives for Information
and Related Technology (COBIT)1 from ISACA. Though this is unlikely to be an exhaustive approach or “scientific” in the sense that it
would be preferred from a statistically representative perspective, it can at least help to ensure there are no significant gaps left in the
assessment due to omissions by its authors. Frameworks, such as the Capability Maturity Model Integration (CMMI),2 which allows
each area of control to be expressed on a spectrum in terms of the level of rigor, has been applied within organizations. Such ap
proaches pivot from a focus on threats and threat actors to impacts and compliance risk, but many of these controls are designed to
prevent specific threats.
Even if high-quality breach data are made available, an appropriate application of statistical methods would be imperative for using
them effectively (Peng et al., 2018). There is a long history of literature advocating for a more rigorous “science of cybersecurity,”
which has been hampered by a lack of data, among other concerns (e.g., Siponen and Willison, 2007; Catlett, 2008; McMorrow, 2010;
Schneider, 2012; Herley and Van Oorschot, 2017).
The Factor Analysis of Information Risk (FAIR) methodology.allows risks to be decomposed into constituent contributing factors
with the factors assigned to probability distributions to appropriately express the actual level of confidence in their estimated values.
Monte Carlo simulation is then used to “simulate” a large number of hypothetical time periods from these distributions to give a
probable loss curve. This approach not only allows for a fully quantitative approach to assessing risk that can be readily expressed in
monetary terms, but also makes evident the level of uncertainty to ensure decision-makers are not misled by artificial precision. This
methodology requires a significant amount of work – essentially building a probability model for each risk, which might be worthwhile
when risks need to be more rigorously analyzed. To make the best use of this methodology, input data on the factors contributing to
each risk are needed. Very broad estimates will “work”, but might lead to very imprecise results. The better data available to feed such
models, the more effective they will be.
With respect to the immediate value of breach reports, consistent, mandatory reporting of breaches is a factor in making cyber
breach data available (Eling and Schnell, 2016). Enterprises could use these data to make more accurate decisions about the allocation
of funds for information security purposes and for broader corporate governance (Sarabi et al., 2016). Existing work shows the promise
of these types of analyses in countering the “fear, uncertainty, and doubt” that often serve as the driving force behind information
security investments, but results to date have been mixed (Edwards et al., 2016; Xu et al., 2018). Data are required for actuarial
purposes as well, namely to ensure that enterprises can obtain appropriate insurance to cover information technology-related events
(Biener et al., 2015; Marotta et al., 2017; Eling and Loperfido, 2017).
Combined with techniques such as the FAIR methodology, such data allows decision-makers to make “Return on Security In
vestment” (ROSI) decisions in a similar way to any other investment (e.g., Schatz and Bashroush, 2017). When evaluating multiple
potential controls or preventive measures that can be built or bought, the decision-maker can select those that are expected to make the
biggest reduction to the probable loss. This moves what might otherwise be a very technology-focused decision-making process into
the realm of financial portfolio management and has the potential to greatly increase the efficiency of capital and labor utilization.
Organizations may find themselves in multiple national, state, or local jurisdictions, that may have vague, conflicting, or a lack of
requirements for breach reporting. Understanding the challenges to standardizing breach reporting can aid in understanding why this
has not happened to date and helps us to understand the information content, or lack thereof, within cyber breach reports.
3. Research methodology
To better understand the relationship among the reports, and to identify any information content that benefits the internal control
structure of breached entities, it is necessary to decompose further the information provided in the context of a cyber-threat (or risk)
model. The impact of breach reports is viewed from the perspective of cybersecurity decision makers, such as Chief Information Se
curity Officers (CISO). Such risk models are a foundational component of IT risk management and are built from an enumeration of the
types of threats that exist, the assets at risk, and the interplay between them (NIST Special Publication 800 (United States National
Institute of Standards and Technology, 2012).
Within the accounting profession, COSO provides guidance on internal controls to company leaders in order to ensure that financial
1
https://www.isaca.org/resources/cobit.
2
https://www.itgovernanceusa.com/capability-maturity-model.
3
B. Blakely et al. International Journal of Accounting Information Systems 46 (2022) 100568
statements are based upon accurate data that has not been manipulated so as to reflect a material misstatement. As such, we evaluate
the detailed analysis of the cyber breach reports to estimate the information content of the reports as pertains to internal controls that
may have been related to a breach within the context of COSO ERM Principle 15, which is focused on the ability for an entity to adapt to
internal and external substantial changes in an industry.
Fig. 1 was created by the authors to portray the relationships among the incident activities and terminology utilized in breach
reports. To build a quantitative and accurate risk model that can be utilized to evaluate internal controls, we require data points for
each of the elements that comprise the model. We have summarized several available standards, or guidelines, regarding risk
modeling, and the relationship between the elements and the components of this model We apply the standards to a sample of breach
reports to identify the strengths and weaknesses that exist in our sample of breach reports and propose remedies that flow from the
analysis.
Threat Sources, also known as Threat Actors, represent the specific individual or organization that initiates a malicious action. As
noted within COSO ERM Principle 15, threats may also constitute natural occurrences (e.g., a hurricane) and internal threats (e.g., a
hazardous waste spill). For our purposes, we limit the definition to actors with motive and intent. The lower right box in Fig. 1
identifies “risk” as the residual risk that remains once an evaluation of the implications associated with a breach report have been
completed. Inherent risk is controlled via the overall internal control framework, using COBIT, COSO, COSO ERM, NIST, or similar
frameworks. However, the existence of breaches indicates that there exist residual risk that may be addressed with information
contained within a breach report.
The topic of threat vector, or intelligence, exists in an attempt to gain information about potential adversaries that might target a
given organization type. Products and subscriptions are sold to provide such information, and Information Sharing and Analysis
Centers (ISACs) are set up for a number of sectors to distribute timely and relevant warnings to peer institutions.
A threat is any circumstance or event with the potential to impact organizational operations (including mission, functions, image,
or reputation), organizational assets, or individuals adversely through an information system via unauthorized access, destruction,
disclosure, modification of information, or denial of service (“FIPS PUB 200, Minimum Security Requirements for Federal Information
and Information Systems”, 2006). Also known as the threat vector or attack vector, we will primarily be concerned with the tactics or
tools that a threat agent, i.e., an attacker, uses to perform a malicious action, such as to exploit a vulnerability.
Knowledge of threat vectors comes largely from incident investigations, as it is difficult to connect the dots between a specific
breach event and the root cause through fully automatic means. This data is a component of most of the surveyed reports and is of
particular value to decision makers who determine where to allocate defensive investments. As such, information contained in a cyber
breach report may provide accountants and risk professionals with information to strengthen internal controls.
The Attack Surface of an information system is the exposed area that makes a system more vulnerable to cyberattacks (United States
National Institute of Standards and Technology, 2013). Every system has some degree of attack surface – another term for this might be
“external interface.” A system with no way to interact with any other entity is of very limited use. For example, the network ports that
are allowed to traverse a firewall and are monitored by a service constitute an attack surface. Much of the work that goes into
hardening IT systems is focused on reducing the attack surface, i.e., leading to vulnerabilities, by limiting the ways in which any person
or application can interact with an asset by disabling or blocking any unneeded functionality (or not including it in the design).
A vulnerability is a weakness in an information system, system security procedures, internal control, or implementation that could
be exploited or triggered by a threat source (“FIPS PUB 200, Minimum Security Requirements for Federal Information and Information
Systems”, 2006). Analogous to a flaw in the attack surface of a system, a vulnerability is what an attacker must find and exploit to break
inside an information network, and internal control systems must prevent, detect, and correct in a timely fashion once a vulnerability is
discovered.
Countermeasures are the actions, devices, procedures, techniques, or other measures that reduce the vulnerability of an infor
mation system and are otherwise known as security controls or safeguards (“FIPS PUB 200, Minimum Security Requirements for
Federal Information and Information Systems”, 2006). Much as in an accounting context, where controls are in place throughout a
process to ensure the prevention, detection, and correction of unacceptable outcomes, countermeasures mitigate the risk and limit the
potential damage of vulnerabilities that have been left exposed. Numerous frameworks exist to list “best practices” in this area, such as
NIST 800–53, ISO 27001, and the Center for Internet Security (CIS) Controls.3 However, determining the level of effectiveness of any
particular control against any particular threat is more challenging. Cyberattacks are dynamic and multi-faceted, and measuring the
impact of any specific countermeasure in isolation from the broader environment may not always be possible.
Breach reports may be difficult to monitor because there are several types of organizations that release reports with varying
motivations and levels of maturity, and the reports are not produced in accordance with an industry or academic standard. This leads
to a level of instability in methodologies, report contents, formats, publication cycles, and other details yearly. Some reports are based
on mandatory reporting, some on voluntary participation in surveys, and some on internal data about security vendor clients of
companies that provide security software or services. We also note inconsistent reporting requirements across industries that may
undermine the response to coordinated attacks and reduce the information content for practitioners focused on improving internal
3
https://www.cisecurity.org/controls/.
4
B. Blakely et al. International Journal of Accounting Information Systems 46 (2022) 100568
Fig. 1. Author Created Diagram of KeyTterms and Actions related to a Data Breach.
controls. For example, the US Department of Human Services has different reporting timeline requirements based upon the perceived
size of the breach.4 That implies that there may be inconsistencies or delays in information available pertaining to ongoing attack
campaigns that would otherwise provide useful information to security professionals.
There is a variety of documents that could be considered as the “breach report” corpus of information. It is challenging to ensure the
comprehensiveness of such a study because there is no single source from which to query these reports. They are commercial or
governmental in nature, not subject to peer review, and there is no central registry. Thus, we have used our own experience, broad
internet searches, and common industry-brand names to generate a candidate population.
We utilize the criteria based upon our experiences as professionals in cyber security and accounting to determine our sample of
included reports for this study. The authors collectively have advanced degrees in accountancy and computer engineering, with
approximately thirty (30) years of combined experience as a security professional and as security and technology leaders with private
entities, pre-IPO entities, a R1 university, government, and public companies. We capture what we believe to be the primary re
quirements for any such breach reporting data, i.e., the data is available to practitioners without barriers of cost or accessibility, the
data can be considered as a population for the purposes of statistical evaluation (i.e., no overlap of multiple populations or obvious
gaps) and the data can be utilized for quantitative methods (i.e., findings are presented numerically in a manner amenable to
incorporate in risk models). For example, the ENISA Threat Landscape (European Union Agency for Network and Information Security,
2019) report is often referenced, but fails our second criterion as it is a meta-study that does not lend itself to analysis of specific
empirical data. Insert Table 1.
Though the analyzed reports are relevant to ascertaining the prevalence and impact of breach events, they are developed in
different contexts. Table 2 describes the properties that are used to classify these reports and represents a high-level taxonomy that can
be applied to similar documents. Table 2 utilizes a qualitative approach to security risk analysis as portrayed by Freund and Jones
(2015) to explain the Factor Analysis of Information Risk (FAIR) Institute approach to quantifying risk. The qualitative properties are
relevant to an attack model that may provide insight into internal control weaknesses, such as discussed below and shown in Fig. 1, as
well as developing quantitative models.
The categories identified and included for analysis are: source, frequency, mandatory or voluntary reporting, audience, sector
coverage, and geographic coverage. Source and mandatory reporting help determine the degree to which the analysis might be subject
to sampling biases. Frequency provides an indication of how current the data is – cyber threats change very fast and documents that are
more than a couple of years old quickly lose relevance. Audience gives context as to the intended usage of these documents, which is
relevant to trying to understand the underlying methodology. Sector and Geographic Coverage determine the potential scope of
applicability of the report. Vector Coverage tells us the “type” of attacks being considered, deviations of which can make combining
knowledge from different reports challenging. The results of applying these categories and definitions to our data are shown in Table 4.
4
https://www.hhs.gov/hipaa/for-professionals/breach-notification/breach-reporting/index.html.
5
B. Blakely et al. International Journal of Accounting Information Systems 46 (2022) 100568
Table 1
Criteria for Breach Report Inclusion.
Item Description
1 Publicly available
2 Based on primary source, empirical data
3 Conclusions drawn from a large and broadly representative dataset
4 Provide quantitative data regarding threat vectors and/or impacts
Table 2
Taxonomy Used to Evaluate Reports.
Item Category Description
1 Source Detail regarding the information used for analysis, to whatever extent is provided in the report
2 Frequency How often the report is issued
3 Mandatory Whether or not the analyzed data is based on mandatory reporting versus surveys, voluntary reporting, etc
Reporting
4 Audience Unless explicitly specified, this is classified based on whether it is a more technical document, apparently targeted at IT Decision
makers, or more accessible for general Risk Managers who may lack an IT background
5 Sector Coverage The extent of coverage by organization type or standard used for classification
6 Geographic The jurisdictions represented in the analyzed data.
Coverage
7 Vector Coverage The method used to classify breaches by cause. If multiple, that which is most closely aligned to the definition of “Attack Method8”
in the DHS Risk Lexicon (United States Department of Homeland Security and Committee, 2010)
8
Manner and means, including the weapon and delivery method, an adversary may use to cause harm on a target.”
Where reporting of a breach to a US Government agency may not be mandatory, or provided by the government in a timely report
to industry, we find it necessary to investigate further the methods of population sampling. Terms used for sector and attack-vector
coverage do not derive from a common lexicon, and so it is necessary to perform a mapping between reports to understand overlap
or gaps, especially as involves the identification of any internal control weakness. Geographic coverage can determine whether the
data are broadly representative or perhaps biased due to regulatory, political, or cultural factors specific to a locality or jurisdiction.
Reports for technical audiences will contain more types of the required detail for granular modeling of cyber risks, and those aimed at
less-technical audiences may contain more information regarding broader financial or legal impacts.
Based on our criteria, the reports in Table 3 were included (parentheticals denote how they are branded by the issuer). Each report
was analyzed in accordance with the previously identified taxonomy. The resulting sample represent a combination of governmental
and private-sector authorship and cover, in total, the aspects of threat modeling discussed in the research methodology section. Insert
Table 3.
Following is a summary of each included report. For the sake of brevity, these summaries are of a high level and should be
combined with the information in Table 4, Table 6, and the original documents, all of which are publicly-available. We classify cyber
breach reports as Governmental Reports, Private Sector Reports, and Private Entity Studies. The distinction between private sector
reports and private sector studies lies in the information source data. Private sector reports rely on electronic monitoring of security
systems; whereas, private entity studies are based upon interviews with technology professionals and governmental reports are based
on compilations by federal agencies.
The “Annual Report to Congress on Breaches of Unsecured Protected Health Information” (US DHHS, 2019) is a result of the
Table 3
Reports included in analysis.
Item Description
1 Annual Report to Congress on Breaches of Unsecured Protected Health Information for Calendar Year 2019, (HHS)
2 Ninth Annual Cost of Cybercrime Study. (Accenture) (Accenture, 2019)
3 2020 Internet Crime Report (IC3). (United States Federal Bureau of Investigation, Internet Crime Compliant Center, 2021)
4 2020 Cost of Data Breach Study. (IBM) (IBM Corporation, 2020)
5 2021 Data Breach Investigations Report. (DBIR) (Verizon, 2021)
6 IBM X-Force Threat Intelligence Index 2020. (XFTI) (IBM Corporation, 2021)
7 Internet Security Threat Report. (ISTR) (Symantec, 2019)
8 M-Trends 2021. (M-Trends) (FireEye, 2021)
9 Microsoft Digital Defense Report. (Microsoft) (Microsoft, 2020)
6
B. Blakely et al. International Journal of Accounting Information Systems 46 (2022) 100568
Table 4
Information Contained in Included Breach Reports.
Item Report Source9 Frequency Mandatory Audience Sector Geographic Vector
(Last) Reporting Coverage Coverage Coverage
collection of data by the US Department of Health and Human Service’s Office for Civil Rights (HHS OCR). As a requirement of the
Health Information Technology for Economic and Clinical Health (HITECH) Act, which is part of the American Recovery and Rein
vestment Act of 2009, “covered entities,” as defined by the Health Insurance Portability and Accountability Act (HIPAA) of 1996, such
as health care service providers, plan administrators, and data clearinghouses, must provide notification of breaches of unsecured
protected health information (PHI).5 The report issued by the Secretary of Health and Human Services to committees in the Senate and
House annually (though at a significant delay, given the age of the most recent publication) provide details on the types and quantity of
reported breaches6 as well as the responsive actions. Thus, as mandatory reports, this dataset is likely to be representative in nature to
the degree that covered entities comply with the HITECH act. However, it is also narrowly scoped to the healthcare sector and United
States.
The Internet Crime Complaint Center (IC3), a component of the US Federal Bureau of Investigation, issues the “Internet Crime
Report” annually. The dataset consists of reports by individual persons (not limited to US citizens) who believe they are the victim of a
cyber-crime. It is intended as a database for law enforcement personnel to use as a resource to assist in investigation and information
sharing. The annual report is made available to the public. Given the focus on criminal activity, it views cyber activity through the lens
that combines cyber and criminological terminology, with a particular focus on attacker motivation and victim impact.
5
This summary modified from the introductory paragraph of the cited report.
6
Defined by HITECH as the “acquisition, access, use, or disclosure of PHI in a manner not permitted by the HIPAA Privacy Rule.”.
7
B. Blakely et al. International Journal of Accounting Information Systems 46 (2022) 100568
The following five reports, though similar, reflect different uses of taxonomies and imply different audience characteristics. They
tend to be written in an informal manner with a large degree of graphical or infographic support. The private sector reports draw on
information from their own operations and those of their partners.
The Verizon “Data Breach Investigations Report” (DBIR) is released annually and draws from a large dataset consisting of both
Verizon’s own investigations and those of a network of contributing partners. DBIR makes use of the Vocabulary for Event Recording
and Incident Sharing (VERIS), a community framework that is designed for use in cybersecurity reports.
The IBM “X-Force Threat Intelligence Index” (XFTI) draws from managed endpoints and servers as well as collections of data from
various deployed sensors or public web content. XFTI uses the Common Attack Pattern Enumeration and Classification (CAPEC)
framework from MITRE as a descriptive framework.
The Symantec “Internet Security Threat Report” (ISTR) draws from Symantec’s various protection services and the “Global In
telligence Network,” which, it claims, is the “largest civilian threat collection network in the world.” ISTR is the only report to
reference the US Securities Exchange Commission (SEC) Standard Industry Classifications (SIC) as a taxonomy for breach victims.
The Mandiant/FireEye “M-Trends” report is the most technical of the survey reports, going so far as to include sample exploit code
in the guidance on Red Teaming (that is, playing the attacker on a friendly network to discover vulnerabilities that can be remediated).
A portion of the M-Trends report is allocated to the Advanced Persistent Threat (APT) actors that Mandiant tracks and reports.
The Microsoft “Digital Defense Report” (Microsoft), like the IBM XFTI report, is a hybrid report and slide deck. The dataset contains
similar information to the IBM and Symantec datasets with the addition of collected data from the Microsoft Azure cloud environment,
one of the largest such cloud infrastructures (Darrow 2017).
The IBM “Cost of a Data Breach Report” is based on interviews with individuals in “IT, data protection, and compliance.” It spe
cifically aims to measure the cost of breaches versus their frequency, and it is focused on the per-customer-record loss, implying a focus
on personally identifiable information (PII) or financial records. It dissects the factors that impact this cost (positively or negatively)
and uses activity-based costing (ABC) as a methodology to calculate the per-breach costs, including elements of detection and esca
lation, post-breach response, notification costs, and lost business cost.
The Accenture “Cost of Cybercrime” report is also based on interviews. Accenture attempts to quantify expenditure on technologies
to prevent cybercrime, i.e., cybersecurity investment, and the resulting cost savings (i.e., the cost of a breach that might have otherwise
occurred). This approach is often referred to as return on security investment (ROSI) in security literature. Accenture makes a number
of recommendations for properly balancing relevant factors to maximize return on security investment decisions. The Accenture report
most closely aligns with the FAIR Institute attempt to quantify risk via the use of their ROSI metric.
Table 4 is a summary of the taxonomic properties of the surveyed reports. Commonalities as well as differences can be observed due
to the slightly different audiences or author perspectives. Most of the reports draw on a large corpus of data, though further discussion
below will consider the extent to which this lends credibility to the findings and to the decision usefulness as pertains to effective
internal controls. Data sets are drawn from voluntary breach reports, interviews or surveys of volunteers, inspection of internal data
from an organization’s own investigations or monitoring services, or a compilation of the above.
Five of the reports analyze the results by industry or sector, but only one references a standard classification system. Six reports are
global in nature, although it is not clear as to the representation of specific international communities. The remaining reports are
identified by the number of represented countries, identified as only containing US data, or identified as representing a number of
geographical regions.
The most difficult category to reconcile is the attack vector. This is a common term used to refer to the method used to exploit a
system vulnerability and relates most directly to failures in COSO recommended internal control. An example is examining the
effectiveness of a general control, such as tricking an employee into revealing their password. However, the exact nature of this concept
Table 5
Report Attributes.
Item Report Technicality Specificity Utility for Cyber Practitioners
8
B. Blakely et al. International Journal of Accounting Information Systems 46 (2022) 100568
is dependent on the vantage point of the author and the intent of the report. No two reports use the same terminology, and only two use
standards-based terminology.
In Table 5, we have developed a three-attribute taxonomy to aid in understanding how these reports may be used. The first attribute
“Technicality” is the degree to which the report gives information that would be directly relevant to cybersecurity decision-makers
versus other audiences with a lesser focus on development and configuration of technical control measures. Reports with details on
technologies, malware strains, vulnerabilities, and similar are ranked high in technicality. Those with data presented regarding po
tential criminal intent, financial impacts only, or otherwise lacking information directly usable in calibrating a cybersecurity control
are considered low in technicality.
“Specificity” indicates the granularity of the information contained within a report. Reports discussing only population-wide
metrics would be low in specificity, reports given detailed breakdowns by various dimensions of attackers, vectors, targets, or im
pacts would be high in specificity. Essentially, we focus on whether the information contained within a report has actionable infor
mation for cybersecurity practitioners.
“Utility for Cyber Practitioners” is the product of the first two measures, where Low, Medium, and High are encoded as 1, 2, and 3
for Technicality as the input features and ranges for the Specificity of 1–3, 4–6, 7–9 are used to reflect Low, Medium, and High utility.
This metric relates most directly to COSO ERM Principle 15 as we impute the information available to practitioners.
Based upon analysis of the “utility” metric, we find that four (4) reports provide a high degree of utility for cyber practitioners
derived from a High coverage in Technicality and Specificity. Four (4) reflect a medium level of utility due to weak coverage of
Technicality (or technical information), and one report is ranked with a low level of utility for practitioners due to a low ranking in
both Technicality and Specificity. While this information does not directly address implications for internal controls, we may conclude
that less than half (four of nine reports) contain any level of information content that may be beneficial in revising internal controls in
other industries.
Table 6 details the coverage of the terms in the surveyed reports. With the exception of vulnerabilities, all elements are covered in at
least one report. Similarly, the basis of the incident is only identified in the Accenture data breach report. Vulnerabilities and details
pertaining to the incident may not be necessary for modeling if sufficient data about threat vectors and attack surfaces are available. It
is worth noting that no single report covers all of the necessary elements for using a risk model as described above in a manner that may
be quantified. As pertains to internal controls, we were unable to identify any vulnerabilities within the cyber breach reports that may
benefit managers of entity security operations over internal controls.
Though various reports can be combined to cover all of the elements, data elements may be incompatible or not presented in a
manner that allows for the necessary type of numerical analysis. For example, lacking relative percentage or portions, baseline
population figures, or incident frequencies.
Tables 1 through 5 provided one approach to quantifying the data contained with publicly available breach reports. The primary
information contained within cyber breach reports appears most relevant to security professionals as the reports focus on the threats
and the attack vectors that have been utilized in reported cyber breaches. However, as shown in Table 6, there is very little information
pertaining to the specific incident and the associated vulnerabilities that benefit an accounting professional interested in strengthening
COSO enterprise risk management controls. Based upon this analysis, we conclude that currently stylized cyber breach reporting does
not contain information content that benefits an entity in designing or redefining internal controls to avoid future breaches.
Improved breach reporting and standardization of required breach reporting terminology may provide increased confidence to
decision-makers, such as a CISO, through easier compilation and more sophisticated analysis of threats, vectors, and impacts. Ulti
mately, improved communication pertaining to the application of internal controls that failed during a cyber breach will reduce the
threat vector of new attacks. It is understood that many of these reports are provided by companies on a best-effort basis, for costless
public consumpltion, for the good of the information security community, and are not intended as academic or scientific works.
However, as these reports are often the only publicly available data and have high visibility, it is important baseline principles and
rigorous methodologies are designed to provide insights pertaining to failed internal controls that permit security professionals to
strengthen applicable controls.
In analyzing breach reports, the challenges and limitations in the use of these data vary. The type of analyses in breach reports,
despite their focus on information technology, are very similar to a typical study in the social sciences. The American Psychological
Association, for example, provides guidance for the manner in which a manuscript describes its methodology (American Psychological
Association, 2018) . In particular, the following limitations noted by the APA were encountered in our analysis:
• Sampling error: Lack of detail regarding participant selection. What portion of eligible organizations7 were included, where was
self-selection conducted versus sampling of an existing data set (and was there an ability for clients to opt-out), etc.
• Non-sampling error: Lack of detail regarding collection. Considerations for self-selection, bias in the sample, bias in survey
questions, and errors or skew in the underlying data sets cannot be addressed.
7
This item is mainly concerned with data that have been drawn from customer bases vs individual persons.
9
B. Blakely et al. International Journal of Accounting Information Systems 46 (2022) 100568
Table 6
Information Content of Breach Reports.
Item Report Threat Threat Attack Vulnerability Information Countermeasure Asset Incident Impact Risk
Source Vector Surface System
1 HHS X X X X
2 IC3 X X X
3 IBM X
4 DBIR X X X X X
5 XFTI X X
6 M-Trends X X X X X
7 ISTR X X X X X X
8 Microsoft X X
9 Accenture X X X X X
• Estimation and inference: Lack of detail regarding the precision of findings. Without information regarding error margins, the
reader cannot determine the amount of confidence to place in the conclusions.
• Inconsistent terminology: The manner in which the report’s present data reflect the background and context of the authoring
organizations. Combining data from multiple reports requires guesswork or is impossible.
• Undefined or overlapping populations: Due to the inability to share the raw data sets underlying breach reports, readers cannot
ascertain details about what population was sampled beyond the statistics presented in the report. Several of these reports are
compilations of others. In the case of those that are based on investigations or reporting from deployed systems, the reader does not
know how exclusive a particular population is as compared with other reports.
While the benefit to accounting practitioners is limited in the current structure of breach reports, there are opportunities to make
the reports more meaningful to a broader audience, and to add specific information that helps practitioners confront existing vul
nerabilities. In addition to adding a strong focus on specific incidents and their associated vulnerabilities, we make the following
recommendations for the reporting of breaches, while noting that there is a need for reports to cater to different audiences.
1. Explicitly state the intended audience and objectives of the report to allow it to be placed in the appropriate context.
2. Make use of standard terminology wherever possible instead of using internally developed taxonomies. There are multiple stan
dards for various taxonomic items, so this will not completely solve the problem. It will, however, allow analysts to lean on existing
work that attempts to harmonize multiple standards, and it creates an opportunity for report authors to negotiate which standards
should be used. Additionally, the standard terminology would benefit identification of internal controls that should be strength
ened to ward off additional attacks.
3. Where standard terminology is unavailable or inappropriate, provide details regarding the methodology for developing internal
taxonomies and place them in the context of standards by providing mappings and indicating where gaps remain.
4. Describe the methodology for collecting and analyzing data in detail. This should include the manner in which participating in
dividuals, institutions, or systems were recruited or selected, how data-gathering instruments (such as surveys and interviews) were
constructed and validated, how data were normalized or filtered, and how the presented statistics were selected and calculated.
5. Identify the incident and vulnerabilities that led to the breach, with recommended methods of remediation.
6. Where possible, make the underlying data set available for direct inspection with necessary protections for confidentiality and
privacy. The Veris Community Database (https://veriscommunity.net/vcdb.html) is an example of one effort to do so.
7. Where impossible to make the underlying data set available, provide not only statistics about the sampled data but also parameters
regarding the population from which they are drawn.
8. When presenting statistical measures, indicate the margin of error. If this cannot be determined, state this explicitly and explain
why.
A limitation in adopting such recommendations it that any deviation in methodology or terminology from one reporting period to
the next might make it difficult to maintain the ability to track trends. For this reason, these recommendations might be of the most use
for the creation of new reports, yet existing reports could incorporate recommendations that do not affect trending, provide mappings
between old and new terminology, or simply choose to favor improved reporting over historical congruency. This last may be more
practical than it first appears, as the value of using historical data that have methodological issues and lack quantitative rigor for
trending purposes may introduce false confidence or even lead to incorrect conclusions.
We also note a lack of consistency in reporting requirements across US entities. Some sectors, such as educational institutions,
report breaches through privately-run information sharing and analysis center (ISAC) organizations. The medical community complies
with reporting requirements of the Department of Health and Human Services. For-profit corporations may ultimately report to the US
Cybersecurity and Infrastructure Security Agency (CISA), and if publicly traded may be required to disclose breaches to shareholders
under Sarbanes-Oxley. Critical infrastructure providers have their own reporting bodies, such as NERC/FERC for electrical generation
and transmission. Though any citizen or corporation can report cyber incidents or crimes to CISA or IC3, the actual rate of reporting is
10
B. Blakely et al. International Journal of Accounting Information Systems 46 (2022) 100568
unknown. There are potential conflicts of interest when it comes to reporting incidents (let alone breaches) as organizations might not
wish to publicly admit to this happening, might not want to incur the cost of investigating further or being involved in investigation or
litigation, or might simply lack the capacity to gather and report on this data in a systematic manner. However, redacting sensitive
information while providing insights as to incidents and vulnerabilities would improve the information content of breach reports and
benefit practitioners desiring to strengthen internal controls.
Standardization of reporting requirements, and centralization of the reporting functions within a single agency, such as CISA (or
NIST from a standards, if not reporting, perspective), that can analyze aggregated data and communicate threats to all industries may
help to increase the utility of these reports for risk assessment and management. It would also aid organizations (and security product
vendors) in knowing what types of information to collect and retain for future reporting purposes. Making such reporting mandatory
can increase confidence in the statistical validity of such datasets, as well as leveling the playing field to remove some of the disin
centives for information sharing.
Information security practitioners, especially those charged with making investment decisions and managing the information
control structures to maximize their defensive posture, have increasingly been pressured for better metrics, justification of their ex
penditures, and to play a role in corporate governance that extends well beyond the traditional information technology job description.
As these shifts continue, or accelerate, it is critical that they be armed with correct, quantitative data to allow for the development of
risk models that provide not only accurate estimates but also estimates of known precision. Improving and standardizing available data
and control remediation are foundational and imperative steps in this direction.
For the accounting practitioner, cyber breach reports serve to educate practitioners as to the technical aspects of attack vectors and
the overall impact of cyber breaches; however, cyber breach reports, as currently structured, do not contain sufficient information
identifying exploited vulnerabilities to permit accounting and security professionals to revise internal controls. As such, existing
breach reporting either needs to be refocused on existing vulnerabilities, or the vulnerability associated with reported breaches, in
order for a revision of internal controls to be implemented.
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to
influence the work reported in this paper.
Acknowledgements
We appreciate the insightful comments and suggestions from anonymous reviewers and participants at the 2021 University of
Waterloo Centre for Information Integrity and Information Systems Assurance Symposium. We also wish to acknowledge that this
work has been supported by Argonne National Laboratory of the U.S. Department of Energy, Office of Science under contract DE-AC02-
06CH11357.
References
11
B. Blakely et al. International Journal of Accounting Information Systems 46 (2022) 100568
Julisch, K., 2013. Understanding and overcoming cyber security anti-patterns. Comput. Networks 57 (10), 2206–2211. https://doi.org/10.1016/j.
comnet.2012.11.023.
March, J.G., Shapira, Z., 2016. Managerial perspectives on risk and risk taking. Manage. Sci. 33 (11), 1404–1418. http://www.jstor.org/stable/2631920.
Marotta, A., Martinelli, F., Nanni, S., Orlando, A., Yautsiukhin, A., 2017. Cyber-insurance survey. Comput. Sci. Rev. 24 (May), 35–61. http://linkinghub.elsevier.com/
retrieve/pii/S1574013716301137.
McMorrow, D., 2010. Science of Cyber-Security. MITRE, McLean, Virginia.
Microsoft, 2020. “Microsoft Digital Defense Report.” 2020. https://www.microsoft.com/en-us/security/business/security-intelligence-report.
Peng, C., Xu, M., Xu, S., Hu, T., 2018. Modeling multivariate cybersecurity risks. J. Appl. Stat. February, 1–23. https://www.tandfonline.com/doi/full/10.1080/
02664763.2018.1436701.
Ruan, K., 2017. Introducing cybernomics: A unifying economic framework for measuring cyber risk. Comput. Security 65. https://doi.org/10.1016/j.
cose.2016.10.009.
Rundle, J., 2021. Industry groups urge lawmakers to streamline cyber breach reporting rules. Wall Street J. https://www.wsj.com/articles/industry-groups-urge-
lawmakers-to-streamline-breach-reporting-rules-11630575000.
Sarabi, A., Naghizadeh, P., Liu, Y., Liu, M., 2016. Risky business: fine-grained data breach prediction using business profiles. J. Cybersecurity 2 (1), 15–28. https://
academic.oup.com/cybersecurity/article-lookup/doi/10.1093/cybsec/tyw004.
Schatz, D., Bashroush, R., 2017. Economic valuation for information security investment: a systematic literature review. Inf. Syst. Front. 19, 1205–1228.
Schneider, F., 2012. Blueprint for a science of cybersecurity. The Next Wave 19 (2), 47–57. http://ecommons.library.cornell.edu/handle/1813/22943.
Siponen, M.T, Willison, R. 2007. “A Critical Assessment of IS Security Research Between 1990-2004.” ECIS 2007 Proceedings, 1: 1551–9. http://openarchive.cbs.dk/
handle/10398/6505.
Symantec. “Internet Security Threat Report.” 2019. 24. https://docs.broadcom.com/docs/istr-24-2019-en.
United States Department of Health and Human Services, 2019. Office for Civil Rights. “Annual Report to Congress on Breaches of Unsecured Protected Health
Information For Calendar Year 2019.” 2019. https://www.hhs.gov/sites/default/files/breach-report-to-congress-2019.pdf.
United States Department of Homeland Security, 2010. Risk Steering Committee. “DHS Risk Lexicon 2010 Edition.” 2010. Washington, DC https://www.dhs.gov/
sites/default/files/publications/dhs-risk-lexicon-2010_0.pdf.
United States Federal Depositors Insurance Corporation, 2021, “Agencies Approve Final Rule Requiring Computer-Security Incident Notification” https://www.fdic.
gov/news/press-releases/2021/pr21095.html.
United States National Initiative for Cybersecurity Careers and Studies Cybersecurity and Infrastructure Security Agency (NICCS CISA) 2022 https://niccs.cisa.gov/
about-niccs/cybersecurity-glossary.
United States National Institute of Standards and Technology, 2006. “FIPS PUB 200, Minimum Security Requirements for Federal Information and Information
Systems.” 2006. FIPS PUB 200. Gaithersburg, Maryland. https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.200.pdf.
United States National Institute of Standards and Technology, 2012. “NIST Special Publication 800-30, Revision 1: Guide for Conducting Risk Assessments.” 2012.
NIST SP 800-30r1. Gaithersburg, Maryland. https://doi.org/10.6028/NIST.SP.800-30r1.
United States National Institute of Standards and Technology, 2013. “NIST Special Publication 800-53, Revision 4: Security and Privacy Controls for Federal
Information Systems and Organizations.” 2013. NIST SP 800-53r4. Gaithersburg, Maryland. https://doi.org/10.6028/NIST.SP.800-53r4.
United States Executive Office of the President [Joseph Biden]. Executive Order 14028: Executive Order on Improving the Nation’s Cybersecurity. 12 May 2021.
Verizon, 2021. Data Breach Investigations Report.” 2021. https://www.verizon.com/business/resources/reports/dbir/.
Xu, M., Schweitzer, K.M., Bateman, R.M., Xu, S., 2018. Modeling and predicting cyber hacking breaches. IEEE Trans. Inf. Foren. Security 13 (11), 2856–2871. https://
doi.org/10.1109/TIFS.2018.2834227.
Young, D., Beebe, N., and Chang, F. 2012. “Prospect Theory and Information Security Investment Decisions,” 9. https://pdfs.semanticscholar.org/8108/
516e0f3259e65ea2d8f2a73c10010b7a2d83.pdf.
12