Attack IQ
Attack IQ
Ending the
Era of Security
Control Failure
A data analytic study of historic security control failures against
top MITRE ATT&CK techniques – and what to do to improve
security program performance.
Introduction: How
ATT&CK techniques that succeeded against
endpoint detection and response (EDR)
security controls. We chose EDR for two
to become a peak reasons: it is the most broadly adopted con-
trol across the industry, and AttackIQ has
wearable devices to track their pulse, opposite. From our laboratory environ-
exercise and steps taken, and oxygen- ment at AttackIQ, we know that leading
ation. The U.S. military trains constantly EDR technologies and our customers
on land, air, sea, space, and cyberspace can stop these top seven techniques
to prepare for potential conflicts. Unlike consistently, and therefore our custom-
the human body, a car engine, or the U.S. ers should be able to do so consistently
military, however, cybersecurity teams as well. The issue is that organizations
have until now lacked a means to exer- aren’t testing enough. Information tech-
cise, measure, and report on their health. nology, like the human body, is not stat-
The result is a mismatch. Even the most ic. Misconfigurations, infrastructure
effective technologies and the most changes, and team transitions all lead to
effective teams will fail to stop the adver- degraded security control performance
sary part of the time if they do not test over time. Only by testing controls
and train. against known threats can teams gen-
erate the data they need to understand
Imagine a World Cup qualifying team performance, tune up, and improve ef-
that made it to the first match but had fectiveness. That’s how they can become
failed to prepare for its opponents. Step- a World Cup team.
ping out on the pitch, the opposing team
would run circles around them. This is
the story in cybersecurity today. The
impact of a lack of continuous assess-
ment is that breaches continue to occur,
and adversaries continue to succeed. In
our historical analysis, we found that the
top 7 techniques have been used over
and over in impactful cyberattacks and
intrusions by adversaries like the Conti
ransomware gang and state-sponsored
actors from Russia, China, Iran, North
Korea, and others to achieve their stra-
tegic objectives. We outline the historic
impact of these techniques below.
The Seven So, what are they not? They are not a top
list of individual techniques by priority or
Deadly Techniques popularity. Other organizations have taken
this approach (to include the Center for
Threat-Informed Defense, of which we
How did we arrive at this 39 percent
are founding research partner). Instead,
statistic? We looked at our customers’
this list represents a solid foundation of
historical performance against a curated
techniques that a customer will see and
list of techniques to see how well they
which our research indicates they will
performed. The goal in selecting these
likely fail to prevent. Our historical data
techniques was to find a sweet spot for
also shows that by testing against these
realistic and popular techniques that could
techniques, customers can adjust their EDR
be prevented by recommended security
configurations to improve security control
configurations but are not currently being
performance.
prevented most of the time. We chose
these key techniques because they fit the Below are the scenarios and techniques
following criteria: that comprise the “Seven Deadly
Techniques.” The column on the left names
1. They match real-word attacks from the AttackIQ scenario (a software-based
threat actors that should concern logical combination of adversary behaviors)
our customers and their engineering that contains the technique within it. The
implementation is accurate. middle column names the known technique
from the MITRE ATT&CK framework of
2. Their usage is common; in other words,
known adversary tactics, techniques, and
they are not edge cases that are
procedures, the world’s leading repository
infrequently reported.
of threat intelligence and threat behavior.
3. They are core functional techniques that The column on the right shows how
help a threat actor achieve their goals. often these techniques are prevented
4. Laboratory evidence shows that the by EDR technologies in our customer’s
recommended configuration settings of environments. Of note, because the data is
EDR solutions should be able to prevent anonymized, we do not have clear visibility
the execution of these techniques. into our customer’s networks to know why
5. Our customers show that these a specific EDR solution failed to prevent a
techniques can be prevented in specific technique. We may pursue such
their environments, proving that our analysis in the future, building on the
prevention measurements are not just research methodologies we developed in the
theoretical but practical and real world. context of our first report.
DATA STUDY REPORT
JScript. This file format is frequently used up their process attack chains or for per-
by threat actors directly as a malicious email sistence to survive reboots and shutdowns.
attachment or a file dropped and executed To help their malicious activities blend in and
by macro-enabled Office documents. The hide from analyst detection, actors will use
Russia-linked Gamaredon group leveraged task names that appear to be legitimate up-
this technique in the cyberattacks during date mechanisms. The cybercriminal group
the start of the invasion of Ukraine. Addi- Wizard Spider has used scheduled tasks
tionally, actors like Mustang Panda have named after Google and Windows
found that the mshta tool can be used to applications to help facilitate their
execute malicious script code directly in the ransomware operations.
command line.
7. Scenario: Stop Windows Defender
5. Scenario: BITS Jobs Script via Encoded Powershell Script
BITS Jobs (T1197): The Background Intelli- Impair Defenses: Disable or Modify Tools
gent Transfer Service (BITS) is a file transfer (T1562.001) | Command and Scripting
mechanism found in Microsoft Windows and Interpreter: PowerShell (T1059.001) |
commonly used by legitimate applications Obfuscated Files or Information (T1027):
to use the system’s available idle bandwidth Threat actors will take overt actions to dis-
without disrupting other applications. This able the security tools that could detect or
functionality has been historically abused by prevent their future operations. PowerShell’s
multiple threat actors during their attacks. ability to integrate with Windows internals
APT41 is a Chinese-sponsored threat actor makes it a key tool to be abused in these
who conducted global cyberespionage at- attacks to help facilitate disabling Windows
tacks throughout 2020 that used bitsadmin. tools or controls. Additionally, actors encode
exe to download their 2nd stage payloads. their PowerShell commands to make the
Additionally, the FBI warned about APT39, initial incident response more difficult. The
an Iranian-sponsored threat actor, using Maze ransomware group used PowerShell
BITS to exfiltrate stolen data during their to disable Windows Defender’s real-time
global attacks. monitoring before their encryption process
was executed. They demanded a $15 million
6. Scenario: Scheduled dollar ransom.
Task Masquerading
Customers that prevented 76% - 99%
of the Seven Deadly Techniques
Customers that prevented 51% - 75%
of the Seven Deadly Techniques
Customers that prevented 26% - 50%
of the Seven Deadly Techniques
Customers that prevented 1% - 25%
of the Seven Deadly Techniques
Customers that prevented 0%
of the Seven Deadly Techniques
Methodology
How and why do we trust these scenarios to like the attackers themselves, all the scenar-
make the 39 percent determination of aver- ios, assessments, and attack graphs in the
age effectiveness? AttackIQ Security Optimization can be used
in production, at scale, and run concurrently
All these scenarios were manually dou- against all a customer’s assets at once.
ble-checked for accuracy to determine how
well they match real world actors’ use of For those interested in the specifics, Ap-
the same techniques. In our data analysis, pendix A outlines a technical analysis of the
we measured how well the techniques per- seven scenarios against real-world exam-
formed against at least 50 percent of our ples. Finally, it should be noted that all these
host agents (which measure security control scenarios were run as system which means
performance) in our customers’ environ- they are running with the highest privileg-
ments. We confirmed that those scenarios es possible on the host. This means that if
were blocked by EDR tools in our internal AttackIQ’s Security Optimization Platform
labs and then validated that our expectation measures a prevention at that level, it would
of a prevented status could be found in our measure a prevention at a lower level as
external customer implementations. Finally, well. When we ran our tests in the AttackIQ
DATA STUDY REPORT
the state of the practice to help improve the world’s cybersecurity effectiveness. Other founding
research partners include JPMorgan Chase, Bank of America, Citi, Fortinet, HCA Healthcare, and
IBM Security.
The impact of continuous security control validation and a threat-informed defense strategy
is significant, and can help drive down security control failures and elevate cybersecurity
effectiveness. A study by the analyst firm IDC of existing AttackIQ customers found that
“substantial benefits were achieved as a result of deploying AttackIQ to test cybersecurity
readiness and validate the effectiveness of security programs. Specifically, they noted significant
improvements in the efficiency of security staff and risk reduction, the importance of purple
teaming, and the value of AttackIQ Academy”, the company’s free online course in breach and
attack simulation and the practice of threat-informed defense. The IDC report of AttackIQ
customers found that continuous security control validation led to 47 percent more efficient
security operations teams, a 44 percent reduction in potential costs of security breaches, and 35
percent less impactful breaches overall.1
Finally, the IDC study found that by aligning red teams and blue teams in a process of purple
teaming for continuous testing, organizations could save at least $4.7 million in threat response
expenditures. When asked why this was the case, one customer said, “Business risk has been re-
duced, because with AttackIQ we can measure where things work well. If something isn’t work-
ing, we can take steps to address that.” By running adversary emulations against an organiza-
tion’s security program, an organization2 can improve its performance against key adversaries like
APT29, the Conti Ransomware Group, or Muddy Waters, to name a few of the organizations that
have employed the Seven Deadly Techniques throughout recent history. By embracing a heuris-
tic-focused defense capability, investing in a defense-in-depth strategy that protects high-value
assets and adopts an assume breach mindset, and by deploying a threat-informed defense strat-
egy for continuous security control validation, customers can generate real-time data to elevate
their security program effectiveness and keep intruders out.
1 Source - IDC Infographic, sponsored by AttackIQ, The Business Value of AttackIQ Security Optimization Platform, doc #US49454222 and July 2022
2 Ibid
DATA STUDY REPORT
Appendix A
This technical appendix shows how our scenarios closely match real-world techniques. It then
discusses mitigation processes and offers sigma rules for improving customers’ detection of
these techniques.
Mitigation process:
1. BITS Job Script: This scenario uses the Background Intelligent Transfer Service (BITS) to
download a remote payload to a temporary directory. This is a mechanism found in Microsoft
Windows and commonly used by legitimate applications to use the system’s available idle
bandwidth to retrieve files without disrupting other applications.
A PowerShell script is executed on the targeted host and executes the following commands to
create and stage a file to be downloaded from a remote server:
Then the following command is executed to begin the BITS Job transfer, monitor progress for
successful transfer, and mark the job complete:
Our scenario activity is a direct match with (or similar to) the following reported examples:
Detection details:
With an EDR or SIEM product, you can create detections to look for suspicious use of the Bitsad-
min tool on windows devices by using the following detection logic:
Mitigation details:
MITRE has provided the following mitigation steps for BITS Jobs (T1197)
• M1037 - Filter Network Traffic
• M1028 - Operating System Configuration
• M1018 - User Account Management
Customer prevention statistics for this scenario: 29 prevented out of 72 run, with a
prevention rate of 40 percent.
Mitigation process:
2. Deobfuscate / Decode Files or Information Script: This scenario uses certutil.exe to decode a
base64 file into a malicious executable. Certutil is a command-line tool natively found on Mic
rosoft Windows systems that is meant to be used to help validate and verify certificate
authority information. The certificate authority files are commonly encoded in base64 so the
tool has the built-in functionality to decode this common encoding format.
A batch file is executed on the targeted host, and it launches certutil.exe with the follow
arguments to decode a base64 encoded executable into a binary named “calc.exe” located in a
temporary directory.
Our scenario activity is a direct match for (or similar to) the following examples:
Detection details:
With an EDR or SIEM product, you can create detections to look for suspicious use of the Certutil
binary on windows devices by using the following detection logic:
Mitigation details:
It is recommended that only administrators and authorized users have access to utilizing system
interpreters such as cmd.exe and powershell.exe, as well as system binaries such as certutil.exe.
This will limit the chance of malicious actors carrying out this technique on compromised
end users.
Customer prevention statistics for this scenario: 24 prevented out of 57 run, with a 42 per-
cent prevention rate.
Mitigation process:
3. Dump SAM hashes with Mimikatz using a Volume Shadow Copy: The Security Account
Manager (SAM) is a database in Microsoft Windows that stores account passwords and can
be used to authenticate local or remote users. The account passwords are hashed and stored
in a registry hive. The file is locked by the operating system and it cannot normally be read by
other applications. A Volume Shadow Copy is service in Windows that makes point-in-time
copies of files including those that are normally locked and unreadable on the host. This
scenario abuses Volume Shadow Copy to make a backup of that locked file which can then be
used by Mimikatz to dump credentials.
Once the Volume Shadow Copy is created, the SAM hive is copied to a temporary directory. Mim-
ikatz is then written to disk and executed with the following arguments:
Our scenario activity is a direct match for (or similar to) the following examples:
Detection details:
For detecting and/or preventing this attack through Anti-Virus, we encourage placing systems in
a Quarantine policy and ensure that prevention is enabled for static and dynamic analysis results.
Additionally, ensuring that AV’s have updated blocklist entries for known Mimikatz signatures will
help ensure that this binary will be quarantined if attempted to be placed on disk.
For detecting and/or preventing this attack through EDR or SIEM products, we encourage using
the below detection details to alert when shadow copies are being created for the Mimikatz pay-
load to read from:
Mitigation details:
MITRE has provided the following mitigation steps for OS Credential Dumping: Security Account
Manager (T1003.002)
• M1028 - Operating System Configuration
• M1027 - Password Policies
• M1026 - Privileged Account Management
• M1017 - User Training
Customer prevention statistics for this scenario are: 59 prevented out of 91 run, with a 64
percent prevention rate.
DATA STUDY REPORT
Mitigation process:
4. Mshta Script: Mshta is a native binary found on Microsoft Windows systems that opens
HTML Application (HTA) files which can contain web scripts written in VBScript or JScript.
Additionally, raw script code can be passed via the command line to be directly executed. Our
scenario uses that technique to get MSHTA to launch another binary.
A batch file is executed that copies a malicious executable to a temporary directory and then
MSHTA is launched with arguments containing VBScript code to open that executable.
- mshta.exe vbscript:CreateObject(“Wscript.Shell”).Run(“%temp%\attack
iq_mshta\binary.exe”,0,true)(window.close) > nul 2>&1
Our scenario activity is a direct match for (or similar to) the following examples:
Detection details:
For detecting and/or preventing this attack through EDR or SIEM products, we encourage using
the below detection details to alert when mshta.exe is being use din a possible malicious manor:
Mitigation details:
MITRE has provided the following mitigation steps for System Binary Proxy Execution: Mshta
(T1218.005)
• M1042 - Disable or Remove Feature or Program
• M1038 - Execution Prevention
Customer prevention statistics for this scenario: 39 prevented out of 81 run, with a 48 per-
cent prevention rate.
DATA STUDY REPORT
Mitigation process:
5. Remote File Copy Script: PowerShell is one of the most common sources of threats detected
on endpoints. Using legitimate built-in functionality, an actor can launch directly from the com
mand line and instruct PowerShell to retrieve a file from a URL and then execute their
malicious payload. This scenario performs that behavior to download a remote file to a
temporary directory.
The following PowerShell commands are executed to download a remote malicious file:
Our scenario activity is a direct match for (or similar to) the following examples:
Detection details:
For detecting and/or preventing this attack through EDR or SIEM products, we encourage
using the below detection details to alert when powershell is being used to download files onto
the system:
Process Name == “powershell.exe”
Command Line CONTAINS ((“DownloadFile” OR “Invoke-WebRequest” OR
“IWR”) AND “http”)
Mitigation details:
MITRE has provided the following mitigation steps for Command and Scripting Interpreter: Pow-
erShell (T1059.001)
• M1049 - Antivirus/Antimalware
• M1045 - Code Signing
• M1042 - Disable or Remove Feature or Program
• M1038 - Execution Prevention
• M1026 - Privileged Account Management
DATA STUDY REPORT
The scenario was prevented in our labs by: Cisco and Cybereason. The scenario was detectedin
our labs by: Microsoft Defender. Customer prevention statistics for this scenario: 13 prevented
out of 38 run, with a prevention rate of 34 percent.
Mitigation process:
6. Scheduled Task Masquerading: Scheduled Tasks can be created to either initially launch a
process at a pre-determined date and time or repeatedly execute commands at specific
intervals. This scenario creates a scheduled task using the native Windows utility “schtasks.
exe”. The created task is named to appear as an AdobeFlashSync update and it launches a
batch file in temporary directory from the System account.
The scheduled task is created and set to run in 60 seconds from its initial creation.
After the task is executed, the schtasks utility is used to check to see if the task still exists and if
it was successfully executed.
Our scenario activity is a direct match for or (similar to) the following examples:
Detection details:
For detecting and/or preventing this attack through EDR or SIEM products, we encourage using
the below detection details to alert when scheduled tasks are being created:
Mitigation details:
Mitigation process:
7. Stop Windows Defender via Encoded Powershell Script: Encoded PowerShell commands
are commonly used to try and obfuscate the code’s true intentions by making it difficult to
read or decipher with an atomic signature. This scenario uses a common PowerShell
obfuscation tool to encode a PowerShell script that disables Windows Defender.
The original script is first encoded with base64 and then it is executed with the -encodedCom-
mand parameter.
When decoded, the following PowerShell cmdlets are executed disabling the various components
of Windows Defender:
Our scenario activity is a direct match for or similar to the following examples:
Detection details:
For detecting encoded powershell being ran, utilize the below query for EDR or SIEM
related products:
For detecting plain text attempts to disable Windows Defender, utilize the below query for EDR or
SIEM related products:
Mitigation details:
Customer prevention statistics: 11 prevented out of 43 run, with a 25 percent prevention rate.
AttackIQ has customers that use the AttackIQ Security Optimization Platform on their premises and disconnected
from the cloud, many in the U.S. government, and we did not have access to those customers data during the course
of this study.