INTELLIGENCE-LED TESTING

www.SELabs.uk
www.SELabs.uk [email protected]
[email protected] @SELabsUK
@SELabsUK www.facebook.com/selabsuk
www.facebook.com/selabsuk blog.selabs.uk
blog.selabs.uk

HOME ANTI-
MALWARE
PROTECTION
OCT - DEC 2017
 SE Labs SE Labs

CONTENTS
Introduction 04
Executive Summary 05
1. Total Accuracy Ratings 06
2. Protection Ratings 08
3. Protection Scores 10
4. Protection Details 11
5. Legitimate Software Ratings 12
SE Labs tested a variety of anti-malware (aka ‘anti-virus’; aka ‘endpoint 6. Conclusions 16
security’) products from a range of well-known vendors in an effort to Appendix A: Terms used 17
judge which were the most effective.
Appendix B: FAQs 18
Appendix C: Product versions 19
Each product was exposed to the same threats, which were a mixture of
targeted attacks using well-established techniques and public email and Appendix D: Attack types 19

web-based threats that were found to be live on the internet at the time
of the test.

The results indicate how effectively the products were at detecting and/
or protecting against those threats in real time. Document version 1.0 Written 1st February 2018

02 OCT - DEC 2017 • Home Anti-Malware Protection Home Anti-Malware Protection • OCT - DEC 2017 03
 SE Labs

INTRODUCTION EXECUTIVE SUMMARY

WILL YOUR ANTI-MALWARE PROTECT Product names
YOU FROM TARGETED ATTACKS? It is good practice to stay up to date with the latest version of your
chosen endpoint security product. We made best efforts to ensure
The news isn’t good. Discover your best options in our latest report.
that each product tested was the very latest version running with
Criminals routinely create ingenious scams and indiscriminate the most recent updates to give the best possible outcome.
Simon Edwards attacks designed to compromise the unlucky and, occasionally,
For specific build numbers, see Appendix C: Product versions on page 19.
Director foolish. But sometimes they focus on a specific target rather than
casting a net wide in the hope of landing something interesting.
WEBSITE www.SELabs.uk EXECUTIVE SUMMARY
TWITTER @SELabsUK Products Tested Protection Legitimate Total
Targeted attacks can range from basic, like an email simply
EMAIL [email protected] Accuracy (%) Accuracy (%) Accuracy (%)
FACEBOOK www.facebook.com/selabsuk
asking you to send some money to an account, through to
BLOG blog.selabs.uk extremely devious and technical. If you received an email from Norton Security 94% 98% 96%
PHONE 0203 875 5000 your accountant with an attached PDF or Excel spreadsheet Kaspersky Internet Security 88% 100% 96%
POST ONE Croydon, London, CR0 0XT would you open it? Most would and all that then stands between ESET Smart Security 79% 100% 93%
them and a successful hack (because the email was a trick and Avira Free Security Suite 77% 100% 92%
MANAGEMENT
Operations Director Marc Briggs contained a dodgy document that gives remote control to the
Avast Free Antivirus 75% 100% 91%
Office Manager Magdalena Jurenko attacker) is the security software running on their PC.
Trend Micro Internet Security 78% 96% 89%
Technical Lead Stefan Dumitrascu
In this test we’ve included indiscriminate, public attacks that come AVG Antivirus Free Edition 70% 100% 89%
TESTING TEAM at victims from the web and via email, but we’ve also included ZoneAlarm Free Antivirus 61% 100% 86%
Thomas Bean
some devious targeted attacks to see how well-protected F-Secure Safe Internet Security 76% 89% 84%
Dimitar Dobrev
Liam Fisher
potential victims would be.
Bitdefender Internet Security 72% 90% 83%
Gia Gorbold
We’ve not created any new types of threat and we’ve not Cisco Immunet 52% 100% 83%
Pooja Jain
Ivan Merazchiev discovered and used ‘zero day’ attacks. Instead we took tools Microsoft Security Essentials 41% 99% 78%
Jon Thompson that are freely distributed online and are well-known to penetration Qihoo 360 Total Security 27% 95% 70%
Jake Warren
testers and criminals alike. We used these to generate threats that
Stephen Withey Products highlighted in green were the most accurate, scoring 85 per cent or more for Total Accuracy.
are realistic representations of what someone could quite easily
Those in yellow scored less than 85 but 75 or more. Products shown in red scored less than 75 per cent.
IT SUPPORT put together to attack you or your business. For exact percentages, see 1. Total Accuracy Ratings on page 6.
Danny King-Smith
The results are extremely worrying. While a few products were
•anti-malware •most
Chris Short
There was a wide spread in how effectively the False positives were not an issue for
excellent at detecting and protecting against these threats many
PUBLICATION products were able to handle products
more were less useful. We will continue this work and report any
Steve Haines
progress that these companies make in improving their products. general threats from cyber criminals… Most of the products were good at correctly classifying
Colin Mackleworth Most products were largely capable of handling public legitimate applications and websites. A slim majority
If you spot a detail in this report that you don’t understand, web-based threats such as those used by criminals allowed all of the legitimate websites and applications.
SE Labs is BS EN ISO 9001 : 2015 certified for 
The Provision of IT Security Product Testing. or would like to discuss, please contact us via our Twitter or to attack Windows PCs, tricking users into running

SE Labs Ltd is a member of the Anti-Malware

Facebook accounts. malicious files or running scripts that download and
run malicious files.
•Symantec
Which products were the most effective?
and Kaspersky Lab products achieved
Testing Standards Organization (AMTSO) SE Labs uses current threat intelligence to make our tests as
extremely good results due to a combination of
realistic as possible. To learn more about how we test, how we
While every effort is made to ensure the
accuracy of the information published in this
define ‘threat intelligence’ and how we use it to improve our tests • .. and targeted attacks were prevented in
many cases.
their ability to block malicious URLs, handle exploits and
correctly classify legitimate applications and websites.
document, no guarantee is expressed or
please visit our website and follow us on Twitter.
A few were particularly competent at blocking
implied and SE Labs Ltd does not accept
more targeted, exploit-based attacks. However, the
liability for any loss or damage that may  We continue to test Microsoft and McAfee business products
arise from any errors or omissions. privately and plan to produce results in the first report of 2018. majority struggled. Simon Edwards, SE Labs, 1st February 2018

04 OCT - DEC 2017 • Home Anti-Malware Protection Home Anti-Malware Protection • OCT - DEC 2017 05
 SE Labs SE Labs

1. TOTAL ACCURACY RATINGS AWARDS

The following products win SE Labs awards:
Judging the effectiveness of an endpoint security it from downloading any further code to the target.
product is a subtle art, and many factors are at play In another case malware might run on the target for a
when assessing how well it performs. To make things short while before its behaviour is detected and its
easier we’ve combined all the different results from this code is deleted or moved to a safe ‘quarantine’ area for
●E
 SET Smart Security
report into one easy-to-understand graph. future analysis. We take these outcomes into account ●N
 orton Security

HOME A

HOME A
CTION

CTION
when attributing points that form final ratings. ● Avira Free Security Suite
The graph below takes into account not only each OCT-DEC 2017 ●K
 aspersky Internet Security
OCT-DEC 2017

TE

TE
● Avast Free Antivirus

NT

N
product’s ability to detect and protect against threats, For example, a product that completely blocks a threat I- I-

O
T
MA R MA R
LWARE P LWARE P
but also its handling of non-malicious objects such as is rated more highly than one that allows a threat to run
web addresses (URLs) and applications. for a while before eventually evicting it. Products that
allow all malware infections, or that block popular
Not all protections, or detections for that matter, are legitimate applications, are penalised heavily. ●T
 rend Micro Internet Security ●F
 -Secure Safe Internet

equal. A product might completely block a URL, which Security

HOME A

HOME A
CTION

CTION
stops the threat before it can even start its intended Categorising how a product handles legitimate objects ●A
 VG Antivirus Free Edition
●B
 itdefender Internet Security
series of malicious events. Alternatively, the product is complex, and you can find out how we do it in OCT-DEC 2017 OCT-DEC 2017
●Z

TE

TE
 oneAlarm Free Antivirus

NT

N
I- I- ●C
 isco Immunet

O
might allow a web-based exploit to execute but prevent 5. Legitimate Software Ratings on page 12.

T
MA R MA R
LWARE P LWARE P

HOME A

CTION
Total Accuracy Ratings ●M
 icrosoft Security Essentials

OCT-DEC 2017

TE
NT
1,108 I-

O
MA R
LWARE P

TOTAL ACCURACY RATINGS

831 Product Total Accuracy Rating Total Accuracy (%) Award
Norton Security 1,069 96% AAA
F-Secure Safe Internet Security
Trend Micro Internet Security

Bitdefender Internet Security

Kaspersky Internet Security 1,060 96% AAA

Microsoft Security Essentials
Kaspersky Internet Security

AVG Antivirus Free Edition

AA
ZoneAlarm Free Antivirus

ESET Smart Security 1,025 93%

Qihoo 360 Total Security
Avira Free Security Suite

554 Avira Free Security Suite 1,017 92% AA

ESET Smart Security

Avast Free Antivirus

Avast Free Antivirus 1,007 91% AA

Trend Micro Internet Security 988.5 89% A

Norton Security

Cisco Immunet

AVG Antivirus Free Edition 988 89% A

277 ZoneAlarm Free Antivirus 953 86% A

F-Secure Safe Internet Security 936 84% B

Bitdefender Internet Security 924 83% B

0 Cisco Immunet 915 83% B

Microsoft Security Essentials 867 78% C

Total Accuracy Ratings combine protection and false positives.
Qihoo 360 Total Security 778 70%

06 OCT - DEC 2017 • Home Anti-Malware Protection Home Anti-Malware Protection • OCT - DEC 2017 07
 SE Labs SE Labs

2. PROTECTION RATINGS
The results below indicate how effectively the products • Neutralised (+1) Rating calculations
dealt with threats. Points are earned for detecting the Products that kill all running malicious processes We calculate the protection ratings using the
threat and for either blocking or neutralising it. ‘neutralise’ the threat and win one point. following formula:

• Detected (+1) • Complete remediation (+1) Protection rating =

If the product detects the threat with any degree of If, in addition to neutralising a threat, the product (1x number of Detected) +
useful information, we award it one point. removes all significant traces of the attack, it gains (2x number of Blocked) +
an additional one point. (1x number of Neutralised) +
• Blocked (+2) (1x number of Complete remediation) +
Threats that are disallowed from even starting their • Compromised (-5) (-5x number of Compromised)
malicious activities are blocked. Blocking products If the threat compromises the system, the product
score two points. loses five points. This loss may be reduced to four The ‘Complete remediation’ number relates to cases of
points if it manages to detect the threat (see Detected, neutralisation in which all significant traces of the attack
above), as this at least alerts the user, who may now were removed from the target. Such traces should not
take steps to secure the system. exist if the threat was ‘Blocked’ and so Blocked results
imply Complete remediation.

These ratings are based on our opinion of how important

these different outcomes are. You may have a different
view on how seriously you treat a ‘Compromise’ or
Protection Ratings ‘Neutralisation without complete remediation’. If you
want to create your own rating system, you can use the
400 raw data from 4. Protection Details on page 11 to roll
your own set of personalised ratings.

PROTECTION RATINGS

Qihoo 360 Total Security

300
Product Protection Rating Protection Rating (%)
Norton Security 377 94%
F-Secure Safe Internet Security
Trend Micro Internet Security

Bitdefender Internet Security

Kaspersky Internet Security 352 88%

Kaspersky Internet Security

AVG Antivirus Free Edition

ZoneAlarm Free Antivirus

200 ESET Smart Security 317 79%

Avira Free Security Suite

Trend Micro Internet Security 312 78%

ESET Smart Security

Avast Free Antivirus

Avira Free Security Suite 309 77%

Security Essentials

F-Secure Safe Internet Security 304 76%

Norton Security

Cisco Immunet

100 Avast Free Antivirus 299 75%

Bitdefender Internet Security 288 72%
Microsoft

AVG Antivirus Free Edition 280 70%

ZoneAlarm Free Antivirus 245 61%

0 Cisco Immunet 207 52%

Protection Ratings are weighted to show that how products handle threats can be subtler than Microsoft Security Essentials 164 41%
just ‘win’ or ‘lose’. Qihoo 360 Total Security 106 27%

Average: 69%

08 OCT - DEC 2017 • Home Anti-Malware Protection Home Anti-Malware Protection • OCT - DEC 2017 09
 SE Labs SE Labs

3. PROTECTION SCORES 4. PROTECTION DETAILS

This graph shows the overall level of protection, For each product we add Blocked and Neutralised These results break down how each product handled an element of the threat but aren’t equipped to stop it.
making no distinction between neutralised and cases together to make one simple tally. threats into some detail. You can see how many Products can also provide protection even if they don’t
blocked incidents. detected a threat and the levels of protection provided. detect certain threats. Some threats abort on detecting
specific endpoint protection software.
Products sometimes detect more threats than they
Protection Scores
protect against. This can happen when they recognise
100

Protection Details

100
75
F-Secure Safe Internet Security

75
Trend Micro Internet Security

Bitdefender Internet Security

Microsoft Security Essentials

F-Secure Safe Internet Security

Kaspersky Internet Security

AVG Antivirus Free Edition

Trend Micro Internet Security

Bitdefender Internet Security

ZoneAlarm Free Antivirus

Microsoft Security Essentials

Qihoo 360 Total Security
50
Avira Free Security Suite

Kaspersky Internet Security

AVG Antivirus Free Edition

ZoneAlarm Free Antivirus

Qihoo 360 Total Security

Avira Free Security Suite
ESET Smart Security

Avast Free Antivirus

50

ESET Smart Security

Avast Free Antivirus

Norton Security

Cisco Immunet

Norton Security

Cisco Immunet
25
25
Defended
Neutralised
Compromised
0 0

Protection Scores are a simple count of how many times a product protected the system. This data shows in detail how each product handled the threats used.

PROTECTION SCORES PROTECTION DETAILS

Product Protection Score Product Detected Blocked Neutralised Compromised Protected
Norton Security 100 Norton Security 100 91 9 0 100
F-Secure Safe Internet Security 96 Kaspersky Internet Security 91 94 2 4 96
Kaspersky Internet Security 96 Trend Micro Internet Security 97 82 14 4 96
Trend Micro Internet Security 96 F-Secure Safe Internet Security 98 79 17 4 96
ESET Smart Security 93 ESET Smart Security 94 87 6 7 93
Avira Free Security Suite 92 Avira Free Security Suite 96 80 12 8 92
Avast Free Antivirus 90 Avast Free Antivirus 100 79 11 10 90
AVG Antivirus Free Edition 90 AVG Antivirus Free Edition 92 79 11 10 90
Bitdefender Internet Security 90 Bitdefender Internet Security 99 71 19 10 90
ZoneAlarm FreeAntivirus 86 ZoneAlarm Free Antivirus 97 64 22 14 86
Cisco Immunet 79 Cisco Immunet 92 67 12 21 79
Microsoft Security Essentials 75 Microsoft Security Essentials 74 70 5 25 75
Qihoo 360 Total Security 72 Qihoo 360 Total Security 73 55 17 28 72

10 OCT - DEC 2017 • Home Anti-Malware Protection Home Anti-Malware Protection • OCT - DEC 2017 11
 SE Labs SE Labs

5. LEGITIMATE SOFTWARE RATINGS 5.1 Interaction Ratings

It’s crucial that anti-malware endpoint products not classifies the application and how it presents that
These ratings indicate how accurately the products We also take into account the prevalence (popularity) of only stop – or at least detect – threats, but that they information to the user. Sometimes the endpoint
classify legitimate applications and URLs, while also the applications and websites used in this part of the allow legitimate applications to install and run without software will pass the buck and demand that the user
taking into account the interactions that each product test, applying stricter penalties for when products misclassifying them as malware. Such an error is decide if the application is safe or not. In such cases
has with the user. Ideally a product will either not misclassify very popular software and sites. known as a ‘false positive’ (FP). the product may make a recommendation to allow
classify a legitimate object or will classify it as safe. To understand how we calculate these ratings, see or block. In other cases, the product will make no
In neither case should it bother the user. 5.3 Accuracy ratings on page 15. In reality, genuine FPs are quite rare in testing. In our recommendation, which is possibly even less helpful.
experience it is unusual for a legitimate application to
be classified as ‘malware’. More often it will be classified If a product allows an application to install and run with
Legitimate Software Ratings
as ‘unknown’, ‘suspicious’ or ‘unwanted’ (or terms that no user interaction, or with simply a brief notification
708
mean much the same thing). that the application is likely to be safe, it has achieved
an optimum result. Anything else is a Non-Optimal
We use a subtle system of rating an endpoint’s approach Classification/Action (NOCA). We think that measuring
to legitimate objects, which takes into account how it NOCAs is more useful than counting the rarer FPs.

Interaction Ratings

F-Secure Safe Internet Security

None Click to allow Click to allow/block (no Click to block None

Trend Micro Internet Security

Bitdefender Internet Security

Microsoft Security Essentials
(allowed) (default allow) recommendation) (default block) (blocked)
Kaspersky Internet Security
AVG Antivirus Free Edition

ZoneAlarm Free Antivirus

354

Qihoo 360 Total Security

Object is safe 2 1.5 1 A
Avira Free Security Suite

Object is unknown 2 1 0.5 0 -0.5 B

ESET Smart Security
Avast Free Antivirus

Object is not classified 2 0.5 0 -0.5 -1 C

Norton Security

Object is suspicious 0.5 0 -0.5 -1 -1.5 D

Cisco Immunet

Object is unwanted 0 -0.5 -1 -1.5 -2 E

Object is malicious -2 -2 F
1 2 3 4 5

0 Products that do not bother users and classify most applications correctly earn
more points than those that ask questions and condemn legitimate applications.
Legitimate Software Ratings can indicate how well a vendor has tuned its detection engine.

INTERACTION RATINGS
LEGITIMATE SOFTWARE RATINGS Product None Click to Block None Click to allow
Product Legitimate Accuracy Rating Legitimate Accuracy (%) (Allowed) (Default Block) (Blocked) (Default Allow)
Avast Free Antivirus 100 0 0 0
Avast Free Antivirus 708 100%
AVG Antivirus Free Edition 708 100% AVG Antivirus Free Edition 100 0 0 0

Avira Free Security Suite 708 100% Avira Free Security Suite 100 0 0 0

Cisco Immunet 708 100% Cisco Immunet 100 0 0 0

ESET Smart Security 708 100% ESET Smart Security 100 0 0 0

Kaspersky Internet Security 708 100% Kaspersky Internet Security 100 0 0 0

ZoneAlarm Free Antivirus 708 100% ZoneAlarm Free Antivirus 100 0 0 0
Microsoft Security Essentials 703 99% Microsoft Security Essentials 98 0 0 2
Norton Security 692 98% Norton Security 98 1 1 0
Trend Micro Internet Security 676.5 96% Trend Micro Internet Security 98 0 2 0
Qihoo 360 Total Security 672 95% Qihoo 360 Total Security 96 2 2 0
Bitdefender Internet Security 636 90% F-Secure Safe Internet Security 94 0 6 0
F-Secure Safe Internet Security 632 89% Bitdefender Internet Security 92 8 0 0

12 OCT - DEC 2017 • Home Anti-Malware Protection Home Anti-Malware Protection • OCT - DEC 2017 13
 SE Labs SE Labs

5.2 Prevalence Ratings 5.3 Accuracy Ratings 5.4 Distribution of

There is a significant difference between an endpoint LEGITIMATE SOFTWARE PREVALENCE We calculate legitimate software accuracy ratings by Impact Categories
product blocking a popular application such as the RATING MODIFIERS multiplying together the interaction and prevalence Endpoint products that were most accurate in
latest version of Microsoft Word and condemning a rare Impact Category Rating Modifier ratings for each download and installation: handling legitimate objects achieved the highest
Iranian dating toolbar for Internet Explorer 6. One is very Very high impact 5 ratings. If all objects were of the highest prevalence,
popular all over the world and its detection as malware High impact 4 Accuracy rating = Interaction rating x Prevalence the maximum possible rating would be 1,000 (100
(or something less serious but still suspicious) is a big Medium impact 3 rating incidents x (2 interaction rating x 5 prevalence rating)).
deal. Conversely, the outdated toolbar won’t have had Low impact 2
a comparably large user base even when it was new. If a product allowed one legitimate, Medium impact In this test there was a range of applications with
Very low impact 1
Detecting this application as malware may be wrong, application to install with zero interaction with the user, different levels of prevalence. The table below shows
but it is less impactful in the overall scheme of things. then its Accuracy rating would be calculated like this: the frequency:
Applications were downloaded and installed during the
With this in mind, we collected applications of varying test, but third-party download sites were avoided and Accuracy rating = 2 x 3 = 6 LEGITIMATE SOFTWARE CATEGORY FREQUENCY
popularity and sorted them into five separate categories, original developers’ URLs were used where possible. Prevalence Rating Frequency
as follows: Download sites will sometimes bundle additional This same calculation is made for each legitimate Very high impact 23
components into applications’ install files, which may application/site in the test and the results are summed
High impact 39
1. Very high impact correctly cause anti-malware products to flag adware. and used to populate the graph and table shown under
Medium impact 17
2. High impact We remove adware from the test set because it is often 5. Legitimate Software Ratings on page 12.
Low impact 11
3. Medium impact unclear how desirable this type of code is.
Very low impact 10
4. Low impact
5. Very low impact The prevalence for each application and URL is Grand total 100

estimated using metrics such as third-party download

Incorrectly handling any legitimate application will sites and the data from Alexa.com’s global traffic
invoke penalties, but classifying Microsoft Word as ranking system.
malware and blocking it without any way for the user
to override this will bring far greater penalties than
doing the same for an ancient niche toolbar. In order
to calculate these relative penalties, we assigned
each impact category with a rating modifier, as shown
in the table above.

14 OCT - DEC 2017 • Home Anti-Malware Protection Home Anti-Malware Protection • OCT - DEC 2017 15
 SE Labs SE Labs

6. CONCLUSIONS APPENDICES
Attacks in this test included threats that affect the
wider public and more closely-targeted individuals
Kaspersky Internet Security came a very close
second. This is because it made no mistakes with the
APPENDIX A: Terms Used
and organisations. You could say that we tested the legitimate software and protected against 96 per cent
products with ‘public’ malware and full-on hacking of the threats. In the cases where it was compromised TERM MEANING
attacks. We introduced the threats in a realistic way (with targeted attacks), it detected the attack and
The attack succeeded, resulting in malware running unhindered on the target. In the
such that threats seen in the wild on websites were removed the threat, although we were still able to Compromised case of a targeted attack, the attacker was able to take remote control of the system
downloaded from those same websites, while threats hack the system even after the initial malicious file and carry out a variety of tasks without hindrance.
caught spreading through email were delivered to our was removed.
Blocked The attack was prevented from making any changes to the target.
target systems as emails.
ESET Smart Security takes third place. It was When a security product misclassifies a legitimate application or website as
False positive
All of the products tested are well-known and should compromised more often than Kaspersky’s product being malicious, it generates a ‘false positive’.
do well in this test. While we do ‘create’ threats by but detected more of the threats. It scores just ahead Neutralised The exploit or malware payload ran on the target but was subsequently removed.
using publicly available free hacking tools, we don’t of two of the free products in this test, those from
If a security product removes all significant traces of an attack, it has achieved
write unique malware so there is no technical reason Avast and Avira. The other free products scored Complete remediation
complete remediation.
why any vendor being tested should do poorly. more humbly in this test.
Target The test system that is protected by a security product.
Consequently, it’s not a shock to see all products Products were awarded the full range of ratings, A program or sequence of interactions with the target that is designed to take some
Threat
handle the email threats very effectively. By and from AAA at the top down through AA, A, B and C. level of unauthorised control of that target.
large the malicious websites were also ineffective, 360 Total Security scored the lowest and failed
Security vendors provide information to their products in an effort to keep abreast
although there were a few that evaded detection. to achieve a rating. It tended to neutralise, rather Update of the latest threats. These updates may be downloaded in bulk as one or more files,
360 Total Security was particularly weak in than block threats, and missed nearly all of the or requested individually and live over the internet.
handling these in comparison to the competition. targeted attacks.
Targeted attacks were handled well by the leaders
in this test, but most products missed a lot and those The leading products from Symantec and Kaspersky
from Cisco, Microsoft and Qihoo 360 (360 Total Lab win AAA awards. AAA awards for their strong
Security) were significantly weaker than the others. overall performance.

Norton Security blocked all of the public and

targeted attacks. It blocked two legitimate
applications, though, so it lost a few points – but not
enough to move it from the number one spot.

16 OCT - DEC 2017 • Home Anti-Malware Protection Home Anti-Malware Protection • OCT - DEC 2017 17
 SE Labs SE Labs

APPENDIX B: FAQs APPENDIX C: Product Versions

A full methodology for this test is available from
our website. Q I am a security vendor. How can I include my
product in your test?
A product’s update mechanism may upgrade the software to a new version automatically
so the version used at the start of the test may be different to that used at the end.

• T he products chosen for this test were selected

by SE Labs.
A Please contact us at [email protected]. We will
be happy to arrange a phone call to discuss
our methodology and the suitability of your
PRODUCT VERSIONS
Provider Product Name Build Version
• The test was not sponsored. This means that no product for inclusion. Avast Avast Free Antivirus 17.8.2318
security vendor has control over the report’s AVG AVG Antivirus Free Edition 17.8.3036

•
content or its publication.
The test was conducted between 26th September Q I am a security vendor. Does it cost money to
have my product tested?
Avira
Bitdefender Internet Security
Avira Free Security Suite
Bitdefender Internet Security
15.0.33.24
22.0.15.189

•
and 6th December 2017
All products had full internet access and were
confirmed to have access to any required or
A We do not charge directly for testing products in
public tests. We do charge for private tests. CISCO
ESET
Cisco Immunet
ESET Smart Security
6.0.6.106.00
10.1.219.0
recommended back-end systems. This was
confirmed, where possible, using the Anti-Malware
Testing Standards Organization (AMTSO) Cloud
Q What is a partner organisation? Can I become
one to gain access to the threat data used in
your tests?
F-Secure
Kaspersky Lab
F-Secure Safe Internet Security
Kaspersky Internet Security
2.93 build 175
18.0.0405 (c)
Microsoft Microsoft Security Essentials 4.10.209.0

•
Lookup Features Setting Check.
Malicious URLs and legitimate applications and
URLs were independently located and verified by 
A Partner organisations support our tests by paying
for access to test data after each test has completed
but before publication. Partners can dispute results and
Symantec
Trend Micro
Norton Security
Trend Micro Internet Security
22.11.2.7
12.0.1191

SE Labs. use our awards logos for marketing purposes. We do not Qihoo 360 Total Security 9.6.0.1017
• Targeted attacks were selected and verified by share data on one partner with other partners. We do ZoneAlarm ZoneAlarm Free Antivirus 15.1.504.17269
SE Labs. They were created and managed by not currently partner with organisations that do not
Metasploit Framework Edition using default engage in our testing.
settings. The choice of exploits was advised APPENDIX D: AttackTypes
by public information about ongoing attacks.
One notable source was the 2016 Data Breach Q So you don’t share threat data with test
participants before the test starts?
The table below shows how each product protected against the different types of attacks used in the test.

•
Investigations Report from Verizon.
Malicious and legitimate data was provided
to partner organisations once the full test
A No, this would bias the test and make the results
unfair and unrealistic.
ATTACK TYPES
Product Web
Download
Targeted
Attack
Email
Attack
Protected
(Total)

•
was complete.
SE Labs conducted this endpoint security testing
on physical PCs, not virtual machines.
Q I am a security vendor and you tested my product
without permission. May I access the threat data
to verify that your results are accurate?
Norton Security
Trend Micro Internet Security
50
50
25
23
25
23
100
96
F-Secure Safe Internet Security 50 24 22 96

A We are willing to share small subsets of data with

non-partner participants at our discretion. A small
administration fee is applicable.
Kaspersky Internet Security
ESET Smart Security
50
50
21
20
25
23
96
93
Avira Free Security Suite 50 18 24 92
AVG Antivirus Free Edition 50 17 23 90
Avast Free Antivirus 49 17 24 90
Bitdefender Internet Security 50 16 24 90
ZoneAlarm Free Antivirus 49 14 23 86
Cisco Immunet 49 5 25 79
Microsoft Security Essentials 47 3 25 75
Qihoo 360 Total Security 45 3 23 71

18 OCT - DEC 2017 • Home Anti-Malware Protection Home Anti-Malware Protection • OCT - DEC 2017 19