AntiVirus Testing

An Approach

1 INTRODUCTION.......................................................................................................................................1
2 INTRODUCTION TO ANTIVIRUS.........................................................................................................3
2.1 WHAT IS VIRUS?.......................................................................................................................................3
2.2 WHAT IS ANTIVIRUS?................................................................................................................................3
3 HOW ANTIVIRUS WORKS.....................................................................................................................3
4 TESTING ANTIVIRUS..............................................................................................................................4
4.1. FUNCTIONAL TESTS....................................................................................................................................4
SYSTEM TESTS.................................................................................................................................................5
INTEROPERABILITY TESTS...................................................................................................................................5
RELEASE CHECKS.............................................................................................................................................6
5 SPECIFICATION DEVELOPMENT APPROACH................................................................................6
5.1. STUDY OF SPECIFICATION DOCUMENTS..........................................................................................................6
5.2. PREPARATION OF COVERAGE MATRIX..........................................................................................................6
5.3. TEST SPECIFICATION DEVELOPMENT............................................................................................................7
5.4. TEST DATA COLLECTION...........................................................................................................................7
6 TEST ENVIRONMENT.............................................................................................................................7
6.1. TEST SIMULATORS.....................................................................................................................................8
6.2. ON DEVICE TESTING...................................................................................................................................8
TESTING VARIOUS RELEASES OF THE SOFTWARE......................................................................9
7.1. ALPHA TESTS............................................................................................................................................9
7.2. BETA TESTS..............................................................................................................................................9
7.3. CANDIDATE RELEASE TESTS.........................................................................................................................9
7.4. RELEASE TESTS.........................................................................................................................................9
TEST CASES ESTIMATE.........................................................................................................................10
TIME ESTIMATE.......................................................................................................................................11
9.1. TEST SPECIFICATION DEVELOPMENT .........................................................................................................11
9.2. TEST LAB SETUP ...................................................................................................................................12
9.3. TESTING................................................................................................................................................12

1 Introduction

This document covers the various anti-virus technologies to be followed to detect the
viruses present in the mobiles to protect the device from any data loss and damage.It
also coves in detail about the the various types of testing that can be performed on
the anti-virus software to ensure that the software is bug-free and to check the
stability of the device while using the it.

Guide to Document
This document is structured as follows:

• Introduction to AntiVirus
• How AntiVirus works
• Testing Antivirus
• Test Specification Development Approach
• Test Environment
• Testing across various releases
• Various time estimates
• References

Acronyms

Acronyms Definition
GSM Global System for Mobile Communications
GPRS General Packet Radio Service
IOT Interoperability tests
PSTN Public Switched Telephone Network
RAS Remote Access Server
PPG Push Proxy Gateway
2 Introduction to ANTIVIRUS

2.1 What is Virus?

Virus is a program or programming code that replicates by being copied or

initiating its copying to another program, computer boot sector or document.

The following are the three main types of virus

• Boot Sector Virus: This is the first sector in the hard disk or internal
drive. The boot sector is referred to every time the device is powered
on, and hence it is a vulnerable place for Virus attacks.

• Macro Virus: This is a most common virus. This virus spreads through
emails, Internet downloads etc.

• File Infecting Virus: This virus infects the executable files loading into
the memory when executed. This is the most interesting virus found
on mobile phones, in addition to worms and Trojan horses.

Virus in Mobiles

Virus in mobiles have been a new phenomenon but are increasingly becoming
a reality due to the large number of features and complexity of the software.
Virus in mobiles work pretty much the same way as in desktops from the user
perspective though the way in which they are transmitted differs greatly.
In desktops, the major source of virus transmission is through mails and web
sites. (network access). For mobiles, in addition to this, the most important
modes of virus transmission will be other interfaces like Bluetooth and
Infrared.
The ultimate result of a virus will still be the same i.e. to corrupt user data
and system data and in a lot of cases render the system unusable. This is a
worrying factor in mobiles considering that the core applications in most
mobiles are written by different vendors increasing the possibility of
weaknesses to be exploited.

2.2 What is AntiVirus?

Anti-Virus is software that is designed to detect the viruses present in the device
and remove them without causing any damage to the device.

3 How AntiVirus Works

The main component of the antivirus software is the Scanning Engine.

 This engine should be capable of performing the various scanning methods in
order to check for the virus present in the device. Some of the most common
scanning methods are mentioned below.

1. Identifying the various virus-laden files using the virus signatures defined by
the scan engine
2. Since finding all the virus files using virus signatures is difficult, especially for
the new viruses, it should be possible to flag the suspicious data structures or
strange behavior which might result in a virus. This helps in detecting the files
that might contain virus in them.

Once the virus is detected, proper action should be taken by the anti-virus
software to ensure that the device is not affected and that there is no data loss.

4 Testing AntiVirus
4.1. Functional tests

These tests should be done to ensure that the anti-virus software functions properly
in the device and all it's features work fine.

The major functionalities to be tested are:

a) Scanning Engine

The scanning Engine has to be tested to ensure that the device is

scanned properly and that the various types of virus are detected and
proper action is taken.

This can be performed by importing some virus files into the device
and then running the anti-virus software.
Once the scanning is done, the details of the scan like the number of
viruses detected, type of the viruses and the action performed details
can be shown to the user.

All the types of actions that can be performed on the detected virus
should be tested. This can be done using various types of virus data
files like:

• Files containing Virus that can be disinfected

automatically

• Files containing virus that might require special

disinfection (In this case, the vendor should provide the
tools to be used to remove the virus).

• Files containing virus that cannot be removed by the

anti-virus software
 b) AntiVirus Software Updation

It should be possible to perform updation to the anti-virus software

present in the device and ensure that the latest virus signature files
are also added to it. This helps in checking the files present in the
device for the latest found viruses also.

c) Logs

Logs should be generated everytime a scan is performed so that the

user can keep a record of the various scans performed till that time.

System Tests

These tests should be performed to ensure that the device is stable while the
virus scan is running in the background and when following interrupts (for
example) occur in between.

• Incoming/Outgoing Call
• Incoming/Outgoing Messages like SMS/EMS/MMS
• Beaming Events – IR and BT
• Alarm events from various applications like Clock, Calendar,
Tasks etc.
• Push Events – SI and SL
• Cell Broadcast Messages
• Synchronization
• Automatic Schedule updates
• Active Browser session and downloading of various files
including virus files

Other type of “active” tests should also be done during System tests. An
example would be sending a virus to the device via. MMS or E-mail as an
attachment or via IR/BT beaming and checking if the running anti-virus
detects it.

Interoperability tests

These tests ensure that the anti-virus software under test is compatible with
the various other applications present in the device.

Some of the applications with which the anti-virus software should be

compatible are:

1. Secure applications provided by various vendors like Symantec, F-Secure,

PointSec etc

For example, the PointSec application can be used to encrypt and lock the
various files present in the device. The behavior of the device on trying to
perform a scan while the PointSec application is active can tested.
 2. Firewall applications: The anti-virus application should be compatible with
the Firewall software present in the device.

Virus detection can be done using the Firewalls also using the Detection
Intrusion feature where the firewalls scan for patterns of network traffic to
detect Virus. The interoperability of the anti-virus application on trying to run
it while the firewall is already active can also be tested.

Release Checks

These include the basic checks to be performed on the Anti-Virus software

before it is going to be released into the market. This includes testing the
basic functionality with a very few and most common interrupts on it ensure
that the software is good and stable

5 Specification Development Approach

The test specification development phase involves the following activities.

• Study of the specification documents

• Preparation of Coverage Matrix
• Test Specification Development
• Test Data collection

5.1. Study of Specification documents

During this phase, the individuals involved in the test specification

development activity are supposed to go through the various documents like
requirements document, specification document, and Use Case documents to
equip themselves with the necessary knowledge on various features in
antivirus to develop the necessary documents like coverage matrix, test
specification.

5.2. Preparation of Coverage Matrix

The aim of this document is to have a complete coverage of the features

mentioned in the specification document and the Use Case document if any.

The following types of test cases would be identified.

1. Positive test cases- These test cases test directly the positive functionality
mentioned in the specification document or in the UC document.

For e.g. – Testing the various types of possible scan methods

2. Negative test cases – Test Cases with invalid scenarios

 For e.g. like trying to scan for a file which is not present in the device or
which is not valid or trying to update the antivirus software using corrupted
virus signature files etc.

5.3. Test Specification Development

The test specification is a consolidated report that lists all the test cases for
testing the antivirus software.

It includes test cases to test the following features

• The various functionalities of the antivirus software

• Stability of the device while various interrupts occur while the antivirus
software is active (i.e. scan is running in the background)
• Interoperability test cases
• Negative scenarios

Every test case will be in the following structure.

Test Case ID This cell contains the test case ID in a specified format to
ensure that the test Case ID's are unique
Prerequisite This cell contains the prerequisites for testing.
Objective Objective of the test case to verify
Description Short description of what the test case actually does
Expected Output The expected output
Reference Reference for this scenario from the specification
documents or Use Case document
Comments Comments regarding this test case
Test Data The Virus test data files required to execute this test
scenario

5.4. Test Data Collection

This phase involves the collection of various virus data files required to test
the antivirus software. The types of data files that are required to test the
various actions of the antivirus software are mentioned in Section 4.1.

These test data files may be supplied by the vendor during the testing. If not
then the test data has to be collected from the Internet.

6 Test Environment

Two types of test environment can be used to test the antivirus software
based on the development life cycle and the phases of the testing
6.1. Test Simulators

During the initial stages, the testing of the antivirus software can be done on
PC based test simulator. All the test cases that deal with checking the
functionality of the antivirus software can be covered using the test simulator.

The tests here will be performed by importing the necessary virus data files
into the test simulator and then running the antivirus software in it.

6.2. On device testing

The test cases related to system testing and those test scenarios that need
network support have to be tested on the device.

The test environment required for this testing is as follows:

Web Servers and Origin Servers are useful to test browser related interrupts
during the testing and also to download the virus data files into the device
while the scan is active.

The virus data files can be imported into the device either by downloading
them from the Web through various mechanisms like E-mail, MMS or by
beaming. Another alternative is to copy the virus data files to the Phone
memory or to the external memory using PC software.

With the help of anti-virus server it is possible to customize the testing of

antivirus update by having various types of virus signature files on the Anti-
Virus server
 To test the automatic updation of anti-virus client software from the server,
the PPG is required in between. The antivirus client software will be
automatically updated with the latest virus signature files present in the
server using the Push technology.

Testing various releases of the software

7.1. Alpha tests

This testing comprises of the initial testing cycles that are performed once the
software is ready.

For the software releases that happen initially, only the functionality of the
software has to be checked. Most of this testing can be done using the PC
based simulators (Refer Section 6.1) since network support will not be
required to test the basic functionalities of the antivirus software.

If the software is found to be stable and most or all of it's functionalities are
met, the software goes for Beta Release testing.

7.2. Beta tests

The software is taken for Beta testing only if it passes the alpha criteria.
This testing concentrates mainly on testing the stability of the device or the
system while using the antivirus software. This release is oriented more
towards the System testing.
These test have to be performed mostly on the device in order to execute the
network related interrupts like incoming call/messages, beaming events etc.
(Refer Section 6.2).
This includes checking the interoperability of the software by executing
various IOT related test scenarios.

7.3. Candidate release tests

Candidate release software is like a preview of the final software.

This testing is performed to ensure that the software is bug free and to find
any bugs present in the software before releasing it into the market.
The entire functionality of the software will be tested to ensure that the
software that is going for the final release is bug-free.

7.4. Release tests

These are the tests to be performed on the software that is ready to be

released into the market. (Refer Section 4.4 for more details).
 Note: Regression testing will be performed during all the above mentioned
test cycles to ensure that the software works fine even after fixing the found
problems and on performing various changes to the existing software.

Test Cases Estimate

The possible number of test cases for each of the types of testing is as
mentioned below:

S.No Test Cases Possible Scenarios Number of Test

cases
Software Installation
Testing the various
Virus scan techniques
Perform scan with
various options set
Updation of antivirus
software using various
mechanisms like
manual/automatic
1 Functional test cases 8
updates
Logs
Checking all the types
of actions that can be
performed on the
virus found after scan
is done
Software Upgrade
Software Uninstall
2 System test cases Performing scan with 15
various interrupts like:

1.Incoming/Outgoing
Call
2.Incoming/Outgoing
Messages like
SMS/EMS/MMS
3.Beaming Events –
IR and BT
4. Alarm events from
various applications
like Clock, Calendar,
Tasks etc.
5.Push Events – SI
and SL
6.Cell Broadcast
Messages
7.Synchronization
8.Automatic Schedule
updates
9.Active Browser
 session and
downloading of
various files including
virus files
Performing scan when
the Phone memory is
full and during low
battery conditions
Negative test
scenarios like:
1. Scanning corrupt
files
2. Try to scan for files
not present in the
device
3.Updation of Virus
Scan software using
corrupted virus
signature files
Check the
compatibility of the 2
antivirus software
with other
softwares/applications
3 IOT test cases present in the mobile
Perform automatic
virus scan update and
automatic email
update at the same
time

Note: This estimate has been prepared considering only the basic
functionalities of the antivirus software. The number of these test cases is
subject to change based on the features that are going to available in the
software that is to be tested.

Time Estimate

9.1. Test Specification Development

Phase Time (in man hours)

Study 24
Coverage Matrix Development 24
Test Specification development 40
Test Data collection 32

Total time estimated for specification development = 120 man hours

 Note: This estimate has been prepared considering only the basic
functionalities of the antivirus software. The number of these test cases is
subject to change based on the features that are going to available in the
software that is to be tested. This might affect the time estimated for each of
the above mentioned phases.

9.2. Test Lab Setup

Time required to set up the servers and have the data files like various virus
signature files in place in the anti-virus server = 40 man hours

Note: The feasibility of the lab set up for the antivirus server has to be
discussed yet. Based on that this time estimate is subject to change.

9.3. Testing

Testing Cycle Type of testing to be Time (in man hours)

performed
Alpha Testing Functional 24
Beta Testing System testing along with 40
checking all the
functionalities, and
Interoperability testing
Candidate Release Testing Complete testing including 40
Functional tests
System tests
Interoperability tests
Final Release Release testing with some 10
basic checks on the
software

Total time estimated for testing across various cycles = 114 man hours

Note: This estimate is made considering on the basic and common features of
the antivirus software and is subject to change.