SOFTWARE TESTING

Module Number: 03

Module Name: Levels of Testing

1
Levels of Testing

AIM:
• To familiarize students with different levels of the Software Testing.

2
Levels of Testing

Objectives:
The Objectives of this module are to understand:
• The key concepts of need for levels of testing.
• The unit test.
• The test harness.
• Different concepts of integration test, acceptance testing, performance testing, regression testing.
• The usability and accessibility testing.
• The configuration testing.

3
Levels of Testing

Outcome:
At the end of this module, you are expected to explain / describe:
• Different types of software testing.
• The need of testing.
• Configure testing.
• Use of usability and accessibility testing.

4
Levels of Testing

Content
• The need for Levels of Testing
• Basic definitions of Unit testing, Integration test, alpha, beta test
• Alpha and Beta test
• Testing the documentation
• Defect bash elimination System Testing
• Website testing

5
Levels of Testing

The need for Levels of Testing

• A level of software testing is a process where every unit or component of a software/system is tested.
• The main goal of system testing is to evaluate the system's compliance with the specified needs.
• These testing levels are designed to recognize missing areas and reconciliation .

6
Levels of Testing

The need for Levels of Testing

7
Levels of Testing

The need for Levels of Testing

• These testing levels are designed to recognize missing areas and reconciliation.
• There are many different testing levels which helps to check behaviour and performance for software
testing.

8
Levels of Testing

Unit Test

9
Levels of Testing

Unit Test

What is Unit Testing?

• UNIT TESTING is a level of software testing, where individual units or components of a
software are tested.
• Unit Testing of software applications is done during the development (coding) of an
application.

10
Levels of Testing

Unit Test
• The purpose is to validate that each unit of the software performs as designed.

• The goal of Unit Testing is to isolate each part of the program and show that the individual parts
are correct.

11
Levels of Testing

Unit Test

Why is it important?
• Proper unit testing done during the development stage saves both time and money in the end.
• Unit test helps to find software bugs as early as possible.

12
Levels of Testing

Unit Test

When and Who will perform the Unit test?

• Unit Testing is the first level of software testing and is performed prior to Integration Testing.

• It is normally performed by software developers or by the software testers.

13
Levels of Testing

Unit Test

Unit test benefits

• Unit testing increases confidence in changing or maintaining the code.
• Codes are more reusable.
• Development is fast.

14
Levels of Testing

Unit Test
• The cost of fixing a defect detected during unit testing is lesser in comparison to that of defects
detected at higher levels.

• Debugging is easy.

• Codes are more reliable.

15
Levels of Testing

Unit Test Planning

• A general unit test plan should be prepared. It may be prepared as a component of the master
test plan or as a stand-alone plan. It should be developed in conjunction with the master test
plan and the project plan for each project.

16
Levels of Testing

Unit Test Planning

• Phase 1: Describe Unit Test Approach and Risks.

In this phase of unit testing planning the general approach to unit testing is outlined.

The test planner:

(i) Identifies test risks.

(ii) Describes techniques to be used for designing the test cases for the units.

17
Levels of Testing

Self Assessment Question

1. Is Unit test is second Level of Software Testing.

a. True

b. False

Answer: B

18
Levels of Testing

Self Assessment Question

2. Unit Test is done by the end User

a. True

b. False

Answer: B

19
Levels of Testing

Self Assessment Question

3. Unit Test helps to find Software Bugs as early as possible.

a. True

b. False

Answer: A

20
Levels of Testing

Phase 1: Describe Unit Test Approach and Risks

iii. Describes techniques to be used for data validation and recording of test results.

iv. Describes the requirements for test harnesses and other software that interfaces with the units to
be tested. For example, any special objects needed for testing object-oriented units.

21
Levels of Testing

Unit Test Planning

Phase 2: Identify Unit Features to be Tested
• This phase requires information from the unit specification and detailed design description.
• The planner determines which features of each unit will be tested. For example: functions,
performance requirements, states, and state transitions, control structures, messages, and data flow
patterns. If some features will not be covered by the tests, they should be mentioned and the risks of
not testing them be assessed.
• Input/output characteristics associated with each unit should also be identified, such as variables with
an allowed ranges of values and performance at a certain level.

22
Levels of Testing

Unit Test Planning

Phase 3: Add Levels of Detail to the Plan
In this phase, the planner refines the plan as produced in the previous two phases. The planner adds
new details to the approach, resource, and scheduling portions of the unit test plan. As an example,
existing test cases that can be reused for this project can be identified in this phase.

23
Levels of Testing

Designing the Unit Tests

• Part of the preparation work for unit test involves unit test design.
It is important to specify the following:
i. The test cases (including input data, and expected outputs for each test case).
ii. The test procedures (steps required run the tests).

24
Levels of Testing

Designing the Unit Tests

• For designing the unit test suitable, tabular formats for test cases are found.
• In the test design specification includes lists of relevant states, messages, exceptions, and
interrupts.

25
Levels of Testing

Designing the Unit Tests

• As part of the unit test design process, developers/testers should also describe the relationships between
the tests.

• Test case design at the unit level can be based on use of the black and white box test design strategies.

26
Levels of Testing

Designing the Unit Tests

• Considering the relatively small size of a unit, it makes sense to focus on white box test design for
procedures/functions and the methods in a class.

• Some black box-based testing is also done at unit level; however, the bulk of black box testing is usually
done at the integration and system levels and beyond.

27
Targets for Unit Test Cases
• Module interface
• Ensure that information flows properly into and out of the
module
• Local data structures
• Ensure that data stored temporarily maintains its integrity during
all steps in an algorithm execution
• Boundary conditions
• Ensure that the module operates properly at boundary values
established to limit or restrict processing
• Independent paths (basis paths)
• Paths are exercised to ensure that all statements in a module
have been executed at least once
• Error handling paths
• Ensure that the algorithms respond correctly to specific error
conditions

Jain (Deemed-to-be University), Department of BCA 28

Levels of Testing

The Test Harness

29
Levels of Testing

The Test Harness

What is Test Harness?

• Test Harness, also known as automated test framework is mostly used by developers.
• A test harness provides stubs and drivers, which will be used to replicate the missing items, which
are small programs that interacts with the software under test.

30
Levels of Testing

The Test Harness

Test Harness Features
• To execute a set of tests within the framework or using the test harness.
• To key in inputs to the application under test.
• Provide a flexibility and support for debugging.
• To capture outputs generated by the software under test.
• To record the test results (pass/fail) for each one of the tests.
• Helps the developers to measure code coverage at code level.

31
Levels of Testing

The Test Harness

Why use Test Harness?

• Automate the testing process.
• Execute test suites of test cases.
• Generate associated test reports.
• Support for debugging.
• To record the test results for each one of the tests.
• Helps the developers to measure code coverage at code level.
• Increase the productivity of the system through automation.
• Enhance the quality of software components and application.
• To handle the complex condition that testers are finding difficult to simulate.
32
Levels of Testing

The Test Harness

Test Harness Benefits
• Increased productivity as automation is in place.
• Improved quality of software as automation helps us to be efficient.
• Provides Tests that can be scheduled.
• It can handle complex conditions that testers are finding it difficult to simulate.

33
Common Computational Errors in Execution Paths

 Misunderstood or incorrect arithmetic precedence

 Mixed mode operations (e.g., int, float, char)
 Incorrect initialization of values
 Precision inaccuracy and round-off errors
 Incorrect symbolic representation of an expression (int vs. float)
 Comparison of different data types
 Incorrect logical operators or precedence
 Incorrect comparison of variables
 Improper or nonexistent loop termination
 Improperly modified loop variables
 Boundary value violations

Jain (Deemed-to-be University), Department of BCA 34

Unit Test Procedure: Drivers and Stubs for Unit Testing
• Because a component/ module is not stand-alone
program, driver and or stub software must be
developed for each unit test.
• Driver
• A simple main program that accepts test case
data, passes such data to the component
being tested, and prints the returned results
• Stubs
• Serve to replace modules that are subordinate
to (called by) the component to be tested
• It uses the module’s exact interface, may do
minimal data manipulation, provides
verification of entry, and returns control to the
module undergoing testing
• Drivers and stubs both represent
overhead
• Both must be written but don’t constitute part of the
installed software product

Jain (Deemed-to-be University), Department of BCA 35

Levels of Testing

Running the Unit tests and Recording results

Unit tests can begin when:
i. The units becomes available from the developers (an estimation of availability is part of the test
plan).
ii. The test cases have been designed and reviewed.
iii. The test harness, and any other supplemental supporting tools are available. The testers then
proceed to run the tests and record results.

36
Levels of Testing

Running the Unit tests and Recording results

• Documents called test logs that can be used to record the results of specific tests. The
status of the test efforts for a unit, and a summary of the test results, could be recorded
in a simple format given (refer slide 37).

37
Levels of Testing

Running the Unit tests and Recording results

38
Levels of Testing
Running the Unit tests and Recording results
• These forms can be included in the test summary report, and are of value at the weekly status
meetings that are often used to monitor test progress.

• It is very important for the tester at any level of testing to carefully record, review, and check test
results. The tester must determine from the results whether the unit has passed or failed the test.

• If the test is failed, the nature of the problem should be recorded in what is sometimes called a test
incident report.

• This gives clues to the developers to help them locate any faults. During testing the tester may
determine that additional tests are required.

• Ideally, when a unit has been completely tested and finally passes all of the required tests it is ready
for integration. Under some circumstances unit may be given a conditional acceptance for integration
39
test.
Levels of Testing

Integration tests

What is Integration Testing?

• INTEGRATION TESTING is a level of software testing, where individual units are combined and
tested as a group.

• In integration Testing, individual software modules are integrated logically and tested as a group.

• A typical software project consists of multiple software modules, coded by different programmers.

40
Levels of Testing

Integration tests
• Integration Testing focuses on checking data communication amongst these modules.

• Hence it is also termed as 'I & T' (Integration and Testing), 'String Testing' and sometimes
'Thread’ .

41
Levels of Testing

Integration tests

When is Integration Testing performed?

• Integration Testing is the second level of testing performed after Unit Testing and before System
Testing.

• System Integration Testing (SIT) is performed to verify the interactions between the modules of a
software system.

42
Levels of Testing

Integration tests

Who performs Integration Testing?

Developers themselves or independent testers perform Integration Testing.

43
Levels of Testing

Integration tests

Why to do System Integration Testing?

System Integration Testing is done because of the following reasons:
• It helps to detect defect early.
• Earlier feedback on acceptability of the individual module will be available.
• Scheduling of defect fixes is flexible, and it can be overlapped with development.
• Correct data flow.
• Correct control flow.
• Correct timing.
• Correct memory usage.
• Correct with software requirements.
44
Levels of Testing

Designing Integration Tests

• Integration tests for procedural software can be designed using a black or white box approach.
• Both are recommended. Some unit tests can be reused. Since, many errors occur at module interfaces,
test designers need to focus on exercising all input/output parameter pairs, and all calling relationships.
• The tester needs to ensure the parameters are of the correct type and in the correct order.
• The author has had the personal experience of spending many hours trying to locate a fault that was
due to an incorrect ordering of parameters in the calling routine.
• The tester must also ensure that once the parameters are passed to a routine they are used correctly.

45
Levels of Testing

Designing Integration Tests

• Testers must ensure that test cases are designed, so that all modules in the structure chart are called at
least once, and all called modules are called by every caller.
• The reader can visualize these as coverage criteria for integration test.

46
Levels of Testing

Designing Integration Tests

• Coverage requirements for the internal logic of each of the integrated units should be achieved during
unit tests.
• Some black box tests used for module integration may be reusable from unit testing.
• However, when units are integrated and subsystems are to be tested as a whole, new tests will have to
be designed to cover their functionality and adherence to performance and other requirements.

47
Levels of Testing

Designing Integration Tests

• Sources for development of black box or functional tests at the integration level are the requirements
documents and the user manual.

• Testers need to work with requirements analysts to ensure that the requirements are testable, accurate, and
complete.

• Black box tests should be developed to ensure proper functionally and ability to handle subsystem stress.

• For example, in a transaction-based subsystem the testers want to determine the limits in number of
transactions that can be handled.

• The tester also wants to observe subsystem actions when excessive amounts of transactions are generated.
Performance issues such as the time requirements for a transaction should also be subjected to test.

• These will be repeated when the software is assembled as a whole and is undergoing system test.
48
Types of Integration Testing

• Unit tested modules combine one at a time to build a

larger structure of the software system based on the
prescribed design. Module integration is done
incrementally.
• While modules are integrated, tests are conducted
to uncover errors associated with the integration.
 Three kinds
 Top-down integration
 Bottom-up integration
 Sandwich integration

Jain (Deemed-to-be University), Department of BCA 49

Top-down Integration
• Modules are integrated by moving
downward through the control hierarchy,
beginning with the main module
• Subordinate modules are incorporated in
either a depth-first or breadth-first fashion
• DF: All modules on a major control path are integrated
• BF: All modules directly subordinate at each level are
integrated
• Advantages
• This approach verifies major control or decision points early
in the test process
• Disadvantages
• Stubs need to be created to substitute for modules that
have not been built or tested yet; this code is later
discarded
• Because stubs are used to replace lower level modules, no
significant data flow can occur until much later in the
integration/testing process. This may require more testing
when lower level modules are integrated.

Jain (Deemed-to-be University), Department of BCA 50

Bottom-up Integration
• Integration and testing starts with
the most atomic modules in the
control hierarchy
• Advantages
• This approach verifies low-level data processing
early in the testing process
• Need for stubs is eliminated
• Disadvantages
• Driver modules need to be built to test the lower-
level modules; this code is later discarded or
expanded into a full-featured version
• Drivers inherently do not contain the complete
algorithms that will eventually use the services of
the lower-level modules; consequently, testing
may be incomplete or more testing may be
needed later when the upper level modules are
available

Jain (Deemed-to-be University), Department of BCA 51

Sandwich Integration
• Consists of a combination of both top-down and bottom-up
integration
• Occurs both at the highest level modules and also at the
lowest level modules
• Proceeds using functional groups of modules, with each
group completed before the next
• High and low-level modules are grouped based on the
control and data processing they provide for a specific
program feature
• Integration within the group progresses in alternating
steps between the high and low level modules of the
group
• When integration for a certain functional group is
complete, integration and testing moves onto the next
group
• Advantage: Reaps the advantages of both types of
integration while minimizing the need for drivers and stubs
• Disadvantage: Requires a disciplined approach so that
integration doesn’t tend towards the “big bang” scenario

Jain (Deemed-to-be University), Department of BCA 52

Levels of Testing

Planning Integration Tests

• Integration test must be planned.

• Planning can begin when high-level design is complete, so that the system architecture is defined.

• Other documents relevant to integration test planning are the requirements document, the user manual,
and usage scenarios.

• These documents contain structure charts, state charts, data dictionaries, cross-reference tables, module
interface descriptions, data flow descriptions, messages and event descriptions, all necessary to plan
integration tests. The strategy for integration should be defined.

53
Levels of Testing

Integration Test Planning

• For procedural-oriented system, the order of integration of the units should be defined. This depends
on the strategy selected.
• Consider the fact that the testing objectives are to assemble components into subsystems and to
demonstrate that the subsystem functions properly with the integration test cases.
• For object-oriented systems, a working definition of a cluster or similar construct must be described,
and relevant test cases must be specified.

54
Levels of Testing

Integration Test Planning

In addition, testing resources and schedules for integration should be included in the test plan.
The plan includes the details of clusters items:
Clusters: This cluster is dependent on the following:
(ii) A natural language description of the functionality of the cluster to be tested.
(iii) List of classes in the cluster.
(iv) A set of cluster test cases.

55
Levels of Testing

Integration Test Planning

• One of the goals of integration test is to build working subsystems, and then combine these into the
system as a whole.
• When planning for integration test, the planner selects subsystems to build based upon the requirements
and user needs.
• Very often subsystems selected for integration are prioritized.
• Those that represent key features, critical features, and/or user-oriented functions may be given the
highest priority.
• Developers may want to show clients that certain key subsystems have been assembled and are
minimally functional.

56
System Testing
• System testing is a type of software testing that evaluates the
overall functionality and performance of a complete and fully
integrated software solution.
• It tests if the system meets the specified requirements and if it is
suitable for delivery to the end-users.

57
Objective of system testing
• The goal of integration testing is to detect any irregularity
between the units that are integrated together.
• System testing detects defects within both the integrated units
and the whole system.
• The result of system testing is the observed behavior of a
component or a system when it is tested.

58
Who carries out system testing and when
• System Testing is basically performed by a testing team that is
independent of the development team that helps to test the
quality of the system impartial.
• It has both functional and non-functional testing.
• System Testing is carried out using black-box testing
approach.
• System Testing is performed after the integration testing and
before the acceptance testing.

59
System Testing Process:
• Test Environment Setup: Create testing environment for the better
quality testing.
• Create Test Case: Generate test case for the testing process.
• Create Test Data: Generate the data that is to be tested.
• Execute Test Case: After the generation of the test case and the test
data, test cases are executed.
• Defect Reporting: Defects in the system are detected.
• Regression Testing: It is carried out to test the side effects of the
testing process.
• Log Defects: Defects are fixed in this step.
• Retest: If the test is not successful then again test is performed.

60
61
Types of System Testing:
• Performance Testing: Performance Testing is a type of software
testing that is carried out to test the speed, scalability, stability and
reliability of the software product or application.
• Load Testing: Load Testing is a type of software Testing which is
carried out to determine the behavior of a system or software
product under extreme load.
• Stress Testing: Stress Testing is a type of software testing performed
to check the robustness of the system under the varying loads.
• Scalability Testing: Scalability Testing is a type of software testing
which is carried out to check the performance of a software
application or system in terms of its capability to scale up or scale
down the number of user request load.

62
• Migration testing. This is conducted to ensure smooth
migration of legacy systems to new systems without
disruptions, data loss or downtimes.
• Usability testing. These are tests to evaluate if a system is
easy to use and functional for the end user. Metrics, including
user error rates, task success rates, the time it takes a user to
complete a task and user satisfaction, are used during testing.

63
• Functionality testing. This is conducted to validate a system's
functionality against its functional and business requirements.
• Recovery testing. This is a type of nonfunctional testing done
to ensure that a system is capable of recovering from certain
system errors, crashes and failures.
• Regression testing. It ensures that all changes introduced into
an application or code during system testing, recent code
changes or updates haven't caused any new bugs or
issues. Regression testing is responsible for the functionality of
the existing features of a system or software.
64
Levels of Testing

Regression Testing

What is Regression Testing?

• Regression Testing is defined as a type of software testing to confirm that a recent program or code
change has not adversely affected existing features.
• Regression Testing is nothing but full or partial selection of already executed test cases which are
re-executed to ensure existing functionalities work fine.
• This testing is done to make sure that new code changes should not have side effects on the existing
functionalities. It ensures that old code still works once the new code changes are done.

65
Levels of Testing

Regression Testing
Need of Regression Testing
Regression Testing is required when there are the following reasons:
• Change in requirements and code is modified according to the requirement.
• New feature is added to the software.
• Defect fixing.
• Performance issue fix.

66
Levels of Testing

Regression Testing
Challenges in Regression Testing:
• With successive regression runs, test suites become fairly large. Due to time and budget
constraints, the entire regression test suite cannot be executed.
• Minimizing test suite while achieving maximum Test coverage remains a challenge.
• Determination of frequency of Regression Tests, i.e., after every modification or every build
update or after a bunch of bug fixes, is a challenge.

67
Levels of Testing

Acceptance testing

68
Levels of Testing

Acceptance testing

What is Acceptance Testing?

A testing technique performed to determine whether or not the software system has met the
requirement specifications.
The main purpose of this test is to evaluate the system's compliance with the business requirements
and verify if it is has met the required criteria for delivery to end users.

69
Levels of Testing

Acceptance testing

When is it performed?
Acceptance Testing is the fourth and last level of software testing performed after System
Testing and before making the system available for actual use.

70
Levels of Testing

Acceptance testing

Who performs it?

Many a times, internal Acceptance Testing is performed by members of the organization that developed
the software, but who are not directly involved in the project (Development or Testing).
Usually, it is the members of Product Management, Sales and/or Customer Support.

71
Levels of Testing

Acceptance testing
• External Acceptance Testing is performed by people who are not employees of the organization
that developed the software.

• Customer Acceptance Testing is performed by the customers of the organization that developed the
software.

• User Acceptance Testing (Also known as Beta Testing) is performed by the end users of the
software.

72
Levels of Testing

Alpha testing

73
Levels of Testing

Alpha testing

What is Alpha Testing?

• Alpha testing takes place at the developer's site by the internal teams, before release to external
customers. This testing is performed without the involvement of the development teams.

• The focus of this testing is to simulate real users by using black box techniques. The aim is to carry out
the tasks that a typical user might perform. Alpha testing is carried out in a lab environment and usually
the testers are internal employees of the organization.

74
Levels of Testing

Alpha testing
Advantages of Alpha Testing:
• Provides better view about the reliability of the software at an early stage.
• Helps simulate real time user behaviour and environment.
• Detect many showstopper or serious errors.
• Ability to provide early detection of errors with respect to design and functionality.

75
Levels of Testing

Beta testing
• Beta Testing of a product is performed by "real users" of the software application in a "real
environment" and can be considered as a form of external User Acceptance Testing.
• Beta testing also known as user testing takes place at the end users site by the end users to validate the
usability, functionality, compatibility, and reliability testing.

76
Beta Testing
• Beta testing is the term used to describe the external
testing process in which the software is sent out to a
select group of potential customers who use it in a
real-world environment.
• Beta testing usually occurs toward the end of the
product development cycle and ideally should just be
a validation that the software is ready to release to
real customers.
• The goals of a beta test can vary considerably from
getting the press to write early reviews of the
software to user interface validation to a last-ditch
effort in finding bugs.
• As a tester, you need to make it known to the person
managing the beta testing what you want to achieve
from it.
• From a test standpoint, there are several things to
think about when planning for or relying on a beta
test:
Who are the beta testers?
• Since a beta test can have different goals, it's
important to understand who the beta participants
are.
• If your area of the software is to be beta tested, make
sure that you define what types of beta testers you
need in the program so that you can receive the most
benefit out of it.
• How will you know if the beta testers even use the
software?
• If a number of beta testers had the software for some
amount of time and reported no problems, would
that mean there were no bugs, that they saw bugs
but didn't report them, or that the disks were lost in
the mail?
• It's not uncommon for beta testers to let the
software sit for days or weeks before they try to use
it, and when they do, only use it for a limited time
and a limited set of features.
• Make sure that you or someone running the beta
program follows up with the participants to make
sure that they're using the software and meeting the
plan's goals.
• Beta tests can be a good way to find compatibility
and configuration bugs.
• As you know it's difficult to identify and test a
representative sample of all the real-world hardware
and software setups.
• If your beta test participants have been wisely chosen
to represent your target customers, they should do a
good job finding configuration and compatibility
problems for you.
• Usability testing is another area that beta testing can
contribute to if the participants are well chosen a
good mix of experienced and inexperienced users.
• They'll be seeing the software for the first time and
will readily find anything that's confusing or difficult
to use.
• Besides configuration, compatibility, and usability,
beta tests are not all that effective ways to find bugs.
• The participants often don't have a lot of time to use
the software, so they won't find much more than
superficial, obvious problems ones that you likely
already know about.
• And, because beta testing usually occurs near the
end of the development cycle, there's not much time
to fix bugs that are found.
• Trying to rely on beta testing as a substitute for real
testing is one of the major pitfalls of software
product development.
• A beta test program can take up a lot of a tester's time.
• A common job for a new tester is to work with the beta
customers to help solve their problems, answer their
questions, and confirm the bugs they find
• If you're assigned this task, you'll also need to work with
your fellow testers to understand how the bugs slipped
through to the beta testers and how to improve the test
cases so that the bugs are found internally in the future.
• All this can be a full-time job, leaving little room to do any
real testing yourself.
• If you and your team plan on holding a beta test
program, make arrangements in advance, preferably
when the product's schedule is being defined.
• Make sure that the beta test's goals mesh with goals
that you and your testing team want out of it and
work closely with the person (or team) managing the
beta program to keep testing's voice heard.
• Beta testing can prove to be a valuable method for
getting solid, independent test data back on your
software, but to be effective it must be properly
defined and managed.
Levels of Testing

Scenario testing

What is Scenario Testing?

• Scenario testing is a variant of software testing, where scenarios are used for testing.
• Scenarios help in an easier way of testing of the more complicated systems.

88
Levels of Testing

Self Assessment Question

6. Integration Testing is a level of Software testing, where individual units are combined and
tested as a group

a. True

b. False

Answer: A

89
Levels of Testing

Scenario testing

Why create Test Scenarios?

Test Scenarios are created for the following reasons:
• Creating test scenarios ensures complete test coverage.
• Test scenarios can be approved by various stakeholders like business analyst, developers,
customers to ensure the application under test is thoroughly tested. It ensures that the software is
working for the most common use cases.

90
Levels of Testing

Scenario testing

Why create Test Scenarios? (Continued)

• They serve as a quick tool to determine the testing work effort and accordingly create a
proposal for the client or organize the workforce.
• They help determine the most important end-to-end transactions or the real use of the software
applications.
• For studying the end-to-end functioning of the program, Test Scenario is critical.

91
Levels of Testing

Scenario testing
Importance of Scenario Testing
As you know that exhaustive testing of any software application is not possible because of the following
reasons:
• Large number of data combinations.
• Large number of possible paths in software

92
Levels of Testing

Scenario Testing example

• By scenario testing, you figure out the most important end-to-end transactions or the real use of the
software applications which ensures that the software is working for the most common user scenarios.
It helps in finding lot of defects which cannot be found with other types of testing.
• Scenario testing is very important for complex transactions and for studying end-to-end functioning of
the program.

93
Levels of Testing

Scenario Testing example

• Let us take an example of Hospital Management System to explain the importance of scenario
testing.
• Nowadays, whenever you go to some hospital the patients records are maintained using HMS
(Hospital Management System) software. This software system manages both outpatient and
inpatient records.
• Outpatients leave hospital on the same day, however inpatients are admitted in hospital and
treated for more number of days before being discharged from hospital.

94
Levels of Testing

Scenario Testing example

• Now, let us take a scenario, where one outpatient comes to hospital to consult bone doctor for pain
in his leg.
• His entry is made in HMS as outpatient and all his previous medical history is entered in HMS.
• After few minutes, he suddenly feels acute chest pain and needs to be admitted in hospital, after
being admitted to hospital his entry is now changed from outpatient to inpatient.
• But after making this change the doctor finds that he is not able to get any previous history of this
patient, ideally the outpatient history of that patient should be available even after changing the
outpatient status to inpatient.
• This happened because the HMS system fails in this scenario (When OPD patients status is changed
to Internal patient).
95
Levels of Testing

Scenario Testing example (Cont…)

• So, if scenario testing would have been done for this test scenario (Patients history should get
transferred when his status is changed from outpatient to Inpatient), this bug would have been
found well in advance.
• So, now you can see why Scenario testing is so important in software testing.

96
Bug bash
• A fun way to share the testing tasks is to schedule a
bug bash.
• A bug bash is a period of time (usually a couple
hours) in which the entire test team stops their
normally assigned testing tasks to participate in the
bash.
• In a bug bash, a specific area of the software is
selected and all the testers concentrate their testing
on that one area or set of features
• The selection might be an area that's been especially
buggy to see if there are still more problems lurking
about.
• it might be an area that's suspiciously bug free. A bug
bash could determine if the bugs have been missed
by normal testing or if it's just well written code.
• There are lots of potential criteria for choosing the
area, but ultimately a bug bash gets many different
people looking at one particular area of the software
for bugs.
• One of your greatest allies in your quest to find bugs
is your product support or customer service team the
people who will talk with customers when they call
or email with questions and problems.
• These people are obviously very sensitive to bugs and
are a great resource to leverage for helping you test.
• Find out who will be supporting your product once
it's released and ask them to participate in your test
sharing activities. You'll be amazed at the bugs they'll
find for you.
• Probably the most common class of calls that product
support people take is in the area of usability
problems.
• Many of the calls are from people simply trying to
figure out how to use the software.
• For this reason, it's a good idea to get your product
support team helping you test the product early in
the design cycle to help identify and fix usability
bugs.
Levels of Testing

Benefits
• Defect bashing is finding defects in a short time before the software is released. It can also bring
out both functional and non-functional defects.

• The life cycle of the bugs can be minimized as the reports can be verified and assigned during the
defects bash.

101
Levels of Testing

Benefits
• Defect bash acts as a basic sort of usability as well as acceptance testing. People can pound the
system from the load in the defect bash.

• Additionally, defect bash can be used to break the system instead of trying to conclude the system
works.

102
Levels of Testing

Benefits
• Defect bash brings different people from different roles together in the organization for testing.

• The boundaries between roles are minimized in a co located session.

• Different roles also help validating the software from end user perspective.

• The end users using a software product will be quite different from each other in many aspects such as
understanding about the product, the manner of using the software.

• Defect bash can bring in people who have different levels of product understanding to test the product
together randomly, which can simulate the different approaches of the end users.

• It is also recognized that fresh eyes have less bias and that fresh eyes can uncover new defects.

103
Levels of Testing

Benefits
• Learning and competitions are also mentioned as benefits of a defect bash.

• The built-in competitive instinct of participants should be stimulated to achieve this.

• Defect bash also helps in learning the product and learning from each other. It can be used
as unofficial demo.

• The learning and competition aspects are also claimed to help in team building inside a
company.

104
Levels of Testing

Limitations
• Limitation 1: Defect bash might cause too many duplicate defect reports. The quality of defect
reports can be low. Time is wasted in investigating, diagnosing and logging the same problem
several times.

105
Levels of Testing

Limitations
• Limitation 2: Defect bash there isn’t much opportunity to learn from each other.

• This is because many people use the system for the first time, at the same time.

• Also the limited time period disables learning. We think that the defect bash in the first time would
be similar to what observed. However after more experience both organizer and participants will
learn how to do a defect bash more efficiently.

106
Levels of Testing

Limitations
• Limitation 3: Defect bash can only predict customer behaviour for the first few hours . Thus, it
cannot offer information of long-term product use. We think that usage by different users even
once or short period is better than nothing, and we maybe should not expect too many feedbacks on
customer behaviours from defect bash as Lindsay’s observation is reasonable.

107
Levels of Testing

Limitations
• Limitation 4: Defect bash causes the strain of available resources for setting test environment with
a big group of people. However, this limitation can be overcome by careful planning of defect
bash.

108
Levels of Testing

Website testing
• Database Server: Make sure queries sent to the database give expected results.

• Test system response when connection between the three layers (Application, Web and
Database) cannot be established and appropriate message is shown to the end user.

• Web Server: Test Web server is handling all application requests without any service denial.

109
Levels of Testing

Website testing
Database Testing

Database is one critical component of your web application and stress must be laid to test it thoroughly.
Testing activities will include:

• Test if any errors are shown while executing queries.

• Data Integrity is maintained while creating, updating or deleting data in database.

• Check response time of queries and fine tune them if necessary.

• Test data retrieved from your database is shown accurately in your web application.

110
Levels of Testing

Website testing
Compatibility testing

Compatibility tests ensures that your web application displays correctly across different devices. This
would include:

• Browser Compatibility Test: Same website in different browsers will display differently. You need
to test if your web application is being displayed correctly across browsers, JavaScript, AJAX and
authentication is working fine. You may also check for Mobile Browser Compatibility.

111
Levels of Testing

Website testing
• The rendering of web elements like buttons, text fields etc. changes with change in Operating System.
Make sure your website works fine for various combination of Operating systems such as Windows,
Linux, Mac and Browsers such as Firefox, Internet Explorer, Safari etc.

112
Levels of Testing

Website testing

Performance Testing:

• Website application response times at different connection speeds.

• Load test your web application to determine its behaviour under normal and peak loads.

113
Levels of Testing

Website testing
Performance Testing

• Stress test your website to determine its break point when pushed to beyond normal loads at peak
time.

• Test if a crash occurs due to peak load. How does the site recover from such an event?

• Make sure optimization techniques like g zip compression, browser and server side cache enabled
to reduce load times.

114
Levels of Testing

Website testing
Security testing

• Security Testing is vital for e-commerce website that stores sensitive customer information like
credit cards. Testing Activities will include-

• Test unauthorized access to secure pages should not be permitted.

• Restricted files should not be downloadable without appropriate access.

• Check sessions are automatically killed after prolonged user inactivity.

• On use of SSL certificates, website should re-direct to encrypted SSL pages.

115
Levels of Testing

Website testing
Crowd Testing

• You will select a large number of people (crowd) to execute tests which otherwise would have
been executed a select group of people in the company. Crowd sourced testing is an interesting
and upcoming concept and helps unravel many a unnoticed defects.

• This concludes almost all testing types applicable to your web application.

116
Levels of Testing

Crowd Testing
• As a Web-tester, it is important to note that web testing is quite an arduous process and you are bound
to come across many obstacles. One of the major problems you will face is deadline pressure. The
number of times the code needs changing is also taxing. Make sure you plan your work and know
clearly what is expected of you. It is best to define all the tasks involved in your web testing and
then create a work chart for accurate estimates and planning.

117
Levels of Testing

Defect Bash Elimination System Testing and Testing OO Systems

• Several definitions for defect bash were found. Next we present three definitions that appeared as
the most popular, i.e. these definitions were used in several places.

118
Levels of Testing

Self Assessment Question

13. Pareto Analysis strategy states that 80% of the problem arises due to 20% reasons. State whether True
or False.

a. True
b. False

Answer : True

119
Levels of Testing

Assignment
1. Define Pareto Analysis and Fishbone Analysis in detail.
2. List out few Review and Inspection in defect detection strategy.
3. Case study on the Defect Repository and Test Design.
4. Explain Typographical Defects in details.
5. Draw the flow chart on Error detection strategy.

120
Levels of Testing

Performance testing

What is Performance Testing?

• Performance testing, a non-functional testing technique performed to determine the system

parameters in terms of responsiveness and stability under various workload.

• Performance testing measures the quality attributes of the system, such as scalability, reliability
and resource usage.

121
Levels of Testing

Performance testing
Attributes of Performance Testing:

• Speed

• Scalability

• Stability

• Reliability

122
Levels of Testing

Performance testing

Why should you do Performance Testing?

• Performance Testing is done to provide stakeholders with information about their application -
regarding speed, stability and scalability. More importantly, Performance Testing uncovers
what needs to be improved before the product goes to market.

• Without Performance Testing, software is likely to suffer from issues such as: running slow
while several users use it simultaneously, inconsistencies across different operating systems
and poor usability. Performance testing will determine whether or not their software meets
speed, scalability and stability requirements under expected workloads.

123
Levels of Testing

Performance testing
• Applications sent to market with poor performance metrics due to non existent or poor performance
testing are likely to gain a bad reputation and fail to meet expected sales goals. Also, mission critical
applications like space launch programs or life saving medical equipments should be performance
tested to ensure that they run for a long period of time without deviations.

124
Levels of Testing

Internationalization testing

What is an Internationalization Testing?

• Internationalization testing is the process of verifying the application under test to work
uniformly across multiple regions and cultures.
• It is a process of designing and developing a product, application or document content such
that it enables localization for any given culture, region, or language.
• Internationalization is also written in 18n, where 18 is the number of letters between me and n
in the word internationalization..

125
Levels of Testing

Internationalization testing

What is the purpose of Internationalization testing?

• The main purpose of internationalization is to check if the code can handle all international
support without breaking functionality that might cause data loss or data integrity issues.
Globalization testing verifies if there is proper functionality of the product with any of the locale
settings.
• Internalization is a challenge and a thorough understanding of its development concepts is
important to test it efficiently. Also writing and maintaining automation test scripts for
internationalized software is not tedious as the script written to automate testing of the base
version of the application can be used to test any supported language.

126
Levels of Testing

Ad hoc Testing

127
Levels of Testing

Ad hoc Testing
What is Ad hoc Testing?
When a software testing is performed without proper planning and documentation, it is said to be Ad hoc
Testing. Such kind of tests are executed only once unless we uncover the defects.

128
Levels of Testing

Ad hoc Testing
• AD HOC TESTING, also known as Random Testing or Monkey Testing, is a method of software
testing without any planning and documentation. The tests are conducted informally and randomly
without any formal expected results.
• The tester improvises the steps and arbitrarily executes them. Though defects found using this method
are more difficult to reproduce, sometimes very interesting defects are found which would never have
been found, if written test cases existed and were strictly followed. This method is normally used
during Acceptance Testing.

129
Levels of Testing

Ad hoc Testing

When to Execute Ad hoc Testing ?

Ad hoc testing can be performed when there is limited time to do exhaustive testing and usually
performed after the formal test execution. Ad hoc testing will be effective only if the tester has in-depth
understanding about the system under test.

130
Levels of Testing

Ad hoc Testing
Forms of Ad hoc Testing
Buddy Testing: Two buddies, one from development team and one from test team mutually work on
identifying defects in the same module. Buddy testing helps the testers develop better test cases
while development team can also make design changes early. This kind of testing happens usually
after completing the unit testing.

131
Levels of Testing

Ad hoc Testing
• Pair Testing: Two testers are assigned the same modules and they share ideas and work on the same
systems to find defects. One tester executes the tests while another tester records the notes on their
findings.
• Monkey Testing: Testing is performed randomly without any test cases in order to break the system.

132
Levels of Testing

Self Assessment Question

9. AD HOC TESTING, also known as Random Testing or Monkey Testing

a. True

b. False

Answer: A

133
Levels of Testing

Self Assessment Question

10. Alpha testing is done at the developer side.

a. True

b. False

Answer: A

134
Levels of Testing

Self Assessment Question

11. Beta testing is done at the developer side.

a. True

b. False

Answer: B

135
Levels of Testing

Self Assessment Question

12. Beta testing is done at the end user’s side

a. True

b. False

Answer: A

136
Levels of Testing

Beta testing
Advantages Beta Testing
• Reduces product failure risk via customer validation.
• Beta Testing allows a company to test post-launch infrastructure.
• Improves product quality via customer feedback.
• Cost effective compared to similar data gathering methods.
• Creates goodwill with customers and increases customer satisfaction.

137
Levels of Testing

Beta testing

Disadvantages Beta Testing

• Test Management is an issue. As compared to other testing types, which are usually executed
inside a company in a controlled environment, beta testing is executed out in the real world where
you seldom have control.
• Finding the right beta users and maintaining their participation could be a challenge.

138
Levels of Testing

Usability and Accessibility testing

139
Levels of Testing

Usability and Accessibility testing

• User-friendliness and accessibility are critical attributes that contribute to the market value of an
application or website. It is vital that software applications should be tested for the ease with
which they can be used by the end-users (the customers). If average users find using a particular
application or website jarring, they will leave off using it.

140
Levels of Testing

Usability and Accessibility testing

• Usability Testing is a type of black box testing carried out to determine how useful the software or
website is, to an end-user. It is based on testing, the software on the following attributes: Learn
ability, Efficiency, Memo ability, Errors, and Satisfaction.
• A well-designed usability test gauges the actual performance of the website/application on crucial
user-directed tasks.
• Accessibility Testing is a sub-set of usability testing, to check whether people with disabilities can
access the application or website.

141
Levels of Testing

Usability and Accessibility testing

How do we conduct accessibility usability testing?
We apply industry-standard techniques for usability evaluations, including:
• With input from your team, designing and preparing tasks for participants.
• Recruiting users with a range of disabilities, as suitable for the product – blind, low vision, motor
impaired, deaf and those with cognitive disabilities such as dyslexia and learning disabilities.

142
Levels of Testing

Usability and Accessibility testing

How do we conduct accessibility usability testing? (Continued)

• Conduct a pilot test to identify any issues with the testing protocol.

• Conducting the testing either in a usability tab or through remote testing.

• Analyzing and prioritizing the findings, making recommendations.

143
Levels of Testing

Configuration testing
• Configuration testing is testing the performance of the system under development against various
combinations of software and hardware to find out the best configurations, under which the system
can work with any flaws or issues while matching its functional requirements.

• Usually, Configuration Testing is a time-consuming process; since usually sets of software and
hardware forming the system have many variables which result in a large number of combinations,
also installing and uninstalling software and hardware can be a very time-consuming task. That’s
why planning is usually an essential phase of Configuration Testing process.

144
Levels of Testing

Configuration testing
Configuration Testing Example

• Let us understand this with an example of a Desktop Application:

• Generally, desktop applications will be of 2 tier or 3 tier. Here, we will consider a 3 tier desktop
application which is developed using asp.Net and consists of client, business logic server and database
server where each component supports the mentioned platforms.

145
Levels of Testing

Configuration testing
• Client Platform - Windows XP, Window7 OS, windows 8 OS, etc.

• Server Platform - Windows Server 2008 R2, Windows Server 2008 R2, Windows Server 2012R2.

• Database –SQL Sever 2008, SQL Server 2008R2, SQL Server 2012, etc.

A tester has to test the Combination of Client, Server and Database with combinations of the above-
mentioned platforms and database versions to ensure that the application is functioning properly and
does not fail.

146
Levels of Testing

Configuration testing
• Configuration testing is not only restricted to Software but also applicable for Hardware, which is why
it is also referred as a Hardware configuration testing, where we test different hardware devices like
Printers, Scanners, Web cams, etc. that supports the application under test.

147
Levels of Testing

Compatibility testing
Compatibility Testing is a type of Software testing to check whether your software is capable of
running on different hardware, operating systems, applications, network environments
or Mobile devices.

148
Levels of Testing

Compatibility testing
Types of Compatibility Tests

• Hardware

• Operating System

• Software

• Network

• Browser

• Devices

• Mobile

• Versions
149
Levels of Testing

Compatibility testing
There are two types of Compatibility testing

• Backward compatibility Testing

• Forward compatibility Testing

150
Levels of Testing

Compatibility testing
• Backward compatibility Testing is to verify the behaviour of the developed hardware/software
with the older versions of the hardware/software.

• Forward compatibility Testing is to verify the behaviour of the developed hardware/software

with the newer versions of the hardware/software.

151
Levels of Testing

Testing the documentation

• Documentation Testing involves testing of the documented artifacts that are usually developed before
or during the testing of Software.

• Documentation for Software testing helps in estimating the testing effort required, test coverage,
requirement tracking/tracing, etc.

152
Levels of Testing

Testing the documentation

The description of some commonly used documented artifacts related to Software development and
testing, such as:

• Test Plan

• Requirements

• Test Cases

• Traceability Matrix

153
Levels of Testing

Website testing
Web Testing in simple terms is checking your web application for potential bugs before its made live or
before code is moved into the production environment.

154
Levels of Testing

Functionality Testing
• This is used to check, if your product is as per the specifications you intended for it as well as the
functional requirements you charted out for it in your developmental documentation.

Testing Activities Included:

• Test all links in your web pages are working correctly and make sure there are no broken
links.

155
Levels of Testing

Usability testing
• Usability Testing has now become a vital part of any web based project. It can be carried out by
testers like you or a small focus group similar to the target audience of the web application.

Test the site Navigation:

• Menus, buttons or Links to different pages on your site should be easily visible and consistent on all
web pages.

156
Levels of Testing

Interface testing
Three areas to be tested here are - Application, Web and Database Server

• Application: Test requests are sent correctly to the Database and output at the client side is displayed
correctly. Errors if any by the application, must be only shown to the administrator and not the end
user.

157
Levels of Testing

Document Links

Topics URL Notes

Testing as an
https://worldofcse.wordpress.com/2014/07/13/tes This link gives an introduction to
Engineering
ting-as-an-engineering-activity/ Testing
Activity
Testing as a https://www.tutorialspoint.com/software_testing/s This link explains process of a
Process oftware_testing_quick_guide.htm testing
http://www.informit.com/articles/article.aspx?p=1 This link will help to explain
Testing axioms
9796 about testing axioms
Basic definitions
http://softwaretestingfundamentals.com/software This link helps basic definitions
of software
-testing-basics of software testing
testing
Software Testing https://www.utest.com/articles/seven-testing- This link explains seven principle
Principles principles of software testing

158
Levels of Testing

Document Links

Topics URL Notes

The Tester’s Role in a
https://worldofcse.wordpress.com/2014 This link explains detail s about
Software
/07/19/the-testers-role-in-a-software- Tester’s role in a software
Development
development-organization/ development.
Organization
The Defect http://www.brainkart.com/article/Defec
In this link, you will learn about the
Repository and Test t-classes--the-defect-repository-and-
Defect Repository and Test Design
Design test-design_9142/
Defect Prevention https://www.softwaretestinghelp.com/d This link explains about Defect
strategies. efect-prevention-methods/ prevention strategies.

159
Levels of Testing

Video Links

Topics URL Notes

Defect Prevention https://www.youtube.com/watch?v=aTE This video explains Defect

strategies z5bVBNdE Prevention strategies.

The Tester’s Role in a

https://www.youtube.com/watch?v=ijCF This video explains about the
Software Development
5-h9ZoU tester’s role in a organization
Organization
The Defect Repository This video explains about the test
https://slideplayer.com/slide/8790183/
and Test Design design
Software Testing https://www.guru99.com/software- This video explains about software
Principles testing-seven-principles.html testing principles.

160
Levels of Testing

E-Book Links

Topics URL Page number

Srinivasan Desikan and
https://www.oreilly.com/library/view/software-
Gopalaswamy Ramesh,
testing-
“Software Testing – Principles 6 to 45
principles/9788177581218/xhtml/chapter010.x
and Practices”, Pearson
html#ch10.1
Education, 2006
Naresh Chauhan, Software https://access.redhat.com/documentation/en-
Testing Principle and Practices, us/red_hat_enterprise_linux/7/html/storage_a 46 to 90
Oxford University Press. dministration_guide/index
Ron Patton, “Software Testing”,
https://shekharsk.files.wordpress.com/2016/01
Second Edition, Sams Publishing, 91 to 120
/ron-patton-software-testing.pdf
Pearson Education, 2007.

161