Chapter 23
Testing Conventional
Applications
R.Bhavani
AP/CSE/SRC
Introduction
The primary objective for test-case design
is to derive a set of tests that have the
highest likelihood for uncovering errors in
software.
To accomplish this objective, two different
categories of test-case design techniques
are used: white-box testing and black-
box testing.
Software Testing
•Graph-Based
Testing
•Basis path •Equivalence
testing white-box black-box partitioning
•Condition methods methods • Boundary
testing value
• Data flow •analysis
Orthogonal array
testing testing
• Loop
testing
Methods
Strategies
White-box testing
White-box (Glass box/structural ) tests focus on the
program control structure.
The test cases can be derived to
(1) Guarantee that all independent paths within a
module have been exercised at least once,
(2) Exercise all logical decisions on their true and
false sides,
(3) Execute all loops at their boundaries and within
their operational bounds, and
(4) Exercise internal data structures to ensure their
validity
Basis Path Testing
The basis path method enables the test-case
designer to derive a logical complexity measure
of a procedural design and use this measure as a
guide for defining a basis set of execution paths.
Test cases derived to exercise the basis set are
guaranteed to execute every statement in the
program at least one time during testing.
Basis Path Testing
Step – 1: Construct the flow graph from the source
code or flow charts.
Step – 2: Identify independent paths.
Step – 3: Calculate Cyclomatic Complexity, V(G).
Step – 4: Design the test cases.
Flow graph notation
Sequence UNTIL
IF
CASE
WHILE
Procedures to construct flow graph from
flow chart
Each circle, called a flow graph node, represents one or more procedural
statements
A sequence of process boxes and a decision diamond (predicate node)
can map into a single node.
The arrows on the flow graph, called edges or links, represent
flow of control and are analogous to flowchart
arrows.
Area bounded by edges and nodes are called regions.
When counting regions, we include the area outside the1 graph as region.
Edge
1
2,3 Node
2
6 4,5
R2
3
7 R3 8
6 4
R1 Region
9
7 8 5 R4
9 10
10
8
11
11
Flow graph
Region
4
edges Region 6
Region 5
Region 3
Region 1
Region 2
Flow graph – Compound Statement
Predicate
node a
.
. b x
.
IF a OR b
then procedure y x
x
else procedure
y
ENDIF
A compound condition occurs when one or more Boolean
operators (logical OR, AND, NAND, NOR) is present
in a conditional statement.
Program design language (PDL) segment translates into the flow
graph shown.
A separate node is created for each of the conditions a and b in
the statement IF a OR b.
Independent path programs & Cyclomatic
Complexity
An independent path is any path through the program
that introduces at least one new set of processing
statements or a new condition.
Path 1: 1-11
1 Path 2: 1-2-3-4-5-10-1-11
Edge
Path 3: 1-2-3-6-8-9-10-1-11
Node
2,3
Path 4: 1-2-3-6-7-9-10-1-11
6
R2 4,5 Each new path introduces a new
7 R3 8 edge.
R1 Region
9
10
R4 But the path
Path 5: 1-2-3-4-5-10-1-2-3-6-8-9-
11 10-1-11 – Combination of already
specified
Independent path programs & Cyclomatic
Complexity
Cyclomatic Complexity - software metric that provides
a quantitative measure of the logical complexity of a
program.
Defines the number of independent paths in the basis
set of a program
Provides with an upper bound for the number of tests
that must be conducted to ensure that all statements
have been executed at least once.
Cyclomatic Complexity
Calculation of Cyclomatic Complexity V(G) by three methods:
Method 1
V(G) = e – n + 2 ( Where “e” are edges & “n” are nodes)
Method 2
V(G) = P + 1 (Where P- predicate nodes with out-degree = 2)
Method 3
V(G) = Number of regions
1
Edge
2,3 Node
1.V(G)= 11edges - 9nodes + 2 = 4
6 4,5
R2
7 R3 8
R1 Region
2.V(G) = 3 predicate nodes + 1 = 4
9
R4
10
3.V(G) = 4 regions = 4
11
Example 2: Sample Program
void foo (float y, float a *, int n)
{
float x = sin (y) ;
if (x > 0.01)
z = tan (x) ;
else
z = cos
(x) ;
for (int i = 0 ; i < x ; + + i)
{ a[i] = a[i] * z ;
Cout < < a [i]; }
}
Example 2
Step 1: Drawing flow graph from simple
Predicate
code nodes
1
void foo (float y, float a *, int n) 1
{
float x = sin (y) ; R1
2 3
if (x > 0.01) 2
z = tan (x) ;
else 3 Predicate
4
R3
z = cos (x) ; nodes
4 5 6
for (int i = 0 ; i < x ; + + i) { 5 R2
a[i] = a[i] * z ; 6
6
Cout < < a [i]; } 6
} 7 7
Example 2
Step 2: Identify independent paths
• There are 3 paths in this program which are
independent paths and they form a basis-set.
• These paths are described below
Path 1: 1 – 2 – 4 – 5 - 7
Path 2: 1 – 3 – 4 – 5 - 7
Path 3: 1 – 2 – 4 – 5 – 6 – 5 – 7
Path 4: 1 – 3 – 4 – 5 – 6 – 5 – 7
• We must execute these paths at least once in order to test the
program thoroughly.
• Accordingly we can design the test cases (no of test cases =>
3)
Example 2
Step 3: Calculate cyclomatic complexity,
V(G)
Calculation of V(G) by three methods:
Method 1
V(G) = e – n + 2 ( Where “e” are edges & “n” are
nodes) V(G) = 8 – 7 + 2 = 3
Method 2
V(G) = P + 1 (Where P- predicate nodes with out-degree = 2)
V(G) = 2 + 1 = 3 (The predicate nodes are labeled)
Method 3
V(G) = Number of regions = 3
Example 3 Example 4
Flow graph Flow chart
1 1
2 2
3 3
8 4
5 6
4 5
6 7
7
8
9
Example 3
Step 2: Identify independent paths
1
3
8
4 5
6
7 Path 1: 1-9
Path 2: 1-2-8-7-1-9
Path 3: 1-2-3-4-6-7-1-9
9 Path 4: 1-2-3-5-6-7-1-9
Example 3
Step 3: Calculate cyclomatic complexity, V(G)
V(G) = E - N + 2
V(G) = 11 - 9 + 2 = 4
E= no. of edges,
2 N= no. of
nodes
1
3
4
V(G) = P + 1
V(G) = 3 + 1 =
4
P= no. of predicate nodes
(conditions) # Regions = 4
Example 4
Flow chart
3
4
5 6
25
Example 4
Step 2: Identify independent paths
Predicate
1
nodes
Path 1: 1,2,3,6,7,8
Path 2: 1,2,3,5,7,8
Path 3: 1,2,4,7,8
2
Path 4: 1,2,4,7,2,4,...7,8
R1
3
4
5 6 R4
R2
R3 Predicate
nodes edges
7
Predicate
nodes
8
Example 4
Step 3: Calculate cyclomatic complexity,
V(G)
Calculation of V(G) by three methods:
Method 1
V(G) = e – n + 2 ( Where “e” are edges & “n” are
nodes) V(G) = 9 – 7 + 2 = 4
Method 2
V(G) = P + 1 (Where P- predicate nodes with out-degree = 2)
V(G) = 3 + 1 = 4 (The predicate nodes are labeled)
Method 3
V(G) = Number of regions = 4
Graph Matrices
A graph matrix is a square matrix whose size (i.e.,
number of rows and columns) is equal to the number
of nodes on a flow graph
Each row and column corresponds to an identified
node, and matrix entries correspond to connections
(an edge) between nodes.
By adding a link weight to each matrix entry, the
graph matrix can become a powerful tool for
evaluating program control structure during testing
Graph Matrices
Connected
1 to node
Node 1 2 3 4 5
a
1 a
3
e 2
b
3 d b
5 f
4 d
4 c f
c
g 5 g e
2
The link weight is 1 (a connection exists) or 0 (a connection does not exist).
But link weights can be assigned other, more interesting properties:
The probability that a link (edge) will be executed.
The processing time expended during traversal of a link
The memory required during traversal of a link
The resources required during traversal of a link.
Control Structure Testing
Condition testing
Condition testing is a test-case design method that
exercises the logical conditions contained in a program
module.
In short, it tests relational operator like <, ≤, >,
≥, =, ≠ and Boolean operators like OR (|), AND (&), NOT(⌐)
that exist in your program.
Control Structure Testing
Condition testing
i<n
i == n
S1 i := 2 i>n
C1 while (i is less than or equal to n) do
S2 j := i - 1
C2 while ((j is greater than or equal to 1)
and (A[j] is greater than A[j+1])) do
S3 temp := A[j]
A[j] := A[j+1]
S4 A[j+1] := temp
j := j-1
S5
S7 end while
i := i + 1
S6 end while
Control Structure Testing
Condition testing
(j < 1) && (A[j] < A[j+1])
(j < 1) && (A[j] == A[j+1])
(j < 1) && (A[j] > A[j+1])
(j == 1) && (A[j] < A[j+1])
(j == 1) && (A[j] == A[j+1])
(j == 1) && (A[j] > A[j+1])
(j > 1) && (A[j] < A[j+1])
(j > 1) && (A[j] == A[j+1])
(j > 1) && (A[j] > A[j+1])
C2 (j >= 1) && (A[j] > A[j+1])
all true-false combinations of conditions be exercised at least once.
Data flow testing
The data flow testing method selects test paths of a
program according to the locations of definitions and
uses of variables in the program.
DEF(S) = {X | statement S contains the definition of X}
USE(S) = {X | statement S contains the use of X}
Data flow testing
1. read x, y;
2. if(x>y)
3. a = x+1
else
4. a = y-1
5. print a;
DEFINED AT USED AT
VARIABL
NODE NODE
E
x 1 2, 3
y 1 2, 4
a 3, 4 5
Loop testing
Loop testing is a white-box testing technique
that focuses exclusively on the validity of
loop constructs.
Examples of types of loop tested are,
Simple loop
Nested loop
Concatenated loop
Unstructured loop
Simple Loop testing
The following set of tests can be applied to simple loops, where n is
the maximum number of allowable passes through the loop
Skip the entire loop
Make 1 passes through the loop
Make 2 passes through the loop
m passes through loop where m<n.
(n-1), n, and (n+1) passes through the loop.
Nested Loop testing
Start with inner loop. Set all
other loops to minimum values.
Conduct simple loop testing on
inner loop.
Work outwards
Continue until all loops
tested.
Concatenated Loop testing
If independent loops, use
simple loop testing.
If dependent, treat as nested
loops. (first loop counter is
used as initial value for loop
2)
Concatenated Loop testing
For unstructured loops, it
requires restructuring of the
design to reflect the use of
the structured programming
constructs.
Black-box testing
Black-box tests are designed
validate
to functional requirements
without regard to the
workings of a program. internal
Behavioural testing or
functional testing
Derive input conditions to
exercise all functional
requirements
Not an alternative, complement
to WBT to uncover different
class of error
WBT: early, BBT: later
Black-box testing
Black-box testing attempts to find errors in the following
categories:
incorrect or missing functions
interface errors
errors in data structures or external database access
behavior or performance errors
initialization and termination error
Design - Black-box tests
Tests are designed to answer some of
the following questions:
What classes of input will make good test
cases?
Is the system particularly sensitive to certain
input values?
How are system behavior and performance
tested?
How is functional validity tested?
How are the boundaries of a data
class
isolated?
What data rates and data volume can the
system tolerate?
What effect will specific
combinations of data have on system
Black-box techniques
Graph Based Testing
Equivalence Partitioning
Boundary Value Analysis
Orthogonal Array Testing
Graph-based testing
Begins by creating a graph of important
objects and their relationships
Devising a series of tests that will cover the
graph so that each object and relationship is
exercised and errors are uncovered.
Links: diff forms
Graph
Directed: relationship in one
Nodes (objs)
direction
Links (relationship)
Bidirectional / symmetric:
Node weights (properties of a node)
relationship in both directions
Link weights (characteristic of a link)
Parallel: no. of diff
relationships
44
Graph-based testing - Uses number
of behavioral testing methods
Modeling Nodes Links
Transaction flow Steps in some Logical connections between
(DFD: used) (online transaction steps
airline reservation) (flight information (validation,availability,
input) processing)
Finite state User observable states Transitions between states
(State dia: used) of the software (order info verified, inventory
(user observable (each entry on the availability, customer billing info)
states) (order entry screen)
through phone order)
Data flow Data objects Transformations from one data
(tax calculation) (Tax – computed based on object to another
annual income) (tax = 20 % of Annual income)
Timing Program objects Sequential connections
between these objects
Link weights: execution
times
Equivalence partitioning
Divides the input domain into classes of data that are likely to
exercise a specific software function.
Derive ideal test cases from input domain to uncover
classes of errors
•Test-case design for equivalence partitioning is based on an evaluation of
equivalence classes for an input condition
Set of valid and invalid states for input conditions
specific numeric value, range of values, set of related values, or
boolean condition
How do I define equivalence classes for testing?
Input condition Equivalence classes defined
Range (1 to 100) 1 valid (50), 2 invalid (0,110)
specific value (100) 1 valid (100), 2 invalid (50, 110)
member of a set {a,e,i,o,u} 1 valid {i}, 1 invalid {b}
Boolean (0,1)/(true,false) 1 valid {1}, 1 invalid {2}
Example:
Function takes a parameter “month”
Valid range for the month: 1 to 12
representing January to December
Valid range: partition
Invalid ranges: 2 partitions
Invalid Valid Invalid
x<1 1 ≤ x ≤ 12 12 < x
Example:
Pizza Ordering System
Pizza values 1 to 10 is considered valid. A
success message is shown.
While value 11 to 99 are considered invalid for
order and an error message will appear, "Only 10
Pizza can be ordered"
Test Condition:
Invalid Valid Invalid
x<1 1 ≤ x ≤ 10 Any Number greater than 10 entered in
the Order Pizza field(let say 11) is
considered invalid.
x >10 (two digits)
Any 3 Digit Number say -100 is
invalid. x >99 (three digits)
Example: Equivalence partitioning
Boundary Value Analysis
Boundary value analysis leads to a selection of
test cases that exercise bounding values.
More errors occur on the "boundary" of an input
domain rather than on the "center“
Focus is not only on i/p but also on o/p
BVA: more complete, higher likelihood for error
detection
Boundary Value Analysis
The basic idea in boundary value testing is to
select input variable values at their:
Minimum
Just below the minimum
A nominal value
Just above the maximum
Maximum
Pizza Ordering System
Boundary Value Analysis
Test Case Design Techniques
Boundary Value Checking
Robustness Testing Method
Worst-Case Testing Method
Robustness Worst-Case Testing Method
Boundary Value Checking
Boundary Value Checking – 4n+1 (n represents
no. of input variables & 1 is the nominal value)
Single input variable
Min value (Min)
Value just above the minimum (Min+)
Maximum value (Max)
Value just below the maximum value (Max-)
Boundary Value Checking
If there are two inputs (A) and (B), then
extreme values can be selected at:
1.
Anom, Bmin
2.
3. Anom, Bmin+
4.
Anom, Bmax
5.
6. Anom, Bmax-
7.
Amin, Bnom
8.
9. Amin+, Bnom
EX: Boundary Value Checking
A program reads an integer number (1,100)
and checking whether it is a prime number.
Min = 1
Min+ = 2
Max = 100
Max- = 99
Nominal Value = 50-55
BVC= 4n +1 = 5
Test Case ID Test Input Expected Output
1 1 Not a Prime number
2 2 Prime Number
3 100 Not a Prime Number
4 99 Not a Prime Number
5 53 Prime Number
Robustness Testing Method
Robustness Testing Method – 6n+1 (n
represents no. of input variables & 1 is the
nominal value)
Single Variable
Value just below the minimum value (Min-)
Min value (Min)
Value just above the minimum (Min+)
Value just below the maximum value (Max-)
Maximum value (Max)
Value just above the maximum (Max+)
Robustness Testing Method
Two input variables: Along with the test
inputs from BVC, values at Max+ and Min-
are considered:
1.
2.
Anom, Bmin 10.Amax+, Bnom
3. Anom, Bmin+ 11.Amin-, Bnom
4. 12.Anom, Bmax+
Anom, Bmax
5.
6. Anom, Bmax-
13.Anom, Bmin-
7.
Amin, Bnom
8.
9. Amin+, Bnom
EX: Robustness Testing Method
A program reads an integer number (1,100)
and checking whether it is a prime number.
Min- = 0 Max = 100
Min = 1 Max- = 99
Min+ = 2 Max+=101
Nominal Value = 50-55
RTM = 6n+1 = 7
Test Case ID Test Input Expected Output
1 0 Invalid
2 1 Not a Prime number
3 2 Prime Number
4 100 Not a Prime Number
5 99 Not a Prime Number
6 101 Invalid
7 53 Prime Number
Worst-Case Testing Method
Worst-Case Testing Method– 5n(n
represents no. of input variables & 1 is the
nominal value)
Single I/P variable
Min value (Min)
Value just above the minimum (Min+)
Value just below the maximum value (Max-)
Maximum value (Max)
Worst Case Testing Method
Along with the test inputs from BVC, values
with combo of Min, Max, Max+ and Min- are
considered:
1. 10. Amin, Bmin
Anom, Bmin
19. Amin+, Bmax
2. 11. Amin+, Bmin 20. Amin, Bmax-
3. Anom, Bmin+ 12. Amin, Bmin+ 21. Amin, Bmax-
4. 13. Amax, Bmin
Anom, Bmax 22. Amin+, Bmax
5. 14. Amax, Bmin 23. Amax-, Bmax
6. Anom, Bmax-
15. Amax-, Bmin+ 24. Amax, Bmax-
7.
Amin, Bnom 16. Amax, Bmin+ 25. Amax-, Bmax-
8.
17. Amax-, Bmin+
9. Amin+, Bnom
18. Amin, Bmax
EX: Worst-Case Testing Method
A program reads an integer number (1,100)
and checking whether it is a prime number.
Since it is a single input, it takes the same
form of Robustness (i.e.5n = 51 = 5)
Max = 100
Min = 1
Min+ = 2
Max- = 99
Nominal Value = 50-55
Test Case ID Test Input Expected Output
1 1 Not a Prime number
2 2 Prime Number
3 100 Not a Prime Number
4 99 Not a Prime Number
5 53 Prime Number
Robustness Worst-Case
Testing Method
Robustness Worst-Case Testing Method– 7n(n
represents no. of input variables & 1 is the
nominal value)
Further we can add more number of test cases
to the worst case method – Two i/ps 49 test
cases
Value just below the minimum value (Min-)
Min value (Min)
Value just above the minimum (Min+)
Value just below the maximum value (Max-)
Maximum value (Max)
Value just above the maximum (Max+)
EX: Robustness Worst-Case Testing Method
A program reads an integer number (1,100) and
checking whether it is a prime number.
Since it is a single input, it takes the same form
of Robustness (i.e.7n = 71 = 7)
Min- = 0 Max = 100
Min = 1 Max- = 99
Min+ = 2 Max+=101
Nominal Value = 50-55
Test Case ID Test Input Expected Output
1 0 Invalid
2 1 Not a Prime number
3 2 Prime Number
4 100 Not a Prime Number
5 99 Not a Prime Number
6 101 Invalid
7 53 Prime Number
Orthogonal Array Testing
Used when the number of input parameters is
small and the values that each of the parameters
may take are clearly bounded
The orthogonal array testing method is
particularly useful in finding region faults —an
error category associated with faulty logic within a
software component
Conventional “one input item at a time”
approaches, consider a system that has three
input items, X, Y, and Z (with 3 inputs each).
There are 33 = 27 possible test cases.
When orthogonal array testing occurs, an L9
orthogonal array of test cases is created.
The L9 orthogonal array has a “balancing
property” . Test coverage across the input
Example
consider the send function for a fax application.
Four parameters, P1, P2, P3, and P4, are passed to
the send function. Each takes on three discrete
values.
For example, P1 takes on values:
P1 = 1, send it now
P1 = 2, send it one hour later
P1 = 3, send it after midnight
P2, P3, and P4 would also take on values of 1, 2,
and 3, signifying other send functions.
one input item at a time testing strategy (P1, P2, P3,
P4) would be specified: (1, 1, 1, 1), (2, 1, 1, 1), (3,
1, 1, 1), (1, 2, 1, 1), (1, 3, 1, 1), (1, 1, 2, 1), (1, 1, 3,
1), (1, 1, 1, 2), and (1, 1, 1, 3). The number of tests
required is 34= 81
Orthogonal Array Testing
Detect and isolate all single mode faults. In case of errors,
one can identify which parameter values cause the fault
Detect all double mode faults.
Multimode faults.
EX: Orthogonal Array Testing
A microprocessor's functionality has to be tested:
Temperature: 100C, 150C and 200C.
Pressure : 2 psi,5psi and 8psi
Doping Amount :4%,6% and 8%
Deposition Rate : 0.1mg/s , 0.2 mg/s and 0.3mg/s
By using the Conventional method we need = 81 test cases to
cover all the inputs.
Let's work with the OATS method:
No. of factors = 4 (temperature, pressure, doping amount and
Deposition rate)
Levels = 3 levels per factor (temperature has 3 levels-100C,
150C, and 200C and likewise other factors too have levels)
EX: Orthogonal Array Testing
1. Columns with the No. of factors
Test case # Temperature Pressure Doping Deposition
rate
amount
2. Enter the number of rows is equal to levels per factor. i.e
temperature has 3 levels. Hence, insert 3 rows for each level for
temperature
Test case # Temperature Pressure Doping amount Deposition rate
1 100C
2 100C
3 100C
4 150C
5 150C
6 150C
7 200C
8 200C
9 200C
EX: Orthogonal Array Testing
3. Now split up the pressure, doping amount and the deposition
rates in the columns.
For eg: Enter 2 psi across temperatures 100C,150C and 200C
likewise enter doping amount 4% for 100C,150C and 200C and
so on. Hence, in OAs, we need 9 Test cases to cover.
Test case # Temperature Pressure Doping Deposition
rate
amount
1 100C 2 psi 4% 0.1 mg/s
2 100C 5 psi 6% 0.2 mg/s
3 100C 8 psi 8% 0.3 mg/s
4 150C 2 psi 4% 0.1 mg/s
5 150C 5 psi 6% 0.2 mg/s
6 150C 8 psi 8% 0.3 mg/s
7 200C 2 psi 4% 0.1 mg/s
8 200C 5 psi 6% 0.2 mg/s
9 200C 8 psi 8% 0.3 mg/s
Model-Based Testing
Uses information contained in the requirements model
as the basis for the generation of test cases.
uses UML state diagrams - behavioral model, as the
basis for the design of test cases.
1. evaluate all use cases to fully
understand the sequence of interaction
within the system,
2. identify events that drive the
interaction sequence and how understand
these
relate to specific objects, events
3. create a sequence for each use case,
4. build a UML state diagram for the system and
5. review the behavioral model to verify accuracy
and consistency.
Model-Based Testing
Traverse the behavioral model and specify
the inputs - make the transition from state to
state.
Review the behavioral model and note the
expected outputs as the software makes the
transition from state to state.
Execute the test cases.
Compare and expected results
take corrective action
actual andas required.
TESTING FOR REAL-TIME SYSTEMS
The time-dependent, asynchronous nature of many
real- time applications adds a new and potentially
difficult element to the testing mix—time.
the test-case designer have to consider not only
conventional test cases but also event handling (i.e.,
interrupt processing), the timing of the data, and the
parallelism of the tasks (processes) that handle the
data.
Testing for Real-Time Systems – Four
Step strategy
Task testing: The first step in the testing of real-time software
is to test each task independently.
Behavioral testing. Using system models created with
automated tools, it is possible to simulate the behavior of a real-
time system and examine its behavior as a consequence of
external events.
Intertask testing. Once errors in individual tasks and in system
behavior have been isolated, testing shifts to time-related errors.
Asynchronous tasks are tested with different data rates and
processing load to determine intertask synchronization errors
In addition, tasks that communicate via a message queue or data
store are tested to uncover errors in the sizing of these data storage
areas.
Testing for Real-Time Systems...
System testing. Software and hardware are integrated, and a full range
of system tests are conducted in an attempt to uncover errors at the
software- hardware interface.
Are interrupt priorities properly assigned and properly handled?
Is processing for each interrupt handled correctly?
Does the performance (e.g., processing time) of each interrupt-handling
procedure conform to requirements?
Does a high volume of interrupts arriving at critical times create problems
in function or performance?
Thank You!