Module - 4
1
Syllabus
Path Testing, Data flow testing, Levels of Testing,
Integration Testing
DD Paths, Test coverage metrics, Basis path testing,
guidelines and observations,
Definition Use testing, Slice based testing, Guidelines
and observations.
Traditional view of testing levels, Alternative life
cycle models, the SATM systems, separating
integration and system testing, Guidelines and
observations.
2
Path Testing
Also known as glass box, structural, clear box,
white box and open box testing.
A software testing technique whereby explicit
knowledge of the internal workings of the
item being tested are used to select the test data.
Unlike black box testing that is using the
program specification to examine outputs,
white box testing is based on specific
knowledge of the source code to define the
test cases and to examine outputs.
3
Structural Testing
• Structural testing methods are very
amenable to:
– Rigorous definitions
• Data flow, control flow, objectives,
coverage criteria, relation to
programming language semantics
– Mathematical analysis
• Graphs, path analysis
– Precise measurement
• Metrics, coverage analysis 4
Program Graph - Definition
“Given a program written in an imperative
programming language, its Program
Graph, is a directed labeled graph in
which nodes are either groups of
entire statements or fragments of a
statement, and edges represent flow
of control”
5
Program Graph
• If i, j, are nodes in the program graph, there
is an edge from node i, to node j in the
program graph if and only if, the statement
corresponding to node j, can be executed
immediately after the last statement of the
group of statement(s) that correspond to
node i.
• The groups of statements that make up a node in
the Program Graph is called a basic block.
• There is a straightforward algorithm to segment a
code fragment into basic blocks and create the
corresponding Program Graph. 6
Style Choices for Program
Graphs
• Deriving a program graph from a given program is
an easy process.
• It is illustrated here with four of the basic
structured programming constructs and also with
our pseudocode implementation of the triangle
program.
• Line numbers refer to statements and statement
fragments.
7
Style Choices for Program
Graphs
8
Style Choices for Program
Graphs
9
Style Choices for Program
Graphs
10
Style Choices for Program
Graphs
11
White-box Testing: Determining the Basic Blocks
FindMean (FILE ScoreFile)
{ float SumOfScores = 0.0;
int NumberOfScores = 0;
1
float Mean=0.0; float Score;
Read(ScoreFile, Score);
2 while (! EOF(ScoreFile) {
3 if (Score > 0.0 ) {
SumOfScores = SumOfScores + Score;
4
NumberOfScores++;
}
5
Read(ScoreFile, Score); 6
}
/* Compute the mean and print the result */
7 if (NumberOfScores > 0) {
Mean = SumOfScores / NumberOfScores;
printf(“ The mean score is %f\n”, Mean); 8
} else
printf (“No scores found in file\n”); 9
} 12
Constructing the Logic Flow Diagram
Start
F
2
T
3
T F
4 5
7
T F
8 9
Exit
13
Program Graph - Example
14
Program Graph - Example
15
Program Graph - Example
16
Simple (Unstructured) Program - Example
17
DD-Paths (1)
• The best-known form of structural testing is based on a
construct known as a decision-to-decision path.
• A DD-Path is a chains obtained from a program graph, where
a chain is a path in which the initial and terminal nodes
are distinct, and every interior node has indegree = 1,
and outdegree = 1.
• We show in the next slide on selecting the nodes from a
program graph to form a DD-Path chain. DD-Paths are used
to create DD-Path Graphs.
• Note that the initial node is 2-connected to every other node in
the chain, and there are no instances of 1- or 3- connected
nodes.
• An example of a chain is shown below:
18
Initial node Internal nodes Final node
DD-Paths (2)
• More formally a DD-Path is a chain obtained from a
program graph such that:
– Case1: it consists of a single node with indeg = 0.
– Case2: it consists of a single node with outdeg = 0.
– Case3: it consists of a single node with indeg ≥ 2 or
outdeg ≥ 2
– Case4: it consists of a single node with indeg =1, and
outdeg = 1
– Case5: it is a maximal chain of length ≥ 1
19
DD-Paths (2)
• Cases 1 and 2 establish the unique source and sink
nodes of the program graph of a structured program
as initial and final DD-paths.
• Case 3 deals with complex nodes; it assures that no
node is contained in more than one DD-path.
• Case 4 is needed for “short branches”; it also
preserves the one-fragment, one DD-path principle.
• Case 5 is the “normal case,” in which a DD-path is a
single entry, single-exit sequence of nodes (a chain).
The “maximal” part of the case 5 definition is used
to determine the final node of a normal (nontrivial)
chain. 20
DD-Path Graph
• Given a program written in an imperative
language, its DD-Path graph is a labeled
directed graph, in which nodes are DD-Paths
of its program graph, and edges represent
control flow between successor DD-Paths.
• In this respect, a DD-Path is a condensation
graph.
• For example, 2-connected program graph nodes
are collapsed to a single DD-Path graph node.
21
DD-Path Graph
This is a complex definition, so we will apply it to the
program graph of triangle program.
•Node 4 is a case 1 DD-path; we will call it “first.”
Similarly, node 23 is a case 2 DD-path, and we will call
it “last.”
•Nodes 5 through 8 are case 5 DD-paths. We know that
node 8 is the last node in this DD-path because it is the
last node that preserves the 2-connectedness property
of the chain.
•If we go beyond node 8 to include node 9, we violate
the indegree = outdegree = 1 criterion of a chain. If we
stop at node 7, we violate the “maximal” criterion. 22
DD-Path Graph
• Nodes 10, 11, 15, 17, 18, and 21 are case 4 DD-
paths.
• Nodes 9, 12, 13, 14, 16, 19, 20, and 22 are case 3
DD-paths.
• Finally, node 23 is a case 2 DD-path.
• All this is summarized in Figure shown in next
slide.
23
DD-Path Graph
24
Test Coverage Metrics
• The motivation of using DD-paths is that they
enable very precise descriptions of test
coverage.
• In our quest to identify gaps and redundancy in
our test cases as these are used to exercise
(test) different aspects of a program we use formal
models of the program structure to reason about
testing effectiveness.
• Test coverage metrics are a device to
measure the extent to which a set of test
cases covers a program. 25
Test Coverage Metrics
Metric Description of Coverage
C0 Every Statement
C1 Every DD-Path
C1P Every predicate to each outcome
C2 C1 Coverage + loop coverage
Cd C1 Coverage + every dependent pair of DD-Paths
CMCC Multiple condition coverage
Cik Every program path that contains up to k repetitions of a loop
(usually k=2)
Cstat “Statistically significant” fraction of paths
C∞ All possible execution paths
26
Statement and Predicate Coverage
Testing
• Statement coverage based testing aims to devise test
cases that collectively exercise all statements in a
program.
• Predicate coverage (or branch coverage, or decision
coverage) based testing aims to devise test cases that
evaluate each simple predicate of the program to True and
False.
• Here the term simple predicate refers to either a single
predicate or a compound Boolean expression that is
considered as a single unit that evaluates to True or False.
This amounts to traversing every edge in the DD-Path
graph.
• For example in predicate coverage for the condition
if(A or B) then C we could consider the test cases A=True,
B= False (true case), and A=False, B=False (false case).
Note if the program was encoded as if(A) then C we would
27
not detect any problem.
DD-Path Graph Edge Coverage C1
( Every DD-path )
1 2
Here a T,T and
F,F combination P1
will T F
suffice to have
DD-Path Graph
edge coverage T P2 F
or
Predicate
coverage C1
28
DD-Path Coverage Testing C1P
( Every Predicate to each outcome )
• This is the same as
the C1 but now we T P1 F
must consider test
cases that exercise
all all possible
outcomes of the P2
choices T,T, T,F, F,T, T F
F,F for the predicates
P1, and P2
respectively, in the
DD-Path graph.
29
Multiple Condition Coverage Testing
• Now if we consider that the predicate P1 is a
compound predicate (i.e. (A or B)) then Multiple
Condition Coverage Testing requires that
each possible combination of inputs be tested
for each decision.
• Example: “if (A or B)” requires 4 test cases:
A = True, B = True
A = True, B = False
A = False, B = True
A = False, B = False
• The problem: For n conditions, 2n test cases are
needed, and this grows exponentially with n. 30
Loop Coverage
• The simple view of loop testing coverage is that we must
devise test cases that exercise the two possible
outcomes of the decision of a loop condition that is one
to traverse the loop and the other to exit (or not
enter) the loop.
• An extension would be to consider a modified boundary
value analysis approach where the loop index is given a
minimum, minimum +, a nominal, a maximum -, and a
maximum value or even robustness testing.
• Once a loop is tested, then the tester can collapse it into
a single node to simplify the graph for the next loop
tests. In the case of nested loops we start with the
inner most loop and we proceed outwards.
• If loops are knotted then we must apply data flow
analysis testing techniques, that we will examine later
31
in
the course.
Loop Coverage
32
Path Testing
• Path Testing is focusing on test techniques that are
based on the selection of test paths through a
program graph. If the set of paths is properly
chosen, then we can claim that we have achieved a
measure of test thoroughness.
• The fault assumption for path testing techniques
is that something has gone wrong with the software
that makes it take a different path than the one
intended.
• Structurally, a path is a sequence of statements
in a program unit. Semantically, a path is an
execution instance of the program unit.
• For software testing we are interested in entry-exit
33
paths.
Path Testing Process
• Input:
– Source code and a path selection criterion
• Process:
– Generation of a CFG
– Selection of Paths
– Generation of Test Input Data
– Feasibility Test of a Path
– Evaluation of Program’s Output for the
Selected Test Cases
34
McCabe Algorithm to Determine Basis
Paths
• The algorithm is straightforward:
– The method begins with the selection of a “baseline
path”, which should correspond to a normal execution
of a program (from start node to end node, and
has as many decisions as possible).
– The algorithm proceeds by retracing the paths
visited and flipping the conditions one at a time.
– The process repeats up to the point all flips have
been considered.
• The objective is to generate test cases that exercise
these “basis” paths.
35
McCabe CFG Algorithm
36
McCabe Algorithm to Determine Basis Paths
37
McCabe Algorithm to Determine Basis Paths
38
McCabe Algorithm to Determine Basis Paths
• Table 9.3 shows the edges traversed by each path, and also
the number of times an edge is traversed.
• We can force this to begin to look like a vector space by
defining notions of addition and scalar multiplication: path
addition is simply one path followed by another path, and
multiplication corresponds to repetitions of a path.
• The path A, B, C, B, E, F, G is the basis sum p2 + p3 – p1, and
the path A, B, C, B, C, B, C, G is the linear combination 2p2 –
p1.
• We can check the independence of paths p1 – p5 by examining
the first five rows of this incidence matrix. The bold entries
show edges that appear in exactly one path, so paths p2 – p5
must be independent.
• Path p1 is independent of all of these, because any attempt to
express p1 in terms of the others introduces unwanted edges.
39
Program Graph - Example
40
DD-Path Graph
41
Program Graph to DD-Path Graph
first
A
4 5 6 7 8
B
9 C D
10 11 E
12
F
13
H
14 G
21 I J
15 16
17 18 K L
20 N
22 O
19 M
23 last
42
Topological Paths
43
Feasible Paths
• If we assume that the logic of the program
dictates that,
“if node C is traversed, then we must traverse
node H”,
and
“if node D is traversed, then we must traverse
Node G”
– These constraints will eliminate Paths 4, 5, 6 and 7
– We are left to consider four feasible paths:
1. first – A – B – C – E – F – H – J – K – M – N – O – last
2. first – A – B – C – E – F – H – J – L – M – N – O – last
3. first – A – B – C – E – F – H – I – N – O – last
8. first – A – B – D – E – F – G – O – last
44
Structured Programming Constructs
s1 s1 s1 s1 s1 s1
s2 s2 s3 s2 s3 s4 s2 s2 s2
s3 s4 s5 s3
Sequence if-then-else case if-then loop-pre-test loop-post-test
45
Violations of Structured
Programming Constructs
s1 s1 s1 s1
s0 s0
s2 s2 s2 s2
s0
s3 s3 s3 s3 s1
Branching into loop Branching out of loop Branching into decision Branching out of decision
46
Condensing A Program Graph
47
Guidelines and Observations
• Functional testing focuses on properties that are “too far” or disassociated
from the source code being tested
• Path testing focuses too much on the graph and not the logic of the code
• Path testing very useful to measure the quality of the testing (coverage
criteria)
• Basis path testing is giving us a lower bound of how much testing is necessary
• Path testing is providing a set of metrics that act as cross checks to functional
testing
– When a program path is traversed by several functional test cases we suspect
redundancy
– When we fail to attain DD-Path graph coverage, we know that there are gaps in the
functional test cases
• Coverage metrics can be used as
– Setting minimum testing standards and as
– Mechanism to selectively test portions of the code more rigorously than others 48
