Path & Fault Testing
Path & Fault Testing
Path Testing
• Path testing is a Structural Testing method that involves using
the source code of a program to attempt to find every
possible executable path.
• The idea is that are able to test each individual path from
source code is as many way as possible in order to maximize
the coverage of each test case.
1 1
2 2
3
3
6 4 6 4
7 8 5
7 8 5
9
9
11 10 11 10
Flow Graph Notation
Notation Description
10
Flow Graphs consist of
2
6 7
N Y Y
8
N
Switch case 9
While Loop Until Loop
10
Flow Graphs consist of
2
6 7
Goto
8
Connector
10
Flow Graphs consist of
2
Sequence
6 7
10
Flow Graph Example(2)
1
2
1. scanf(“%d %d”,&x, &y);
2. If (y<0) 3 4
3. pow = -y;
else 5
4. pow = y;
5. Z = 1.0 6
6. While (pow != 0) {
7. z = z*x; 7
8. pow = pow -1; }
9. If (y <0) 8
10. z = 1.0 /z;
11. printf(“%f”,z); 9
10
1
What is Path?
• A path through a program is a sequence of statements that
starts at an entry, junctions, or decision and ends.
• For example: 3
Path1 = 2-3-4-5-6-8-9-10
4
6 7
10
What is Path?
2
• For example: 3
Path2 = 2-3-4-5-7-8-9-10
4
6 7
10
Path Testing Strategies
1
• Statement Testing
a • 100% statement / node coverage
b
2 • Branch Testing
T F c • 100% branch/ link coverage
3 4
• Path Testing
d e • 100% path coverage
5
f T Fg
6 7
h i
8
j
9
Path Testing Strategies
1
• Statement Testing
a • 100% statement / node coverage
b
2 • Branch Testing
T F c • 100% branch/ link coverage
3 4
• Path Testing
d e • 100% path coverage
5
f T Fg
6 7
h i
8
j
9
Path Testing Strategies
1
• Statement Testing
a • 100% statement / node coverage
b
2 • Branch Testing
T F c • 100% branch/ link coverage
3 4
• Path Testing
d e • 100% path coverage
5
f T Fg
6 7
h i
8 Statement Testing < Branch Testing < Path Testing
j
9
Path Testing Strategies
1
• Statement Testing
a • 100% statement / node coverage
b
2 • Branch Testing
T F c • 100% branch/ link coverage
3 4
• Path Testing
d e • 100% path coverage
5
f T Fg
6 7
h i
8 Statement Testing < Branch Testing < Path Testing
j
9
Path Testing Strategies
1
• Statement Testing
a • 100% statement / node coverage
b
2 • Branch Testing
T F c • 100% branch/ link coverage
3 4
• Path Testing
d e • 100% path coverage
5
f T Fg
6 7
h i
8 Statement Testing < Branch Testing < Path Testing
j
9
Path Testing Strategies
1
• Statement Testing
a • 100% statement / node coverage
b
2 • Branch Testing
T F c • 100% branch/ link coverage
3 4
• Path Testing
d e • 100% path coverage
5
f T Fg
6 7
h i
8 Statement Testing < Branch Testing < Path Testing
j
9
A Coverage Table
Decisions Process-Link
1 Paths 2 5 a b c d e f g h i j
a
1-2-3-5-6-8- T T
2
b c 9
T F
(a-b-d-f-h-j)
3 4
1-2-4-5-7-8- F F
d e 9
5
f T F g (a-c-e-g-i-j)
6 7 1-2-3-5-7-8- T F
h i 9
8
(a-b-d-g-i-j)
j
1-2-4-5-6-8- F T
9
9
(a-c-e-f-h-j)
Exmple Flow Graph
1. Program Triangle
4 5 6 7 8
2. Dim a, b,c As Integer
3. Dim IsTriangle As Boolean
9
4. Output ( “enter a,b, and c integers”)
5. Input (a,b,c) 10 11
6. Output (“side 1 is”, a)
7. Output (“side 2 is”, b)
12
8. Output (”side 3 is”, c)
13. If IsTriangle
17 18
14. then if (a=b) AND (b=c)
15. then Output (“equilateral”)
16. else if (a != b) AND (a != b) AND (b != c) 20 19
17. then Output ( “Scalene”)
18. else Output (“Isosceles”) 22
19. endif
20. endif
23
21. else Output (“not a triangle”)
22. endif
23. end Triangle2
Basis Path Testing
4 5 6 7 8
• For example: 10 11
4-5-6-7-8-9-10-12-13-21-22-23 12
13
21 14
15 16
17 18
20 19
22
23
Test Case for Path Coverage
4-5-6-7-8-9-10-12-13-21-22-23
4-5-6-7-8-9-11-12-13-14-15-20-22-23
4-5-6-7-8-9-11-12-13-14-16-17-19-20-22-23
4-5-6-7-8-9-11-12-13-14-16-18-19-20-22-23
indegree =1
outdegree =1
DD-Paths
Characteristic of DD-Path has 5 cases:
• Case 1: Single node with indeg =0
• Case 2: Single node with outdeg = 0
• Case 3: Single node with indeg >= 2 or outdeg >=2
• Case 4: Single node with indeg = 1 and outdeg =1
• Case 5: Maximum chain of length >=1
Condensation of Code to Table then to Flow Graph
Path/node DD-path
code statement name Case
1. Program Triangle
2. Dim a, b,c As Integer Skip 1- 3
3. Dim IsTriangle As Boolean (or w/4)
4 first 1
4. Output ( “enter a,b, and c integers”) 5–8 A 5
5. Input (a,b,c)
6. Output (“side 1 is”, a) 9 B 3
7. Output (“side 2 is”, b)
8. Output (”side 3 is”, c)
10 C 4
11 D 4
9. If (a<b+c) AND (b<a+c) And (c<b+a) Def of 12 E 3
10. then IsTriangle = True DD-paths on 13 F 3
11. else IsTriangle = False 14 H 3
12. endif 15 I 4
16 J 3
13. If IsTriangle
14. then if (a=b) AND (b=c)
17 K 4
15. then Output (“equilateral”) 18 L 4
16. else if (a NE b) AND (a NE b) AND (b NE c) 19 M 3
17. then Output ( “Scalene”) 20 N 3
18. else Output (“Isosceles”) 21 G 4
19. endif 22 O 3
20. endif 23 last 2
21. else Output (“not a triangle”)
22. endif
23. end Triangle2
DD-Path
First
4 5 6 7 8
A
9
B
10 11
C D
12
E
13
F
21 14
G H
15 16
I J
17 18
K L
20 19
N M
22
O
23
Last
Path Analysis
• What is path analysis?
V(G) = e-n+1 B D
e : the number of edges F T
n : the number of nodes T E F
C F
Example: 11-7+1 = 5
G
How to select path
• Independent Path
• McCabe’s Baseline
Independent Path
• Independent program paths an independent path is
any path through the program that introduces at least
one new set of processing statements or a new
condition.
• Cyclomatic complexity is a software metric that
provides a quantitative measure of the logical A
complexity if a program
• No count on traverse edge between sink node-G to
source node-A B D
• P1: A-B-C-G F T
• P2: A-B-C-B-C-G T E F
• P3: A-B-E-F-G
• P4: A-D-E-F-G C F
• P5: A-D-F-G
G
Independent Path
C F
G
McCabe’s Baseline
To determine a set of basis paths,
1. Pick a "baseline" path that corresponds to normal execution.
(The baseline should have as many decisions as possible.)
3. Repeat this until all decisions have been flipped. When you
reach V(G) basis paths, you're done.
• P1: A-B-C-B-E-F-G A
• P2: A-D-E-F-G
• P3: A-D-F-G B D
• P4: A-B-E-F-G F T
T E F
• P5: A-B-C-G
C F
G
McCabe’s Path of Triangle Program
First
Results
B
Original P1: First-A-B-C-E-F-H-J-K-M-N-O- Scalene
Last C D
at B Last path
F
Flip P1 P3: First-A-B-C-E-F-G-O-Last Infeasible
G H
at F path
I J
Flip P1 P4: First-A-B-C-E-F-H-I-N-O-Last Equilateral
at H K L
Last
McCabe’s Path
1 3 4 5 Scalene P1
2 4 1 2 Not a Triangle P6
3 5 5 5 Equilateral P4
4 3 2 2 Isosceles P5
Steps of path testing
5 6
7 8
9 10
11 12
13
14
Basis Path Testing Example
• V(G) = E - N + 2
= 19 - 14 + 2
= 7
Basis Path Testing Example
DD-Path Testing:
• When every DD-path is traversed (the C1 metric), we know that
each predicate outcome has been executed; this amounts to
traversing every edge in the DD-path graph (or program graph).
• For if–then and if–then–else statements, this means that both the
true and the false branches are covered (C1p coverage). For CASE
statements, each clause is covered.
• Coupling effect hypothesis: state that a test data set that detects all
simple faults in program is so sensitive that will also detect more
complex faults.
Fault-based adequacy criteria
Given a program and test suit T ,mutation analysis consists of following steps
1. Mutant can be distinguish from original program but test suite T doesn’t
contain a test case that distinguish them.
Therefore the test adequacy for above program is 25% since only one
mutant killed out of the 4 by the selected test suits TS.
65
Test Execution
Designing test cases and test suites is creative, executing test cases
should be automatic with the help of test automation .
Scaffolding includes:
1. Test driver(substituting for main or calling program)
2. Test harnesses(substituting for parts of the deployment
environment)
3. Stubs(substituting for functionality called or used by the
software under test)
Figure shows the When self-checks are embedded in the program, test
cases need not include predicted outputs.
A program or module specification describes all correct program
behaviors, so an oracle based on a specification need not be paired
with a particular test case. Instead, the oracle can be incorporated
into the program under test, so that it checks its own work.
The cost of a runtime assertion check is too high - most tools for
assertion processing also provide controls for activating and
deactivating assertions.
Capture and Replay
The savings from automated retesting with a captured log
depends on how many build and-test cycles we can continue
to use it in, before it is invalidated by some change to the
program.
Capturing events at a more abstract level suppresses
insignificant changes. For example, if we log only the actual
pixels of windows and menus, then changing even a typeface
or background color can invalidate an entire suite of
execution logs.
The value of a captured log can be obtained by varying the
logged events to obtain additional test cases. Creating
meaningful and well-formed variations also depends on the
abstraction level of the log.
Basic Principles
1. Sensitivity
Human developers make errors, producing faults in software.
Faults may lead to failures, but faulty software may not fail on
every execution. The sensitivity principle states that it is better to
fail every time than sometimes.
Consider the cost of detecting and repairing a software fault. If it
is detected immediately (e.g., by an on-the-fly syntactic check in a
design editor), then the cost of correction is very small, and in fact
the line between fault prevention and fault detection is blurred. If a
fault is detected in inspection or unit testing, the cost is still
relatively small.
A failure detected in integration testing, the cost of correction is
much greater. If the first failure is detected in system or acceptance
testing, the cost is very high indeed, and the most costly faults are
those detected by customers in the field.
2. Redundancy
Redundancy is the opposite of independence. If one part of a software
artifact (program, design document, etc.) constrains the content of
another, then they are not entirely independent, and it is possible to
check them for consistency.
In communication, redundancy can be introduced into messages in
the form of error detecting and error-correcting codes to guard against
transmission errors.
The redundancy can be introduced or exploited with an automatic,
algorithmic check for consistency, it has the advantage of being much
cheaper and more thorough than dynamic testing or manual
inspection.
Checkable redundancy is not limited to program source code, nor is it
something that can be introduced only by programming language
designers.
For example, software design tools typically provide ways to check
consistency between different design views or artifacts.
Figure 3.2: Can the variable k ever be uninitialized the first time i is added
to it? The property is undecidable, so Java enforces a simpler, stricter
property.
4. Partition
Partition, often also known as "divide and conquer," is a general
engineering principle. Dividing a complex problem into subproblems
to be attacked and solved independently is probably the most common
human problem-solving strategy.
Software engineering in particular applies this principle in many
different forms and at almost all development levels, from early
requirements specifications to code and maintenance.
Partitioning can be applied both at process and technique levels. At
the process level, we divide complex activities into sets of simple
activities that can be attacked independently.
5. Visibility
Visibility means the ability to measure progress or status against
goals. In software engineering, one encounters the visibility principle
mainly in the form of process visibility, and then mainly in the form
of schedule visibility: ability to judge the state of development against
a project schedule.
Quality process visibility also applies to measuring achieved (or
predicted) quality against quality goals. The principle of visibility
involves setting goals that can be assessed as well as devising
methods to assess their realization.
Visibility is closely related to observability, the ability to extract
useful information from a software artifact.
6. Feedback