Black-Box and White-Box

Testing

1
 Testing Objective
• Primary Objective
– Testing is a process of executing a program with the intent
of finding an error.
– A good test case is one that has a high probability of
finding an as-yet undiscovered error.
– A successful test is one that uncovers an as-yet-
undiscovered error

• Objective is to design tests that systematically uncover

different classes of errors and to do so with a minimum
amount of time and effort.
 Software Testing
Two major categories of software testing:
• Black-box testing
– Knowing the specified function that a product has been designed to
perform, test to see if that function is fully operational and error free
– Includes tests that are conducted at the software interface
– Not concerned with internal logical structure of the software

• White-box testing
– Knowing the internal workings of a product, test that all internal
operations are performed according to specifications and all
internal components have been exercised
– Involves tests that concentrate on close examination of procedural
detail
– Logical paths through the software are tested
– Test cases exercise specific sets of conditions and loops
3
 White-box Testing
• White-box testing, sometimes called glass-box testing, is
a test case design method that uses the control structure
of the procedural design to derive test cases.

• These test cases

– Guarantee that all independent paths within a module
have been exercised at least once
– Exercise all logical decisions on their true and false
sides
– Execute all loops at their boundaries and within their
operational bounds
– Exercise internal data structures to ensure their
validity 4
 Basis path testing
• Basis path testing is a white-box testing technique

• Test cases derived to exercise the basis set are guaranteed to

execute every statement in the program at least one time during
testing.

• Defines a basic set of execution paths based on logical complexity

of a procedural design

Methods:
1. Flow graph notation
2. Independent program paths or Cyclomatic complexity
3. Deriving test cases
4. Graph Matrices
 Flow Graph Notation
• Start with simple notation for the representation of control flow
(called flow graph). It represent logical control flow.
 Fig. A represent program control structure and fig. B maps
the flowchart into a corresponding flow graph.

In fig. B each circle, called flow graph node, represent one

or more procedural statement.
 Flow Graph Notation
• A circle in a graph represents a node, which stands for a sequence
of one or more procedural statements
• A sequence of process boxes and decision diamond can map into
a single node.
• An edge, or a link, is a an arrow representing flow of control in a
specific direction
– An edge must start and terminate at a node
– An edge does not intersect or cross over another edge
• Areas bounded by a set of edges and nodes are called regions
• When counting regions, include the area outside the graph as a
region, too.
• A node containing a simple conditional expression is referred to as
a predicate node
– Each compound condition in a conditional expression containing one
or more Boolean operators (e.g., and, or) is represented by a separate
predicate node
– A predicate node has two edges leading out from it (True and False) 8
• When we translating PDL segment into flow graph,
separate node is created for each condition.
• Each node that contains a condition is called predicate
node and is characterized by two or more edges comes
from it.
 Independent program paths or
Cyclomatic complexity
• An independent path is any path through the program that
introduces at least one new set of processing statement or new
condition.
• For example, a set of independent paths for flow graph:
– Path 1: 1-11
– Path 2: 1-2-3-4-5-10-1-11 Basis Set
– Path 3: 1-2-3-6-8-9-1-11
– Path 4: 1-2-3-6-7-9-1-11
• Note that each new path introduces a new edge.
• The path 1-2-3-4-5-10-1-2-3-6-8-9-10-1-11 is not considered to be
an independent path because it is simply a combination of already
specified paths and does not traverse any new edges.
• Test cases should be designed to force execution of these paths
(basis set).
• Every statement in the program should be executed at least once
and every condition will have been executed on its true and false.
• How do we know how many paths to looks for ?
 Cyclomatic complexity is a software metrics that provides a
quantitative measure of the logical complexity of a program.
• It defines no. of independent paths in the basis set and also
provides number of test that must be conducted.
• One of three ways to compute cyclomatic complexity:
1. The no. of regions corresponds to the cyclomatic complexity.
2. Cyclomatic complexity, V(G), for a flow graph, G, is defined as
V(G) = E - N + 2
where E is the number of flow graph edges, N is the number of
flow graph nodes.
3. Cyclomatic complexity, V(G), for a flow graph, G, is also
defined as
V(G) = P + 1
where P is the number of predicate nodes edges
So the value of V(G) provides us with upper bound of test cases.
 Deriving Test Cases
• It is a series of steps method.
• The procedure average depicted in PDL.
• Average, an extremely simple algorithm, contains
compound conditions and loops.
To derive basis set, follow the steps.
1. Using the design or code as a foundation, draw a
corresponding flow graph.
• A flow graph is created by numbering those PDL
statements that will be mapped into corresponding flow
graph nodes.
Deriving Test Cases
Flow graph for the procedure average
2. Determine the cyclomatic complexity of the resultant flow
graph.
V(G) can be determined without developing a flow graph by
counting all conditional statements in the PDL (for the
procedure average, compound conditions count as two) and
adding 1
V(G) = 6 regions
V(G) = 17 edges - 13 nodes + 2 = 6
V(G) = 5 predicate nodes + 1 = 6

3. Determine a basis set of linearly independent paths

The value of V(G) provides the number of linearly independent
paths through the program control structure.
path 1: 1-2-10-11-13
path 2: 1-2-10-12-13
path 3: 1-2-3-10-11-13
path 4: 1-2-3-4-5-8-9-2-. . .
path 5: 1-2-3-4-5-6-8-9-2-. . .
path 6: 1-2-3-4-5-6-7-8-9-2-. . .
The ellipsis (. . .) following paths 4, 5, and 6 indicates that any path
through the remainder of the control structure is acceptable.
4. Prepare test cases that will force execution of each
path in the basis set.
– Data should be chosen so that conditions at the
predicate nodes are appropriately set as each path
is tested.
– Each test case is executed and compared to
expected results.
– Once all test cases have been completed, the tester
can be sure that all statements in the program have
been executed at least once.
 Graph Matrices
 A graph matrix is a square matrix whose size (i.e.,
number of rows and columns) is equal to the number of
nodes on the flow graph.
 Each row and column corresponds to an identified node, and matrix
entries correspond to connections (an edge) between nodes.

 Each node on the flow graph is identify by numbers, while each

edge is identify by letters.

 The graph matrix is nothing more than a tabular representation of a

flow graph.

 By adding a link weight to each matrix entry, the graph matrix can
become a powerful tool for evaluating program control structure
during testing.

 The link weight provides additional information about control flow. In

its simplest form, the link weight is 1 (a connection exists) or 0 (a
connection does not exist).
Connection matrix
• Each letter has been replaced with a 1, indicating that a
connection exists (this graph matrix is called a
connection matrix).

• In fig.( connection matrix) each row with two or more

entries represents a predicate node.

• We can directly measure cyclomatic complexity value by

performing arithmetic operations

• Connections = Each row Total no. of entries – 1.

• V(G)= Sum of all connections + 1

 Control Structure testing
• Basis path testing is simple and effective
• It is not sufficient in itself

 Control structure broadens the basic test coverage and

improves the quality of white box testing
– Condition Testing
– Data flow Testing
– Loop Testing

21
 Condition Testing
--Exercise the logical conditions contained in a
program module
--Focuses on testing each condition in the program
to ensure that it does not contain errors

Simple condition
E1<relation operator>E2
Compound condition
simple condition<Boolean operator>simple
condition
22
 Data flow Testing
• Selects test paths according to the locations of
definitions and use of variables in a program
• Aims to ensure that the definitions of variables and
subsequent use is tested
• First construct a “definition-use graph” from the control
flow of a program
• DEF(S) = {X | statement S contains a definition of X}
USE(S) = {X | statement S contains a use of X}

• Every DU chain be tested at least once.

23
 Loop Testing - General
• A white-box testing technique that focuses exclusively on
the validity of loop constructs

• Four different classes of loops exist

– Simple loops
– Nested loops
– Concatenated loops
– Unstructured loops

• Testing occurs by varying the loop boundary values

24
 Loop Testing

Simple
loop
Nested
Loops

Concatenated
Loops Unstructured
Loops
25
 Testing of Simple Loops
1) Skip the loop entirely
2) Only one pass through the loop
3) Two passes through the loop
4) m passes through the loop, where m < n
5) n –1, n, n + 1 passes through the loop

‘n’ is the maximum number of allowable passes through the loop

26
 Testing of Nested Loops
1) Start at the innermost loop; set all other loops to minimum values
2) Conduct simple loop tests for the innermost loop while holding the
outer loops at their minimum iteration parameter values; add other
tests for out-of-range or excluded values
3) Work outward, conducting tests for the next loop, but keeping all
other outer loops at minimum values and other nested loops to
“typical” values
4) Continue until all loops have been tested

27
 Testing of Concatenated Loops
• For independent loops, use the same approach as for simple loops
• For dependent loops, use the approach applied for nested loops

Testing of Unstructured Loops

• Redesign the code to reflect the use of structured programming
practices
• Depending on the resultant design, apply testing for simple loops,
nested loops, or concatenated loops

28
 Black box testing
• Also called behavioral testing, focuses on the functional
requirements of the software.
• It enables the software engineer to derive sets of input
conditions that will fully exercise all functional
requirements for a program.
• Black-box testing is not an alternative to white-box
techniques but it is complementary approach.
• Black-box testing attempts to find errors in the following
categories:
– Incorrect or missing functions,
– Interface errors,
– Errors in data structures or external data base access.
– Behavior or performance errors,
– Initialization and termination errors.
• Black-box testing purposely ignored control structure, attention is
focused on the information domain. Tests are designed to answer the
following questions:
– How is functional validity tested?
– How is system behavior and performance tested?
– What classes of input will make good test cases?

• By applying black-box techniques, we derive a set of test cases that

satisfy the following criteria
– Test cases that reduce the number of additional test cases that
must be designed to achieve reasonable testing (i.e. minimize
effort and time)
– Test cases that tell us something about the presence or absence
of classes of errors

 Black box testing methods

– Graph-Based Testing Methods
– Equivalence partitioning
– Boundary value analysis (BVA)
– Orthogonal Array Testing
 Graph-Based Testing Methods

• To understand the objects that are modeled in software and the

relationships that connect these objects.
• Next step is to define a series of tests that verify “all objects have
the expected relationship to one another.
• Stated in other way:
– Create a graph of important objects and their relationships
– Develop a series of tests that will cover the graph
• So that each object and relationship is exercised and errors are
uncovered.
• Begin by creating graph –
– a collection of nodes that represent objects
– links that represent the relationships between objects
– node weights that describe the properties of a node
– link weights that describe some characteristic of a link.
Symbolic Representation of Graph
• Nodes are represented as
circles connected by links
that take a number of
different forms.
• A directed link (represented
by an arrow) indicates that a
relationship moves in only
one direction.
• A bidirectional link, also
called a symmetric link,
implies that the relationship
applies in both directions.
• Parallel links are used when
a number of different
relationships are established
between graph nodes.
Example
• Object #1 = new file menu select
• Object #2 = document window
• Object #3 = document text

Referring to example figure, a menu select on new file

generates a document window.
• The link weight indicates that the window must be
generated in less than 1.0 second.
• The node weight of document window provides a list of
the window attributes that are to be expected when the
window is generated.
• An undirected link establishes a symmetric relationship
between the new file menu select and document text,
• parallel links indicate relationships between document
window and document text
• Number of behavioral testing methods that can make
use of graphs:
• Transaction flow modeling.
– The nodes represent steps in some transaction and the links
represent the logical connection between steps
• Finite state modeling.
– The nodes represent different user observable states of the
software and the links represent the transitions that occur to
move from state to state. (Starting point and ending point)
• Data flow modeling.
– The nodes are data objects and the links are the transformations
that occur to translate one data object into another.
• Timing modeling.
– The nodes are program objects and the links are the sequential
connections between those objects.
– Link weights are used to specify the required execution times as
the program executes.
 Equivalence Partitioning
• Equivalence partitioning is a black-box testing method that divides
the input domain of a program into classes of data from which test
cases can be derived.

• Test case design for equivalence partitioning is based on an

evaluation of equivalence classes for an input condition.

• An equivalence class represents a set of valid or invalid states for

input conditions.

• Typically, an input condition is either a specific numeric value, a

range of values, a set of related values, or a Boolean condition.
 Guidelines for Defining
Equivalence Classes
• If an input condition specifies a range, one valid and two invalid equivalence
classes are defined
– Input range: 1 – 10 Eq classes: {1..10}, {x < 1}, {x > 10}

• If an input condition requires a specific value, one valid and two invalid
equivalence classes are defined
– Input value: 250 Eq classes: {250}, {x < 250}, {x > 250}

• If an input condition specifies a member of a set, one valid and one invalid
equivalence class are defined
– Input set: {-2.5, 7.3, 8.4} Eq classes: {-2.5, 7.3, 8.4}, {any other x}

• If an input condition is a Boolean value, one valid and one invalid class are
define
– Input: {true condition} Eq classes: {true condition}, {false condition}

37
Boundary Value Analysis (BVA)
• A greater number of errors occur at the boundaries of
the input domain rather than in the "center“

• Boundary value analysis is a test case design method

that complements equivalence partitioning
– It selects test cases at the edges of a class
– It derives test cases from both the input domain and
output domain

Example
• If 0.0<=x<=1.0
• Then test cases (0.0,1.0) for valid input and (-0.1 and
1.1) for invalid input
 Guidelines for BVA
1. If an input condition specifies a range bounded by
values a and b, test cases should be designed with
values a and b and just above and just below a and b.

1. If an input condition specifies a number of values, test

cases should be developed that exercise the minimum
and maximum numbers. Values just above and below
minimum and maximum are also tested.

1. Apply guidelines 1 and 2 to output conditions.

1. If internal program data structures have prescribed

boundaries be certain to design a test case to exercise
the data structure at its boundary
 Orthogonal Array Testing
• The number of input parameters is small and the values that each of
the parameters may take are clearly bounded.

• When these numbers are very small (e.g., three input parameters
taking on three discrete values each), it is possible to consider every
input permutation .

• However, as the number of input values grows and the number of

discrete values for each data item increases (exhaustive testing
occurs)

• Orthogonal array testing can be applied to problems in which the

input domain is relatively small but too large to accommodate
exhaustive testing.

 Orthogonal Array Testing can be used to reduce the number of

combinations and provide “maximum coverage with a minimum
number of test cases.”
 Example
• Consider the send function for a fax application.
• Four parameters, P1, P2, P3, and P4, are passed to the send
function. Each takes on three discrete values.
• P1 takes on values:
– P1 = 1, send it now
– P1 = 2, send it one hour later
– P1 = 3, send it after midnight
• P2, P3, and P4 would also take on values of 1, 2 and 3, signifying
other send functions.
• OAT is an array of values in which each column represents a
Parameter - value that can take a certain set of values called levels.
• Each row represents a test case.
• Parameters are combined pair-wise rather than representing all
possible combinations of parameters and levels
L9 orthogonal testing will reduce the number of test case to 9 as
shown below