Chapter Four
Test Case Design
Test Case Design
• The goal of the test case design process is to create a set of
test cases that are effective in discovering program defects
and showing that the system meets its requirements.
White box testing
• The general white box testing process is:
• The software’s implementation is analyzed.
• Paths through the software are identified.
• Inputs are chosen to cause the software to execute
selected paths. This is called path sensitization.
Expected results for those inputs are determined.
• The tests are run.
• Actual outputs are compared with the expected
outputs.
• A determination is made as to the proper
functioning of the software.
• White box testing can be applied at all levels of
system development
• unit,
• integration, and
• system.
Control flow testing
• This testing approach identifies the execution paths
through a module of program code and then creates
and executes test cases to cover those paths.
Control Flow Graphs
• Control flow graphs are the foundation of control flow
testing.
• Control flow graphs are the foundation of control flow
testing.
• These graphs document the module's control structure.
• Modules of code are converted to graphs, the paths through
the graphs are analyzed, and test cases are created from that
analysis.
• Control flow graphs consist of a number of elements
• Process block
• A process block is a sequence of program statements that execute
sequentially from beginning to end.
• No entry into the block is permitted except at the beginning.
• No exit from the block is permitted except at the end.
• Once the block is initiated, every statement within it will be
executed sequentially.
• Decision Point
• A decision point is a point in the module at which the control flow can
change.
• Most decision points are binary and are implemented by if-then-else
statements.
• Multi-way decision points are implemented by case statements.
• They are represented by a bubble with one entry and multiple exits.
• Junction Point
• A junction point is a point at which control flows join together.
Example
• In control flow testing we define coverage at a number of different
levels.
• Level 1
• The lowest coverage level is "100% statement coverage“ or statement
coverage.
• This means that every statement within the module is executed, under test,
at least once.
• While this may seem like a reasonable goal, many defects may be missed with this level
of coverage.
• Consider the following code snippet:
if (a>0)
{
x=x+1;
}
if (b==3)
{
y=0;
}
• This code can be represented in graphical form as:
Graphical representation of the above code
• Four execution paths.
• While a single test case is sufficient to test every line of code
in this module (for example, use a=6 and b=3 as input), it is
apparent that this level of coverage will miss testing many
paths.
• Thus, statement coverage, while a beginning, is generally not
an acceptable level of testing.
• Even though statement coverage is the lowest level of coverage, even
that may be difficult to achieve in practice.
• Often modules have code that is executed only in exceptional
circumstances—low memory, full disk, unreadable files, lost connections, etc.
• Testers may find it difficult or even impossible to simulate these
circumstances and thus code that deals with these problems will
remain untested.
• (Statement coverage = No of statements Executed/Total no of
statements in the source code * 100)
• Example:
Read A
Read B
if A > B
Print “A is greater than B”
else
Print "B is greater than A"
endif
• Set 1: If A = 5, B = 2
• No of statements Executed: 5
• Total no of statements in the source code: 7
• Statement coverage =5/7*100 = 71.00 %
• Set 2: If A = 2, B = 5
• No of statements Executed: 6
• Total no of statements in the source code: 7
• Statement coverage =6/7*100 = 85.20 %
• Level 2
• The next level of control flow coverage is "100% decision
coverage.“ or "branch coverage."
• At this level enough test cases are written so that each decision
that has a TRUE and FALSE outcome is evaluated at least once.
• In the previous example this can be achieved with two test cases
(a=-2, b=2 and a=4, b=3).
• Decision coverage= number of decision outcomes exercised/total
number of decision outcomes *100
• Example
Example (int x)
{
if (x < 10)
print("x < 10")
else
print("x >= 10")
}
• For X=7
• Coverage=1/2∗100=50%
• For x=13
• Coverage=21∗100=50%
• Level 3
• It is called 100% condition coverage or condition coverage
• At this level enough test cases are written so that each condition that has a
TRUE and FALSE outcome that makes up a decision is evaluated at least once.
• Example :
if (a>0 && c==1) {x=x+1;}
if (b==3 || d<0) {y=0;}
• This level of coverage can be achieved with two test cases (a>0, c=1, b=3, d<0
and a≤0, c≠1, b≠3, d≥0).
• Condition coverage is usually better than decision coverage because every
individual condition is tested at least once while decision coverage can be
achieved without testing every condition.
• Level 4
• At this level test cases are created for every condition and every decision
100% decision/condition
if(x&&y) {conditionedStatement;}
// note: && indicates logical AND
• for x=TRUE, y=FALSE and x=FALSE, y=TRUE condition coverage
• But conditionedStatement will never be executed so…. decision/condition
• Level 5- multiple condition coverage.
• Consider how the programming language compiler
actually evaluates the multiple conditions in a
decision.
• Use that knowledge to create test cases yielding
100% multiple condition coverage.
f (a>0 && c==1) {x=x+1;}
if (b==3 || d<0) {y=0;}
// note: || means logical OR
• This level of coverage can be achieved with four test cases:
• Achieving 100% multiple condition coverage also achieves decision
coverage, condition coverage, and decision/condition coverage
Exercise
if ((A || B) && C)
{
<< Few Statements >>
}
else
{
<< Few Statements >>
}
Data Flow Testing
• Data flow testing is a powerful tool to detect improper use of data
values due to coding errors.
referencing the value
of a variable without
first assigning a value
to it.
• Three possibilities exist for the first occurrence of a variable through a
program path:
• 1.~d the variable does not exist (indicated by the ~), then it is defined (d)
• 2.~u the variable does not exist, then it is used (u)
• 3.~k the variable does not exist, then it is killed or destroyed (k)
• The first is correct.
• The variable does not exist and then it is defined.
• The second is incorrect.
• A variable must not be used before it is defined.
• The third is probably incorrect.
• Destroying variable before it is created is indicative of a
programming error.
• Examine time-sequenced pairs of defined, used, and killed variable
references.
Dd-Defined and defined again not invalid but suspicious Probably a programming error
Du- Defined and used perfectly correct The normal case
Dk- Defined and then killed not invalid probably a programming error
ud Used and defined acceptable
uu Used and used again acceptable
uk Used and killed acceptable
kd Killed and. defined—acceptable A variable is killed and then
redefined.
KU- Killed and used a serious defect Using a variable that does not exist or is
undefined is always an error
kk Killed and killed Invalid probably a programming error
• A data flow graph is similar to a control flow graph in
that it shows the processing flow through a module.
• It is done in two ways:
• Static Data Flow Testing
• Dynamic Data Flow Testing
Static Data Flow Testing
Define-use-kill information for each
of the variables used in the module.
• The define-use-kill patterns for x (taken in pairs as we
follow the paths) are:
• ~define correct, the normal case
• define-define suspicious, perhaps a programming error
• define-use correct, the normal case
• The define-use-kill patterns for y (taken in pairs as we
follow the paths) are:
• ~use major blunder
• use-define acceptable
• define-use correct, the normal case
• use-kill acceptable
• define-kill probable programming error
• The define-use-kill patterns (taken in pairs as we follow the
paths) are:
• ~kill programming error
• kill-use major blunder
• use-use correct, the normal case
• use-define acceptable
• kill-kill probably a programming error
• kill-define acceptable
• define-use correct, the normal case
• In performing a static analysis on this data flow model the following problems
have been discovered:
• x: define-define
• y: ~use
• y: define-kill
• z: ~kill
• z: kill-use
• z: kill-kill
Drawback
• Arrays are defined and destroyed as a unit but specific elements of
the array are used individually. In the general case, static analysis
cannot determine whether the define-use-kill rules have been
followed properly unless each element is considered individually.
• In complex control flows it is possible that a certain path can never be
executed.
• In this case an improper define-use-kill combination might exist but
will never be executed and so is not truly improper
• For this reason, we now turn to dynamic data flow testing
Dynamic Data Flow Testing
• Because data flow testing is based on a module's control flow, it
assumes that the control flow is basically correct.
• The data flow testing process is to choose enough test cases so that:
• Every "define" is traced to each of its "uses"
• Every "use" is traced from its corresponding "define"
• To do this, enumerate the paths through the module
Black Box Testing Techniques
• The general black box testing process is:
• The requirements or specifications are analyzed.
• Valid inputs are chosen based on the specification to
determine that the software under test (SUT) processes
them correctly.
• Invalid inputs must also be chosen to verify that the SUT
detects them and handles them properly.
• Expected outputs for those inputs are determined.
• Tests are constructed with the selected inputs.
• The tests are run.
• Actual outputs are compared with the expected outputs.
• A determination is made as to the proper functioning of
the SUT
• Black box testing can be applied at all levels of system development—
unit, integration, system, and acceptance.
• When using black box testing, the tester can never be sure of how
much of the system under test has been tested.
• Even though we can't test everything, formal black box testing
directs the tester to choose subsets of tests that are both efficient
and effective in finding defects.
Equivalence Class Testing
• Equivalence class testing is a technique used to reduce the number of
test cases to a manageable level while still maintaining reasonable
test coverage.
• An equivalence class consists of a set of data that is treated the same
by the module or that should produce the same result.
• A group of tests forms an equivalence class if you
believe that:
• They all test the same thing.
• If one test catches a bug, the others probably will too.
• If one test doesn't catch a bug, the others probably won't
either
• We an decide the equivalent classes using the
following two designing logic.
• Testing-by-contract : Creates test cases only for the
situations in which the pre-conditions are met.
• modules are defined in terms of pre-conditions and post-conditions.
• Post-conditions define what a module promises to do (compute a
value, open a file, print report, update a database record, change the
state of the system, etc.).
• Pre-conditions define what that module requires so that it can meet
its post-conditions.
• Defensive testing: an approach that tests under both
normal and abnormal pre-conditions
• If the normal preconditions are met, the module will
achieve its normal post-conditions.
• If the normal pre-conditions are not met, the module will
notify the caller by returning an error code or throwing an
exception
• Guidelines for Equivalence Partitioning :
• If the range condition is given as an input, then one valid and
two invalid equivalence classes are defined.
• If a specific value is given as input, then one valid and two
invalid equivalence classes are defined.
• If a member of set is given as an input, then one valid and one
invalid equivalence class is defined.
• If Boolean no. is given as an input condition, then one valid
and one invalid equivalence class is defined.
Example
• the Amount of test field accepts a Range (100-400) of values
Boundary Value Testing
• Equivalence class testing is the most basic test design
technique.
• It helps testers choose a small subset of possible test cases
while maintaining reasonable coverage.
• Equivalence class testing has a second benefit.
• It leads us to the idea of boundary value testing
• Boundary value testing focuses on the boundaries because that is
where so many defects hide.
If (applicantAge >= 0 && applicantAge <=16)
hireStatus="NO";
If (applicantAge >= 16 && applicantAge <=18)
hireStatus="PART"; The interesting values on or near the boundaries in
If (applicantAge >= 18 && applicantAge <=55) this example are
{-1, 0, 1}, {15, 16, 17},
hireStatus="FULL"; {17, 18, 19}, {54, 55, 56}, and {98, 99, 100}.
If (applicantAge >= 55 && applicantAge <=99) Other values, such as {-42, 1001, FRED, %$#@}
might be included depending on the module's
hireStatus="NO"; documented preconditions.
• Steps
• First, identify the equivalence classes.
• Second, identify the boundaries of each equivalence class.
• Third, create test cases for each boundary value by choosing one point on the
boundary, one point just below the boundary, and one point just above the
boundary.
Decision Table Testing
• Decision tables are an excellent tool to capture certain kinds
of system requirements and to document internal system
design.
• They are used to record complex business rules that a system
must implement.
• In addition, they can serve as a guide to creating test cases.
Rule 1 Rule 2 ……. Rule p
Conditions
Condition 1
Condition 2
…..
Condition m
Action
Action 1
Action 2
…
Action n
The general form of a decision table.
• An auto insurance company gives discounts to drivers who are
married and/or good students.
• Let's begin with the conditions.
• The following decision table has two conditions, each one of which
takes on the values Yes or No.
• Decision tables may specify more than one action for each rule.
• Again, these rules may be unique or may be shared.
• Each rule (vertical column) becomes a test case but values satisfying
the conditions must be chosen.
Applicability and Limitations
• Decision Table testing can be used whenever the
system must:
• implement complex business rules
• when these rules can be represented as a combination of
conditions and
• when these conditions have discrete actions associated
with them
Use Case Testing
• Use cases comes with a description that explains them in detail.
• Each use case has been through an inspection process before it was
implemented.
• To test the implementation, the basic rule is to create at least one test case
for the main success scenario and at least one test case for each extension.
• Because use cases do not specify input data, the tester must select it.
• Typically we use the equivalence class and boundary value techniques
described earlier.
Example
Pairwise testing with orthogonal array
• It is software testing technique that uses orthogonal arrays
to create test cases.
• It is statistical testing approach especially useful when
system to be tested has huge data inputs.
• Orthogonal array testing helps to maximize test coverage by
pairing and combining the inputs and testing the system with
comparatively less number of test cases for time saving.
Example
• A Web page has three distinct sections (Top, Middle, Bottom) that can
be individually shown or hidden from a user
• No of Factors = 3 (Top, Middle, Bottom)
• No of Levels (Visibility) = 2 (Hidden or Shown)
• If we go for Conventional testing technique, we need test cases like 2
X 3 = 6 Test Cases
• If we go for OAT Testing we need 4 Test cases as shown below:
Example :
• A microprocessor’s functionality has to be tested:
• Temperature: 100C, 150C and 200C.
• Pressure : 2 psi,5psi and 8psi
• Doping Amount :4%,6% and 8%
• Deposition Rate : 0.1mg/s , 0.2 mg/s and 0.3mg/
State-Transition Testing
• State-Transition diagrams, like decision tables, are another
excellent tool to capture certain types of system
requirements and to document internal system design.
• These diagrams document the events that come into and are
processed by a system as well as the system's responses.
• These diagrams are also vital tools in the tester's personal
toolbox
Example of state transition diagram (ticket
Reservation)
State-Transition Tables
• Create a set of test cases such that all transitions are exercised at
least once under test.
• This level of testing provides a good level of coverage without
generating large numbers of tests.
• This level is generally the one recommended
Example
• Below is a state-transition diagram for the "enroll in a
course" and "drop a course" process. Determine a set
of test cases that you feel adequately cover the enroll
and drop process.
Reading Assignment
Domain Analysis Testing
