Basic Definitions

Error—People make errors. A good synonym is mistake. When people make mistakes while coding,
we call these mistakes bugs. Errors tend to propagate; a requirements error may be magnified during
design and amplified still more during coding.

Fault—A fault is the result of an error. It is more precise to say that a fault is the representation of an
error, where representation is the mode of expression, such as narrative text, Unified Modeling
Language diagrams, hierarchy charts, and source code.

Failure—A failure occurs when the code corresponding to a fault executes.

Incident—When a failure occurs, it may or may not be readily apparent to the user (or customer or
tester). An incident is the symptom associated with a failure that alerts the user to the occurrence of a
failure.

Test—Testing is obviously concerned with errors, faults, failures, and incidents. A test is the act of
exercising software with test cases. A test has two distinct goals: to find failures or to demonstrate
correct execution.

Test case—A test case has an identity and is associated with a program behavior. It also has a set of
inputs and expected outputs.

Test Cases

The essence of software testing is to determine a set of test cases for the item to be tested. A test case
is (or should be) a recognized work product. A complete test case will contain a test case identifier, a
brief statement of purpose (e.g., a business rule), a description of preconditions, the actual test case
inputs, the expected outputs, a description of expected postconditions, and an execution history.

Test case execution entails establishing the necessary preconditions, providing the test case inputs,
observing the outputs, comparing these with the expected outputs, and then ensuring that the
expected postconditions exist to determine whether the test passed. From all of this, it becomes clear
that test cases are valuable—at least as valuable as source code. Test cases need to be developed,
reviewed, used, managed, and saved.

Insights from a Venn Diagram

Testing is fundamentally concerned with behavior, and behavior is orthogonal to the code-based view
common to software (and system) developers. A quick distinction is that the code-based view focuses
on what it is and the behavioral view considers what it does.
Consider a universe of program behaviors. (Notice that we are forcing attention on the essence of
testing.) Given a program and its specification, consider the set S of specified behaviors and the set P
of programmed behaviors.

some possibilities arise: either such a test case is unwarranted, the specification is deficient, or the
tester wishes to determine that specified non-behavior does not occur.

Identifying Test Cases

Two fundamental approaches are used to identify test cases; traditionally, these have been called
functional and structural testing. Specification-based and code-based are more descriptive names, and
they will be used here. Both approaches have several distinct test case identification methods; they
are generally just called testing methods.

• Specification-Based Testing

The reason that specification-based testing was originally called “functional testing” is that any
program can be considered to be a function that maps values from its input domain to values in its
output range. This notion is commonly used in engineering, when systems are considered to be black
boxes. This led to another synonymous term—black box testing, in which the content
(implementation) of the black box is not known, and the function of the black box is understood
completely in terms of its inputs and outputs.

With the specification-based approach to test case identification, the only information used is the
specification of the software. Therefore, the test cases have two distinct advantages: (1) they are
independent of how the software is implemented, so if the implementation changes, the test cases are
still useful; and (2) test case development can occur in parallel with the implementation, thereby
reducing the overall project development interval.
 • Code-Based Testing

Code-based testing is the other fundamental approach to test case identification. To contrast it with
black box testing, it is sometimes called white box (or even clear box) testing. The clear box metaphor
is probably more appropriate because the essential difference is that the implementation (of the black
box) is known and used to identify test cases. The ability to “see inside” the black box allows the tester
to identify test cases on the basis of how the function is actually implemented.

• Specification-Based versus Code-Based Debate

Given the two fundamentally different approaches to test case identification, it is natural to question
which is better. If you read much of the literature, you will find strong adherents to either choice. The
Venn diagrams presented earlier yield a strong resolution to this debate. Recall that the goal of both
approaches is to identify test cases (Figure 1.7). Specification-based testing uses only the specification
to identify test cases, while code-based testing uses the program source code (implementation) as the
basis of test case identification
Error And Fault Taxonomies

Our definitions of error and fault hinge on the distinction between process and product: process
refers to how we do something, and product is the end result of a process. The point at which testing
and Software Quality Assurance (SQA) meet is that SQA typically tries to improve the product by
improving the process. In that sense, testing is clearly more product oriented. SQA is more concerned
with reducing errors endemic in the development process, whereas testing is more concerned with
discovering faults in a product.

Levels of Testing

Thus far, we have said nothing about one of the key concepts of testing—levels of abstraction. Levels
of testing echo the levels of abstraction found in the waterfall model of the software development life
cycle. Although this model has its drawbacks, it is useful for testing as a means of identifying distinct
levels of testing and for clarifying the objectives that pertain to each level. A diagrammatic variation of
the waterfall model, known as the V-Model in ISTQB parlance; this variation emphasizes the
correspondence between testing and design levels.

Generalized Pseudocode

Pseudocode provides a language-neutral way to express program source code. This version is loosely
based on Visual Basic and has constructs at two levels: unit and program components. Units can be
interpreted either as traditional components (procedures and functions) or as object-oriented
components (classes and objects). This definition is somewhat informal; terms such as expression,
variable list, and field description are used with no formal definition.
The Triangle Problem

The triangle problem is the most widely used example in software testing literature. Some of the
more notable entries in three decades of testing literature are Gruenberger (1973), Brown and Lipov
(1975), Myers (1979), Pressman (1982) and subsequent editions, Clarke and Richardson (1983,
1984), Chellappa (1987), and Hetzel (1988).

❖ Problem Statement:

• Simple version: The triangle program accepts three integers, a, b, and c, as input. These are
taken to be sides of a triangle. The output of the program is the type of triangle determined by
the three sides: Equilateral, Isosceles, Scalene, or NotATriangle.

• Improved version: The triangle program accepts three integers, a, b, and c, as input. These are
taken to be sides of a triangle.

❖ Discussion: Perhaps one of the reasons for the longevity of this example is that it contains
clear but complex logic. It also typifies some of the incomplete definitions that impair
communication among customers, developers, and testers.

❖ Traditional Implementation

The traditional implementation of this grandfather of all examples has a rather FORTRAN-like style.
The flowchart for this implementation appears in Figure 2.1. Figure 2.2 is a flowchart for the
improved version. The flowchart box numbers correspond to comment numbers in the
(FORTRANlike) pseudocode program given next.
The NextDate Function

The complexity in the triangle program is due to the relationships between inputs and correct
outputs. We will use the NextDate function to illustrate a different kind of complexity—logical
relationships among the input variables. 2.3.1

❖ Problem Statement: NextDate is a function of three variables: month, date, and year. It
returns the date of the day after the input date. The month, date, and year variables have
 integer values subject to these conditions (the year range ending in 2012 is arbitrary, and is
from the first edition): c1. 1 ≤ month ≤ 12 c2. 1 ≤ day ≤ 31 c3. 1812 ≤ year ≤ 2012

❖ Discussion: Two sources of complexity exist in the NextDate function: the complexity of the
input domain discussed previously, and the rule that determines when a year is a leap year. A
year is 365.2422 days long; therefore, leap years are used for the “extra day” problem. If we
declared a leap year every fourth year, a slight error would occur.

The Commission Problem

Our third example is more typical of commercial computing. It contains a mix of computation and
decision making, so it leads to interesting testing questions. Our main use of this example will be in
our discussion of data flow and slice-based testing. 2.4.1 Problem Statement A rifle salesperson in the
former Arizona Territory sold rifle locks, stocks, and barrels made by a gunsmith in Missouri. Locks
cost $45, stocks cost $30, and barrels cost $25.

❖ Discussion: This example is somewhat contrived to make the arithmetic quickly visible to the
reader. It might be more realistic to consider some other additive function of several variables,
such as various calculations found in filling out a US 1040 income tax form.

The SATM System

To better discuss the issues of integration and system testing, we need an example with larger scope
(Figure 2.3). The ATM described here is minimal, yet it contains an interesting variety of functionality
and interactions that typify the client side of client–server systems.

❖ Problem Statement: The SATM system communicates with bank customers via the 15 screens
shown in Figure 2.4. Using a terminal with features as shown in Figure 2.3, SATM customers
can select any of three transaction types: deposits, withdrawals, and balance inquiries. For
simplicity, these transactions can only be done on a checking account.

When a bank customer arrives at an SATM station, screen 1 is displayed. The bank customer accesses
the SATM system with a plastic card encoded with a personal account number (PAN), which is a key to
an internal customer account file, containing, among other things, the customer’s name and account
information. If the customer’s PAN matches the information in the customer account file, the system
presents screen 2 to the customer. If the customer’s PAN is not found, screen 4 is displayed, and the
card is kept.

❖ Discussion: A surprising amount of information is “buried” in the system description just

The Currency Converter The currency conversion program is another event-driven program that
emphasizes code associated with a GUI. A sample GUI is shown in Figure 2.5.
The application converts US dollars to any of four currencies: Brazilian reals, Canadian dollars,
European Union euros, and Japanese yen. Currency selection is governed by the radio buttons (option
buttons), which are mutually exclusive. When a country is selected, the system responds by
completing the label; for example, “Equivalent in …” becomes “Equivalent in Canadian dollars” if the
Canada button is clicked.

Saturn Windshield Wiper Controller

The windshield wiper on some Saturn automobiles is controlled by a lever with a dial. The lever has
four positions: OFF, INT (for intermittent), LOW, and HIGH; and the dial has three positions,
numbered simply 1, 2, and 3. The dial positions indicate three intermittent speeds, and the dial
position is relevant only when the lever is at the INT position. The decision table below shows the
windshield wiper speeds (in wipes per minute) for the lever and dial positions.