3.

Static Testing

1
 1.1 What is Testing?
Terms

Ad Hoc Reviewing A review technique is performed informally without a structured process.

Checklist-Based
A review technique guided by a list of questions or required attributes.
Reviewing

Dynamic Testing Testing that involves the execution of the test item.

2
 A type of review that follows a defined process with a formally
Formal Review documented output.

A type of review that does not follow a defined process and has no
Informal Review formally documented output.

A type of formal review to identify issues in a work product, which

provides measurement to improve the review process and the software
Inspection
development process.

A review technique in which a work product is evaluated from the

Perspective-Base perspective of different stakeholders with the purpose to derive other
d Reading work products.

3
 A type of static testing in which a work product or process is evaluated by
Review one or more individuals to detect defects or to provide improvements.

A review technique in which a work product is evaluated from the

Role-Based Reviewing perspective of different stakeholders.

A review technique in which a work product is evaluated to determine its

Scenario-Base ability to address specific scenarios.
d Reviewing

The process of evaluating a component or system without executing it,

Static Analysis based on its form, structure, content, or documentation.

4
 Static Testing Testing a work product without the work product code being executed.

A type of formal review by a team of technically-qualified personnel that

Technical Review examines the quality of a work product and identifies discrepancies from
specifications and standards.

A type of review in which an author leads members of the review through

Walkthrough a work product and the members ask questions and make comments
about possible issues.

5
 3.1 Static Testing Basics
Static testing techniques – techniques that test software without executing the code.

Static testing

Review (manual examination of work

Static Analysis (tool-driven evaluation of the
products)
code)
Used to find and remove errors and
Enables code to be analyzed for structural
ambiguities in documents before they are
defects or systematic programming
used in the development process.
weaknesses that may lead to defects.
Completed Completed
manually. automatically.

Examples (areas) of using static analysis:

▪ Safety-critical computer systems (aviation, medical, or nuclear software);
▪ Security testing;
▪ Automated build and delivery systems (agile development, continuous delivery, and continuous
deployment).

6
 3.1.1 Work Products that сan be
Examined by Static Testing
Examples of products that can be examined using
static testing:
▪ Specifications, including business Applying of static
requirements, functional requirements, and testing:
security requirements.

▪ Epics, user stories, and acceptance criteria. Reviews Static Analysis

▪ Architecture and design specifications.
The work product that • Work product with a formal
▪ Code. structure (typically code or
the participants know
▪ Testware, including test plans, test cases, models) for which static
how to read and
test procedures, and automated test scripts analysis tool exists.
understand.
▪ User guides. ▪ Work products are written in
▪ Web pages. natural language such as
▪ Contracts, project plans, schedules, and requirements (e.g., checking
budgets. for spelling, grammar, and
▪ Models, such as activity diagrams, which
readability)
may be used for Model-Based testing.
7
 3.1.2 Benefits of Static Testing

1. Static testing applied early in the software development lifecycle enables the
early detection of defects before dynamic testing is performed (e.g., in
requirements or design specifications reviews, product backlog refinement,
etc.).

2. Defects found early are often much cheaper to remove than defects found
later in the lifecycle, especially compared to defects found after the software
is deployed and in active use.

3. Fixing those defects promptly is almost always much cheaper for the organization
than using dynamic testing to find defects in the test object and then fixing them,
especially when considering the additional costs associated with updating other
work products and performing confirmation and regression testing.

8
 Additional benefits of static testing may include:

▪ Detecting and correcting defects more efficiently, and prior to dynamic test execution.

▪ Identifying defects that are not easily found by dynamic testing.

▪ Preventing defects in design or coding by uncovering inconsistencies, ambiguities,
contradictions, omissions, inaccuracies, and redundancies in requirements.

▪ Increasing development productivity (e.g., due to improved design, more maintainable code).

▪ Reducing development cost and time.

▪ Reducing testing cost and time.

▪ Reducing total cost of quality over the software’s lifetime, due to fewer failures later in the
lifecycle or after delivery into operation.
▪ Improving communication between team members in the course of participating in reviews.

9
 3.1.3 Differences between Static and
Dynamic Testing
Distinctions between static testing
and dynamics
Objectives related to both static testing and
testing:
dynamic testing:
• Static testing defects
▪ providing an assessment of the quality of the
dynamic testing
work products;
failures
▪ identifying defects as early as possible.
Static and dynamic testing complement each other
by finding different types of defects.
Compared with dynamic testing, typical defects
found by static testing are easier and cheaper to
find and fix. • Static testing
dynamic
testing

10
 Typical defects found by static testing include:

▪ Requirement defects (e.g., inconsistencies, ambiguities, contradictions, omissions, inaccuracies,

and redundancies).

▪ Design defects (e.g., inefficient algorithms or database structures, high coupling, low cohesion)

▪ Coding defects (e.g., variables with undefined values, variables that are declared but never used,
unreachable code, duplicate code).

▪ Deviations from standards (e.g., lack of adherence to coding standards).

▪ Incorrect interface specifications (e.g., different units of measurement used by the calling system
than by the called system).

▪ Security vulnerabilities (e.g., susceptibility to buffer overflows).

▪ Gaps or inaccuracies in test basis traceability or coverage (e.g., missing tests for an acceptance
criterion).

Most types of maintainability defects can only be found by static testing:

▪ Improper modularization;

▪ Poor reusability of components;

11
▪ Code that is difficult to analyze and modify without introducing new defects.
 3.2 Review Process
Review

Formal Informal

• Team participation • Not following a defined process

• Documented results of the review • Not having formal documented output
• Documented procedures for conducting
the review

Factors on which the formality of The focus of a review depends on the agreed objectives of the
review
the review depends on:
Review objectives
▪ Software development lifecycle model ▪ Finding defects
▪ Maturity of the development process ▪ Gaining understanding
▪ Complexity of the work product to be reviewed ▪ Generating discussion
▪ Any legal or regulatory requirements ▪ Decision-making by consensus
▪ Need for an audit trail

ISO standard (ISO/IEC 20246) contains more in-depth

descriptions of the review process for work products, including
12 roles and review techniques.
 Figure 3.1 Activities of Planning
Formal Review

Initiat
Fixing
and
reporting
3.2 e
review

Review
Process

Review Individu
meeting la
preparat
i on
13
 Main activities of the review
process
Planning Initiate review (or kick-off)
▪ Defining the scope (the purpose of the ▪ Distributing the work product (physically
review, documents or parts of documents or by electronic means) and other
to review, the quality characteristics to be material, such as issue log forms,
evaluated). checklists, and related work products;

▪ Estimating effort and timeframe. ▪ Explaining the scope, objectives, process,

▪ Identifying review characteristics such as roles, and work products to the
the review type with roles, activities, and participants;
checklists. ▪ Answering any questions that participants
▪ Selecting the people to participate in the may have about the review.
review and allocating roles. The goal of this meeting is to get everybody on
▪ Defining the entry and exit criteria for more the same wavelength regarding the document
formal review types (e.g., inspections). under review and to commit to the time that will
▪ Checking that entry criteria are met be spent on checking.
(for more formal review types).
14
 Individual review (i.e., individual preparation) Issue communication and analysis
▪ Communicating identified potential defects
▪ Reviewing all or part of the work
(e.g., in a review meeting).
product.
▪ Analyzing potential defects, and
assigning ownership and status
▪ Noting potential defects,
to them.
recommendations, and questions.
▪ Evaluating and documenting quality
Checking rate – the number of pages
characteristics.
checked per hour – critical success factor for
▪ Evaluating the review findings against the exit
this stage.
criteria to make a review decision (reject;
major changes needed; accept, possibly
with minor changes).

15
 •

Fixing
and •

reporting
•

16
 3.2.2 Roles and responsibilities
in a formal review
Author Manager Facilitator (often called moderator)
▪ Creates the work product under ▪ Is responsible for review planning. ▪ Ensures effective running of
review. review meetings (when held).
▪ Decides on the execution of reviews.
▪ Fixes defects in the work product ▪ Mediates, if necessary, between the
▪ Assigns staff, budget, and time.
under review (if necessary) . various points of view.
▪ Monitors ongoing cost-effectiveness.
▪ Is often the person upon whom
▪ Executes control decisions in the
the success of the review
event of inadequate outcomes.
depends.

17
 Review leader Reviewers Scribe (or recorder)
▪ May be subject matter experts, persons
▪ Takes overall responsibility for ▪ Collates potential defects found
working on the project, stakeholders with an
the review. during the individual review activity.
interest in the work product, and/or
▪ Decides who will be involved and ▪ Records new potential defects, open
organizes when and where it individuals with specific technical or points, and decisions from the
will take place. business backgrounds. review meeting (when held).
▪ Identify potential defects in the work product
under review. (with the advent of tools to support the
▪ May represent different perspectives (e.g., review process, especially the
tester, programmer, user, operator, logging of defects, open points,
business analyst, usability expert, etc.). and decisions, there is often no
need for a scribe)

In some review types, one person may play more than one role, and the actions associated with
each role may also vary based on the review type.

18 More detailed roles are described in ISO standard (ISO/IEC 20246).

 Reasons for less successful reviews:

▪ The required persons are not available or do not have the required qualifications
or technical skills.
▪ Inaccurate estimates during resource planning by management may result in

time pressure.

▪ Lack of preparation, because the wrong reviewers were chosen.

▪ Missing or insufficient documentation.
▪ Lack of management support.

19
 3.2.3 Review
Types Hig
Reviews can be used for various purposes. h
One of the main objectives of the review is to
uncover defects.

All review types can aid in defect

Inspection
detection The selected review type
should be based on different selection
criteria, such as:
▪ Needs of the project Technical review
▪ Available resources
▪ Product type and Walkthrough
risks
▪ Business domain
▪ Company culture Informal
Lo
The four most common types of reviews are w
in

20
 Informal review (least formal, e.g.,
buddy check, pairing, pair review)
Key characteristics:

▪ Main purpose: to find defects.

▪ Possible additional purposes: generating new ideas or solutions, quickly solving minor
problems.
▪ Not based on a formal (documented) process.
▪ May not involve a review meeting.
▪ May be performed by a colleague of the author (buddy check) or by more people.
▪ Results may be documented.
▪ Varies in usefulness depending on the reviewers.
▪ Use of checklists is optional.
▪ Very commonly used in Agile development.

21
 Walkthrough
Walkthrough – a type of review in which an author leads members of the review through a work
product and the members ask questions and make comments about possible issues.

Key characteristics:

▪ Main purposes: find defects, improve the software product, consider alternative implementations, and
evaluate conformance to standards and specifications.
▪ Possible additional purposes: exchanging ideas about techniques or style variations, training of
participants, and achieving consensus.
▪ Individual preparation before the review meeting is optional.

▪ Review meeting is typically led by the author of the work product.

▪ Scribe is mandatory.

▪ Use of checklists is optional.

▪ May take the form of scenarios, dry runs, or simulations.

▪ Potential defect logs and review reports may be produced.

22 ▪ May vary in practice from quite informal to very formal.

 Technical review
Technical review – a type of formal review by a team of technically-qualified personnel that examines the
quality of a work product and identifies discrepancies from specifications and standards.

Key characteristics:
▪ Main purposes: gaining consensus, and detecting potential defects.
▪ Possible further purposes: evaluating quality and building confidence in the work product,
generating new ideas, motivating and enabling authors to improve future work products, and
considering alternative implementations.
▪ Reviewers should be technical peers of the author and technical experts in the same or other
disciplines.

▪ Individual preparation before the review meeting is required.

▪ Review meeting is optional, ideally led by a trained facilitator (typically not the author).

▪ Scribe is mandatory, ideally not the author.

▪ Use of checklists is optional.

23 ▪ Potential defect logs and review reports are typically produced.

 Inspection
Inspection is the most formal review type.

Key characteristics:

▪ Main purposes: detecting potential defects, evaluating quality and building confidence in the work product, preventing future
similar defects through author learning and root cause analysis.
▪ Possible further purposes: motivating and enabling authors to improve future work products and the software
development process, achieving consensus.
▪ Follows a defined process with formally documented outputs, based on rules and checklists.

▪ Uses clearly defined roles and may include a dedicated reader (who reads the work product aloud during the review
meeting).

▪ Individual preparation before the review meeting is required.

▪ Reviewers are either peers of the author or experts in other disciplines that are relevant to the work product.
▪ Specified entry and exit criteria are used.
▪ Scribe is mandatory.
▪ Review meeting is led by a trained facilitator (not the author).
▪ Author cannot act as the review leader, reader, or scribe.
▪ Potential defect logs and review reports are produced.
24 ▪ Metrics are collected and used to improve the entire software development process, including the inspection process.
 3.2.4 Applying Review Techniques
There are a number of review techniques that can be applied during the individual review (i.e., individual preparation) activity
to uncover defects.

These techniques can be used across the review types described above.

The effectiveness of the techniques may differ depending on the type of review used.

Ad hoc Checklist-based
▪ No guidance for reviewers on how this task should ▪ Reviewers detect issues based on checklists that are distributed at
be done; review initiation;
▪ Reviewers often read the work product
▪ Checklist consists of a set of questions based on potential defects
sequentially, identifying and documenting
(which may be derived from experience);
issues as they encounter them;
▪ Checklists should be specific to the type of work product under
▪ Technique needs little preparation and depends on review;
the reviewer’s skills; ▪ Checklists should be maintained regularly to cover issue types missed
▪ Disadvantage: technique is highly dependent in previous reviews;
on reviewer skills and may lead to many ▪ Advantage: a systematic coverage of typical defect types;
duplicate issues reported by different ▪ Care should be taken not to simply follow the checklist in
reviewers. individual reviewing, but also to look for defects outside the
checklist.

25
 Scenarios and dry runs
Role-based
▪ Reviewers are provided with structured
▪ In a role-based review, similar to
guidelines on how to read through the
the perspective-based reading,
work product;
reviewers take on different
▪ Reviewers are supported in performing
stakeholder viewpoints in
“dry runs” on the work product based on
individual reviewing.
expected usage of the work product;
▪ Roles include specific end user
▪ Reviewers have strong guidelines on
types (experienced,
how to identify specific defect types
inexperienced, senior, child,
(much better than simple checklists);
etc.), and specific roles in the
▪ Reviewers are not constrained to the
organization (user administrator,
documented scenarios in order not to
system administrator,
miss other defect types (e.g., missing
performance tester, etc.).
features).

26
 Perspective-based

▪ Reviewers take on different stakeholder viewpoints in individual reviewing.

▪ Stakeholder viewpoints include end-user, marketing, designer, tester, or operations.

▪ Using different stakeholder viewpoints leads to more depth in individual reviewing with less duplication of
issues across reviewers.
▪ Perspective-based reading also requires the reviewers to attempt to use the work product under review to
generate the product they would derive from it.
For example, a tester would attempt to generate draft acceptance tests if performing a perspective-based
reading on a requirements specification to see if all the necessary information was included.
▪ In perspective-based reading, checklists are expected to be used.

Empirical studies have shown perspective-based reading to be the most effective general
technique for reviewing requirements and technical work products.

A key success factor is including and weighing different stakeholder viewpoints appropriately,
based on risks.
27
 3.2.5 Success Factors for
Reviews
Organizational success factors for reviews include:

▪ Each review has clear objectives, defined during review planning, and used as measurable exit criteria.

▪ Review types are applied which are suitable to achieve the objectives and are appropriate to the type and level of
software work products and participants.
▪ Any review techniques used, such as checklist-based or role-based reviewing, are suitable for effective defect
identification in the work product to be reviewed.
▪ Any checklists used address the main risks and are up to date.
▪ Large documents are written and reviewed in small chunks so that quality control is exercised by providing authors with
early and frequent feedback on defects.

▪ Participants have adequate time to prepare.

▪ Reviews are scheduled with adequate notice.

▪ Management supports the review process (e.g., by incorporating adequate time for review activities in project schedules).

▪ Reviews are integrated in the company's quality and/or test policies.

28
 People-related success factors for reviews include:

▪ The right people are involved to meet the review objectives, for example, people with different skill sets or perspectives,
who may use the document as a work input.
▪ Testers are seen as valued reviewers who contribute to the review and learn about the work product, which enables
them to prepare more effective tests, and to prepare those tests earlier.

▪ Participants dedicate adequate time and attention to detail.

▪ Reviews are conducted in small chunks so that reviewers do not lose concentration during individual reviews and/or the
review meeting (when held).
▪ Defects found are acknowledged, appreciated, and handled objectively.

▪ The meeting is well-managed so that participants consider it a valuable use of their time.

▪ The review is conducted in an atmosphere of trust; the outcome will not be used for the evaluation of the participants.

▪ Participants avoid body language and behaviors that might indicate boredom, exasperation, or hostility to other
participants.

▪ Adequate training is provided, especially for more formal review types such as inspections.

▪ A culture of learning and process improvement is promoted.

29