Chapter – 4 Software Testing Artefacts

Certificate in Software Testing Skill

Page 1 of 46
Page 2 of 46
 Confidentiality Statement

This document should not be carried outside the physical and virtual boundaries of TCS and
its client work locations. Sharing this document with any person other than a TCS associate
would tantamount to violation of confidentiality agreement signed by you while joining
TCS.

Notice
The information given in this course material is merely for reference. Certain third party
terminologies or matter that may be appearing in the course are used only for contextual
identification and explanation, without an intention to infringe.
Certificate in Software Testing Skill TCS Business Domain Academy

Contents
Chapter-4 Software Testing Artefacts ............................................................................ 5
4.1 Traceability Matrix, Test Strategy and Test Plan....................................................6
4.2 Test Design, Test case and Test Script. ................................................................ 19
4.3 Few more Testing Artefacts ................................................................................. 41
Summary ........................................................................................................................ 43
References ...................................................................................................................... 45

Page 4 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Chapter-4 Software Testing Artefacts

Introduction
The application developed into different phases of software testing life cycle and shared
with the stake holders are known as Test Artefacts. Normally the software test team should
make these artefacts and they are to take sign off on those artefacts from the stake holders
to ensure that there is no communication gaps between customer and testing team.

Learning Objective
After reading this chapter, you will be able to understand:
• Traceability Matrix
• Test Strategy
• Test Plan
• Test Design
• Test Case
• Test Scripts
• Test Scenario
• Test fixture & Data
• Test Harness
• Test Summery Report
• Requirement Mapping to Test Case

Page 5 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

4.1 Traceability Matrix, Test Strategy and Test Plan.

4.1.1 Traceability Matrix

A traceability matrix is term used for a document, which is generally in form of a table,
which correlates any two baseline documents based on many to many relationship to
validate the completeness of the relationship. It is often used along with high level
requirements documents which are part of project documents.

In other words, a requirements traceability matrix (RTM) is a document that traces and
maps user requirements (requirement Ids from requirement specification document) with
the test case ids to meet the purpose that all the requirements are covered in test cases so
that while testing no functionality can be missed.

A RTM is generally used to check if the necessary current project requirements are being
met. Further, it helps in the creation of variety of project deliverable documents such as
request for proposal (RFP), project plan and so on. Traceability is an important, relatively
simple testing concept. Tools handle traceability matrix mechanics, but not content.
Human beings must indicate each cross-reference to the tool, e.g., each module where the
requirement is implemented and each test of the module's implementation of the
requirement .This is where traceability matrix issues arise and capabilities are over-stated.
These comprises of marketing requirements and detailed requirements of the software
product which are linked with other artefacts such as high level design, detailed design, test
plan and test cases.

Basically, tracing requirements to where they are implemented and then to where they are
tested helps assure all requirements have been met and can identify extra usable work
needed without corresponding requirements.

Usage of traceability matrix ensures that requirements captured are complete and all lower
level requirements originate from higher level requirements which are the linked with each
other. It is also used in managing change control process and provides basis for test
planning as well.

Page 6 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Types of traceability matrix

Forward Traceability: Mapping of the requirements to test cases.
Backward Traceability: Mapping of test cases to requirements.
Bi-Directional Traceability: Bi-Directional Traceability contains both Forward & Backward
Traceability

Bi-Directional Traceability contains both Forward & Backward Traceability. This is helpful in
identifying test cases that do not trace to any requirement and should be removed.
Backward traceability is very helpful in cases where you are required to identify the number
of requirements which a particular test case is covering. Forward traceability helps in
identifying which requirements covered in which test case. Forward traceability ensures
that right product is being build which will meet the end-user requirements completely.

Figure 1 Forward and Backward Traceability

Benefits of using Traceability Matrix

Following are benefits of using Traceability matrix:
• Gives confidence to customer that software is being developed as per requirement.
• Ensures that all requirements are covered by test cases and unnecessary test cases
are not part of test plan.
• Helps in locating missing functionalities.
• Change management becomes easier as once a requirement is changed, it can be
easily found which test cases needs to be updated.

Absence of traceability matrix may result in poor test coverage resulting in more defects in
production. It may also result in bugs from previous test cycles getting carried forward to

Page 7 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

current cycle which may impact cost and schedule of the project. Further, it may lead of
inefficient project planning, conflicts between different team over project dependencies
and delay in project completion.

Steps to create Traceability Matrix

Step 1. Make use of MS-excel to create Traceability Matrix docs:
Step 2. Define following columns:
• Base Requirement ID (If any)
• Requirement ID
• Requirement description
• TC 001
• TC 002
• TC 003 and so on.

Step 3. Identify all the testable requirements in granular level from requirement document.
Typical requirements you need to capture are as follows:
• Used cases (all the flows are captured)
• Error Messages
• Business rules
• Functional rules
• SRS
• FRS and so on.

Step 4. Identity all the test scenarios and test flows.

Step 5. Map Requirement IDs to the test cases. Assume (as per below table), Test case “TC
001” is your one flow/scenario. Now in this scenario, Requirements SR-1.1 and SR-1.2 are
covered. So mark “x” for these requirements.
Now from below Table 1 you can conclude –
Requirement SR-1.1 is covered in TC 001
Requirement SR-1.2 is covered in TC 001
Requirement SR-1.5 is covered in TC 001, TC 003 [Now it is easy to identify, which test cases
need to be updated if there is any change request].

Page 8 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

TC 001 Covers SR-1.1, SR, 1.2 [we can easily identify that test cases covers which
requirements].
TC 002 covers SR-1.3 and so on.

Table 1 Basic Traceability Matrix format

Requirement ID Requirement description TC 001 TC 002 TC 003
SR-1.1 User should be able to x
do this
SR-1.2 User should be able to x
do that
SR-1.3 On clicking this, x
following message
should appear
SR-1.4 x
SR-1.5 x x
SR-1.6 x
SR-1.7 x

This is a very basic traceability matrix format. We can add more following columns and
make it more effective.

4.1.2 Test Strategy

Test strategy refers to the test approach which defines how testing will be carried out.
There are two techniques for creating test strategy:
• Proactive: In proactive approach, the process of test design is started as early as
possible in software development lifecycle (SDLC). This enables to find and fix the
defects much before the release build is created.
• Reactive: In reactive approach, the process of test design is not stated until design
and coding phase of SDLC is not completed

Test Strategy document is a high level document and generally developed by project
manager. Test Strategy documents defines testing approach to achieve testing objectives
and it’s normally derived from the Business requirement specification documents.

Test strategy document is not updated too often and hence also referred as a static
document. It enforces the standards for entire testing process and other related activities

Page 9 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

such as test plan creation. Few organizations includes the test strategy as part of test plan
for small projects. However, for large projects, there is usually one test strategy document
and different number of test plan for each level of testing.

Components of the Test Strategy documents are

• Scope and Objectives

• Business issues
• Roles and responsibilities
• Communication and status reporting
• Test deliverables
• Industry standards to follow
• Test automation and tools
• Testing measurements and metrics
• Risks and mitigation
• Defect reporting and tracking
• Change and configuration management
• Training plan

4.1.3 Test Plan

A software project test plan is a document that describes the objectives, scope, approach,
and focus of a software testing effort.

Test planning is the most important activity to ensure that there is initially a list of tasks and
milestones in a baseline plan to track the progress of the project and also defines the size of
the test effort.

Test Plan Types

Below are three types of test plan
• Mater Test Plan: - A single high level test plan for product/project that unifies all
other test plans.
• Specific Test Plan (Testing Level): - Plans for each level of testing.

Unit Test Plan

Integration Test Plan

System Test Plan

Page 10 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Acceptance Test plan

• Specific Test Plan (Testing Type): - Plan for major types of testing like security and
performance test plan.

Test Plan Template/Identifiers

The format of software test plan and its content vary depending on the set of process,
standards and tools used for test execution. The following format is based on Institute of
Electrical and Electronics Institute (IEEE) standard for software test plan documentation.
1. Test Plan identifier: This refers to Unique identify in to reference test plane.

2. Introduction: This provides a brief introduction of the project

3. Test items: This refers to a software test item which is part of s the application to

be tested test.
4. References: This refers to list the related documents with links to them if available,
including, but not limited to, project plan and configuration Management plan.
5. Features to be tested: This refers to set of feature that are to be tested

6. Features not to be tested: This refers to the features which are not to be tested
and alsoand the reasons for not including as part of testing.
7. Approach: This refers to details about the overall approach to testing.

8. Item pass/fail criteria: This refers to documentation instance indicating whether a

software item has passed or failed its test.
9. Test deliverables: This refers to the he deliverables that are delivered as part of the

testing process, such as test plans, test specifications and test summary reports.
10. Testing tasks: This refers to activities which are documented for planning and
executing the testing.
11. Environmental needs: This refers to the environmental requirements such as

hardware, software, OS, network configurations, tools required.

12. Responsibilities: This refers to lists of the roles and responsibilities of the team

members.
13. Staffing and training needs: This refers to documenting the actual staffing
requirements and any specific skills and training requirements.
14. Schedule: This refers to the important project delivery dates and key milestones.
15. Risks and Mitigation: This refers to high-level project risks, assumptions and a
mitigating plan for each identified risk.

Page 11 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

16. Assumptions and Dependencies: This refers to list all the assumptions &
dependencies that have been made during the preparation of this plan
17. Approvals: This refers to list of all approvers of the document, their titles and the

sign off date.

Test Plan Benefit

• Test plan assists in calculating the effort which is required to validate the quality of
the software application being tested.
• Test plan provides insight to application to team (such as developers, business
managers and so on) external to testing team testing the software.
• Test plan drives thought process of developers and testers and acts as a rulebook
which should be followed to deliver a quality artefact.
• Test plan covers significant aspects of project, such as test estimation, test scope,
test strategy and so on, which can be monitored by management team and can be
used as benchmark for other projects.

How to make a Test Plan

Following are the steps to prepare a test plan.
Step 1: Analyse the product
Step 2: Design test Strategy
Step 3: Define Test Objectives
Step 4: Define Test Criteria
Step 5: Resource planning
Step 6: Plan test environment
Step 7: Schedule & Estimation
Step 8: Determine Test Deliverables

Step 1: Analyse the product

Without any information about product you cannot test product so you must learn a
product thoroughly before testing it.. One should perform thorough research about end
user and client of product to understand their expectations from product. One can use the
following approach Ref. below Pics.) to analyse.

Page 12 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Figure 2 : Approach for analyze

If one reviews the product documentation, it helps to understand different features of the
products in addition to how to use the same. In case of any ambiguity, one may introspect
the same with either customer, developer or designer to get more information about the
same.

Step 2: Design test Strategy

Test strategy a high level document, is a critical step in making a test plan and usually
develop by Test Manager. This document provide details about the project objective to test
the deliverables and also how to achieve the same with required cost and effort.
For developing Test strategy you should follow steps below

Figure 3 : Steps for Test Strategy

Defining Scope of Testing

Scope should me know before starting any test activities. It is important to define scope of
testing project as it is essential for all stakeholders.

To determine scope, one must assess and document the following:

• Product Specification
• Project Budget
• Precise customer requirement

Page 13 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

• Skills & talent of your test team

Identify Testing Type

A Testing Type refers to a test procedure that gives an expected test outcome. Each testing
type is formulated to identify a specific type of product defects. But, all Testing Types are
aimed to achieve one common goal which is “Early detection of all the defects before
releasing the product to the customer”.

Documents Risk & Issues

Risk refers to an uncertain event which may occur in future and is associated with a
probability of occurrence and a potential for loss. When the risk actually manifests, it
becomes an issue to be handled carefully without impacting the project outcome.

Create Test Logistics Document Risks & Issues

In test logistic the test Manager should answerable for the question “who will test.? “, and
“When will the test occur?”

Who will test?

One may not be able to know exactly which tester will test what feature but it is beneficial if
the type of tester needed can be identified in advance. To select the right member for right
task, one may have to consider if his/her skill is match for the activity to be performed. This
also helps to estimate the budgetary requirements as each test resource is associated with a
cost. Mismatch in assigning tester may cause delay in project schedule and may even result
in project failure.

Following skills are desirable in tester to perform the testing activity:

• Ability to understand customer’s point of view. One should be able to understand
the point of view of end-user i.e. customer.
• One should be aware of quality aspect of software
• One should pay attention to minute details while performing his activities
• One should co-operate with different stakeholders to achieve the project objective.

In the project, tester is team member who will take charge for test execution and related
activities such as reporting, retesting, preparing reports and so on. He/she can be part of the

Page 14 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

organization or external to organization in case activity is being outsourced. When will the
test occur?

Test activities must be co-related with development activities. One should start test only
when all necessary inputs as shown in Figure 4 are available.

Figure 4: Test Activities

Step 3: Define Test Objectives

Test objective refers to complete goal and achievement of the test execution process. The
key objective of the test activity is to find as many software defects as possible so that the
final released software is but free.
To define the test objectives, one should perform the following steps:
• Prepare a list of all features (such as functionality, performance, interface and so on)
which are to be tested
• Articulate the final objective of the testing activity based on feature list.

Step 4: Define Test Criteria

.Test criteria refers to a standard or rule on which a test procedure can be based.
Following are two types of test criteria.

1. Suspension Criteria

In case the suspension criteria is met during the testing process, the active test cycle should
be suspended and should be resumed only when the suspension criteria is resolved.

For example, if one test team member reports that 60% of test cases have failed, one
should suspend testing till the development team fixes all the defects so that test cases are
passed.

Page 15 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Figure 5 Suspension Criteria

2. Exit Criteria

Exit criteria refers to the criteria that testifies a successful completion of a test phase.
Unless the exit criteria is not satisfied, one should not move to next phase of activity. For
example, one test criteria could be that 95% of critical test cases must pass before moving
to next phase of project activity. In few cases, exit criteria are specified in terms of targeted
run rate and pass rate.
• Run rate refers to ratio between numbers of test cases executed and total number
of test cases. For example, let’s assume that test specification has 120 test cases but
the tester has executed only 100 test cases so far, then the run rate is 100/120 = .83
(83 %).
• Pass rate refers to ration between number of test cases passed and total number of
test cases executed. For example, in previous example, out of 100 test cases, 80 has
been passed. Then the pass rate is 80/100 = 0.8 (80%).

For example, if test tam has already completed the test case execution, they report the test
result and may ask you to confirm if the exit criteria is met or not. In the above case, the run
rate is mandatorily 100% but the test team has completed only 90% of test cases. Hence
the run rate is not satisfied and hence you cannot confirm that exit criteria has been met.

Page 16 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

The test team has to complete the remaining test cases to enable you to confirm that exit
criteria has been met and test team can move to next stage/phase of project.

Step 5: Resource planning

Resource planning refers to detailed description of all resources needed to complete the
project activity. The resources can be human resources, physical equipment and machines
needed or any other resource which is associated a well-defined cost. Resource planning is
significant factor of the test planning phase because it assists in determining the number of
resources needed for the project. This in-turn helps test manager to create accurate
schedule and estimation for the project. Test manager, Tester, Developer in test, Test
Administrator, SOA members are required under Human resource and system resources
like Computer, server, testing tools, Network planning required under Hardware stuff.

Step 6: Plan test environment

Test environment refers to the physical combination of software and hardware on which
the test team is supposed to perform test execution. Such an environment consists of actual
business environment such as servers, databases and networks. Test environment emulates
the actual environment where the end use might perform different activities upon the end
product.

Step 7: Schedule & Estimation

Creation of schedule and estimation is common activity in project management. By
creating a realistic schedule and estimation (both cost and time), test manager can use the
same to effectively track and monitor the progress of the project and prevent cost and
schedule overrun within the project. Test manager requires several inputs to create the test
schedule and estimation which are highlighted below:
• Employee availability and project timeline: This includes the number of working
days for which test resource is available, the timeline by which the project has to be
completed and so on.
• Project estimation: Based on initial estimation made by stakeholders, test manager
has to validate the estimate and refine it if needed. This is must do activity with due
diligence so that a realistic project schedule is prepared.
• Project risk: This includes the possible risks which can manifest during project. Each
risk needs to assess for its probability and impact and subsequently, a mitigation

Page 17 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

plan to be identified. Project schedule should also be appropriately refined taking

the risk into account.

Step 8: Determine Test Deliverables

Test deliverables refer to list of all documents, tools and other artefacts that are to be
created and maintained to support the testing activity. Each phase of software
development lifecycle results in different test deliverables/artefacts.

Figure 6 : Test deliverable stage

Following deliverables are needed/generated by test team before the test phase begins:
• Test cases documents
• Test plans document.
• Test design specifications

Following deliverables are needed/generated by test team during the test phase:
• Test Scripts
• Simulators.
• Test Data
• Test Trace-ability Matrix
• Error logs and execution logs.

Following deliverables are needed/generated by test team after testing phase is over.
• Test Results/reports
• Defect Report
• Installation/ Test procedures guidelines
• Release notes

Page 18 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

4.2 Test Design, Test case and Test Script.

4.2.1 Test Design

Design refers to activity where a plan is created for how to implement an idea and/or
identifying and documenting a method to perform a task. Test design means identifying
and documenting what set of inputs for the given software will result in what set of
expected outputs. This ensures that system is working as expected and it can be released
for use by an end user. Test design techniques are used to identify test conditions and test
scenarios which will result in effective and efficient test cases. Using test design techniques
is better approach instead of identifying test cases out of the air. Such techniques also result
in achieving high test coverage.

Test design techniques are classified into three categories:

• Specification based (Black box) technique

• Structure based (White box) technique
• Experience based technique

Specification based technique

Specification based technique is also called as Black box testing where test cases are
derived based on the requirement specification.

Below are the technique used while deriving test cases from a requirement specification
document. Equivalence Partitioning (EP)
• Boundary Value Analysis(BVA)
• Decision Table Testing
• State Transition Diagrams
• Use Case Testing
Equivalence Partitioning (EP): Equivalence partitioning refers to the software testing
technique that segregates the input test data needed for testing application, into multiple
partition in such as a way that at least once of equivalent data from which test cases can be
derived. It is also called as equivalence class partitioning and abbreviated as ECP.

This technique assists in identifying test cases that may expose different class of errors
which results in reducing the total number of test cases to be developed. In few cases, this

Page 19 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

technique is applied to output of a software component but otherwise, it is applied to the

inputs of software components. The equivalence partitions are usually derived from the
requirements specifications for input data which influences the processing of the test
object. An input has pre-defined range which are valid and other set of ranges which are
invalid. In this context, invalid data does not mean that data is incorrect. Instead it means
that the data lies outside of specific partition.

An advantage of this approach is that the time required for testing of a software is reduced
due to decreased number of test cases.

Test design guide lines

• If an input condition specifies range, one valid and two invalid equivalence classes
are needed.
• If a condition requires a specific value, one valid and two invalid equivalence classes
are needed.
• If an input condition specifies a member of a set, one valid and one invalid
equivalence class are needed.
• If an input condition is Boolean, one valid and one invalid equivalence class are
needed.

Example:
EP may be best explained with an example of a function which has the pass parameter
"month" of a date. Total number of months is 12 and the input ranges from 1 to 12.As
discussed we have one valid range and two invalid ranges. The valid range for the month
would be 1 to 12, standing for January to December and this valid range is called a valid
partition. Moving further we can extract two invalid ranges.

The first range would be <= 0 and the second would be >= 13
.... -2, -1, 0, 1 ...................12, 13, 14, 15.....
---------------|-------------------|---------------------
Invalid partition1 valid partition Invalid partition2

Page 20 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Equivalence partitioning is no standalone method to determine test cases. It has to be

supplemented by another method say Boundary value analysis method.

Boundary Value Analysis (BVA):

BVA is a black box testing technique where tests are performed using the boundary values.
The behaviour of system at the edge of each equivalence partition is more likely to be
incorrect, hence boundaries are the areas where testing is likely to yield defects. Values can
be on the edge of an equivalence partition or at the smallest value on either side of an edge.
BVA is applicable at all different test levels. It is relatively easy to apply and its defect
finding capability is high. Sometimes, this technique is also considered as an extension of
equivalence partitioning.

Guide lines
• Focus on the boundaries of the input
• If input condition specifies a range bounded by a certain values, say a and b, then
test cases should include
- The values for a and b
- The values just above and just below a and b
• If an input condition specifies any number of values, test cases should be
- The minimum and maximum numbers,
- The values just above and just below the minimum and maximum values.
Example:
Consider the same example discussed earlier where function has the pass parameter
"Month" of a date. As we know
Valid Class is 1<= month <=12
Invalid Class 1 is month <1
Invalid Class 2 is month >12

EP says select any test case within a range and any on either side of it. In this case between
1 and 12, <1 and >12 .But in BVA the emphasis is on the edges of the partitions.
In this example we have,
-> 1 and 12 as the edges of the valid class
-> 7(a middle value) for the valid Class
-> 0 and 13 as edge values for the Invalid class

Page 21 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Experience has revealed that test cases derived based on boundary values or just around it
are most likely to detect more defects.

Decision Table Testing: In decision table testing test cases are derived in order to execute
the combination of inputs. Decision tables are good way to capture system requirements
comprising logical conditions. It are also used to record complex business rules that a
system is to implement. It assists in documenting internal system design. The given
specifications are analysed and conditions and actions of the system are identified and
subsequently documented. The decision table also comprise of the triggering conditions
and the resulting actions for each combination of conditions. Each column of the table
corresponds to a business rules which identifies a unique combination of logical conditions
that result in the execution of the actions associated with that rule. The coverage standard
usually used with decision table testing is to have at least one test per column which usually
results in covering all combinations of triggering conditions.

The four quadrants of decision table are 1.Conditions, 2.Condition alternatives, 3.Actions,
and 4.Action entries.

Decision table captures all the combinations of inputs and outputs. Decision table is
extracted based on the state transition graphs or Cause effect graph. We have explained
State Transition diagram but what is this Cause effect graph? No need to wonder as here
comes the information about C-E graph which is as simple as it looks

A Cause-Effect graph is a directed graph that maps a set of causes to a set of effects.
Captures the relationships between specific combinations of inputs (causes) and outputs
(effects). The causes can be considered as input to the program and the effect can be
considered as output. Generally, the graph depicts the nodes representing the causes on
the left side and the nodes representing the effects on the right side. There may be
intermediate nodes in between. These intermediate nodes usually combines inputs using
Boolean logical operators such as AND and OR operator. The graph's direction is as follows:

Figure 7 : Cause Effect Graph Flow

Page 22 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Advantages of C-E graph are

• Deals with specific cases,
• Avoids combinatorial explosion
• Explore combinations of possible inputs

This Cause-Effect graph method captures the relationships between specific combinations
of inputs (causes) and outputs (effects).
• Deals with specific cases,
• Avoids combinatorial explosion.
• Explore combinations of possible inputs

Causes/effects are represented as nodes of a cause effect graph. The graph also includes a
number of intermediate nodes linking causes and effects.

Guide lines to create Cause-Effect graph:

1. Study the functional requirements and divide the requirements into workable
pieces
2. Identify causes and effects in the specifications
a. Causes: distinct input conditions
b. Effects: an output condition or a system transformations.
3. Assign unique number to each cause and effect
4. Use the semantic content of the spec and transform it into a Boolean graph
5. Annotate the graph with constrains describing combinations of causes and/or
effects
6. Convert the graph into a limited-entry decision table
7. Decision table rules are converted to test cases

Representation of C-E graph with an example

Consider the following Example regarding the Requirements for Calculating Car Insurance
Premiums:

Note: R00101-R00104 represents the requirement number

R00101: For females less than 65 years of age, the premium is $500
R00102: For males less than 25 years of age, the premium is $3000

Page 23 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

R00103: For males between 25 and 64 years of age, the premium is $1000
R00104: For anyone 65 years of age or more, the premium is $1500
Input conditions and the output conditions for this set of requirements can be mapped as.

Figure 8: Input and Output Conditions

Figure 9 : C-E Graph

Page 24 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Figure 10 : Constraint Symbols in C-E Graph

“O” constraint example

Figure 11: Example of “O” constraint

Page 25 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Guidelines for Decision table

The below rules can be followed for Test Case Generation
• Conditions: inputs
• Actions: outputs
• Rules: test cases

The above conditions have the following implications

• Rules are complete i.e., every combination of decision table values including default
combinations are inherent in the decision table
• The rules are consistent i.e., there is not two actions for the same combinations of
conditions

A decision table is consists of a number of columns (rules) that comprise all test situations.
This is how decision table looks for the above example.

Figure 12: Decision Table

Deriving the Test Cases

Some CEGs may result in more than one test case being created. For example, in below
table CEG #3 resulted in columns 3 and 4 and CEG#4 resulted in columns 5 and 6 in the
decision table. The columns in the decision table are converted into test cases.

Page 26 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Figure 13: Test cases

State Transition Diagrams: This is a black-box testing technique where output is triggered
by changes to input condition which is also referred as changes to the state of the system.

A system may produce different responses depending on the current or previous conditions
(history or its stage). This aspect of the system can be depicted using State Transition
Diagram. It allows the tester to visualize the software in terms of its states, inputs or events
that results in transitions, outcome of these transitions and transition between different
states. A state table describes the relationship between the states and inputs and also the
states which are invalid. Test cases can be designed to cover all different states and their
transitions. This technique is best suited for modelling a business object having specific
states or testing screen dialogue flows.

When to use state transition diagrams?

• The project team have well documented the sequence of events that may occur and
all conditions associated with each of these events.

Extracting Test cases:

• Step1: This involves interpreting different state and their transitions. Further, each
state transition is marked as either valid or invalid.
• Step3: In this stage, each of state which has been visited during traversal should be
documented. Step4: Steps 2 and 3 should be repeated until all states have been
visited and all transitions traversed.

Page 27 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

• Step5: In this stage, actual input and output values for all test cases has to be
generated and documented

An advantage of this approach are as follows:

• It familiarize the testers with the software design to enable them to create and
design effective and efficient test cases
• It enables testers to cover all states (including unplanned or invalid states)

Example:
State transition of a system can be diagrammatically as illustrated in Figure 14.

Figure 14: State transition of a system

The tests are derived from the above state and transition and Figure 15 describes the
possible scenarios that need to be tested.

Figure 15: Possible scenarios for testing.

Use Case Testing:

In software engineering, use case refers to description of steps or actions between a user
(also referred as actor) and a software system. The user or actor can be either a person or
any abstract entity such as an external software system or a manual process.

Page 28 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Simply stated, use case testing is functional black box technique. This technique assists
testers in identifying test scenarios that describes the entire system on transaction basis
from begin till end of system execution.
Test cases are derived to execute user scenarios. Use cases describe the process flow in a
system and hence the test cases derived from use cases are most useful in identifying
defects in the process flows during real-world use of the system. They also help to find out
integration defects caused by the interaction and interference of different components,
which individual component testing would not see.
Use cases, often referred to as scenarios, are very useful for designing acceptance tests with
customer/user participation. Best example can be a Chocolate Vending Machine.

Structure based technique

Structure based technique is also known as White box testing. Test cases are derived based
on the identified structure on which software has built.
• Following are possible techniques that can be used while deriving test cases directly
from the structural description of system. Data flow testing
• Decision testing and coverage
• Path testing
• Statement testing and coverage

Data flow testing: Data-flow testing looks at the life-cycle of a particular piece of data (i.e. a
variable) in an application .Test cases are designed to check availability of the data. By
looking for patterns of data usage, risky areas of code can be found and more test cases can
be applied.

Decision Testing and Coverage: Decision coverage, related to branch testing, is the
assessment of the percentage of decision outcomes that have been exercised by a test case
suite. E.g. the True and False options of an IF statement. Decision testing derives test cases
to execute specific decision outcomes, normally to increase decision coverage. Decision
testing is a form of control flow testing as it generates a specific flow of control through the
decision points.

Page 29 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Statement testing and coverage: In component testing, statement coverage is the

assessment of the percentage of executable statements that have been exercised by a test
case suite. It indicates the total number of statements being executed. Statement testing
derives test cases to execute specific statements, normally to increase statement coverage

Path Testing: Path testing has been one of the first test methods implemented in white box
testing process. This procedure is similar to walk through activity. First of all, a selected
path through the application program is chosen. This is followed by documenting possible
set of inputs and corresponding correct outputs. Then the program is executed manually
and the output result is compared to the predefined result.

Experience based technique

Experience based technique is completely based on the past experience and knowledge
about the software and its errors.
• Error Guessing
• Exploratory Testing

Error Guessing:
This is a test case design technique where tester has to guess in advance what possible
errors may occur and then design the test cases accordingly to represent/cover them. This
technique relies on ability of tester to guess based on his prior experience. This is an adhoc
method to identify tests which are likely to expose errors based on experience and intuition.
The key objective of this technique is to focus on areas that have not been handled by other
more formal techniques such as equivalence partitioning and boundary value analysis. This
process involves making an educated guess based on items such as data from previous
texting experience, tester’s capability to identify situations in Functional Design
Specifications or Detailed design specifications. Most of such items are not addressed
clearly during testing lifecycle.

Exploratory Testing In test lifecycle, there are situations where there is tough trade-off
between time and planned schedule, this technique is adopted which combines the
expertise of testers along with a well-defined and structured approach to testing.
Exploratory testing often performed as a black box testing technique, the tester learns
things that together with experience and creativity generate new good tests to run.

Page 30 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

It can be described as simultaneous learning test design and test execution. It provides
tester the freedom to undergo learning of test design, test execution and test result
interpretation in parallel with the project. While the software is being tested, the tester
learns things that together with experience and creativity generates new good tests to run.
The key here is the cognitive engagement of the tester, and the tester's responsibility for
managing his or her time.

Need of Test Design

The Quality of Testing depends on how good the Test Design i.e. Success of testing
depends on the way in which the test cases are designed. Test cases are derived from the
Input Documents called as Test Basis using Formal and Customized Test specification
techniques. Deriving of Test Cases using specific techniques helps to achieve.
• Proper Coverage of testing during functional and system testing
• Good depth while testing individual modules or functions of the system
• Uniformity among the test specifications prepared by members of the team
• Easy manage of Test Cases.

4.2.2 Test Case

A test case in software engineering is a set of conditions or variables under which a tester
will determine whether an application or software system is working correctly or not. It has
components that describes an input, action or event and an expected response, to
determine if a feature of an application is working correctly. The process of developing test
cases can also help find problems in the requirements or design of an application.

Stating differently, a test case is an artefact (or document) which comprises of test data, set
of preconditions, expected results and post conditions for a specific test scenario. This is
used to verify compliance of the system against a specific and well defined requirement.

Test cases are often referred to as test scripts, particularly when written. Written test cases
are usually collected into test suites

Page 31 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Objective of writing Test cases

• The basic objective of writing test cases is to validate the testing coverage of the
application
• Test cases ensure the proper test coverage of an application
• In some scenarios, writing test cases brings some sort of standardization and
minimizes the ad-hoc approach in testing
• Gives better reliability in estimating the test effort
• Improvement in the productivity during test execution
• Easy understanding of the application’s operation

In practice, we have two classes of test cases

• Formal
• Informal

Formal test cases are written based on the requirement document of the application
provided.

Informal test cases are written for the applications or systems without formal
requirements. In such case, test cases are written based on the accepted operation of the
application or system.

Test Case Template

A test case can have the following elements. However, that normally a test management
tool is used by companies and the format is determined by the tool used.

Table 2 Test Case Template description

Test Suite ID The ID of the test suite to which this test case belongs.
Test Case ID The ID of the test case.
Test Case
The summary / objective of the test case.
Summary
Related
The ID of the requirement this test case relates/traces to.
Requirement

Prerequisites Any prerequisites or preconditions that must be fulfilled prior to executing

Page 32 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

the test.

Test Procedure Step-by-step procedure to execute the test.

The test data, or links to the test data, that are to be used while
Test Data
conducting the test.
Expected Result The expected result of the test.

Actual Result The actual result of the test; to be filled after executing the test.

Pass or Fail. Other statuses can be ‘Not Executed’ if testing is not

Status
performed and ‘Blocked’ if testing is blocked.

Remarks Any comments on the test case or test execution.

Created By The name of the author of the test case.

Date of Creation The date of creation of the test case.

Executed By The name of the person who executed the test.

Date of
The date of execution of the test.
Execution
Test The environment (Hardware/Software/Network) in which the test was
Environment executed.

Table 3 Test Case Sample

Test Suite ID TS001

Test Case ID TC001
Test Case
To verify that clicking the Generate Coin button generates coins.
Summary
Related
RS001
Requirement
1. User is authorized.
Prerequisites 2. Coin balance is available.

1. Select the coin denomination in the Denomination field.

2. Enter the number of coins in the Quantity field.
Test Procedure
3. Click Generate Coin.

Page 33 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

1. Denominations: 0.05, 0.10, 0.25, 0.50, 1, 2, 5

Test Data 2. Quantities: 0, 1, 5, 10, 20

1. Coin of the specified denomination should be produced if the

specified Quantity is valid (1, 5)
Expected Result 2. A message ‘Please enter a valid quantity between 1 and 10’ should
be displayed if the specified quantity is invalid.

1. If the specified quantity is valid, the result is as expected.

2. If the specified quantity is invalid, nothing happens; the expected
Actual Result
message is not displayed

Status Fail
Remarks This is a sample test case.
Created By Manish Varma
Date of Creation 02/14/2016
Executed By VP Gulati
Date of
02/25/2016
Execution
• OS: Windows Y
Test
• Browser: Chrome N
Environment

Writing a good test cases

• Test cases should be written in such a manner that one test only one thing at a time.
There should be no overlap or complicates test cases. Overall, one should attempt
to make the test cases ‘atomic’.
• One should ensure that all test scenarios are covered (both positive and negative)
• One should use simple and easy to understand language.
• Following are characteristics of a good test case:
o It should be accurate and test the feature as developed/documented
o There should be no unnecessary information
o It should be traceable to requirements
o It can be used to perform the same test repeatedly

Page 34 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

o It should be reusable (if needed in future)

o It should be self-explanatory with minimal ambiguity to tester during
execution
o It should be well commented. Comments should also be included in test
scripts to be used for purpose of testing.

Tips on Test Case Writing

• Always tend to write short test cases(limit to maximum of 10 steps)
• Maintain Coherence. There should exist a logical flow between test cases
• Don't skip steps assuming that it is “understood”
• Don't confuse between Action & Results. For e.g.
• “The button is pressed”
• Use spread sheet or table format to write test cases instead of writing the same in
paragraph format
• Follow a logical flow , but start writing test cases for those requirements that are
frozen and make a mark like TBD(to be defined) for those unclear sections
Sample Condition to write a test case

Figure 16 User Registration Screen

Page 35 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Sample Test Case of above

Figure 17 Sample Test case for user registration Screen

Requirement mapping to test cases using testing tool Quality center (QC)
Brief note on QC
Quality Centre is a test management tool used to manage the entire test process. It is a web
based centralized repository for the management of TDLC. It provides methodology,
organization, structure and documentation for all phases of application testing process.

Different phases in QC are

1. Application model
2. Releases
3. Business component
4. Requirements
5. Test plan
6. Test lab
7. Defects
8. Report
9. Dash Board

Page 36 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Once requirements are gathered together we map them to corresponding test cases and
execute them in the test lab. Requirements are mentioned along with its unique id and
description in the requirements tab.

Figure 18: screen shot from a live project HP QC Screen.

As we can see in Figure 18,
• Entity Name field has T.F.SC.Req.1 – Customers as its entry. This is the
Requirement Id
• Req. Description field refers to the brief explanation of customer requirements

Mapping of Requirements to test cases

Once mapping is done test cases along with the status is displayed when we click on the
particular requirement in the requirement tree. In QC mapping of Requirement to Tests is
done as explained below
1. In requirements TAB select coverage view.
2. Select requirement by clicking on Parent/Child or grand Child.
3. On right hand side (in Coverage View Window) another window will appear.
4. It has two TABS (a) Tests Coverage (b) Details. Test Coverage TAB will be
selected by default or you click it.
5. Click on Select Tests Button. A new window will appear on right hand side and
you will see a list of all tests. You can select any test case you want to map with you
requirement.

Page 37 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Snap shot of QC after mapping requirement to its test cases

Figure 19 Snap shot of QC after mapping requirement to its test cases

Page 38 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Figure 20 Screen Shot QC

4.2.3 Test Scripts

A test script refers to a set of instructions which are written using a scripting/programming
language. These set of instructions are performed on system under test to validate that the
system behaviour is as expected. Test scripts are widely used in performing automated
testing. In few cases, the set of instructions can be written normal language. This is
prevalent in manual testing. However, in such cases, it should be referred as test case

Page 39 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

instead of test script. Some of the scripting/programming language used in automated

testing are as follows:
• Tcl/Tk
• Perl
• Python
• JavaScript
• UNIX shell script
• VBScript
There are also many test automation tools/frameworks which generate the test scripts
without the need for manual coding. Most of these tools have their own scripting
languages. For example, Sikuli is a GUI automation tool which uses Sikuli script which in
turn is based on Python language. A sample script is shown below:
Def sample script (s1);
(“Text B”);
Click (RadioButton);
AssertExist (ImageResultB);

A term known as test harness is often used in testing projects. This term collectively refers
to a test execution engine and a repository of test scripts along with test data.

Prioritization of test cases

This technique involves scheduling test cases such that it results in improved performance
of regression testing. These techniques organize the test cases in a test suite by ordering
such that the most promising test cases are executed first thereby resulting in increased
effectiveness of testing. One of the performance goal is to find a measure of how quickly
faults are detected during the testing process (known as fault detection rate) and how to
improve it. Test cases which has high impact in production environment and high business
criticality are given higher priority.

Page 40 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

4.3 Few more Testing Artefacts

In this section we will cover Test Scenario, Difference between Test Case & Scenario, Test
fixture & Data, Test Harness, Test Completion Report and Requirement Mapping to Test
Case.

4.3.1 Test Scenario

The end to end testing is not possible in projects where large number of data combinations
are present and accordingly, the number of possible path in software is high. Scenario
testing is done to ensure that end-to-end functionality of application under test is done.
Further, it also tests if all the business flows are working as expected by end user. Test has
to put himself/herself in customer’s foot and then study how they will use the application
under different conditions. In this type of testing, the preparation of scenarios is most
significant part of this activity. The tester has to consult with multiple stakeholders such as
developers, clients and others. For example:
Requirement: is to test your phone Wi-Fi.

Test Scenario: Verify that device automatically connects to Wi-Fi if user creates new

profile.
Difference between Test Case & Scenario

Test Case Test Scenario

Test case consist of test case name, Test scenario consists of a detailed test
Precondition, steps / input condition, procedure. We can also say that a test
expected result. scenario has many test cases associated with
it. Before executing the test scenario we
need to think of test cases for each scenario.
Test cases are low level actions and it can be Test scenarios are the high level
derived from test scenarios. classification of test requirement grouped
depending on the functionality of a module
and it can be derived from use cases.
Test case is give detailed information about if Test scenario is one liner statement which
any pre-condition, what to test, how to test tell us about what to test.
and expected result etc.
Test case means detailed documenting the Test Scenario means talking and thinking
cases which help executing while testing. requirements in detail.

Page 41 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

Test Case Test Scenario

Test cases are set of steps which performed Test scenario is thread of operations.
on system to verify the expected output.
Test cases are more importance in case Test scenarios are more important when
where development is happening onsite and time to write test cases is no sufficient and
QA is happening Off shored. It will help to team members are agree with the detailed
understand and make both developer and one liner scenario.
QA in sync.
Writing test cases is one time effort which In new software testing generation it is new
can be used in future while executing idea and time saver activity. The addition
regression test case. and modification (easy maintainability) of
test scenarios is easy and independent on
specific person.
While reporting defects it will help tester to …
link the defect with test case id.
The detailed test case document is full proof One of the most positive point about test
guard for new software tester. If developer scenario is good test scenarios reduces the
missed something then it is easy to catch complexity and repeatability of product.
while executing these full-proof test cases.
It requires more time and resources due to If the test scenario not detailed enough then
detailed test case which talks about how to it might take some time to discussion and
test and what to test. understand what test scenario is exactly
talking about.

4.3.2 Test fixture & Data

In most of the cases, multiple set of data are used to test the same functionality of a
particular feature of the product. Hence, all the test values and changeable environmental
components are collected in separate files and stored as test data. It is often useful to
provide this data to client along with the end result of the project. This data can be
subsequently re-used in future projects, thereby reducing time and cost effort.

4.3.3 Test Harness

Test harness is also referred as automated test collection. This is a collection of software
and associated test data which is configured to test the application by executing it under

Page 42 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

varying conditions and subsequent monitoring and documenting its behaviour and output.
There are two parts of test harness. First is test execution engine and second is test script
repository. They key objectives of test harness are as follows:
• To automate the end-to-end testing process
• To execute a collection of test cases (often referred as test suites) collectively
• To generate the results of test execution (often referred as test report)

4.3.4 Test Completion Report

Test completion reporting is a process where, test metrics are reported in summarized
format to update the stakeholders which enables them to take an informed decision.
Test Completion Report Format
• Test Summary Report Identifier
• Summary
• Variances
• Summary Results
• Evaluation
• Planned vs Actual Efforts
• Sign off
Significance of Test Completion Report
• An indication of the quality
• Measure outstanding risks
• The level of confidence in tested software

Summary

• A traceability matrix is term used for a document, which is generally in form of a

table, which correlates any two baseline documents based on many to many
relationship to validate the completeness of the relationship.
• Usage of traceability matrix ensures that requirements captured are complete and
all lower level requirements originate from higher level requirements which are the
linked with each other.
• Test strategy refers to the test approach which defines how testing will be carried
out.
• Test Strategy document is a high level document and generally developed by
project manager.

Page 43 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

• Test Strategy documents defines testing approach to achieve testing objectives and
it’s normally derived from the Business requirement specification documents.
• A software project test plan is a document that describes the objectives, scope,
approach, and focus of a software testing effort.
• Test planning is the most important activity to ensure that there is initially a list of
tasks and milestones in a baseline plan to track the progress of the project and also
defines the size of the test effort.
• Test design means identifying and documenting what set of inputs for the given
software will result in what set of expected outputs.
• Test design techniques are used to identify test conditions and test scenarios which
will result in effective and efficient test cases.
• A test case in software engineering is a set of conditions or variables under which a
tester will determine whether an application or software system is working correctly
or not.
• Test cases are often referred to as test scripts, particularly when written. Written
test cases are usually collected into test suites
• A test script refers to a set of instructions which are written using a
scripting/programming language.
• Scenario testing is done to ensure that end-to-end functionality of application
under test is done.
• Test harness is a collection of software and associated test data which is configured
to test the application by executing it under varying conditions and subsequent
monitoring and documenting its behaviour and output.
• Test completion reporting is a process where, test metrics are reported in
summarized format to update the stakeholders which enables them to take an
informed decision.

Page 44 of 46
Certificate in Software Testing Skill TCS Business Domain Academy

References

• https://tcsltd.skillport.com/skillportfe/main.action#summary/VIDEOS/RW$92271:_
ss_video:87845 - Watch Video about “Identify a Software Testing Strategy” for
additional knowledge.
• https://tcsltd.skillport.com/skillportfe/main.action#summary/VIDEOS/RW$88458:_
ss_video:87846 - Watch Video about “Static Techniques and the Test Process” for
additional knowledge.
• https://tcsltd.skillport.com/skillportfe/main.action#summary/VIDEOS/RW$88505:_
ss_video:87833 - Watch Video about “Software Testing Foundations: The Code of
Ethics” for additional knowledge.
• https://tcsltd.skillport.com/skillportfe/main.action#summary/VIDEOS/RW$88518:_
ss_video:87854 - Watch Video about “Equivalence Partitioning” for additional
knowledge.

Page 45 of 46