Software Testing

Presenter: Khanh Do

1
What is a Test Plan?
● A Test Plan is a detailed document that describes the test strategy,
objectives, schedule, estimation and deliverables and resources required
for testing.
● Test Plan helps us determine the effort needed to validate the quality of
the application under test.
● The test plan serves as a blueprint to conduct software testing activities as
a defined process which is minutely monitored and controlled by the test
manager.

2
Importance of Test Plan
● Help people outside the test team such as developers, business
managers, customers understand the details of testing.
● Test Plan guides our thinking. It is like a rule book, which needs to be
followed.
● Important aspects like test estimation, test scope, Test Strategy are
documented in Test Plan, so it can be reviewed by Management Team
and re-used for other projects.

3
How to write a Test Plan
● Analyze the product
● Design the Test Strategy
● Define the Test Objectives
● Define Test Criteria
● Resource Planning
● Plan Test Environment
● Schedule & Estimation
● Determine Test Deliverables

4
1. Analyze the product
● Analyze the banking website
http://demo.guru99.com/V4
● Who will use the website?
● What is it used for?
● How will it work?
● What are software/ hardware
the product uses?

5
6
How to write a Test Plan
● Analyze the product
● Design the Test Strategy
● Define the Test Objectives
● Define Test Criteria
● Resource Planning
● Plan Test Environment
● Schedule & Estimation
● Determine Test Deliverables

7
2. Develop Test Strategy
● Test Strategy is a critical step in making a Test Plan. A Test Strategy
document, is a high-level document, which is usually developed by Test
Manager. This document defines:
○ The project’s testing objectives and the means to achieve them
○ Determines testing effort and costs

8
2.1 Define scope of Testing
● The components of the system to be tested (hardware, software,
middleware, etc.) are defined as "in scope"
● The components of the system that will not be tested also need to be
clearly defined as being "out of scope"

→ Values

● Give everyone a confidence & accurate information of the testing you are
doing
● All project members will have a clear understanding about what is tested
and what is not
9
2.1 Define scope of Testing
● How do you determine scope your project?
○ Precise customer requirement
○ Project Budget
○ Product Specification
○ Skills & talent of your test team
● As the software requirement specs, the project Guru99 Bank only focus
on testing all the functions and external interface of website Guru99
Bank (in scope testing)
● Nonfunctional testing such as stress, performance or logical database
currently will not be tested. (out of scope testing)

10
2.2 Identify Testing Type
● A Testing Type is a standard test procedure that gives an expected test
outcome.
● Each testing type is formulated to identify a specific type of product bugs.
● → “Early detection of all the defects before releasing the product to the
customer".
● There are tons of Testing Types for testing software product. Your team
cannot have enough efforts to handle all kind of testing. As Test
Manager, you must set priority of the Testing Types
○ Which Testing Types should be focused for web application testing?
○ Which Testing Types should be ignored for saving cost?

11
12
2.3 Document Risk & Issues
● Risk is future’s uncertain event with a probability of occurrence and a
potential for loss. When the risk actually happens, it becomes the ‘issue’.
● Risk analysis is the process of analyzing the risks associated with your
Testing Project.
● For the success of your project, Risk should be identified and
corresponding solutions should be determined before the start of the
project.

13
14
The importance of Risk analysis

15
How to Perform Risk Analysis?
1. Identify the Risks
2. Analyze Impact of each Identified Risk
3. Take countermeasures for the identified & Analyzed risk

16
1. Identify Risk
● Project risk can be defined
as an uncertain event or
activity that can impact the
project's progress. The
impact has a positive or
negative effect on the
prospects of achieving
project objectives.

17
1. Identify Risk
● Product risk is the possibility
that the system or software
might fail to satisfy or fulfill
the expectation of the
customer, user, or
stakeholder. This risk is
related to the functionality
of the product such as
Performance Issues,
Security Issues, Crash
Scenarios, etc.
18
1. Identify Risk - Project Risk

19
Project Risk - Organizational Risk
● It is a risk related to your
human resource or your
Testing team.
● For example, in your
project, lack of
technically skilled
members is a risk. Not
having enough
manpower to complete
the project on time is
another risk.
20
Project Risk - Technical Risk
● Technical Risk is the probability of loss incurred during the execution of a
technical process such as untested engineering, wrong testing
procedure…etc.

● Here is an example of technical risk. Your task in this project is testing a

banking website. You have to set up proper test environments which
mirror real business environments. If the Test Environment is not setup
properly, the product will be not be tested correctly and many defects will
not be detected.
21
Project Risk - Business Risk
● The risk involves an external entity. It is the risk which may come from
your company, your customer but not from your project.

22
Product Risk
● The software skips some key function that the customers specified in the
users' requirement
● The software is unreliable and frequently fails to work.
● Software fail in ways that cause financial or other damage to a user or the
company that uses the software.
● The software has problems related to a particular quality characteristic
such as security, reliability, usability, maintainability or performance.

23
How to Perform Risk Analysis?
1. Identify the Risks
2. Analyze Impact of each Identified Risk
3. Take countermeasures for the identified & Analyzed risk

24
2. Analyze the impact of the risk occurring
● Each risk should be classified on the basis of following two parameters
○ The probability of occurrence
○ The impact on the project

Probability

High (3) Has very high probability to occur, may impact to the whole
project

Medium (2) 50% chance to occur

Low (1) Low probability of occurrence

25
2. Analyze the impact of the risk occurring
● Each risk should be classified on the basis of following two parameters
○ The probability of occurrence
○ The impact on the project

Impact

High (3) Cannot continue with project activity if it is not solved

immediately

Medium (2) Cannot continue the project activity if it is not solved

Low (1) Need to solve it but it is possible to take alternative solution

for a while
26
2. Take countermeasures for the identified &
Analyzed risk

27
Risk Response

28
Register Risk
● All the risk must be recorded, documented and acknowledged by project
managers, stakeholder and the project member. The risk register should
be freely accessible to all the members of the project team.
● There’re some useful to register risk such as Redmine, MITRE... etc.

29
Monitor and Control Risk
● Risks can be monitored on a continuous basis to check if any changes are
made. New risk can be identified through the constant monitoring and
assessing mechanisms.

30
2.4 Create Test Logistics
● In Test Logistics, the Test Manager should answer the following questions:
○ Who will test?
○ When will the test occur?

31
Who will test?
● Skill is qualified for the task or not, also estimate the project budget.
Selecting wrong member for the task may cause the project to fail or
delay.
● Person having the following skills is most ideal for performing software
testing:
○ Ability to understand customers point of view
○ Strong desire for quality
○ Attention to detail
○ Good cooperation
● In your project, the member who will take in charge for the test execution
is the tester. Base on the project budget, you can choose in-source or
outsource member as the tester. 32
When will the test occur?
● Test activities must be matched with associated development activities.

33
How to write a Test Plan
● Analyze the product
● Design the Test Strategy
● Define the Test Objectives
● Define Test Criteria
● Resource Planning
● Plan Test Environment
● Schedule & Estimation
● Determine Test Deliverables

34
3. Define Test Objective
● Test Objective is the overall goal and achievement of the test execution.
The objective of the testing is finding as many software defects as
possible; ensure that the software under test is bug free before release.

● To define the test objectives, you should do 2 following steps

○ List all the software features (functionality, performance, GUI…) which may
need to test.
○ Define the target or the goal of the test based on above features

35
Test objective of Guru99 Bank testing project
● Choose the ‘TOP-DOWN’ method to find the website’s features which may
need to test. In this method, you break down the application under test to
component and sub-component.
● Check that whether website Guru99 functionality (Account, Deposit…) is
working as expected without any error or bugs in real business
environment
● Check that the external interface of the website such as UI is working as
expected and & meet the customer need
● Verify the usability of the website. Are those functionalities convenient for
user or not?

36
37
How to write a Test Plan
● Analyze the product
● Design the Test Strategy
● Define the Test Objectives
● Define Test Criteria
● Resource Planning
● Plan Test Environment
● Schedule & Estimation
● Determine Test Deliverables

38
4. Define Test Criteria
● Test Criteria is a standard or rule on which a test procedure or test
judgment can be based. There are 2 types of test criteria as following
○ Suspension Criteria
○ Exit Criteria

39
Suspension Criteria
● Specify the critical
suspension criteria for a test.
If the suspension criteria are
met during testing, the active
test cycle will be suspended
until the criteria are resolved.

40
Exit Criteria
● It specifies the criteria that denote a successful completion of a test
phase. The exit criteria are the targeted results of the test and are
necessary before proceeding to the next phase of development.
● Example: 95% of all critical test cases must pass.

41
Exit Criteria
● Some methods of defining exit criteria are by specifying a targeted run
rate and pass rate.
● Run rate is ratio between number test cases executed/total test cases
of test specification.
○ For example, the test specification has total 120 TCs, but the tester only
executed 100 TCs, So the run rate is 100/120 = 0.83 (83%)
● Pass rate is ratio between numbers test cases passed / test cases
executed.
○ For example, in above 100 TCs executed, there are 80 TCs that passed, so the
pass rate is 80/100 = 0.8 (80%)

42
Exit Criteria

43
How to write a Test Plan
● Analyze the product
● Design the Test Strategy
● Define the Test Objectives
● Define Test Criteria
● Resource Planning
● Plan Test Environment
● Schedule & Estimation
● Determine Test Deliverables

44
5. Resource Planning
● Resource plan is a detailed summary of all types of resources required to
complete project task. Resource could be human, equipment and
materials needed to complete a project

● The resource planning is important factor of the test planning because

helps in determining the number of resources (employee, equipment…) to
be used for the project. Therefore, the Test Manager can make the correct
schedule & estimation for the project.

45
5. Resource Planning
Member Tasks

Test Manager ● Manage the whole project

● Define project directions
● Acquire appropriate resources

Tester ● Identifying and describing appropriate test

techniques/tools/automation architecture
● Verify and assess the Test Approach
● Execute the tests, Log results, Report the defects.
● Tester could be insourced or outsourced members, base
on the project budget
● For the task which required low skill, I recommend you
choose outsourced members to save project cost. 46
5. Resource Planning
Member Tasks

Developer in ● Implement the test cases, test program, test suite etc.
Test

Test ● Builds up and ensures Test Environment and assets are

Administrator managed and maintained
● Support Tester to use the test environment for test
execution

SQA members ● Take in charge of quality assurance

● Check to confirm whether the testing process is meeting
specified requirements
47
5. Resource Planning - System Resource
Resources Descriptions

Server ● Install the web application under test

● This includes a separate web server, database server, and
application server if applicable

Test Tool ● The testing tool is to automate the testing, simulate the
user operation, generate the test results
● There are tons of test tools you can use for this project
such as Selenium, QTP…etc.

Network ● You need a Network include LAN and Internet to simulate

the real business and user environment
48
Computer ● The PC which users often use to connect the web server
How to write a Test Plan
● Analyze the product
● Design the Test Strategy
● Define the Test Objectives
● Define Test Criteria
● Resource Planning
● Plan Test Environment
● Schedule & Estimation
● Determine Test Deliverables

49
6. Plan Test Environment
● A testing environment is a setup of software and hardware on which the
testing team is going to execute test cases. The test environment consists
of real business and user environment, as well as physical environments,
such as server, front end running environment.
● Some questions:
○ What is the maximum user connection which this website can handle at the
same time?
○ What are hardware/software requirements to install this website?
○ Does the user's computer need any particular setting to browse the website?

50
6. Plan Test Environment

51
How to write a Test Plan
● Analyze the product
● Design the Test Strategy
● Define the Test Objectives
● Define Test Criteria
● Resource Planning
● Plan Test Environment
● Schedule & Estimation
● Determine Test Deliverables

52
7. Schedule & Estimation

53
7. Schedule & Estimation
● Complete the project Guru99 in one month

54
How to write a Test Plan
● Analyze the product
● Design the Test Strategy
● Define the Test Objectives
● Define Test Criteria
● Resource Planning
● Plan Test Environment
● Schedule & Estimation
● Determine Test Deliverables

55
8. Test Deliverables
● Test Deliverables is a list of all the documents, tools and other
components that has to be developed and maintained in support of the
testing effort.

56
8. Test Deliverables - Before Testing

● Test Scripts
● Simulators.
● Test Data
● Test Traceability Matrix
● Error logs and execution logs.

57
8. Test Deliverables - During Testing

● Test plans document.

● Test cases documents
● Test Design specifications.

58
8. Test Deliverables - After Testing

● Test Results/reports
● Defect Report
● Installation/ Test procedures guidelines
● Release notes

59
Thanks
60