LECTURE 7: SOFTWARE TESTING

1
 Software Testing
Testing is the process of exercising a program with the specific intent
of finding errors prior to delivery to the end user.
What Testing Shows
errors

requirements conformance

performance

an indication
of quality

2
 Who Tests the Software?
Developer Tester

Understands the system but, Must learn about the system, but, will
will test "gently“ and, is driven attempt to break it and, is driven by
by "delivery" quality

3
 Testing Strategy

Unit test Integration test

System test
User Acceptance test

 We begin by ‘testing-in-the-small’ and move to ‘testing-in-the-large’

 For conventional software
– The module (component) is our initial focus
– Integration of modules follows
 For OO software
– our focus when “testing in the small” on class that encompasses
attributes and operations and implies communication and collaboration
4
 Unit Testing

results
module
to be
tested

Interface
local data structures
Software boundary conditions
Engineer independent paths
test cases error handling paths

5
 Integration Testing Strategies

Big Bang Integration Testing is an approach in which all software components

(modules) are combined at once and Tested.
Advantages Drawbacks
The whole system is tested It is hard to separate modules when the bug is detected
Requires minor planning Ineffective for large systems
Suitable for small systems High risk to miss some crucial issues while testing the whole
system
Failures occur more frequently because of the simultaneous check
of numerous modules
A single mistake can influence the results of the whole integration
testing

6
 Top Down Integration Bottom-Up Integration
A top module is tested with A
stubs

B F G B F G

stubs are replaced one at drivers are replaced

a time, "depth first" one at a time,
C C
"depth first"
as new modules are integrated, worker modules are
some subset of tests is re-run
D E grouped into
D E builds and integrate

cluster
Test stub to simulate the behaviors of software components that a module undergoing
tests depends on.
A Driver is to simulate the behavior of the upper level modules that are not yet
integrated. 7
Sandwich Testing

A
Top modules are
tested with stubs
B F G

C
Worker modules are grouped into
builds and integrated
D E

cluster

8
 OOT Strategy
 Class testing is the equivalent of unit testing
– Operations within the class are tested
– The state behavior of the class is examined

 Integration applied three different strategies

– Thread-based testing—integrates the set of classes required to respond
to one input or event
– Use-based testing—integrates the set of classes required to respond to
one use case
– Cluster testing—integrates the set of classes required to demonstrate
one collaboration

9
 Smoke Testing
 Is a type of software testing that comprises of a non-exhaustive set of tests that aim
at ensuring that the most important functions work.
 The result of this testing is used to decide if a build is stable enough to proceed
with further testing. A common approach for creating “daily builds” for product
software
Smoke testing steps:
1. Software components that are translated into code are integrated into a “build.”
– A build includes all data files, libraries, reusable modules, and engineered
components that are required to implement one or more product functions.
§ A series of tests is designed to expose errors that will keep the build from properly
performing its function.
– The intent should be to uncover “show stopper” errors that have the highest
likelihood of throwing the software project behind schedule
§ The build is integrated with other builds and the entire product (in its current form)
is smoke tested daily.
1. The integration approach may be
top down or bottom up.

10
Software Testing: Definition
The execution of software for purposes of validation and verification

Requires test data (inputs) and knowledge of the expected output.

A test case is the tuple (i,o),where i is the test data and o=S(i) and S the
software under test.
Test Cases Example
Spec of software to test: capitalize a given word.
Set of 5 test data:
Spurs, Tottenham Hotspur, keaneR, spurs3, spurs Rule OK
• Corresponding test case are the 5 tuples:
– (Spurs, SPURS),
– (Tottenham Hotspur, error),
– (keaneR , KEANER),
– (spurs3, SPURS3),
– (spurs Rule OK, error)

11
Some really boring observations
 It is impossible to completely test any nontrivial module or any system
 Testing can only show the presence of bugs, not their absence (Dijkstra)

Testing: the ‘standard’ approach

1. Write a pile of code
2. Get scared enough about some aspect of it to feel the need to ‘try it
out’
3. Write some kind of driver that invokes the bit of code. Add a few
print statements to verify it’s doing what you think it should
4. Run the test, look at the output and then delete (or comment out) the
print statements
5. Go back to step 1

12
Good testing takes creativity
 Testing is done best by independent testers

 Programmers often stick to the data set that makes the program work

 A program often does not work when tried by somebody else.

Testing in development
Component testing
– Unit or module testing
– Integration testing
Functional testing: to validate the software against the functional rqts.
System testing

13
Testing after development
 Acceptance testing: Formal testing conducted to determine whether or not a
system satisfies its acceptance criteria and to enable the acquirer to determine
whether or not to accept the system.

 Acceptance testing is a final stage of testing that is performed on a system before the
system being delivered to a live environment.

 The acceptance testing process is designed to replicate the anticipated real-life use of
the product to ensure that what the consumer or end user receives is fully functional
and meets their needs and expectations.

14
Testing after development
 Alpha test: A preliminary, in-house testing of a product to
discover obvious faults.

 At the developer’s site. Developers observe the users and note

problems.

 Alpha testing is typically performed by a group that is independent of

the design team, but still within the company, e.g. in-house software
test engineers, or software QA engineers.

 Alpha testing is final testing before the software is released to the

customer. It has two phases:
– In the first phase: in-house developers.
– In the second phase: software QA staff.

15
Testing after development
 Beta test: Typically the last step in the testing of a product before
it is officially released. A beta test is often conducted with
customers in their offices.

– Beta testing is the last stage of testing and one of the Acceptance
Testing types

– Can involve sending the software to beta test sites outside the

company for real-world exposure or

– Beta testing is often preceded by a round of testing called alpha

testing.

 Installation Testing or Implementation Testing: About testing the

installation procedures that are needed to run the software. 
16
 Testing Methods
1. Dynamic testing (dynamic analysis)
2. Static Testing ("Dry Run Testing")
1. Dynamic testing (dynamic analysis) tests the code dynamic behavior.

 Dynamic refers to the examination of the physical response from the

system to variables that change with time and are not consistent.

 Some of the Dynamic Testing methodologies include Unit Tests,

Integration Tests, System Tests and Acceptance Tests.

1. In dynamic testing the software must actually be compiled and run vs

static testing.

17
 Testing Methods

2. Static Testing ("Dry Run Testing") tests the software that isn't actually used.
Syntax checking and manually reading the code to find errors are methods of
static testing.
Is mostly used by the developer (who designed or code the module).
It is usually the first type of testing done on any system (done as unit
testing).
'Static testing' is generally not taken as detailed testing, but checks mainly for
the sanity of the code/algorithm, and to assess if program is ready for more
detailed testing, where methods like code review, inspection and walkthrough
are used.
Static testing also applies to White box testing techniques.

18
Testing Methods, Cont…

3. Black Box Testing vs White Box Testing

4. Statistical Testing:
– Developers can utilize well-known statistics to direct the application of
testing, thereby reducing redundant testing,

– Focusing testing on portions of the software with the biggest impact on

the system, and reducing the amount of testing required overall.

– Statistical testing can also be used to determine when it is time to stop

testing a software product, through reliability and entropy metrics.

19
Project Name:
Test Case Template

Test Case ID: Fun_10 Test Designed by: <Name>

Test Priority (Low/Medium/High): Med Test Designed date: <Date>
Module Name: Google login screen Test Executed by: <Name>
Test Title: Verify login with valid username and password Test Execution date: <Date>
Description: Test the Google login page

Pre-conditions: User has valid username and password

Dependencies:

Step Test Steps Test Data Expected Result Actual Result Status (Pass/Fail) Notes

1 Navigate to login page User= [email protected] User should be able to login User is navigated to Pass
2 Provide valid username Password: 1234 dashboard with successful
3 Provide valid password login
4 Click on Login button

Post-conditions:
User is validated with database and successfully login to account. The account session details are logged in database.
 Regression testing
 Retesting to ensure previously working code does not fail as a result of
changes. Should be applied irrespective of testing strategy or method.

 It is a normal part of software development in larger companies,

performed by code specialists.

 Before the release of a new version of a software product, old test

cases run against a new version to make sure that old capabilities are
still working.

21
 Exhaustive Testing

53 binary conditions
1 condition with 3 options
2^53 *3= 9,007,199,254,740,992*3 = 207,021,597,764,222,976 possible
combinations of conditions
At one second per test execution: 856,849,244 years to test all possible combinations.22
Selective Testing

Selected path

loop < 20 X

23
 White-Box Testing

... our goal is to ensure that all

statements and conditions have
been executed at least once ...

Black-box testing involves testing the functionality of a software component without

knowing the details of its internal logic.

White-box testing involves testing the independent logic paths with full
implementation knowledge.

24
References

 Bernd Bruegge & Allen H. Dutoit, Object-Oriented Software

Engineering: Using UML, Patterns, and Java
 Software Engineering, Ivan Marsic, 2020
 Sommerville, I. (2015). Software Engineering 10. Pearson.