Hierarchy of Testing

Testing

Program Testing System Testing Acceptance

Testing
Unit Testing Integration Testing Function
Benchmark
Black Box Properties
Top Down
Performance Pilot
Equivalence
Bottom Up
Boundary Reliability Alpha
Big Bang
Decision Table Availability
Beta
State Transition Sandwich Security

Use Case Usability

Domain Analysis Documentation

White Box Portability

Control Flow Data Flow Capacity 1

White box Testing
White box testing is a testing technique, that examines the
program structure and derives test data from the program
logic/code.
White Box Testing is the testing of a software solution's
internal coding and infrastructure. It focuses primarily on
strengthening security, the flow of inputs and outputs through
the application, and improving design and usability. White
box testing is also known as clear box testing, open box
testing, logic driven testing or path driven testing or
structural testing and glass box testing.

White box testing deal with the code, code structure and
internal design and how they are coded.
White box testing needs knowledge of the internal program
design and code required and these types of tests are based on
coverage of code statement, branches, paths and condition.
White box Testing involves
Testing all independent paths within a module at least
once.
Testing all logical decision on their True and False
sides.
Testing internal data structures to ensure their
validity.
Testing all loops at their boundaries and within their
operational bounds.
Types:
1)Code Coverage Testing
A)Function Coverage=(Number of functions Exercised/Total
number of functions in program)*100
B)Statement Coverage=(Number of Statements Exercised/Total
number of executable statement in program)*100
For Example:
Assume ,program has 100 statements,
Testes exercise 80 statements then,
Statements coverage=80%

C)Decision Coverage= (Number of decision outcomes exercised/Total

number of decision outcomes in program)*100
For Example:
Assume ,program has 100, decision outcomes
Testes exercise 80 decision outcomes then,
decision outcomes=80%
D)Path Coverage
E)Condition Coverage
2)Code Complexity Testing
Techniques :
1)Basic Path Testing
2)Loop Testing
3)Control Structure Testing
-----What do you verify in White Box Testing ?
White box testing involves the testing of the
software code for the following:
Internal security holes
Broken or poorly structured paths in the coding
processes
The flow of specific inputs through the code
Expected output
The functionality of conditional loops
Testing of each statement, object and function on an
individual basis
Advantages
1)Verification
2)Followed Processes
3)Early Warnings
4)Increased Effectiveness
5)Early Defect Identification
6)No Waiting
Disadvantages
1)User Requirements
2)Set of Requirements
3)Dominated
4)Difficult to Scale
5)Difficult to maintain
6)Cultural Stress
Black box Testing
Black box testing treats the system as a “black-box”, so it doesn’t
explicitly use Knowledge of the internal structure or code. Or in
other words the Test engineer need not know the internal
working of the “Black box” or application.
Main focus in black box testing is on functionality of the
system as a whole. The term ‘behavioral testing’ is also used
for black box testing and white box testing is also sometimes
called ‘structural testing’.
Black-box testing is a method of software testing that examines
the functionality of an application based on the specifications. It
is also known as Specifications based testing. Independent
Testing Team usually performs this type of testing during the
software testing life cycle.
This method of test can be applied to each and every level of
software testing such as unit, integration, system and acceptance
testing.
Example
Customer : Excuse me Mr. I want to buy ”lock and key”.
Can you show me a sample?
Shopkeeper : Of course sir , this is a very good brand . Please
have look.
Customer : Ok. When I turn it clockwise ,it locks. When I turn it
anticlockwise , it unlocks. Also it is of a medium weight. Good
I will Purchase this.

Moral: The customer has only checked the function of the lock n
key not the internal manufacturing details like usage of how
many nuts , how much steel and as such.
Black Box Testing Techniques/Types
1)Equivalence Partitioning /
(Equivalence class Partition )
2)Requirements Based Techniques/Testing
3)Positive Testing
4)Negative Testing
5)Boundary value Analysis(BVA)
6)Cause Effect Techniques/Testing
Types of Black Box Testing
1)Functional Testing
2)System Testing
3)Integration testing
4)Incremental Integration testing
5)End-to-End Testing
6)Comparison Testing
7)Regression Testing
8)Stress Testing
Advantages
1)Efficient Testing
2)Code access not required
3)Easy to Execute
4)Easy of use
5)No coding skill required
6)Quicker Test Case Development
7)Simplicity
Disadvantages
1)The Test cases are difficult to design
2)Redundant
3)Lack of Introspection
4)Testing every possible input ,is not possible because
it would take a lot of time and therefore many of the
program paths go untested.
5)Blind coverage
6)Script Maintenance
 Gray Box Testing is a technique to test the software product or
application with partial knowledge of the internal workings of an
application.
 In this process, context specific errors that are related to web
systems are commonly identified. It will increase the testing
coverage by concentrating on all of the layers of any complex system.
 Gray Box Testing is a software testing method, which is a
combination of both White Box Testing and Black Box Testing
method.
 In White Box testing internal structure (code) is known
 In Black Box testing internal structure (code) is unknown
 In Grey Box Testing internal structure (code) is partially known

 Best Suited Applications:

 Grey-box testing is a perfect fit for Web-based applications.
 Grey-box testing is also a best approach for functional or domain
testing.
Why Gray Box Testing
Gray Box Testing is performed for the following reason,
It provides combined benefits of both black box testing and
white box testing both
It combines the input of developers as well as testers and
improves overall product quality
It reduces the overhead of long process of testing functional
and non-functional types
It gives enough free time for developer to fix defects
Testing is done from the user point of view rather than
designer point of view
Example of Gray Box Testing: While testing
websites feature like links or orphan links, if tester
encounters any problem with these links, then he can
make the changes straightaway in HTML code and can
check in real time.
Steps to perform Grey box Testing are:
Step 1: Identify inputs
Step 2: Identify outputs
Step 3: Identify major paths
Step 4: Identify Subfunctions
Step 5: Develop inputs for Subfunctions
Step 6: Develop outputs for Subfunctions
Step 7: Execute test case for Subfunctions
Step 8: Verify correct result for Subfunctions
Step 9: Repeat steps 4 & 8 for other Subfunctions
Step 10: Repeat steps 7 & 8 for other Subfunctions
The test cases for grey box testing may include, GUI related,
Security related, Database related, Browser related,
Operational system related, etc.
 Advantages:
 Grey-box testing provides combined benefits of both white-box and black-
box testing
 It is based on functional specification, UML Diagrams, Database
Diagrams or architectural view
 Grey-box tester can handle complex design test scenario more intelligently
 The added advantage of grey-box testing is that it maintains the boundary
between independent testers and developers
 Disadvantages :
 In grey-box testing, complete white box testing cannot be done due to
inaccessible source code/binaries.
 It is difficult to associate defects when we perform Grey-box testing for a
distributed system.
Unit Testing
Unit testing is a software development process in which the
smallest testable parts of an application, called units, are
individually and independently scrutinized for proper
operation. Unit testing is often automated but it can also be
done manually.
 A unit is the smallest testable part of an application like
functions, classes, procedures, interfaces. Unit testing is a
method by which individual units of source code are tested
to determine if they are fit for use.
Unit tests are basically written and executed by software
developers to make sure that code meets its design and
requirements and behaves as expected.
The goal of unit testing is to segregate each part of the
program and test that the individual parts are working
correctly.
Unit testing is basically done before integration.
Method Used for unit testing: White Box Testing method is used
for executing the unit test.
Unit Testing Lifecycle:
Objectives :To test the function of a program or unit of
code such as a program or module.
 To test internal code/logic
To verify internal design
To test path and condition and error handling
 Done By:
Programmers
Debuggers
Tracers
Done : After modules have been coded
Input: Internal Application description
Unit Test Plan
Output: Unit Test Reports
Types of Unit Testing
 1)Static Testing
A)Code Review
B)Desk Checking
C)Code Walkthrough
D)Code Inspection

 2)Code Coverage Testing

 A)Function Coverage
 B)Statement Coverage
 C)Decision Coverage
 D)Path Coverage
 E)Condition Coverage
 3)Code Complexity Testing
A)Code Review
Code review check list:
1)Data Faults:
 Are all program variables initialized before their
values are used?
Is there any possibility of buffer Overflow?
Have all Constants been named?
If character strings are used is a delimiter explicitly
assigned?
2)Computation Faults
Do any calculation that use variables have different
data types such as adding an integer to a floating-point
number.
Do any calculation use variables of same data type but
different sizes such as adding a byte to a word?
3)Comparison Faults
Less than, greater than, not equal to, true or false are
few comparison and decision errors that are very likely
to be susceptible to boundary condition problems.
Are the comparisons correct because there is always a
confusion about whether to use < or <= and > or >=
Does Boolean calculation work as expected?
4)Control Faults
For each condition statement is the condition correct?
Is each loop certain to terminate?
Are compound statements correctly bracketed?
If a break is required after each case in case statements
,has it been included?
5)Subroutine or Parameter Faults
Does the subroutine have multiple entry points?
If global variables are used, then do they have similar
definition and attributes in all referencing
subroutines?
6)Input/Output Faults
Are all input variables used?
Are all output variables assigned a value before they
are output?
 Can unexpected inputs cause corruption?
 B)Desk Checking
Advantage
1)The programmer who is good in programming
language is able to read and understand his systems
code.
Disadvantages
1)A developer is not the right person to detect problems
in his or her own code.
2)Developers generally prefer to write new code rather
than do any form of testing.
 C)Code Walkthrough
 D)Code Inspection
Steps :
1)Planning
2)Overview
3)Preparation
4)Inspection/Meeting
5)Rework and follow up
Tips to Perform Unit Testing.
Find a tool/ framework for your language.
Do not create test cases for everything. Instead, focus on
the tests that impact the behavior of the system.
Isolate the development environment from the test
environment.
Aim at covering all paths through the unit. Pay
particular attention to loop conditions.
Make sure you are using a version control system to keep
track of your test scripts.
In addition to writing cases to verify the behavior, write
cases to ensure performance of the code.
Perform unit tests continuously and frequently.
ONE MORE REASON
Let’s say you have a program comprising of two units and
the only test you perform is system testing. [You skip unit
and Integration testing.] During testing, you find a bug.
Now, how will you determine the cause of the problem?
Is the bug due to an error in unit 1?
Is the bug due to an error in unit 2?
Is the bug due to errors in both units?
Is the bug due to an error in the interface between the
units?
Is the bug due to an error in the test or test case?
Unit testing is often neglected but it is, in fact, the most
important level of testing.
Advantages
1)Faster Debugging
2)Better Feedback Mechanism
3)Good Regression Tool
4)Reduce Future Cost
5)Better Design and Documentation
Disadvantages
1)No time was Allocated for Unit Testing
2)Difficult to write
3)Difficult to Manage
Integration Testing
Integration testing (sometimes
called integration and testing) is the phase in
software testing in which individual software
modules are combined and tested as a group. It occurs
after unit testing and before validation testing.
Integration testing tests integration or interfaces
between components, interactions to different parts of
the system such as an operating system, file system
and hardware or interfaces between systems.
Also after integrating two different components together we do the integration testing. As displayed in the image
below when two different modules ‘Module A’ and ‘Module B’ are integrated then the integration testing is done.
Objectives: To verify the interfacing between
modules and within sub-systems.
Done by: Developers Testers.
Done: After modules are unit tested and also all the
functions of each component are tested.
Input: Internal and External application
descriptions.
Integration test plan
Output: Integration test Report.
Methods: Top-down and Bottom-up testing
techniques.
Integration Testing Types:

1)Top-down Approach
2)Bottom –Up Approach
3)Big-Bang Approach
4)Sandwich Approach
1)Top-down Approach
Advantages:
The tested product is very consistent because the integration
testing is basically performed in an environment that almost
similar to that of reality.
Fault Localization is easier.
Possibility to obtain an early prototype.
Critical Modules are tested on priority; major design flaws
could be found and fixed first.
Detects the flow of design.
Useful when major flaws occur at the top of the program.
Disadvantages:
Basic functionality is tested at the end of cycle.
Needs many Stubs.
2)Bottom –Up Approach
Advantages:
Useful when major flaws occur at the bottom of the
program
Useful for integration the Object-oriented Systems.
Test condition are easier to create
Observation of test results is easier.
 In this approach development and testing can be done
together so that the product or application will be efficient
and as per the customer specifications.
 Fault localization is easier.
 No time is wasted waiting for all modules to be developed
unlike Big-bang approach.
Disadvantages:
Integration errors are found later than earlier.
Design flaws that could need major reconstruction are
found last.
We can catch the Key interface defects at the end of cycle.
It is required to create the test drivers for modules at all
levels except the top control.
Critical modules (at the top level of software architecture)
which control the flow of application are tested last and
may be prone to defects.
Early prototype is not possible.
 3)Big-Bang Approach
Advantages:
This approach is well suited in scenarios where the
majority of components are already available and very
few components get added or modified.
Disadvantages:
When some failure is found out during integration, it is
very difficult to trace the module where the problems
existed.
Certain sub-systems may take lot of time to be
corrected.
It is also difficult to find the developer who developed
that module which again makes it difficult to make
modification so as to correct that defect.
4)Sandwich Approach
 Process of sandwich Testing:
Upward
Downward
Middle Layer
 Advantages:
Top and Bottom Layer Tests can be done in Parallel
Useful for very large Projects
Development Schedule
More Resources
 Disadvantages:
Does not test the individual subsystems thoroughly before integration
of the sub components
High cost of Testing
Cannot be used for smaller Systems
Different skill sets.
Why do Integration Testing?
Although each software module is unit tested, defects still exist
for various reasons like,
A Module in general is designed by an individual software
developer whose understanding and programming logic may
differ from other programmers. Integration testing becomes
necessary to verify the software modules work in unity.
At the time of module development, there are wide chances of
change in requirements by the clients. These new requirements
may not be unit tested and hence system integration testing
becomes necessary.
Interfaces of the software modules with the database could be
erroneous
External Hardware interfaces, if any, could be erroneous
Inadequate exception handling could cause issues.
Integration Testing Procedure:
The integration test procedure irrespective of the test
strategies
Step1: Prepare the Integration Tests Plan
Step2: Design the Test Scenarios, Cases, and Scripts.
Step3: Executing the test Cases followed by reporting
the defects.
Step4: Tracking & re-testing the defects.
Step5: Steps 3 and 4 are repeated until the completion
of Integration is successfully.
Brief Description of Integration Test Plans:
It includes following attributes:
Methods/Approaches to test (as discussed above).
Scopes and Out of Scopes Items of Integration Testing.
Roles and Responsibilities.
Pre-requisites for Integration testing.
Testing environment.
Risk and Mitigation Plans.
Entry and Exit Criteria
 Entry and Exit Criteria to Integration testing phase in any software
development model

Entry Criteria:
 Unit Tested Components/Modules
 All High prioritized bugs fixed and closed
 All Modules to be code completed and integrated successfully.
 Integration tests Plan, test case, scenarios to be signed off and documented.
 Required Test Environment to be set up for Integration testing

Exit Criteria:
 Successful Testing of Integrated Application.
 Executed Test Cases are documented
 All High prioritized bugs fixed and closed
 Technical documents to be submitted followed by release Notes.
Best Practices/ Guidelines for Integration Testing:
First determine the Integration Test Strategy that could be
adopted and later prepare the test cases and test data
accordingly.
Study the Architecture design of the Application and identify
the Critical Modules. These need to be tested on priority.
Obtain the interface designs from the Architectural team and
create test cases to verify all of the interfaces in detail.
Interface to database/external hardware/software application
must be tested in detail.
After the test cases, it's the test data which plays the critical
role.
Always have the mock data prepared, prior to executing. Do
not select test data while executing the test cases.