1

Software Reliability
Basic Concepts
There are three phases in the life of any hardware component i.e.,
burn-in, useful life & wear-out.
In burn-in phase, failure rate is quite high initially, and it starts
decreasing gradually as the time progresses.
During useful life period, failure rate is approximately constant.
Failure rate increase in wear-out phase due to wearing out/aging of
components. The best period is useful life period. The shape of this
curve is like a bath tub and that is why it is known as bath tub
curve. The bath tub curve is given in Fig.1.

Software Reliability

Fig. 1: Bath tub curve of hardware reliability.

3

Software Reliability
We do not have wear out phase in software. The expected curve for
software is given in fig. 2.

Fig. 2: Software reliability curve (failure rate versus time)

Software Reliability
Software may be retired only if it becomes obsolete. Some of
contributing factors are given below:

change in environment
change in infrastructure/technology
major change in requirements
increase in complexity
extremely difficult to maintain
deterioration in structure of the code
slow execution speed
poor graphical user interfaces

Software Reliability
What is Software Reliability?
Software reliability means operational reliability. Who cares how
many bugs are in the program?
As per IEEE standard: Software reliability is defined as the ability of
a system or component to perform its required functions under
stated conditions for a specified period of time.

Software Reliability
Software reliability is also defined as the probability that a software
system fulfills its assigned task in a given environment for a
predefined number of input cases, assuming that the hardware and
the inputs are free of error.
It is the probability of a failure free operation of a program for a
specified time in a specified environment.

Software Reliability

Failures and Faults

A fault is the defect in the program that, when executed under

particular conditions, causes a failure.
The execution time for a program is the time that is actually spent by
a processor in executing the instructions of that program. The
second kind of time is calendar time. It is the familiar time that we
normally experience.

Software Reliability
There are four general ways of characterising failure occurrences in
time:
1. time of failure,
2. time interval between failures,
3. cumulative failure experienced up to a given time,
4. failures experienced in a time interval.

Software Reliability
Failure Number

Failure Time (sec)

Failure interval (sec)

18

10

25

36

11

45

57

12

71

14

86

15

104

18

10

124

20

11

143

19

12

169

26

13

197

28

14

222

25

15

250

28

Table 1: Time based failure specification

10

Software Reliability
Time (sec)

Cumulative Failures

Failure in interval (30 sec)

30

60

90

120

150

11

180

12

210

13

240

14

Table 2: Failure based failure specification

11

Software Reliability
Value of random
variable (failures
in time period)

Probability
Elapsed time tA = 1 hr

Elapsed time tB = 5 hr

0.10

0.01

0.18

0.02

0.22

0.03

0.16

0.04

0.11

0.05

0.08

0.07

0.05

0.09

0.04

0.12

0.03

0.16

0.02

0.13

Table 3: Probability distribution at times tA and tB

12

Software Reliability
Value of random
variable (failures
in time period)

Probability
Elapsed time tA = 1 hr

Elapsed time tB = 5 hr

10

0.01

0.10

11

0.07

12

0.05

13

0.03

14

0.02

15

0.01

Mean failures

3.04

7.77

Table 3: Probability distribution at times tA and tB

13

Software Reliability
A random process whose probability distribution varies with time to
time is called non-homogeneous. Most failure processes during test
fit this situation. Fig. 3 illustrates the mean value and the related
failure intensity functions at time tA and tB. Note that the mean
failures experienced increases from 3.04 to 7.77 between these two
points, while the failure intensity decreases.
Failure behaviour is affected by two principal factors:

the number of faults in the software being executed.

the execution environment or the operational profile of
execution.
14

Software Reliability

Fig. 3: Mean Value & failure intensity functions.

15

Software Reliability
Environment
The environment is described by the operational profile. The
proportion of runs of various types may vary, depending on the
functional environment. Examples of a run type might be:

1. a particular transaction in an airline reservation system or a

business data processing system,
2. a specific cycle in a closed loop control system (for
example, in a chemical process industry),
3. a particular service performed by an operating system for a
user.
16

Software Reliability
The run types required of the program by the environment can be
viewed as being selected randomly. Thus, we define the operational
profile as the set of run types that the program can execute along
with possibilities with which they will occur. In fig. 4, we show two of
many possible input states A and B, with their probabilities of
occurrence.
The part of the operational profile for just these two states is shown
in fig. 5. A realistic operational profile is illustrated in fig.6.

17

Software Reliability

Fig. 4: Input Space

18

Software Reliability

Fig. 5: Portion of operational profile

19

Software Reliability

Fig. 6: Operational profile

20

Software Reliability
Fig.7 shows how failure intensity and reliability typically vary during
a test period, as faults are removed.

Fig. 7: Reliability and failure intensity

21

Software Reliability
Uses of Reliability Studies
There are at least four other ways in which software reliability
measures can be of great value to the software engineer, manager
or user.
1. you can use software reliability measures to evaluate software
engineering technology quantitatively.
2. software reliability measures offer you the possibility of
evaluating development status during the test phases of a
project.
22

Software Reliability
3. one can use software reliability measures to monitor the
operational performance of software and to control new features
added and design changes made to the software.
4. a quantitative understanding of software quality and the various
factors influencing it and affected by it enriches into the
software product and the software development process.

23

Software Reliability
Software Quality
Different people understand different meanings of quality like:

conformance to requirements
fitness for the purpose
level of satisfaction

24

Software Reliability

25

Software Reliability

Fig 8: Software quality attributes

26

Software Reliability
1

Reliability

The extent to which a software performs its intended

functions without failure.

Correctness

The extent to
specifications.

Consistency &
precision

The extent to which a software is consistent and give

results with precision.

Robustness

The extent to which a software is tolerates the

unexpected problems.

Simplicity

The extent to which a software is simple in its

operations.

Traceability

The extent to which an error is traceable in order to

fix it.

Usability

The extent of effort required to learn, operate and

understand the functions of the software

which

software

meets

its

27

Software Reliability
8

Accuracy

Meeting specifications with precision.

Clarity &
Accuracy of
documentation

The extent to which documents are clearly & accurately

written.

10

Conformity of
operational
environment

The extent to which a software is in conformity of

operational environment.

11

Completeness

The extent to which a software has specified functions.

12

Efficiency

The amount of computing resources and code required

by software to perform a function.

13

Testability

The effort required to test a software to ensure that it

performs its intended functions.

14

Maintainability

The effort required to locate and fix an error during

maintenance phase.
28

Software Reliability
15

Modularity

It is the extent of ease to implement, test, debug and

maintain the software.

16

Readability

The extent to which a software is readable in order to

understand.

17

Adaptability

The extent to which a software is adaptable to new

platforms & technologies.

18

Modifiability

The effort required to modify a software during

maintenance phase.

19

Expandability

The extent to which a software is expandable without

undesirable side effects.

20

Portability

The effort required to transfer a program from one

platform to another platform.

Table 4: Software quality attributes

29

Software Reliability

McCall Software Quality Model

Fig: Software quality factors

30

Software Reliability
i.

Product Operation

Factors which are related to the operation of a product are

combined. The factors are:

Correctness

Efficiency

Integrity

Reliability

Usability

These five factors are related to operational performance,

convenience, ease of usage and its correctness. These factors play
a very significant role in building customers satisfaction.
31

Software Reliability
ii. Product Revision
The factors which are required for testing & maintenance are
combined and are given below:

Maintainability

Flexibility

Testability

These factors pertain to the testing & maintainability of software.

They give us idea about ease of maintenance, flexibility and testing
effort. Hence, they are combined under the umbrella of product
revision.
32

Software Reliability
iii. Product Transition
We may have to transfer a product from one platform to an other
platform or from one technology to another technology. The factors
related to such a transfer are combined and given below:

Portability

Reusability

Interoperability

33

Software Reliability
Most of the quality factors are explained in table 4. The remaining
factors are given in table 5.
Sr.No.

Quality Factors

Purpose

Integrity

The extent to which access to software or data by the

unauthorised persons can be controlled.

Flexibility

The effort required to modify an operational program.

Reusability

The extent to which a program can be reused in

other applications.

Interoperability

The effort required to couple one system with

another.

Table 5: Remaining quality factors (other are in table 4)

34

Quality criteria

Fig 10: McCalls quality model

35

Software Reliability

Table 5(a):
Relation
between quality
factors and
quality criteria

36

Software Reliability
1

Operability

The ease of operation of the software.

Training

The ease with which new users can use the

system.

Communicativeness The ease with which inputs and outputs can be

assimilated.

I/O volume

It is related to the I/O volume.

I/O rate

It is the indication of I/O rate.

Access control

The provisions for control and protection of the

software and data.

Access audit

The ease with which software and data can be

checked for compliance with standards or other
requirements.

Storage efficiency

The run time storage requirements of the software.

Execution efficiency

The run-time efficiency of the software.

37

Software Reliability
10

Traceability

The ability to
requirements.

link

software

components

to

11

Completeness

The degree to which a full implementation of the

required functionality has been achieved.

12

Accuracy

The precision of computations and output.

13

Error tolerance

The degree to which continuity of operation is ensured

under adverse conditions.

14

Consistency

The use of uniform design and implementation

techniques and notations throughout a project.

15

Simplicity

The ease with which the software can be understood.

16

Conciseness

The compactness of the source code, in terms of lines

of code.

17

Instrumentation

The degree to which the software provides for

measurements of its use or identification of errors.
38

Software Reliability
18

Expandability

The degree to which storage requirements or

software functions can be expanded.

19

Generability

The breadth of the potential application of software

components.

20

Selfdescriptiveness

The degree to which the documents are self

explanatory.

21

Modularity

The provision of highly independent modules.

22

Machine
independence

The degree to which software is dependent on its

associated hardware.

23

Software system
independence

The degree to which software is independent of its

environment.

24

Communication
commonality

The degree to which standard protocols and

interfaces are used.

25

Data commonality The use of standard data representations.

Table 5 (b): Software quality criteria

39

Software Reliability

Boehm Software Quality Model

Fig.11: The Boehm software quality model

40

Software Reliability
ISO 9126

Functionality

Reliability

Usability

Efficiency

Maintainability

Portability

41

Software Reliability
Characteristic/
Attribute

Short Description of the Characteristics and the

concerns Addressed by Attributes

Functionality

Characteristics relating to achievement of the basic

purpose for which the software is being engineered

Suitability

The presence and appropriateness of a set of functions for

specified tasks

Accuracy

The provision of right or agreed results or effects

Interoperability

Softwares ability to interact with specified systems

Security

Ability to prevent unauthorized access, whether accidental

or deliberate, to program and data.

Reliability

Characteristics relating to capability of software to maintain

its level of performance under stated conditions for a
stated period of time

Maturity

Attributes of software that bear on the frequency of failure

by faults in the software
42

Software Reliability
Fault tolerance

Ability to maintain a specified level of performance in cases

of software faults or unexpected inputs

Recoverability

Capability and effort needed to reestablish level of

performance and recover affected data after possible
failure.

Usability

Characteristics relating to the effort needed for use, and on

the individual assessment of such use, by a stated implied
set of users.

Understandability The effort required for a user to recognize the logical

concept and its applicability.
Learn ability

The effort required for a user to learn its application,

operation, input and output.

Operability

The ease of operation and control by users.

Efficiency

Characteristic related to the relationship between the level

of performance of the software and the amount of
resources used, under stated conditions.
43

Software Reliability
Time behaviour

The speed of response and processing times and

throughout rates in performing its function.

Resource
behaviour

The amount of resources used and the duration of such

use in performing its function.

Maintainability

Characteristics related to the effort needed to make

modifications, including corrections, improvements or
adaptation of software to changes in environment,
requirements and functions specifications.

Analyzability

The effort needed for diagnosis of deficiencies or causes

of failures, or for identification of parts to be modified.

Changeability

The effort needed for modification, fault removal or for

environment metal change.

Stability

The risk of unexpected effect of modifications.

Testability

The effort needed for validating the modified software.

44

Software Reliability
Portability

Characteristics related to the ability to transfer the

software from one organization or hardware or software
environment to another.

Adaptability

The opportunity for its adaptation to different specified

environments.

Installability

The effort needed to install the software in a specified

environment.

Conformance

The extent to which it adheres to standards or

conventions relating to portability.

Replaceability

The opportunity and effort of using it in the place of other

software in a particular environment.

Table 6: Software quality characteristics and attributes The ISO 9126 view

45

Software Reliability

Fig.12: ISO 9126 quality model

46

Software Reliability
Software Reliability Models

Basic Execution Time Model

( ) 0

1
V0

(7.1)

Fig.13: Failure intensity as a

function of for basic model
47

Software Reliability
d 0

d
V0

(7.2)

& for basic model

Fig.14: Relationship between

48

Software Reliability
For a derivation of this relationship, equation 7.1 cab be written as:

d ( )
( )

0 1
d
V0

The above equation can be solved for ( ) and result in :

( ) V0

0
1 exp
V0

(7.3)

49

Software Reliability
The failure intensity as a function of execution time is shown in
figure given below

0
( ) 0 exp
V0

Fig.15: Failure intensity versus execution time for basic model

50

Software Reliability

Derived quantities

Fig.16: Additional failures required to be experienced to reach the objective 51

Software Reliability

Fig.17: Additional time required to reach the objective

This can be derived in mathematical form as:

V0 P

Ln
0 F

52

Software Reliability
Example- 1
Assume that a program will experience 200 failures in infinite time. It has
now experienced 100. The initial failure intensity was 20 failures/CPU hr.
(i) Determine the current failure intensity.
(ii) Find the decrement of failure intensity per failure.
(iii) Calculate the failures experienced and failure intensity after 20 and 100
CPU hrs. of execution.
(iv)Compute addition failures and additional execution time required to
reach the failure intensity objective of 5 failures/CPU hr.
Use the basic execution time model for the above mentioned calculations.
53

Software Reliability
Solution
Here

Vo=200 failures

100 failures
0 20 failures/CPU hr.
(i) Current failure intensity:

( ) 0

1
V0

100

20 1
20(1 0.5) 10 failures/CPU hr
200

54

Software Reliability
(ii) Decrement of failure intensity per failure can be calculated as:

d 0
20

0.1 / CPU hr.

d
V0
200
(iii) (a) Failures experienced & failure intensity after 20 CPU hr:

( ) V0 1 exp
V0

20 20
200 1 exp

200

200(1 exp(1 2))

200(1 0.1353) 173failures

55

Software Reliability

20 exp

( ) 0 exp
V0
20 20
20 exp(2) 2.71 failures / CPU hr
200

(b) Failures experienced & failure intensity after 100 CPU hr:

( ) V0

0
1 exp
V0

20 100
200 1 exp

200

200 failures(almost)

0
( ) 0 exp
V0

56

Software Reliability
20 100
20 exp
0.000908 failures / CPU hr
200

(iv) Additional failures required to reach the failure intensity

objective of 5 failures/CPU hr.

V0
200
P F

(10 5) 50 failures
20
0

57

Software Reliability
Additional execution time required to reach failure intensity objective
of 5 failures/CPU hr.

V0
P

Ln

F
0
200 10

Ln
6.93 CPU hr.
20
5

58

Software Reliability

Logarithmic Poisson Execution Time Model

( ) 0 exp( )

Fig.18: Relationship between

59

Software Reliability
d
0 exp( )
d

d

d

Fig.19: Relationship between

60

Software Reliability
1
( ) Ln(0 1)

( ) 0 /(0 1)
1 P

Ln
F
1 1
1


F P

(7.3)

61

Software Reliability
Example- 2
Assume that the initial failure intensity is 20 failures/CPU hr. The failure
intensity decay parameter is 0.02/failures. We have experienced 100
failures up to this time.
(i) Determine the current failure intensity.
(ii) Calculate the decrement of failure intensity per failure.
(iii) Find the failures experienced and failure intensity after 20 and 100 CPU
hrs. of execution.
(iv)Compute the addition failures and additional execution time required to
reach the failure intensity objective of 2 failures/CPU hr.
Use Logarithmic Poisson execution time model for the above mentioned
calculations.

62

Software Reliability
Solution

0 20 failures/CPU hr.
100 failures
0.02 / failures
(i) Current failure intensity:

( ) 0 exp( )
= 20 exp (-0.02 x 100)
= 2.7 failures/CPU hr.
63

Software Reliability
(ii) Decrement of failure intensity per failure can be calculated as:

d

d
= -.02 x 2.7 = -.054/CPU hr.
(iii) (a) Failures experienced & failure intensity after 20 CPU hr:

( )

1
Ln 0 1

Ln(20 0.02 20 1) 109 failures

0.02
64

Software Reliability
( ) 0 / 0 1
(20) /(20 .02 20 1) 2.22 failures / CPU hr.
(b) Failures experienced & failure intensity after 100 CPU hr:

( )

1
Ln 0 1

Ln(20 0.02 100 1) 186 failures

0.02

( ) 0 / 0 1
(20) /( 20 .02 100 1) 0.4878 failures / CPU hr.
65

Software Reliability
(iv) Additional failures required to reach the failure intensity
objective of 02 failures/CPU hr.

1
P
1
2.7
Ln

Ln
15 failures

F 0.02 2
1 1
1
1 1 1

6.5 CPU hr.

F P 0.02 2 2.7

66

Software Reliability
Example- 3
The following parameters for basic and logarithmic Poisson models are
given:
(a) Determine the addition failures and additional execution time required to
reach the failure intensity objective of 5 failures/CPU hr. for both models.
(b) Repeat this for an objective function of 0.5 failure/CPU hr. Assume that
we start with the initial failure intensity only.

67

Software Reliability
Solution
(a) (i) Basic execution time model

V0
(P F )
0
100

(10 5) 50 failures
10

(Present failure intensity) in this case is same as

failure intensity).
Now,

V0 P

Ln
0 F

(initial

68

Software Reliability
100 10

Ln
6.93 CPU hr.
10
5
(ii) Logarithmic execution time model
1 P

Ln
F

1
30
Ln
71.67 Failures
0.025 5

1 1
1

F P

1
1 1

Ln 6.66 CPU hr.

0.025 5 30

69

Software Reliability
Logarithmic model has calculated more failures in almost some duration of
execution time initially.

(b) Failure intensity objective

F = 0.5 failures/CPU hr.

(i) Basic execution time model

V0
P F
0
100

(10 0.5) 95 failures

10

V0 P

Ln
0 F
100 10

Ln
30 CPU /hr
10
0.05
70

Software Reliability
(ii) Logarithmic execution time model

1 P

Ln
F
1
30

Ln
164 failures
0.025 0.5
1 1
1


F P
1 1
1

78.66 CPU/hr

0.025 0.5 30

71

Software Reliability

Calendar Time Component

The calendar time component is based on a debugging process

model. This model takes into account:
1. resources used in operating the program for a given
execution time and processing an associated quantity of
failure.
2. resources quantities available, and
3. the degree to which a resource can be utilised (due to
bottlenecks) during the period in which it is limiting.
Table 7 will help in visualizing these different aspects of the
resources, and the parameters that result.
72

Software Reliability
Resource usage
Usage parameters
requirements per
Resource

Planned parameters

CPU hr

Failure

Quantities
available

Utilisation

Failure identification
personnel

PI

Failure correction
personnel

Pf

Pf

Computer time

Pc

Pc

Fig: Calendar time component resources and parameters

73

Software Reliability
Hence, to be more precise, we have

X C c c

(for computer time)

X f f

(for failure correction)

X I I I

(for failure identification)

dxT / d r r
74

Software Reliability
Calendar time to execution time relationship

dt / d (1 / Pr pr )dxT / d
dt / d ( r r ) / Pr pr

75

Software Reliability

Fig.20: Instantaneous calendar time to execution time ratio

76

Software Reliability

Fig.21: Calendar time to execution time ratio for different

limiting resources

77

Software Reliability
Example- 4
A team run test cases for 10 CPU hrs and identifies 25 failures. The effort
required per hour of execution time is 5 person hr. Each failure requires 2
hr. on an average to verify and determine its nature. Calculate the failure
identification effort required.

78

Software Reliability
Solution
As we know, resource usage is:

X r r r
Here

Hence,

r 15 person hr.

25 failures

10 CPU hrs.

r 2 hrs./failure

Xr = 5 (10) + 2 (25)
= 50 + 50 = 100 person hr.
79

Software Reliability
Example- 5
Initial failure intensity ( ) for a given software is 20 failures/CPU hr. The
0
failure intensity objective ( ) of 1 failure/CPU hr. is to be achieved.
F

Assume the following resource usage parameters.

Resource Usage
Failure identification effort
Failure Correction effort
Computer time

Per hour

Per failure

2 Person hr.

1 Person hr.

5 Person hr.

1.5 CPU hr.

1 CPU hr.

80

Software Reliability
(a) What

resources

must

be

expended

to

achieve

the

reliability

improvement? Use the logarithmic Poisson execution time model with a

failure intensity decay parameter of 0.025/failure.
(b) If the failure intensity objective is cut to half, what is the effect on
requirement of resources ?

81

Software Reliability
Solution
(a)

1 P

Ln
F
1
20

Ln
119 failures
0.025 1

1 1
1


F P

1 1 1
1
1 0.05 38 CPU hrs.

0.025 1 20 0.025
82

Software Reliability
Hence

X1 1 1
= 1 (119) + 2 (38) = 195 Person hrs.

X F F
= 5 (119) = 595 Person hrs.

X C c c
= 1 (119) + (1.5) (38) = 176 CPU hr.
83

Software Reliability
(b)

F 0.5 failures/CPU hr.

1
20
Ln
148 failures
0.025 0.5

1 1
1

78 CPU hr.

0.025 0.5 20
So,

XI = 1 (148) + 2 (78) = 304 Person hrs.

XF = 5 (148) = 740 Person hrs.
XC = 1 (148) + (1.5)(78) = 265 CPU hrs.
84

Software Reliability
Hence, if we cut failure intensity objective to half, resources requirements
are not doubled but they are some what less. Note that

is

approximately double but increases logarithmically. Thus, the resources

increase will be between a logarithmic increase and a linear increase for
changes in failure intensity objective.

85

Software Reliability
Example- 6
A program is expected to have 500 faults. It is also assumed that one fault
may lead to one failure only. The initial failure intensity was 2 failures/CPU
hr. The program was to be released with a failure intensity objective of 5
failures/100 CPU hr. Calculated the number of failure experienced before
release.

86

Software Reliability
Solution
The number of failure experienced during testing can be calculated using
the equation mentioned below:

V0
P F
0
Here

V0 500 because one fault leads to one failure

0 2 failures/CPU hr.

F 5 failures/100 CPU hr.

0.05 failures/CPU hr.
87

Software Reliability

So

500
2 0.05

2
= 487 failures

Hence 13 faults are expected to remain at the release instant of

the software.

88

Software Reliability

The Jelinski-Moranda Model

(t ) ( N i 1)
where
= Constant of proportionality
N = Total number of errors present
I = number of errors found by time interval ti

89

Software Reliability

Fig.22: Relation between t &

90

Software Reliability
Example- 7
There are 100 errors estimated to be present in a program. We have
experienced 60 errors. Use Jelinski-Moranda model to calculate
failure intensity with a given value of =0.03. What will be failure
intensity after the experience of 80 errors?

91

Software Reliability
Solution
N = 100 errors
i = 60 failures
= 0.03
We know

(t ) 0.03(100 80 1)
= 0.03(100-60+1)
= 1.23 failures/CPU hr.

After 80 failures

(t ) 0.03(100 80 1)
= 0.63 failures/CPU hr.

Hence, there is continuous decrease in the failure intensity as the

92
number of failure experienced increases.

Software Reliability

The Bug Seeding Model

The bug seeding model is an outgrowth of a technique used to

estimate the number of animals in a wild life population or fish in a
pond.

Nt
nt

N N t n nt
n
N Nt
nt

n
N Ns
ns

93

Software Reliability

Capability Maturity Model

It is a strategy for improving the software process, irrespective of the

actual life cycle model used.

Fig.23: Maturity levels of CMM

94

Software Reliability
Maturity Levels:
Initial (Maturity Level 1)
Repeatable (Maturity Level 2)
Defined (Maturity Level 3)
Managed (Maturity Level 4)
Optimizing (Maturity Level 5)
95

Software Reliability
Maturity Level

Characterization

Initial

Adhoc Process

Repeatable

Basic Project Management

Defined

Process Definition

Managed

Process Measurement

Optimizing

Process Control

Fig.24: The five levels of CMM

96

Software Reliability

Key Process Areas

The key process areas at level 2 focus on the software projects

concerns related to establishing basic project management controls,
as summarized below:

97

Software Reliability
The key process areas at level 3 address both project and
organizational issues, as summarized below:

98

Software Reliability

99

Software Reliability
The key process areas at level 4 focus on establishing a quantitative
understanding of both the software process and the software work
products being built, as summarized below:

100

Software Reliability
The key process areas at level 5 cover the issues that both the
organization and the projects must address to implement continuous
and measurable software process improvement, as summarized
below:

101

Software Reliability

Common Features

102

Software Reliability

ISO 9000

The SEI capability maturity model initiative is an attempt to improve

software quality by improving the process by which software is
developed.
ISO-9000 series of standards is a set of document dealing with
quality systems that can be used for quality assurance purposes.
ISO-9000 series is not just software standard. It is a series of five
related standards that are applicable to a wide variety of industrial
activities, including design/ development, production, installation,
and serving. Within the ISO 9000 Series, standard ISO 9001 for
quality system is the standard that is most applicable to software
development.
103

Software Reliability

Mapping ISO 9001 to the CMM

1. Management responsibility
2. Quality system
3. Contract review
4. Design control
5. Document control
6. Purchasing
7. Purchaser-supplied product

104

Software Reliability
8. Product identification and traceability
9. Process control
10. Inspection and testing
11. Inspection, measuring and test equipment
12. Inspection and test status
13. Control of nonconforming product
14. Corrective action
105

Software Reliability
15. Handling, storage, packaging and delivery
16. Quality records
17. Internal quality audits
18. Training
19. Servicing
20. Statistical techniques
106

Software Reliability

Contrasting ISO 9001 and the CMM

There is a strong correlation between ISO 9001 and the CMM,

although some issues in ISO 9001 are not covered in the CMM, and
some issues in the CMM are not addressed in ISO 9001.
The biggest difference, however, between these two documents is
the emphasis of the CMM on continuous process improvement.
The biggest similarity is that for both the CMM and ISO 9001, the
bottom line is Say what you do; do what you say.

107

Multiple Choice Questions

Note: Choose most appropriate answer of the following questions:
7.1 Which one is not a phase of bath tub curve of hardware reliability
(a) Burn-in
(b) Useful life
(c) Wear-out
(d) Test-out
7.2 Software reliability is
(a) the probability of failure free operation of a program for a specified time
in a specified environment
(b) the probability of failure of a program for a specified time in a specified
environment
(c) the probability of success of a program for a specified time in any
environment
(d) None of the above
7.3 Fault is
(a) Defect in the program
(c) Error in the program

(b) Mistake in the program

(d) All of the above

7.4 One fault may lead to

(a) one failure
(c) many failures

(b) two failures

(d) all of the above

108

Multiple Choice Questions

7.5 Which time unit is not used in reliability studies
(a) Execution time
(b) Machine time
(c) Clock time
(d) Calendar time
7.6 Failure occurrences can be represented as
(a) time to failure
(b) time interval between failures
(c) failures experienced in a time interval (d) All of the above
7.7 Maximum possible value of reliability is
(a) 100
(b) 10
(c) 1
(d) 0
7.8 Minimum possible value of reliability is
(a) 100
(b) 10
(c) 1
(d) 0
7.9 As the reliability increases, failure intensity
(a) decreases
(b) increases
(c) no effect
(d) None of the above

109

Multiple Choice Questions

7.10 If failure intensity is 0.005 failures/hour during 10 hours of operation of a
software, its reliability can be expresses as
(a) Four portions
(b) Three portions
(c) Five portions
(d) Two portions
7.11 Software Quality is
(a) Conformance to requirements
(c) Level of satisfaction

(b) Fitness for the purpose

(d) All of the above

7.12 Defect rate is

(a) number of defects per million lines of source code
(b) number of defects per function point
(c) number of defects per unit of size of software
(d) All of the above
7.13 How many product quality factors have been proposed in McCall quality model?
(a) 2
(b) 3
(c) 11
(d) 6
110

Multiple Choice Questions

7.14 Which one is not a product quality factor of McCall quality model?
(a) Product revision
(b) Product operation
(c) Product specification
(d) Product transition
7.15 The second level of quality attributes in McCall quality model are termed as
(a) quality criteria
(b) quality factors
(c) quality guidelines
(d) quality specifications
7.16 Which one is not a level in Boehm software quality model ?
(a) Primary uses
(b) Intermediate constructs
(c) Primitive constructs
(d) Final constructs
7.17 Which one is not a software quality model?
(a) McCall model
(b) Boehm model
(c) ISO 9000
(d) ISO 9126
7.18 Basic execution time model was developed by
(a) Bev.Littlewood
(b) J.D.Musa
(c) R.Pressman
(d) Victor Baisili
111

Multiple Choice Questions

7.19 NHPP stands for
(a) Non Homogeneous Poisson Process (b) Non Hetrogeneous Poisson Process
(c) Non Homogeneous Poisson Product (d) Non Hetrogeneous Poisson
Product
7.20 In Basic execution time model, failure intensity is given by

(a ) ( ) 0 1
V
0

(b) ( ) 0 1
V0

V
(c) ( ) 0 1 0

V
(d ) ( ) 0 1 02

7.21 In Basic execution time model, additional number of failures required to

achieve a failure intensity objective ( ) is expressed as

V0
( P F )
0
0
(c) (F P )
V0
(a )

V0
( F P )
0
0
(d ) (P F )
V0
(b)

112

Multiple Choice Questions

7.22 In Basic execution time model, additional time required to achieve a failure
intensity objective ( ) is given as

(a )

0 F

Ln
V0 P

(b)

0 P

Ln
V0 F

( c )

V0 F

Ln
0 P

(d )

V0 P

Ln
0 F

7.23 Failure intensity function of Logarithmic Poisson execution model is given as

(a ) ( ) 0 LN ( )

(b) ( ) 0 exp( )

(c) ( ) 0 exp( )

(d ) ( ) 0 log( )

7.24 In Logarithmic Poisson execution model, is known as

(a) Failure intensity function parameter (b) Failure intensity decay parameter
(c) Failure intensity measurement
(d) Failure intensity increment parameter
113

Multiple Choice Questions

7.25 In jelinski-Moranda model, failure intensity is defined aseneous Poisson
Product
(a ) (t ) ( N i 1)
(b) (t ) ( N i 1)

(c) (t ) ( N i 1)
7.26 CMM level 1 has
(a) 6 KPAs
(c) 0 KPAs
7.27 MTBF stands for
(a) Mean time between failure
(c) Minimum time between failures
7.28 CMM model is a technique to
(a) Improve the software process
(c) Test the software

(d ) (t ) ( N i 1)
(b) 2 KPAs
(d) None of the above
(b) Maximum time between failures
(d) Many time between failures
(b) Automatically develop the software
(d) All of the above

7.29 Total number of maturing levels in CMM are

(a) 1
(b) 3
(c) 5
(d) 7

114

Multiple Choice Questions

7.30 Reliability of a software is dependent on number of errors
(a) removed
(b) remaining
(c) both (a) & (b)
(d) None of the above
7.31 Reliability of software is usually estimated at
(a) Analysis phase
(b) Design phase
(c) Coding phase
(d) Testing phase

7.32 CMM stands for

(a) Capacity maturity model
(c) Cost management model

(b) Capability maturity model

(d) Comprehensive maintenance model

7.33 Which level of CMM is for basic project management?

(a) Initial
(b) Repeatable
(c) Defined
(d) Managed
7.34 Which level of CMM is for process management?
(a) Initial
(b) Repeatable
(c) Defined
(d) Optimizing
115

Multiple Choice Questions

7.35 Which level of CMM is for process management?
(a) Initial
(b) Defined
(c) Managed
(d) Optimizing
7.36 CMM was developed at
(a) Harvard University
(c) Carnegie Mellon University
7.37 McCall has developed a
(a) Quality model
(c) Requirement model

(b) Cambridge University

(d) Maryland University
(b) Process improvement model
(d) Design model

7.38 The model to measure the software process improvement is called

(a) ISO 9000
(b) ISO 9126
(c) CMM
(d) Spiral model
7.39 The number of clauses used in ISO 9001 are
(a) 15
(b) 25
(c) 20
(d) 10
116

Multiple Choice Questions

7.40 ISO 9126 contains definitions of
(a) quality characteristics
(c) quality attributes

(b) quality factors

(d) All of the above

7.41 In ISO 9126, each characteristics is related to

(a) one attributes
(b) two attributes
(c) three attributes
(d) four attributes
7.42 In McCall quality model; product revision quality factor consist of
(a) Maintainability
(b) Flexibility
(c) Testability
(d) None of the above
7.43 Which is not a software reliability model ?
(a) The Jelinski-Moranda Model
(b) Basic execution time model
(c) Spiral model
(d) None of the above
7.44 Each maturity model is CMM has
(a) One KPA
(c) Several KPAs

(b) Equal KPAs

(d) no KPA
117

Multiple Choice Questions

7.45 KPA in CMM stands for
(a) Key Process Area
(c) Key Principal Area

(b) Key Product Area

(d) Key Performance Area

7.46 In reliability models, our emphasis is on

(a) errors
(b) faults
(c) failures
(d) bugs
7.47 Software does not break or wear out like hardware. What is your opinion?
(a) True
(c) Can not say

(b) False
(d) not fixed

7.48 Software reliability is defined with respect to

(a) time
(b) speed
(c) quality
(d) None of the above
7.49 MTTF stands for
(a) Mean time to failure
(c) Minimum time to failure

(b) Maximum time to failure

(d) None of the above
118

Multiple Choice Questions

7.50 ISO 9000 is a series of standards for quality management systems and has
(a) 2 related standards
(b) 5 related standards
(c) 10 related standards
(d) 25 related stadards

119

Exercises
7.1 What is software reliability? Does it exist?
7.2 Explain the significance of bath tube curve of reliability with the help of
a diagram.
7.3 Compare hardware reliability with software reliability.
7.4 What is software failure? How is it related with a fault?
7.5 Discuss the various ways of characterising failure occurrences with
respect to time.
7.6 Describe the following terms:
(i) Operational profile
(iii) MTBF
(v) Failure intensity.

(ii)
(iv)

Input space
MTTF

120

Exercises
7.7 What are uses of reliability studies? How can one use software reliability
measures to monitor the operational performance of software?
7.8 What is software quality? Discuss software quality attributes.
7.9 What do you mean by software quality standards? Illustrate their essence
as well as benefits.
7.10 Describe the McCall software quality model. How many product quality
factors are defined and why?
7.11 Discuss the relationship between quality factors and quality criteria in
McCalls software quality model.
7.12 Explain the Boehm software quality model with the help of a block
diagram.
7.13 What is ISO9126 ? What are the quality characteristics and attributes?
121

Exercises
7.14 Compare the ISO9126 with McCall software quality model and
highlight few advantages of ISO9126.
7.15 Discuss the basic model of software reliability. How and can be
calculated/
7.16 Assume that the initial failure intensity is 6 failures/CPU hr. The failure
intensity decay parameter is 0.02/failure. We assume that 45 failures have
been experienced. Calculate the current failure intensity.
7.17 Explain the basic & logarithmic Poisson model and their significance in
reliability studies.

122

Exercises
7.18 Assume that a program will experience 150 failures in infinite time. It
has now experienced 80. The initial failure intensity was 10 failures/CPU
hr.
(i) Determine the current failure intensity
(ii) Calculate the failures experienced and failure intensity after 25 and
40 CPU hrs. of execution.
(iii) Compute additional failures and additional execution time required
to reach the failure intensity objective of 2 failures/CPU hr.
Use the basic execution time model for the above mentioned
calculations.
7.19 Write a short note on Logarithmic Poisson Execution time model. How
can we calculate & ?
7.20 Assume that the initial failure intensity is 10 failures/CPU hr. The failure
intensity decay parameter is 0.03/failure. We have experienced 75
failures upto this time. Find the failures experienced and failure intensity
after 25 and 50 CPU hrs. of execution.
123

Exercises
7.21 The following parameters for basic and logarithmic Poisson models are
given:

Determine the additional failures and additional execution time required

to reach the failure intensity objective of 0.1 failure/CPU hr. for both
models.
7.22 Quality and reliability are related concepts but are fundamentally
different in a number of ways. Discuss them.
7.23 Discuss the calendar time component model. Establish the relationship
between calendar time to execution time.
124

Exercises
7.24 A program is expected to have 250 faults. It is also assumed that one
fault may lead to one failure. The initial failure intensity is 5 failure/CPU
hr. The program is released with a failure intensity objective of 4
failures/10 CPU hr. Calculate the number of failures experienced before
release.
7.25 Explain the Jelinski-Moranda model of reliability theory. What is the
relation between t and ' ' ?
7.26 Describe the Mills bug seeding model. Discuss few advantages of this
model over other reliability models.
7.27 Explain how the CMM encourages continuous improvement of the
software process.
7.28 Discuss various key process areas of CMM at various maturity levels.
7.29 Construct a table that correlates key process areas (KPAs) in the CMM
with ISO9000.
7.30 Discuss the 20 clauses of ISO9001 and compare with the practices in
the CMM.
125

Exercises
7.31 List the difference of CMM and ISO9001. Why is it suggested that
CMM is the better choice than ISO9001?
7.32 Explain the significance of software reliability engineering. Discuss the
advantage of using any software standard for software development?
7.33 What are the various key process areas at defined level in CMM?
Describe activities associated with one key process area.
7.34 Discuss main requirements of ISO9001 and compare it with SEI
capability maturity model.
7.35 Discuss the relative merits of ISO9001 certification and the SEI CMM
based evaluation. Point out some of the shortcomings of the ISO9001
certification process as applied to the software industry.

126