Probability and Statistics with

Reliability, Queuing and Computer

Science Applications: Chapter 3
Continuous Random Variables
 Definitions
• Distribution function:

• If FX(x) is a continuous function of x, then X is a

continuous random variable.
– FX(x): discrete in x  Discrete rv’s
– FX(x): piecewise continuous  Mixed rv’s
•
 Definitions (Continued)

Equivalence:

• CDF (cumulative distribution function)

• PDF (probability distribution function)

• Distribution function
• FX(x) or FX(t) or F(t)
 Probability Density Function (pdf)
• X : continuous rv, then,

• pdf properties:
1.

2.
 Definitions
(Continued)

• Equivalence: pdf
– probability density function

– density function

– density
t
dF F (t )   f ( x)dx
– f(t) = 
dt t
  f ( x)dx , For a non-negative
0 random variable
 Exponential Distribution

• Arises commonly in reliability & queuing theory.

• A non-negative random variable
• It exhibits memoryless (Markov) property.
• Related to (the discrete) Poisson distribution
– Interarrival time between two IP packets (or voice calls)
– Time to failure, time to repair etc.
• Mathematically (CDF and pdf, respectively):
CDF of exponentially distributed
random variable with  = 0.0001

F(t)

12500 25000 37500 50000

t
 Exponential Density Function
(pdf)

f(t)

t
 Memoryless property
• Assume X > t. We have observed that the
component has not failed until time t.

• Let Y = X - t , the remaining (residual) lifetime

• The distribution of the remaining life, Y, does not

depend on how long the component has been
operating. Distribution of Y is identical to that of X.
 Memoryless property
• Assume X > t. We have observed that the
component has not failed until time t.
• Let Y = X - t , the remaining (residual)
lifetime
Gt ( y )  P(Y  y | X  t )
 P( X  y  t | X  t )
P(t  X  y  t )  y
  1 e
P( X  t )
 Memoryless property (Continued)
• Thus Gt(y) is independent of t and is identical
to the original exponential distribution of X.
• The distribution of the remaining life does
not depend on how long the component has
been operating.
• Its eventual breakdown is the result of some
suddenly appearing failure, not of gradual
deterioration.
 Reliability as a Function of Time

• Reliability R(t): failure occurs after time ‘t’. Let X

be the lifetime of a component subject to failures.

• Let N0: total no. of components (fixed); Ns(t):

surviving ones; Nf(t): failed one by time t.
 Definitions (Continued)

Equivalence:

• Reliability

• Complementary distribution function

• Survivor function

• R(t) = 1 -F(t)
 Failure Rate or Hazard Rate

• Instantaneous failure rate: h(t) (#failures/10k hrs)

• Let the rv X be EXP( λ). Then,

• Using simple calculus the following apples to any rv,

 Hazard Rate and the pdf
f (t ) f (t )
h (t )  
R (t ) 1  F (t )

h(t) t = Conditional Prob. system will fail in

(t, t + t) given that it has survived until time t
f(t) t = Unconditional Prob. System will fail in
(t, t + t)
• Difference between:
– probability that someone will die between 90
and 91, given that he lives to 90
– probability that someone will die between 90
and 91
 Weibull Distribution
• Frequently used to model fatigue failure, ball bearing failure
etc. (very long tails)

• Reliability: Rt   e
 t
t 0
• Weibull distribution is capable of modeling DFR (α < 1), CFR
(α = 1) and IFR (α >1) behavior.
• α is called the shape parameter and  is the scale parameter
 Failure rate of the weibull
distribution with various values
of  and  = 1

5.0

1.0 2.0 3.0 4.0

Infant Mortality Effects in System
Modeling
• Bathtub curves
– Early-life period

– Steady-state period

– Wear out period

• Failure rate models

 Bathtub Curve
•Until now we assumed that failure rate of equipment is time (age)
independent. In real-life, variation as per the bathtub shape has been
observed
Failure Rate (t)

Infant Mortality
(Early Life Failures) Steady State Wear out

Operating Time
 Early-life Period
• Also called infant mortality phase or reliability
growth phase
• Caused by undetected hardware/software defects
that are being fixed resulting in reliability growth
• Can cause significant prediction errors if steady-
state failure rates are used
• Availability models can be constructed and solved
to include this effect
• Weibull Model can be used
 Steady-state Period
• Failure rate much lower than in early-life
period
• Either constant (age independent) or slowly
varying failure rate
• Failures caused by environmental shocks
• Arrival process of environmental shocks can
be assumed to be a Poisson process
• Hence time between two shocks has the
exponential distribution
 Wear out Period
• Failure rate increases rapidly with age
• Properly qualified electronic hardware do
not exhibit wear out failure during its
intended service life (Motorola)
• Applicable for mechanical and other
systems
• Weibull Failure Model can be used
 Bathtub curve
h(t) DFR phase: Initial design, constant bug fixes
CFR phase: Normal operational phase
IFR phase: Aging behavior
(burn-in-period) (wear-out-phase)

CFR
(useful life)

DFR IFR

Decreasing failure rate Increasing fail. rate

 Failure Rate Models
•We use a truncated Weibull Model

7
Failure-Rate Multiplier

6
5
4
3
2
1
0
0 2,190 4,380 6,570 8,760 10,950 13,140 15,330 17,520
Operating Times (hrs)

•Infant mortality phase modeled by DFR Weibull and the

steady-state phase by the exponential
 Failure Rate Models (cont.)
• This model has the form:
W ( t )  C 1 t  1  t  8,760
  SS t  8,760
• where:
• C 1  W 1,  SS  steady-state failure rate
•  is the Weibull shape parameter
• Failure rate multiplier = W ( t)  SS
 Failure Rate Models (cont.)
• There are several ways to incorporate time dependent failure rates in
availability models
• The easiest way is to approximate a continuous function by a
decreasing step function

7
Failure-Rate Multiplier

6
5 1
4

2
3
2
1
 SS
0
0 2,190 4,380 6,570 8,760 10,950 13,140 15,330 17,520
Operating Times (hrs)
 Failure Rate Models (cont.)
•Here the discrete failure-rate model is
defined by:
W ( t )   1 0  t  4,380
 2 4,380  t  8,760
  ss t  8,760
 Uniform Random Variable

• All (pseudo) random generators generate random

deviates of U(0,1) distribution; that is, if you
generate a large number of random variables and
plot their empirical distribution function, it will
approach this distribution in the limit.
• U(a,b)  pdf constant over the (a,b) interval and
CDF is the ramp function
 Uniform density
U(0,1) pdf

1.2

0.8
cdf

0.6

0.4

0.2

0
0 0.1 0.2 0.3 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 2.1 2.2
tim e
 Uniform distribution

• The distribution function is given by:

0 , x < a,

{
F(x)= xa ,
ba
1 ,
a <x<b

x > b.
 cdf

0.2
0.4
0.6
0.8
1.2

0
1
0
0.08
0.16
0.24
0.32
0.4
0.48
0.56
0.64
0.72
0.8

time
0.88
U(0,1) cdf

0.96
1
1.04
1.08
1.16
1.24
Uniform distribution (Continued)

1.32
1.4
1.48
U(
 HypoExponential

• HypoExp: multiple Exp stages in series.

• 2-stage HypoExp denoted as HYPO(λ1, λ2). The
density, distribution and hazard rate function are:

• HypoExp results in IFR: 0  min(λ1, λ2)

• Disk service time may be modeled as a 3-stage
Hypoexponential as the overall time is the sum of
the seek, the latency and the transfer time
 HypoExponential used in software
rejuvenation models

• Preventive maintenance is useful only if failure rate is increasing

• A simple and useful model of increasing failure rate:
Failure
probable
Robust state Failed state
state

Time to failure: Hypo-exponential distribution

Increasing failure rate aging
 Erlang Distribution

• Special case of HypoExp: All stages have same rate.

• [X > t] = [Nt < r] (Nt : no. of stresses applied in (0,t])

and Nt is Possion (param λt). This interpretation gives,
 Erlang Distribution

• Is used to approximate the deterministic one

since if you keep the same mean but increase
the number of stages, the pdf approaches the
delta function in the limit
• Can also be used to approximate the uniform
distribution
 probability density functions (pdf)

If we vary r keeping r/ constant, pdf of r-stage Erlang approaches

an impulse function at r/ .
 cumulative distribution functions (cdf)

And the cdf approaches a step function at r/. In other words

r-stage Erlang can approximate a deterministic variable.
 Comparison of
1.8
probability density functions (pdf)
1.6

1.4

1.2

1
3-stage Erlang pdf
pdf

U(0,1) pdf
0.8

0.6

0.4

0.2

0
T=1
0

9
06

12

18

24

36

42

48

54

66

72

78

84

96

02
0.

0.

0.
0.

0.

0.

0.

0.

0.

0.

0.

0.

0.

0.

0.

0.

1.
time
 Comparison of cumulative distribution
1.2
functions (cdf)
1

0.8

3-stage Erlang cdf

cdf

0.6
U(0,1) cdf

0.4

0.2

0
T=1
0

9
06

12

18

24

36

42

48
54

66

72

78

84

96

02
0.

0.

0.
0.

0.

0.

0.

0.

0.

0.
0.

0.

0.

0.

0.

0.

1.
time
 Gamma Random Variable

• Gamma density function is,

• Gamma distribution can capture all three failure modes,

viz. DFR, CFR and IFR.
– α = 1: CFR
– α <1 : DFR
– α >1 : IFR
• Gamma with α = ½ and  = n/2 is known as the chi-
square random variable with n degrees of freedom
 HyperExponential Distribution
• Hypo or Erlang  Sequential Exp( ) stages.
• Alternate Exp( ) stages  HyperExponential.

• CPU service time may be modeled as HyperExp

• In workload based software rejuvenation model
we found the sojourn times in many workload
states have this distribution
 Log-logistic Distribution
• Log-logistic can model DFR, CFR and IFR failure models
simultaneously, unlike previous ones.

• For, κ > 1, the failure rate first increases with t (IFR); after
momentarily leveling off (CFR), it decreases (DFR) with
time. This is known as the inverse bath tub shape curve
• Use in modeling software reliability growth
Hazard rate comparison
 Defective Distribution

• If
• Example:

• This defect (also known as the mass at infinity) could

represent the probability that the program will not terminate
(1-c). Continuous part can model completion time of
program.
• There can also be a mass at origin.
 Pareto Random Variable
• Also known as the power law or long-tailed
distribution
• Found to be useful in modeling
– CPU time consumed by a request
– Webfile sizes
– Number of data bytes in FTP bursts
– Thinking time of a Web browser
 Gaussian (Normal) Distribution
• Bell shaped pdf – intuitively pleasing!
• Central Limit Theorem: mean of a large number of
mutually independent rv’s (having arbitrary
distributions) starts following Normal distribution
as n 

• μ: mean, σ: std. deviation, σ2: variance (N(μ, σ2))

• μ and σ completely describe the statistics. This is
significant in statistical estimation/signal
processing/communication theory etc.
 Normal Distribution (contd.)

• N(0,1) is called normalized Guassian.

• N(0,1) is symmetric i.e.
– f(x)=f(-x)
– F(z) = 1-F(z).
• Failure rate h(t) follows IFR behavior.
– Hence, N( ) is suitable for modeling long-term wear or
aging related failure phenomena.
 Functions of Random Variables

• Often, rv’s need to be transformed/operated upon.

– Y = Φ (X) : so, what is the density of Y ?
– Example: Y = X2

– If X is N(0,1), then,

– Above Y is also known as the χ2 distribution (with 1-

degree of freedom).
 Functions of RV’s (contd.)

• If X is uniformly distributed, then, Y= -λ-1ln(1-X)

follows Exp(λ) distribution
– transformations may be used to generate random
variates (or deviates) with desired distributions.
 Functions of RV’s (contd.)

• Given,
• A monotone differentiable function,
•

• Above method suggests a way to get the random variates with desired
distribution.
– Choose Φ to be F.
– Since, Y=F(X), FY(y) = y and Y is U(0,1).
– To generate a random variate with X having desired distribution, generate
U(0,1) random variable Y, then transform y to x= F-1(y) .
– This inversion can be done in closed-form, graphically or using a table.
 Jointly Distributed RVs

• Joint Distribution Function:

• Independent rv’s: iff the following holds:

Joint Distribution Properties
Joint Distribution Properties (contd)
Order statistics: kofn, TMR
 Order Statistics: KofN
X1 ,X2 ,..., Xn iid (independent and identically distributed)
random variables with a common distribution function
F().
Let Y1 ,Y2 ,...,Yn be random variables obtained by
permuting the set X1 ,X2 ,..., Xn so as to be in
increasing order.
To be specific:
Y1 = min{X1 ,X2 ,..., Xn} and
Yn = max{X1 ,X2 ,..., Xn}
 Order Statistics: KofN
(Continued)

• The random variable Yk is called the k-th ORDER

STATISTIC.
• If Xi is the lifetime of the i-th component in a system of n
components. Then:
– Y1 will be the overall series system lifetime.
– Yn will denote the lifetime of a parallel system.
– Yn-k+1 will be the lifetime of an k-out-of-n system.
 Order Statistics: KofN (Continued)
To derive the distribution function of Yk, we note that the
probability that exactly j of the Xi's lie in (- ,y] and (n-j)
lie in (y, ) is:
n j n j
  F ( y ) [1  F ( y )]
 j
hence
n
n j
FYk ( y )     F ( y ) [1  F ( y )] n j

j k  j 
 Applications of order statistics
• Reliability of a k out of n system
n
Rkofn (t )   ( nj )[ R(t )] j [1  R(t )]n  j
j k

• Series system: n
Rseries (t )  [ R(t )] n
or  Ri (t )
i 1

• Parallel system: R parallel(t )  1  [1  R(t )]n

• Minimum of n EXP random variables is special case
of Y1 = min{X1,…,Xn} where Xi~EXP(i)
Y1~EXP( i)
• This is not true (that is EXP dist.) for the parallel case
 Triple Modular Redundancy (TMR)
R(t)

R(t) Voter

R(t)

• An interesting case of order statistics occurs when

we consider the Triple Modular Redundant (TMR)
system (n = 3 and k = 2). Y2 then denotes the time
until the second component fails. We get:
RTMR (t )  3R (t )  2R (t )
2 3
 TMR (Continued)

• Assuming that the reliability of a single

component is given by,
 t
e

we get:
2 t 3t
RTMR (t )  3e  2e
 TMR (Continued)

• In the following figure, we have plotted

RTMR(t) vs t as well as R(t) vs t.
TMR (Continued)
 Cold standby (dynamic redundancy)

X Y

Lifetime of Lifetime of
Active Spare
EXP()
EXP()
Total lifetime 2-Stage Erlang

 t EXP() EXP()
R(t )  (1  t )e
Assumptions: Detection & Switching perfect; spare
does not fail
 Sum of RVs: Standby Redundancy
• Two independent components, X and Y
– Series system (Z=min(X,Y))
– Parallel System (Z=max(X,Y))
– Cold standby: the life time Z=X+Y
 Sum of Random Variables

• Z = Φ(X, Y)  ((X, Y) may not be independent)

• For the special case, Z = X + Y

• The resulting pdf is (assuming independence),

• Convolution integral (modify for the non-negative

case)
 Convolution (non-negative case)

Z = X + Y, X & Y are independent random

variables (in this case, non-negative)
t
f Z (t )   f X ( x) fY (t  x) dx
0

• The above integral is often called the

convolution of fX and fY. Thus the density of
the sum of two non-negative independent,
continuous random variables is the
convolution of the individual densities.
 Cold standby derivation
• X and Y are both EXP() and independent.
• Then
t
f t (t )   e e
 x  ( t  x )
dx
0
t

 dx
2  t
 e
0
 t
  te , t  0
2
 Cold standby derivation
(Continued)

• Z is two-stage Erlang Distributed

t
FZ (t )   f Z ( z )dz  1  (1  t )e  t
0

R(t )  1  F (t )
 t
 (1  t )e , t  0
 Convolution: Erlang
Distribution
• The general case of r-stage Erlang Distribution

• When r sequential phases have independent

identical exponential distributions, then the

resulting density is known as r-stage (or r-phase)

Erlang and is given by:

Convolution: Erlang (Continued)

EXP() EXP() EXP()

r r 1  t
t e
f (t ) 
(r  1)!
(  t ) k  t
r 1
F (t )  1   e
k 0 k!
 Warm standby
•With Warm spare, we have:
•Active unit time-to-failure: EXP()
•Spare unit time-to-failure: EXP()

EXP(+ ) EXP()

2-stage hypoexponential distribution

 Warm standby derivation
• First event to occur is that either the active or the spare wil
fail. Time to this event is min{EXP(),EXP()} which is
EXP( + ).

• Then due to the memoryless property of the exponential,

remaining time is still EXP().

• Hence system lifetime has a two-stage hypoexponential

distribution with parameters

1 =  +  and 2 =  .
 Warm standby derivation
(Continued)

• X is EXP(1) and Y is EXP(2) and are

independent
1 = 2
• Then fZ(t) is
t
f Z (t )   1e  1 x
2e  2 ( t  x )
dx
0

12  t 12  t
 e  e 2 1

1  2 2  1
 Hot standby
•With hot spare, we have:
•Active unit time-to-failure: EXP()
•Spare unit time-to-failure: EXP()

EXP(2) EXP()

2-stage hypoexponential
 TMR and TMR/simplex
as hypoexponentials

TMR/Simplex

EXP(3) EXP()

TMR
EXP(3) EXP(2)
 Hypoexponential: general case
r
• Z=  X , where X
i 1
i 1 ,X2 , … , Xr
are mutually independent and Xi is exponentially
distributed with parameter i (i = j for i = j).
Then Z is a r-stage hypoexponentially
distributed random variable.

EXP(1) EXP(2) EXP(r)

Hypoexponential: general case
 KofN system lifetime as a
hypoexponential
At least, k out of n units should be operational for the
system to be Up.

EXP(n) EXP((n-1)) ... EXP(k) EXP((k-1)) ... EXP()

Y1 Y2 Yn-k+1 Yn-k+2 Yn
 KofN with warm spares

At least, k out of n + s units should be operational

for the system to be Up. Initially n units are active
and s units are warm spares.
EXP(n EXP(n ... EXP(n
s) +(s-1) ) + )
EXP(n) ... EXP(k)
 Sum of Normal Random Variables

• X1, X2, .., Xk are normal ‘iid’ rv’s, then, the rv

Z = (X1+ X2+ ..+Xk) is also normal with,

• X1, X2, .., Xk are normal. Then,

follows Gamma or the χ2 (with n-degrees
of freedom) distribution