Simulation Chapter 3
Simulation Chapter 3
CSIT
Chapter 3
3.1 Introductions
Random numbers are a necessary basic ingredient in the simulation of almost all discrete
systems. Most computer languages have a subroutine, object, or function that will generate a
random number. Similarly simulation languages generate random numbers that arc used to
generate event limes and other random variables.
The random number in random number table exactly obey two random number properties:
uniformity and independence so random number generated form table also called true random
numbers.
Table of random numbers are used to create a Radom sample. A random number table is also
called random sample table.
There are many physical devices or process that can be used to generate a sequence of uniformly
distributed random numbers i.e. true random numbers. For example: An electrical pulse
generator can be made to drive a counter cycling from 0 to 9. Using an electronic noise generator
or radioactive source the pulse can be generated as random numbers.
Pseudo means false, so false random numbers are being generated. The goal of any generation
scheme is to produce a sequence of numbers between zero and 1 which simulates, or imitates, the
ideal properties of uniform distribution and independence as closely as possible. When
generating pseudo-random numbers, certain problems or errors can occur.
3. The mean of the generated numbers may be too high or too low.
(c) Several numbers above the mean followed by several numbers below the mean.
1. The routine should be fast. . The total cost can be managed by selecting a computationally
efficient method of random-number generation.
2. The routine should be portable to different computers, and ideally to different programming
languages .This is desirable so that the simulation program produces the same results wherever it
is executed.
3. The routine should have a sufficiently long cycle. The cycle length, or period, represents the
length of the random-number sequence before previous numbers begin to repeat themselves in an
earlier order. Thus, if 10,000 events are to be generated, the period should be many times that
long, A special case cycling is degenerating. A routine degenerates when the same random
numbers appear repeatedly. Such an occurrence is certainly unacceptable. This can happen
rapidly with some methods.
4. The random numbers should be replicable. Given the starting point (or conditions), it should
be possible to generate the same set of random numbers, completely independent of the system
that is being simulated. This is helpful for debugging purpose and is a means of facilitating
comparisons between systems.
5. Most important, and as indicated previously, the generated random numbers should closely
approximate the ideal statistical properties of uniformity and independences
The linear congruential method, initially proposed by Lehmer [1951], produces a sequence of
integers, X1, X2,... between zero and m — 1 according to the following recursive relationship:
The initial value X0 is called the seed, a is called the constant multiplier, c is the increment, and
m is the modulus.
Case 1:
Case 2
When c = 0, the form is known as the multiplicative congruential method. The selection of the
values for a, c, m and Xo drastically affects the statistical properties and the cycle length. . An
example will illustrate how this technique operates.
EXAMPLE 3.1
Use the linear congruential method to generate a sequence of random numbers with X0 = 27, a=
17, c = 43, and m = 100. Here, the integer values generated will all be between zero and 99
because of the value of the modulus. These random integers should appear to be uniformly
distributed the integers zero to 99.
Random numbers between zero and 1 can be generated by Ri =Xi/m, i= 1,2,…… equation (3.2)
X0 = 27
R1=2⁄100=0. 02
R2=77 ⁄100=0. 77
R3=52 ⁄100=0. 52
First, notice that the numbers generated from Equation (7.2) can only assume values from the set
I = {0,1 /m, 2/m,..., (m — l)/m), since each Xi is an integer in the set {0,1,2,..., m -1}. Thus, each
Ri is discrete on I, instead of continuous on the interval [0, 1], This approximation appears to be
of little consequence, provided that the modulus m is a very large integer.
(Values such as m = 231 -1 and m = 248 are in common use in generators appearing in many
simulation languages.)
By maximum density is meant that the values assumed by Ri = 1,2,..., leave no large gaps on
[0,1].
EXAMPLE 3.2
Let m = 102 = 100, a = 19, c = 0, and X0 = 63, and generate a sequence c random integers using
Xi+1 = (a Xi + c) mod m.
X0 = 63
When m is a power of 10, say m = 10b , the modulo operation is accomplished by saving the b
rightmost (decimal) digits.
EXAMPLE 4.4
Let a = 75 = 16,807, m = 231-1 = 2,147,483,647 (a prime number), and c= 0. These choices
satisfy the conditions that insure a period of P = m-1. Further, specify a seed, X0 = 123,457.
X1 = 2,074,941,799
R1= X1 ⁄231
R2 = X2 ⁄231= 0.2607
R3 = X3 ⁄231= 0.7662
By Upendra R. Joshi Page 4
Simulation and Modeling B.Sc.CSIT
This Method requires a sequence of uniformly distributed random numbers. If ui (i=1,2,3,...) are
independent uniformly distributed random numbers over the interval 0 to 1 and F-1(x) is the
inverse of the cumulative distribution function for random variable X then the random variables
generated using inverse transformation method will be xi=F-1(ui). That is to produce random
numbers from given probability function; the inverse cumulative distribution function must be
evaluated with a sequence of uniformly distributed numbers in the interval 0 to 1.
i. e. f(xu)= u 2
u2
xu=f-1(u)
2
u
u2
2
2u1 u2
2
n uniform random numbers u1, u2, .......,un in the interval (0,1) and apply inverse transformation
function f-1(ui) to each.
Example:
Derive an equation to generate non-uniform random numbers from an exponential having pdf
f(x) =λe-λx (x>=0) using inverse transformation method.
Soln:
1. Compute the cdf of the given random variable
cdf=
or, -λx=ln(1-u)
or, x= -1/λ ln (1-u)
this equation is called a random variable generate for the exponential distribution. this equation
can be written as x=f-1(u)
4. Generate uniform number u1,u2,.....un and compute the desired random variables by using
xi= f-1(u) ; for exponential distribution f-1(u)= -1/λ ln (1-u) for (i=1,2,3....)
since ui and 1-ui are uniformly distributed random numbers between 0 and 1, we can replace (1-
ui) by ui. Therefore the equation becomes xi=-1/λ ln (ui).
This method is applicable when the probability density function f(x) has a lower and upper limit
to its range (A,B) and upper bound C. This method for obtaining samples from a given non
uniform distribution basically works by generating uniform random numbers repeatedly and
accepting only those numbers that meet a certain condition. for the rejection method to be
applicable the pdf must be non zero only over a finite interval (A,B).
1. Generate a pair of independent uniformly distributed variables u1 and u2 in the interval (0,1).
4. If Q<=f(x) accept P as the value of a sample from the desired distribution, otherwise reject the
pair and go to step 1. i. e. repeat the above process with a pair of new uniform variables.
In the above process steps 1,2,3 create a random points and the last step relates the points to the
curve of the pdf. If the point P is accepted as the sample from the desired distribution else the
point is rejected and the process as repeated.
Q=Cu2
A P=A+(B-A)u1 B
The desirable properties of random numbers — uniformity and independence To insure that
these desirable properties are achieved, a number of tests can be performed (fortunately, the
appropriate tests have already been conducted for most commercial simulation software}. The
tests can be placed in two categories according to the properties of interest.
The testing for uniformity can be achieved through different frequency test. These tests use the
Kolmogorov-Smirnov or the chi- square test to compare the distribution of the set of numbers
generated to a uniform distribution.
a) Kolmogorov-Smirnov test
Test for independence includes the five types of tests as given below
a) Autocorrelation test Tests the correlation between numbers and compares the sample
correlation to the expected correlation of zero.
b) Gap test. Counts the number of digits that appear between repetitions of particular digit and
then uses the Kolmogorov-Smirnov test to compare with the expected size of gaps,
c) Poker test . Treats numbers grouped together as a poker hand. Then the hands obtained are
compared to what is expected using the chi-square test.
The detail description of each of these tests is given below. In testing for uniformity, the
hypotheses are as follows:
H0: Ri ~ U/[0,1]
H1: Ri ~U/[0,l]
The null hypothesis, H0 reads that the numbers are distributed uniformly on the interval [0, 1].
Failure to reject the null hypothesis means that no evidence of non-uniformity has been detected
on the basis of this test. This does not imply that further testing of the generator for uniformity is
unnecessary.
For each test, a level of significance a must be stated. The level a is the probability of rejecting
the null hypothesis given that the null hypothesis is true, or
The decision maker sets the value of & for any test. Frequently, a is set to 0.01 or O.05.
This test compares the continuous cdf, F(X), of the uniform distribution to the empirical cdf,
SN(x), of the sample of N observations.
By definition,
If the sample from the random-number generator is R1 R2, ,• • •, RN, then the empirical cdf,
SN(X),
is defined by
As N becomes larger, SN(X) should become a better approximation to F(X) , provided that the
The Kolmogorov-Smirnov test is based on the largest absolute deviation or difference between
F(x) and SN(X) over the range of the random variable. I.e. it is based on the statistic
For testing against a uniform cdf, the test procedure follows these steps:
Step 1. Rank the data from smallest to largest. Let R(i) denote the i th smallest observation, so
that R (1) <= R (2) <= • • • <= R (N)
Step 2. Compute
D+ = MAX
D = Max
Step 4. Determine the critical value, Dα, from Table A.8( in your Text book) for the specified
significance level α and the given sample size N.
Step 5.
If the sample statistic D is greater than the critical value, Dα, the null hypothesis that the data are
a sample from a uniform distribution is rejected.
If D <= Da, conclude that no difference has been detected between the true distribution of { R1
R2, ,• • •, Rn } and the uniform distribution. Hence the null hypothesis is accepted.
Example
Suppose that the five numbers 0.44, 0.81, 0.14, 0.05, 0.93 were generated, and it is desired to
perform a test for uniformity using the Kolmogorov-Smirnov test with a level of significance a
of 0.05.
Solution
First, the numbers must be ranked from smallest to largest. I.e the given numbers 0.05, 0.14 0.44
, 0.81, 0.93. The calculations can be facilitated by use of Table below.
For example
At R(3) the value of D+ is given by 3/5 - R(3) = 0.60 - 0.44 =0.16 and of D- is given by R(3) =
2/5 = 0.44 - 0.40 = 0.04. and other value also can be computed similarly.
Now The statistics are computed as D+ = 0.26 (Maximum of the row i/N-Ri) and D- = 0.21
(maximum of the row Ri –(i-1)/N) . Therefore, D = max{0.26, 0.21} = 0.26.
The critical value of D, obtained from Table A.8 (in Text book) for a = 0.05 and N = 5, is 0.565.
Since the computed value, 0.26, is less than the tabulated critical value, 0.565, the hypothesis of
no difference between the distribution of the generated numbers and the uniform distribution is
not rejected.
where Oi; is the observed number in the i th class, Ei is the expected number in the ith class, and
n is the number of classes. For the uniform distribution, Ei the expected number in each class is
given by Ei = N/n ; for equally spaced classes, where N is the total number of observations. It
can be shown that the sampling distribution of χ02 is approximately the chi-square distribution
with n - 1 degrees of freedom.
Example 2.1
Use the chi-square test with a = 0.05 to test whether the data shown below are uniformly
distributed.
0.34, 0.83, 0.96, 0.47, 0.79, 0.99, 0.37, 0.72, 0.06, 0.18,
0.25 0.79 0.77, 0.17 0.23 0.99 0.54 0.56 0.84 0.97 0.89
0.64 ,0.67 0.82 0.19 0.46 0.01 0.97 0.24 0.88 0.87
Solutions
1 8 10 -2 4 0.4
2 8 10 -2 4 0.4
3 10 10 0 0 0.0
4 9 10 -1 1 0.1
5 12 10 2 4 0.4
6 8 10 -2 4 0.4
7 10 10 0 0 0.0
8 14 10 4 16 0.16
9 10 10 0 0 0.0
10 11 10 1 1 0.1
Above Table contains the essential computations for chi square test.The test uses n = 10 intervals
of equal length, namely [0.0, 0.1), [0.1, 0.2), . . . , [0.9, 1.0). The value of χ02 is 3.4.
Here degree of freedom is n-1=10-1=9 and α=0.05. The tabulated value of χ2 0.05, 9 =16.9.Since
χ02 is much smaller than the tabulated value of chi square, the null hypothesis of a uniform
distribution is not rejected.
The tests for autocorrelation are concerned with the dependence between numbers in a sequence.
As an example, consider the following sequence of numbers:
0.12 0.01 0.23 0.28 0.89 0.31 0.64 0.28 0.83 0.93 0.99 0.15 0.33 0.35 0.91 0.41 0.60 0.27 0.75
0.88 0.68 0.49 0.05 0.43 0.95 0.58 0.19 0.36 0.69 0.87
From a visual inspection, these numbers appear random, and they would probably pass all the
tests presented to this point. However, an examination of the 5th, 10th, 15th (every five numbers
beginning with the fifth), and so on. Indicates a very large number in that position.
Now, 30 numbers is a rather small sample size to reject a random-number generator, but the
notion is that numbers in the sequence might be related. In this particular section, a method for
determining whether such a relationship exists is described. The relationship would not have to
be all high numbers. It is possible to have all low numbers in the locations being examined, or
the numbers may alternately shift from very high to very low.
Autocorrelation Test
Autocorrelation test is a statistical test that determines whether a random number generator is
producing independent random number in a sequence.
The test for the auto correlation is concerned with the dependence between number in a
sequence. The test computes the auto correlation between every m numbers (m is also known as
lag) starting with ith index.
m→is the lag, the space between the number being tested.
i+(M+1)m<=N
Where
After computing Z0, do not reject the null hypothesis of independence if - za/2 <= Z0 <= za/2,
where a is the level of significance.
Example 3.1
Test whether the 3rd, 8th, 13th, and so on, numbers in the sequence at the beginning of this
section are auto-correlated. (Use a = 0.05.) Here, i = 3 (beginning with the third number), m = 5
(every five numbers), N = 30 (30 numbers in the sequence).
Solution:
hence M=4
Then,
= -0.1945
And
Z0 = -0.1945/0.1280 = -1.516
Now, the critical value isZ0.025 = 1.96 (Za/2 is taken in this test)
Therefore, the hypothesis of independence cannot be rejected on the basis of this test.
4. Gap test
The gap test is used to determine the significance of the interval between the recurrences of the
same digit. A gap of length x occurs between the recurrences of some specified digit. The
following example illustrates the length of gaps associated with the digit 3:
4, 1, 3, 5, 1, 7. 2, 8, 0, 7, 9, 1, 3. 5, 2, 7, 9, 4, 1, 6, 3 ,3, 9, 6, 3, 4, 8, 2, 3, 1, 9, 4. 4, 6. 8, 4, 1, 3.
There are 7 three’s are there. Thus only six gap can occurs. The first gap is of length 9 and
second gap of length 7 and third gap of length zero. And so on.
Similarly the gap associated with other digits can be calculated. The theoretical probability of
first gap (of lenth 10 for digit 3) can be calculated as
=0.9×0.9×……………..×0.9×0.1
= (0.9)10 ×0.1
The procedure for the test follows the steps below. When applying the test to random numbers,
class intervals such as [0, 0.1), [0.1,0.2),. . . play the role of random digits.
Step 1. Specify the cdf for the theoretical frequency distribution given by Equation (4.1) based
on the selected class interval width.
Step2.Arrange the observed sample of gaps in a cumulative distribution with these same classes.
Step 3. Find D, the maximum deviation between F(x) and SN(X) as in K-S test.
Step 4. Determine the critical value, Da, from Table for the specified value of a and the sample
size N.
Step 5. If the calculated value of D is greater than the tabulated value of Da, the null hypothesis
of independence is rejected.
EXAMPLE 4.13
Based on the frequency with which gaps occur, analyze the 110 digits Below to test whether they
are independent. Use a = 0.05.
4, 1, 3, 5, 1, 7, 2, 8, 2, 0, 7, 9, 1, 3, 5, 2, 7, 9 4, 1, 6, 3 3, 9, 6, 3, 4, 8, 2, 3, 1, 9, 4, 4, 6, 8, 4, 1, 3,
8, 9, 5, 5, 7, 3, 9, 5, 9, 8, 5, 3, 2, 2, 3, 7, 4, 7, 0, 3, 6, 3, 5, 9, 9, 5, 5, 5, 0, 4, 6, 8, 0, 4, 7, 0, 3, 3, 0,
9, 5, 7, 9, 5, 1, 6, 6, 3, 8, 8, 8, 9, 2, 9, 1, 8, 5, 4, 4, 5, 0, 2, 3, 9, 7, 1, 2, 0, 3, 6, 3
Solution
The number of gaps is given by the number of data values minus the number of distinct digits, or
110 —10 = 100 in the example. The number of gaps associated with the various digits are as
follows:
Digit 0 1 2 3 4 5 6 7 8 9
No. of 7 8 8 17 10 13 7 8 9 13
Gaps
Since D = max |F(x) - SN(x)| = 0.0224 is less than D0.05 , we do not reject the hypothesis of
5. Poker Test
The poker test for independence is based on the frequency with which certain digits are repeated
in a series of numbers. The following example shows an unusual amount of repetition:
-the poker test uses the chi square statistics to accept or reject the null hypothesis.
In each case, a pair of like digits appears in the number that was generated. In three-digit
numbers there are only three possibilities, as follows:
The probability associated with each of these possibilities is given by the following
P(three different digits) = P(second different from the first) x P(third different from the first and
second) = (0.9)(0.8) = 0.72
P(three like digits) = P(second digit same as the first) x P (third digit same as the first)
= (0.1)(0.1) = 0.01
Example 5.1
A sequence of 1000 three-digit numbers has been generated and an analysis indicates that 680
have three different digits, 289 contain exactly one pair of like digits, and 31 contain three like
digits. Based on the poker test, are these numbers independent? Let a = 0.05. Test these numbers
using poker test for three digit.
The appropriate degrees of freedom are one less than the number of class intervals. Since 47.65 >
X2 0.05, 2= 5.99 (tabulated value), the independence of the numbers is rejected on the basis of this
test. Here 2 or n-1 is the degree of freedom since there are only 3 (n) class.
Example 5.2
Explain the independence test. A sequence of 1000 four digit numbers has been generated and an
analysis indicates the following combinations and frequencies.
Two pair 32
1000
Based on poker test, test whether these numbers are independent. Use α=0.05 and N=4 is 9.49.
Solution
Here the calculated value of chi-square is 25.185 which is greater than the given value of chi-
square so we reject the null hypothesis of independence between given numbers.