'•J -1 <•

* :
X

<
/
1
r, kfft'. jL
-
'

lwj*
V;
'
>
h
_ < f;
rec(
'1,
Mr •

^ late
'/ P>* ; n; j •
it * the
be
par

of

Q'

0
T

This chapter aims to provide an overview of Queuing Theory . Various models of Queuing Model, their
advantages, disadvantages, comparison of memory models and how models to be selected are described
in this chapter.
t

-^^^ ^ mm - s& tgi*

^
r
4.1 INTRODU ION ; ?*• J: U

Queuing theory is the mathematical study of waiting lines, or queues. In queuing theory' a ino
_: P
ttW constructed so that queue lengths and waiting times can be predicted.
t«
n

0) s
f C

Customers

1 2 ••• M ) Servers
S
b
0
Fig . 4.1 Model Producer -Server

126

Scanned with CamScanner

 r v *! -
Processor Memory Modeling Using Oueuing Theory j/ f 27
']
(
>
- /» - • ' ' ' < •••/
-
mr * »». / i-MJ

Consider a producer server model . A buffer lor queue ) is present between them . Tasks are being
-
received and when one task is finished ( i .e. served ) then the second task is taken up by the server. Now
latency and the response time depend upon how many tasks are present in the queue and how quickly
n

*
they arc served. If there is no task , ahead in the queue the latency would be low and response time would
•

time taken by a
be shorter. Through put depends upon the average number of calls and the service
particular server.
-
Thus to represent the processor memory relationship, queuing theory is explained here. Forevaluation
of computer system designs , open and closed queue models are frequently used.

.
Consider a black ftox. Suppose it represents an I/O controller. At the input, v c have arrival of
different tasks. As one task is done, we have a departure at the output. So in the black box, we
have a server. Now if we expand and open- up the black box, we could see that incoming calls
are coming into the buffer and the output of the buffer is connected to the server. This is an
r .
*

example of "“single server model”. %

* > .4 .
-
5 - X
V V
»

Queuing theory is the mathematics of waiting lines.

f • It is extremely useful in predicting and evaluating system performance.
• Queuing theory has been used for operations research. Traditional queuing theory problems
refer to customers visiting a store , analogous to requests arriving at a device.
Queuing theory deals with problems which involve queuing ( or waiting ).
Typical examples of queuing might be:
ieir • Banks/supermarkcts: Waiting for service
i>ed • Computers: Waiting for a response 1
• Failure situations: Waiting for a failure to occur e.g. in a piece of machinery
• Public transport: Waiting for a train or a bus
*

As Queuing theory deals with waiting processes. Thus queuing theory also plays an important role
in measuring the processor performance . Generally we talk about , running the same job on the same
:1 is processor on 2 different occasions may create different execution times. Thus this theory is a powerful
tool for measuring the performance of I/O systems, multiprocessors, OS ( processes wait for various
resources ) etc.
Queues are formed because resources are limited .

Arrival Serviced
W queue
At the arrival
rate A.
At the
service rate *4 Departure

Fig. 4.2 Queuing Model.

Arrival Process: Arrival processes show how processes arrive e.g . singly or in groups (batch or bulk
arrivals). Important points to be considered are:
How the arrivals are distributed in time (e . g . what is the probability distribution of time between
successive arrivals ( the inter -arrival time distribution ) .The inter - arrival time , a , is the average time
between process arrivals. It is measured in time per process. A common unit would be seconds/ aceess
a = 1/A,.

Scanned with CamScanner

 * sffXGOSXW’
:
are
. Common units
\* i

chite ,
processes arrive
^

—
Advanced Computei ^^
average rate at which
^^
-
irKssv.owKMBw^ -
rate s
*

• Arrival rate ( : »
Arrival
regutar arrivals (i.e. same
constant
access/second. . completely
one where we have
• devices
The input/output

* sr
sing «3
A Poisson stream Of arrivals ==^
s-r
corresponds
c ed

to
to a high speed

arrivals at
processor

“
random. In a Poisson
system with

stream
“
successive
. The
exponentially distributed
^

customers ( processes) arrive - the average

•

Poisson stream is a convenient mathema

arrival rate.
Service Mechanism the resources needed for
service
the service mechanism describes
Service is provided by the system an d
to begin. Service mechanism tells:
.
• how long the service will take called the service time distribution
.
• the number of servers available
whether the servers are in series (each server has a separate queue
all servers)
) or in parallel (one queue for

) • whether preemption is allowed (a server can stop processing a customer to deal with
“emergency” customer)
Certain assumptions considered in queuing theory are : service times for various processes are
independent and do not depend upon the arrival process is common. Another common assumption about
another

service times is that they are exponentially distributed.

Queue Characteristics
In terms of the analysis of queuing situations the types of questions in which we are interested are
typically concerned with measures of system performance and might include:
• How long does a processor expect to wait in the queue before they are served and how loneh will
they have to wait before the service is complete?
’ rya,!l seCr edfbility*'
a » '° lonSer „
a S e rime interval before
"
• What is the average length of the queue
’
• What iis tne prooaDimy that the queue will
*

exceed a certain len 8th? °

• What is the expected utilisation of the &

be fully occupied? In fact if we will

server idle time then

•A
process
• B represents the probability distribution for the
• C represents the number of channels (servers ) service process
• D represents the maximum number of custo mers
served or waiting for service) allowed in queuing
• E represents the maximum number
of customers in
total
^m (either beieing
Scanned with CamScanner
 A
dO
.v

, Q T fbe: P
^^
pft
^e ' 9 ^
n US

^* >

—
distribution) or a exponcr
.
. i ier.arrival
'
and 0 ^ tial tli! I
\ (exponcn °
bUli n
.*
rrwc;

„
!JU T
*

mmon units are Coni^°

n
f a Poiss
n ri
°
l di
* ^ ° #
^
*‘’ “"“consia' wit*
^
’ )
service lil U mcan and variance #
1

.
• D for a genera infinite.
. G for specified Hen « * , >= ,, sjngle channel one server).
a , has a Poisson
e s t queuing system
[ f D and E are
mn queuing sys " nJ (

—^
the
For example time distn

—
- n
nen tial service
^ mem0ry less distributions and binom
^ Thus fl

ludon SON^
e , an exp0 theory
tributed. The
distribution mode ] ing of queuing
^

Note.*
- the average
rX
( distribution are dlsCUSsd
^ Iex system depends on the probability

“ *«“< * J
-
Di : « T Can

the service
of arrivals
ion l
distribution
,

for service like sequential circuits

).

, ,,, "
#ato .
the svsteni does not depend
on the previous state
i
“"
^
ib S circuits) They are sad
For Markovian dist distributions (just like combinational
0
'
a
but nly 00
”
to have a memory to
“ . The Sill
important types of memory less
distribution are:

(one queue for

=H; i
BINOMIAL DISTRIBUTION .*
.

I with another 4.2

The Binomial Distribution is the discrete probability
;
^^
distribution of the number of successes in sequence a
\

processes are of /; independent yes/no experiments, each of which

yields success with probability p.
Now we will discuss the binomial distribution with respect to queuing theory.
jmption about

Let in “7” time interval , V items enter into the system. Out of these /T items , “fc” items request
44

the service each with probability “ p \ Since V’ items are entered in the system , “m” memory modules
rested are are allotted to them.
Thus each request occurs with the probability p = ilm in each memory cycle 7.
tow long will
I For queuing theory we consider the case that there are “Jt” request that are directed to a particular
module (only one will be processed at a time and remaining are
terval before
•

that exactly k out of n request are made to

waiting to be serviced) The probability .
the designated server is:
/ \
n
| / ( ; n, p ) = Pr ( X = k ) = a - p )»- k
'hich he will
* , p
t_

ng time and
for = 0 L 2,
^ - n , where k )

ost. ( \
n n\
is the binomial
coefficient (he ^ - k \ ( n k )\
13 6
^ k ?'e ^ormu a can be underst ^ d'str:'bution) " n choose k'\ also denoted C(n
' ^
‘ »
we wanl k successes (pk ) and n - k ta ur
n
;lHowever
n

‘* -
as follows
the successes
ither being
diff * k succesJ" °
Ways
of distributing
’ P
”

CCUr anywhere am («
ng the n trials, and there are C *
^ distribution is , °
P on
” a
3 SeqUenCe of
” trials
r ,r
averagc ! that expresses the probability of a

^
tS 1s1r1bu110n

^
(

?
.
Urring in a r> ed
tndependently f * ,
ime
0 the ;m
" me Once
•
the last eve .
Ume and/or spacc if these events occur With a k
„
Scanned with CamScanner
 •HO Advanced Computer Architecture r -
w .
1
The probability mass
function of X is given by:
XV *
/(*; X) = Pr ( X = *) = k\
where
• e isis the base of die natural logarithm (c = 2.71828...
)
s a

' °'\|
e0.
h . . £! is the factorial of k.
• k = XT, when the number of events occurring will be observed in the time interval 7 = 1.
‘ion, 9H (j
• The positive real number X is equal to the expected value of X and also to its variance.
X = E( X ) = Var(X).
, He
01
Thus p{ k ) is the probability that exactly k request for service will be made during time 7.
CaH
* Note: K

Hie prt(. Poisson Distribution Poisson Arrival Rate

ils)
are; K 09
Exponential Distribution If customers are arriving at the exponentially
distributed rate X , then the probability that
there will be k customers after time t is:
0.8
0.7
0.6
05
04
Pk (o =
( Vf
k\
.
-x

esina 03
0.2
\
0.1
0
0 0.5 1 1.5 2 2.5 3 35 4 4.5 5
' items
moiy
Poisson Solution Expected Number of Arrivals
•A network printer usually gets 15 print If customers are arriving at the exponentially
jobs every hour. The printer has to be
oa turned off for 10 minutes for maintenance.
.
distributed rate X how many customers
should you expect to arrive in time f?
What is the probability that nobody will
ie pro ^
;
want to use the printer during that time?
• The arrival rate is 15/60 = 0.25 jobs/min. Expected X * t =
For the printer problem with an arrival
p
0 m= ( 0.25 * 10 ) ° e 0.25*10 = 0.082
-
rate A. = 0.25 , in 10 minutes we should
0! expect 2.5 jobs to arrive

..-Si
O SERVICE DISTRIBUTION .
There are 3 Markovian distributions from service point of view. These are

A &\ >
1 - Constant distribution.
2 - Exponential service-time distribution
3. General service- time distribution
C Constant Distribution: As the name implies that all the requests take constant time 7 lor to be
it serviced . As it is said earlier also that the service distribution is represented by p. Thus p - l /T.
/
Scanned with CamScanner
 1
(W .* 1

—
* ^

2 . Exponential service time distribution: I11 this distribution

Processor Memory
-* >
W B x m
jctmnrtn /
Modeling Using Queuing Theory

Prob. ( time between arrivals is < = t ) =

** *
•
"<
— * '
131
'
r^
,
i W;

1 - Probability ( time between arrivals is > t ).

Probability ( time between arrivals is > t ) indicates probability of no
arrivals between 0 and t . Thus
prob ( time between arrivals is < = t ) = 1 - e .
~
^
time , then the
where l /l is the mean of inter-arrival distribution . So if l /|i is the mean interservice
interservicc probability is P( i ) = 1 -
d 2 is
3. General service- time distribution: Service time may be constant or variable . If we consider
the squared coefficient of service time variance or d = variance of service time/( mean service time
2 ) '

d2 = aY
/( l /p)2
then
If we consider the constant 2
=(
^
distribution then d = 0 while for exponential distribution d ~ = 1 thus these
3 service distribution it is shown in the figure as :

^7
d2 = 0 d2 = i
(Constant Distribution ) General Distribution (Exf onential Distribution )
Fig. 4.3 General distribution, constant distribution, exponential distribution.
3Y
It is common to use to use the symbols: : -
> Lamda ( 1 ) to be the mean (or average) number of arrivals per time period , i . e . the mean arrival
\
V-4: - rate
I*' •- l
« -
p. to be the mean (or average) number of customers served per time period, i .e . the mean
service rate.

4.4 MODELING THE PROCESSOR-MEMORY INTERACTIONS

Before discussing the modeling first Little's theorem is explained here.

4.4.1 Little’s result or Theorem

Little ’s result ,or theorem is a theorem given by John Little and it states that :
The long - term average number of customers in a stable system N is equal to the long - term average
effective arrival rate , 1, multiplied by the average time a process (customer ) spends in the system , T; or
expressed algebraically : N = IT .
Example: Imagine a small store with a single counter and an area for browsing, where only one person
can be at the counter at a time, and no one leaves without buying something. So the system is:

.. y *

Entrance -N Browsing > Counter Exit

Fig. 4.4
1

Scanned with CamScanner

 .
- 27 -v •

Advanced Computer Architecture

71
- 32;
- - the store is
wv -
Vhis
^
> ••
pl ®

^
which is called
There are 2 Types
of Systems:
system in w hich the rate at they exit as is
system is the , and the rate at which
1 . Stable System: Stable
the rate at which
arrival rate .
2. Unstable System
and in these systems
they arrive at

!
" “TiTo'‘
the
,
the store
_ - .
. ic m exit rate then

f waking customers
in h e
the system
store will
is called
gradually
unstable

effective
system,
increase towards

arrival rate
infinity . store N , is the
of customers in thesimply:
*V *

the average number , or

spends in the store T
that
Little’s Law tells us a customer
that
A, times the average time /V XT.
- we
stay an average of 0.5 hour. This means
at the rale of 10
per hour and
Assume customers arrive to be 5.
of customers in the store at any time
should find the average number N = 10*0.5 = 5
•v.
to raise the arrival
rate to 2 p
advertising
Now suppose the store is
considering doing more
occupants or must reduce t e time eae
to host an average
of 10 as.er
latter by ringing up t
IC I
The store must either be prepared might achieve the
customer spends in the store to
0.25 hour. The store
or by adding more counters. store . For example, the counter and its
queue.
within the
We can apply Little's Law to systems in the queue and at the counter. We know the
average 2 customers
Assume we notice that there are on spending 0.2 hours on average checking out.
per hour , so customers must be
arrival rate is 10
T = NIX = 2/10 = 0.2
's Law to the counter itself . The
average number of people at the counter
We can even apply Little . In that case,
would be in the range (0, 1 ) since no more
than one person can be at the counter at a time
is also known as the utilisation of the counter.
the average number of people at the counter
limited amount of space, it cannot become
However, because a store in reality generally has a
the exit rate, the store will eventually start to
unstable. Even if the arrival rate is much greater than
be rejected (and forced to go somewhere
overflow, and thus any new arriving customers will simply
else or try again later ) until there is once again free space
available in the store. This is also the
difference between the arrival rate and the effective arrival rate, where the arrival rate
roughly corresponds
to the rate at which customers arrive at the store, whereas the effective arrival rate corresponds
to the
rate at which customers enter the store . However, in a system with an infinite size and no less, the two
are equal .

Little’s results
N = 'k T. , N = { p/1 - p)
N _
1 v»

fi fi(T - p)
"

4.4.2 Simple M/M/1

Generally queuing models are categorized by the triple: Arrival distribution/ service distribution/number
of servers.

Scanned with CamScanner

 F0,y
ei5
I

Modeling Using Quelling Theory i o

^
'
T Ory
processor Mo' '

-.
v

IS
3d

n,
Js
*

Thus M/M 1 h >

exponential service "

Bir:7nVR
In using this
customers tttf
'
«. ^
•
|
islriW"

'
j
. ^
. isis the SIa' s
* and '
|j|\Vcl \ **
singIc q 1 ICUC 10
the servers

V'
: • '
.
*•

.
mS’
.
•
’
•

. ..
'
•
•

*
’
.
-
•
- 0/
e pf °

te ' Importer point

!t
AIBICIDIE “

" 1
Kent's Notation forces
f*
*n7.': .
••
A - V
> 7 •

p %eni.
|
• '
*
•
• '«* -

****** 1 0

^
re nl time g
; o- ,,}
5 • V
Service time distribution
»

PM#. . .-
V *»
p
rr- 'is -.
There ^
.
•• •• •
- • "• • ! r •
*
r 1 . / .«•»1
*rtf

C Number of servers J the system ( waiting and

in service)
r.
! *
!•
2.1
*
'tJk * '

1
Queuing Discipline ( l:CFS, LCTS. SIRO etc.) O '
,

Default is FCFS , ; . Lite's

. ‘ . . .. .
* '
.
.*•

r. I
arrivals , exponentially distributed
M/M/ 1 or M/M/°° is single server queue Wlith Poisson
waiting positions N
service time and infinite number of *‘r
v . •

in the

M/M/ 1 queue model is the most elementary model . This queuing

model represents the queue length
in a system having a single server, where arrivals are determined by a Poisson
process and job service pis
times have an exponential distribution. The model name is written in Kendall s notation .
'

value
An M/M/ 1 queue is a stochastic process whose state space is the set { 0 , L , 2 , 3 , . .. ] where the
corresponds to the number of customers in the system , including any currently in service.
• Arrivals occur at rate X according to a Poisson process and move the process from state i to
i + 1.
• Service times have an exponential distribution with parameter p in the M/M/ 1 queue.
• A single server serves customers or.e at a time from the front of the queue , according to a tu v
come , first -served discipline . When the service is complete the customer leaves the queue and the
number of customers in the system reduces by one .
ISte buffer is of infinite size, so there is no l i m i t on the number of customers it can contain
nth model cite be described as a continuous
time Markov chain with transition rate matrix
f -x
H X
er
Q=
~ <M + X ) X.
* -<U + X) X
^ -(H + X) X

Scanned with CamScanner

 \W s’
\
—
Advanced
»!mm
Computer Architecture
—
ini ni 1
[

t
{0, 1, 2, 3, ... ) . This is the same continuou* t (1
* '

On (he state space Mark v chain asa birth -death

for this chain is as below. ° ii
process. The state space diagram
5.
\ w
&

V
V

Goanic:- 3EQ333C Fig . 4.5

M V 11
•• •

Queue Properties
N

ts Arrival Queue m
; Q
rate \

Tw Ts
1
T

,n % Fig. 4.6
, , , Q , p, X , 6
There are some terms that are used more frequently, these are N Tw Ts {
1. Ratio of request rate to service rate is called r and r Vm
. -
2. Thus average time spent in the system T = TW + Ts Where Ts . is the average service time and Tw
is the average waiting time.
Little's theorem is already defined , thus here again
iistnte
'
. . . ( 1)
N = XT
/ / consists of items in the queue and the item
N is average number of customers in the store, but here '
in the service then . .. ( 2 )
// =Q+p
ntsifc
p is number of items in service.
alien Thus from Eq. (1) and (2)
.] * Q + p AT -
Putting the value of T = TW + TS •
r* but Ts = 1 /p
We get Q + P = X ( TW + Ts )
Thus Q + p = X ( TW + 1/p)
and A/ p ~ p
Q + P Tw + X / p
=
jr
/ r1
Thus Q + p = XTw + p

*4
tf I
p
. rate p - 4 customers
distribution with a mean service rate of 4 customers per minute, Le the service
minute.

Scanned with CamScanner

 Processor Memory Modeling Using
Queuing Theory jap
W 2S.
;; • a

^. . BSusually
/

th A,rival distribution/
described by drree values separated by slashes
‘
v

ia distnbutioivof servers.
*
v
' y ir '+
-
where:
service
. aV
, ^ :/M

.
...... 3kt= Markovian or ejtjobentially
r
distributed
'
'
ji . ** - n*

V v £) Determmh.tic or constant. .
'

= .

: ii
Gi G ’nerii or binomial distribution :
^ for average waiting time)
rw = l /X [[p2 (1 + rf )]/2( l p )] • • • (P - K mean result -
2
for M/M/ 1 d - 1
We have studies three types of service distribution thus
Thus = l /x [p2/( i - P)1 rw
Waiting time of queued items is
- [ using little’s theorem ]
T q = T - 1/ ji. = p/[p.( l p)]
Thus N = t t q = p2/[d p)l -
2nd case when $ = 0 thus we have M /D/1
Tw = [ p /2( l - p ) ] x 1/X
2
Thus
Waiting time of queued items is
[using little's theorem]
T q = T - 1/p
= W, = p-/[20 - P)1
W

Thus N
3rd case
d2
MJDI 1 (Binomial arrival, constant service = ) ( ( p))] *
1) 0)
m Waiting time Tw = [( p - p / 2 l - Ts
where p >= p
2) approximately equals to zero . This
p < pease does not occur. Because if p < p then waiting time
means that in simple-processor models, there is no processo
r request in the memory cycle,
..
l

Summary of Queuing Theory . A :> orr ; . .

'
vi '.
.
*•

5
’

*
*
/
-
r
.V
: •
- &V. ^

- .• V
V

.
9

Single Server - - • J' , . .. 1 1

is

..
"
• \*
•• it
/p /
- -: -

3
^
.- Arrivals with an \ >
Ki
y •V

— -— - V’ vy
Jpiv 'it.
;V
'
average arrivals lK v ..
•
- ••
f'4
.
p? i
i
V - %
- ?.1
rate of X - •
r
- 4 a-
•jr

- > y > .• " .

* ys f •
jAi
P ..
>5
}- V
T,

•f
t
Infinite Buffer i

- -
' • \
. . t x *
s « V ' H XAT* . '*
(infinite number of
• 'J
.
*
Ji '
K

'. waiting positions

*

-
ww - -S; :'.:-.;. .
l \ •V

-.
/ J
) . ci • *.
'

. r • * r • . . : v**

--
f • * 4
•
... \% .
'
mV y i rjrJ - " 1Vr V «•
X •
i

x- - . Assume that, as
1
-. X.
.•r1. v, *
, 0 2?NI V . 4/

.
;
•
; T* -
„ ^ ,
i*

. -
>i
on i
»
• ,
. . J
Amval Process
•

0 rjvali timeA|, XA/

* *
' .-
v '; ; . • .I'
, .
^P{No arrival in ti
* w
«
ier •
*• . V-
ial v
• t

sk L JV . -
'
. . V
Mean Inter-arrival time - ^
= 1 - XA/ >
i «
* V» \
• a
1 •
V

. .. •X
Tie
er Mdrei&ionf •n
SK x25 ,
2
iv il in time Af ) = 0 (( A/ ) ) = 0
.
fr v> r * * *t ^

- Service.Process
J T‘. • *

-
4
) . >
P( a •

-.y
.
1
v
'

i
* i -
I - :,c ; >
a VS
? 1,
x:
r
; vx

i k*
1
'

.
vr:
»

Scanned with CamScanner

 #

< *>* . s»*«ttswn*

mM******

_
$ \\ J

-
veil IOCU ^
--
.. f c« 1-. > •'• vritoSVtfJW
vjiiifjuici
*** 1 8
'
rvi oi meuiuit;
* ./’. wm' - JV *11 ’ • --v .
§. -_ i :
|EIStime .
Mean Service
in time A/ } = pA/ . . . . m» $8fNS -
-
-
P{ One rdeparture . -.- ? • . 1 */
'«
. - :
•
.
- -' - . vT.’--' -
•
• •• •• • '
• • ,•• i
i /

1-p
•

^
'
•

F { No departure in time A
/) = ?
'1 :
2
I

, o /• . : I , .
= 6> ((A/) ) = 0
A

{ More then one departure in

time A/ } ..
. 7 " "-
P
= omfm
,
.

or more departure in time At } -

jP { One or more arrival and one
m.
^vmsmssB
• •
' '
• • • :
f t/ .

Queue ,
Mean Performance Parameters of the
A"
-
- - , - V' y ./ -
1
'';
•
T;
. !» • •
VJ
tfSS§- |
l
^ gap
v

Mean Number m System,

t

• >
•

*
K • « ' •
« ^
i* - M
N^ =
"
>' 3*'A - ^ V • XTI1%
(1 P) =
‘ '
• •
Y% -
^
\
^ '
•
••
• ..
=
'
i -p
'
;
•

>

b ) Mean Number Waiting in Queue, AT

#-sgit®
*

*.
(
j
:

- -
SP: : *« ; :; - - -
.
•’Ay
;v- •
. . *
• • •
^
P p - - -;
it :P (i - Po ) = rI - " -P
'

. .
- .
'

-
P V; "
-
-
'

mi
:
'=1
.
v.

( c ) Mean Time Spent Waiting in Queue .

This will obviously be one mean service time less than W

.. . .
. . •a v:’ *, •> - . t.

1 p
:
*
•

,
v
.
'
.
•
W-
^- P) :
: i ®sai 8fe§@i P-
^
,

(<0 Server Utilization “ Fraction of Jink the server is busy”

r
•• ' '
,
•
w i .

-
-
#

-
•
'
v is
'

P { server is not idle ) '

•
i!

i - Po P
"
* :
•
> •
. . : • v. ; : . =
r T

^
.
t ’.-
'

I 4.5 QUEUING NETWORKS

Queuing Network models are defife/d as the models in which jobs departing from one queue arrive at
4 u

~. V
.. . -.w. .* ^ •
• V '

another queue (or possibly the same queue).

Queuing Models can be:
1 . Open Queue Models
2. Closed Queue Models ,

3 . Mixed Queue Models / i

4.5 .1 Open Queue Mocfels

Open queuing models are the simplest models in which
y ) Number of jobs in the system varies with time . And there are external arnvals and departures

Lb ) Arrival time is independent of the service rate. This results in a queue of unbounded length as

y Thus in open Queue model arrival rate is a function of total service time ( including waiting ).
Thus Throughput = arrival rate
j(d ) Goal: To characterize the distribution of number of jobs in the system.

Scanned with CamScanner

 Processor Memory -. - —
Modeling Using Qn •

»»
f
' 4
•

•'****'
request rate decreases with mer
,« »
. processor s r

^ ^^^
'

jpf? Op
r*x m*rervrt

processor memory interaction , Ming closed queuing neUv

problem can be removed
' '

Tims hi a *
l -$. ; - to wait ). This <fefc
(because it has
ri&ty 4
t
i

memory .
to tintcrlca veil
• •

model is limited
Y V- ^ C
I
foM
*

:
r ;V / .
*

?V Out - Disk
rfr J

A
2 4

•
V .> • In A
’
?*V- Disk
B

Fig. 4.7 Open Queuing Models.

Flores Model: Flores model is used an open-queue memory model . This model allow us
to fix is
initial partition of memory modules using
MJD !\ queue model
In (his model requests are splitted over m modules in m ways .
( binomial arrival and constant

VJir
Xs

i
—
? .> ..

m
Fig . 4.8 Flores model. request
Let h is the total splitted. are
request rate (req
Then Xsf is splitted uests per second) made by processor
‘ over
m modules and I /O devices
uniformly in m ways. ,

X. = Xslm
Then al the module
then M = l / TC
y Here
MIPS 11i used for P a ih/ ni ) x T
f/Milli^n acces5 measuring th» .. .
second (MAPS) Processor instruction
i
C
1 Tc is memory cycle tii ^
* (Mu>S) .f£ 2cessorrefefence
execution rate .

^stmetion 4- MIPS !]9 references

V Instructions )
Scanned with CamScanner
 Total memory access time T H’ + Ta
where Tw = Waiting time
T = Access time
Average si .e of open -queue - m - Qo
- As /

A *^
5 a balance between *** or interleaving factor and performance choose
ilso so that 111 = K
2 ( K is an integer).
, in so that P ~ 0.5 and

Example: Suppose we wish to design a memory system for a processor with peak performance of
= 20 ms and Tc = 100 n sec, m = 16 ,
-
40 MIPS and one instruction decoded per cycle. Also assume Ta
p = 0.47, total of 1.5 references per instruction.
B
Solution: 1. MAPS — x 40 = 60 MAPS (here I/O reference/instruction is ignored)
= 1.5 x 40 = 10
*
2. p= —
m
?;

** * J
*
= —m1 x (100 n sec)
60 x 106 x
odeis ,

= 60 xl 06 x —m
x (100 xlO ) -9

model . This model

( binomial arrivalaal * 1
= eoxio -
m
x
^ ^ . ixio
-6
)

6
m
1 p2 - pp
Tw = • i

\ 2( 1 - p)
here P = -m1
pq - f )
"
X - 2(l - p)
here p = —P
i (i - P) 1
Thus T H' = - =m = —P
11 - 2 ( 1 - p)
~c - -1
p
J Thus T H* = 7C •
(p - P )
2(1 - p)
A
re j*
#
44 ]
- 2 (1 p)
K C>
f‘ lOOx 0.47
1 N

f|tfn TW =
16
2 (1 - 0.47)

Scanned with CamScanner

‘ :
•It.
"J '.

-
_ Processor
Memory Modeling Using
-
»oi r »•« r*. * *•* *’»* i*\ ,n >

100 (0.47 0.0625)

2 (0.53)
• « r

—
Queuing Theory ( 139
IM l » M » r .. - "
.
||
i
'
.
. 11 J
-

100 x 0.4075
1.06
= 38.44 ns
Total memory access time = Tw + Ta
= 38 + 200 ns
= 238 ns
2 ~
Lr )
P PP = 0.18
° 2 (1 — p)
Average open -queue size ( Flore’s model)
= m - Q0 = 16 x 0.18 = 2.88 = 3.
But we know that memory systems are not open queues. Since buffers are of limited size, thus
processor does not make demands at the peak ntofput still this simple model is of interest as it provides
the information of interleaved partition . /
4.5.2 Closed Queue Models
The closed queue models are modeled by queue with a feedback. The features of Closed queuing
network are
1. There are external arrivals or departures
2. Total number of jobs in the system is constant
3. OUT is connected back to ‘IN.’
4. Throughput = flow of jobs in the OUT-to-IN link
Number of jobs is given , determine the throughput.
Closed Queue models have bounded size and waiting time.

Feedback Disk
Out
A

In CPU

Disk
B

Fig. 4.9 Closed Queue Model.

--
.
Open queuing networks have an external input and an external final destination.
-• Closed queuing
networks are
a completely contained and the customers circulate continually
never leaving the network. . .. *. . .
' '

t
>t
4' ' 1 ; •

•'
*
V.
n

Scanned with CamScanner

 <140 1 Advanced Computer Architecture -
T
for X and
Let X be the offered arrival
rate and hi is theadjjev amM
, ^ rate. Let pjsjheoccup
of i ems in_closed _Gc. Qc stands
for closed queues.
^
^
we
^^
pa for ITmisTp P is lhe no
have an // . in system in overall
stability. Then averagg P size includin
^
( g items in
Suppose
-
service ) denoted by A nlm and
dosed Q size Qc nlm - pa p pa
= = -
we know that
From discussion on open queue
N = Q0 + P
Average Q size
occupancy and for MIDI 1 model , Q0 is p / 2( l - p )
“ .
so we have
Since in closed Queue pa is achieved
2
/V nlm pa /2( 1 - pa ) + pa
= =
Solving for pa we have
2
-
pa = ( 1 + nJm ) ( nlm ) + 1
Band width -
B (m, n ) m pa
2
B ( m, n ) = /?i + 7i - n + nr
This solution is called the Asymptotic Solution
Since N nlm is the same as open Queue occupancy p. We can say
=
pa ( 1 + p ) - p2 + 1
= __
Simple Binomial Model: While deriving asymptotic solution , we had assumed m and n to be very
large (approximately reaching to infinity) and used M/D/ 1 model.
For small n or m the binomial is a better characterization of the request distribution rather than
poisson distribution .

Binomial Approximation
Substituting queue size for MB/ D/ 1 ( binomial arrival , constant service )
N = nlm = ( pa 2 - ppa )!2( \ - pa ) + pa
Since Processor makes one request per Tc
p = 1/m ( prob of request to one module )
Substituting this and solving for pa
a. = -
pa 1 + nlm - l /2m - (1 + nlm l /2m)2 - 2n/ m )
r

P
3
- + and
—
B( m, n ) m pa •

£( / 77 , 7? ) = / 77 + 77 - 1 /2 ( /77 + / 7 - 1/ 2)2 2/7777

Binomial approximation is useful whenever we have
fD
- Simple processor memory configuration (a binomial arrival distribution )
- n >- 1 and m >= l.
-Request response behavior: where processor makes exactly n
requests per T .

4.5.3 The (8) Binomial Model

[ MDU: Dec 2012]
The simple binomial model may fail if simple processor is replaced
with a pipelined processor with
buffer (I-buffer, register set , cache etc ).
Simple binomial model can not distinguish between single simple
processor makiing one request per
=
Tc with probability 1, and two processors each making 0.5 requests
per Tc .
In second case there can be contention and both processors may
make request with varying
probability.
 Processor Memory Modeling Using Queuing Theory t 4

Thus 6(DELTA ) binoin

ial model is used 10
equal to =
, so p
correct this problem
§ / m .
. Here
the probability of „
a proc
%t
^
7 is not
X and access during
^
Substituting this we get a more
general definition
B( m, //, 5 ) = rn 4- nil (/// + n - 6/2 2 - 2mn
ms in )
ancy. designs where the source is buffered or makes cias!
This model is is useful in many processor requesis n „
a statistical basis

have
If n is the mean request rate and z is the no. of sources, then 5 = Z . —
• This model can be summarized as follows:
- Processor makes n requests per Tc .
- Each processor request source makes a request with probability 5.
• Offered bandwidth per T , is Bw = niTc = ink and Achieved Bandwidth = B( m, n, 8) per j
• Achieved bandwidth per second
- B( m, /i, 8)/r = m
V

• Achieved Performance = loll * (offered performance)

eiy
Using the 6-Binomial Performance Model
Assume a processor with cycle time of 40ns. Memo request w
0' each cycle are made as per following
• Prob ( IF in any cycle) 0.6 = S
• Prob ( DF in any cycle) = 0.4
T
• Prob ( DS in any cycle) = 0.2
• Execution rate is 1 CPI., = 120 ns, T 120 it
Ta
Determine Achieved Bandwidth/Achieved Perfi =
ns
ormance (Assuming Four way Interleaving)
Solution:
so
= 4, Compute n : ( Mean no of requests per T )
0
" = requests/per cycle x cycles per T
= (0.6 + 0.4 + 0.2) x 120/40
CO

Compute 5:
= 3.6 requests/? .

z = cp x /processor cycle
CL

=r
Where cp is no of processor
So
sources ^ time
3
CO
So * = 3 x 120/40 = 9
5 = n/ z = 3.6/9
n>
Compute B( m , «. 8): = 0.4
B0n, « , 5)
= + /2 On + n
" - 6/2)2 _ 2 mn
So processor , = 2.3 Kequests/r
ffect on processooffers 3.6
memory system can deliver
Performance achieved only 2.3. This has direct
At lcP at 40 = 2- 3/3.6 (offered Perf )
‘ ns cycle .
Achieved PerfrormanceofferedJ = 25 MIPS .
^
= 2.3/3.6( 25 ) = 16
MIPS .
 Advanced Computer Architecture
-• -
'I ** Qfe**l

Queue Models
4.5.4 Mixed
i

-.
M certain systemsjhat fehawasopen queue up to a certain queue
~~

clQS§di °t —— '

' AH jobs of a single class have the same service demands and transition probabilities
. Within each
— C

class , the jobs are indistinguishable

L

jfr:,
Control
subsystem

Fig. 4.10 Mixed Queue Model.

We have studied open queue system and closed queue systems. Now we will study “Buffered Queue
Systems”. In Buffered Queue Systems, the requests made of the processor from the buffered sources.
This is called Mixed Queues System, because with a low request rate, system acts as open queue but once
it refers the critical occupancy , it behaves as an closed -queue system.
Now in buffered queue systems, only one queue is added in the system.

Thus, N = m f -=p

Pl
—
CO
= pfl +
Q.
2 (1 pa )
§
^ here Ba is the achieved queue size in the buffer.
O
As already said that for low request rate, system acts as an open queue thus the offered CCUPan
0 <
-
—_
5
in
size ( )
s same as achieved occupancy this ra for
Ba is less than the actiiajbuffy
2
P+P (v P = Pa >
—
2 (1 p)
pa
P" + 2 (1 - p„ )
One another term defined here i .e. called critical occupancy. Critical occupancy Pc >
tM
31
mean buffer size approaches the physical buffer size. Let Ba
p2
Thus BF =
-
2 (1 Pc )
 ii

— or
Processor Memory Modeling Using Queuing Theory

BF x 2 * [( 1 - pf )|= p2
m
2 * BF - 2 * BF * p . = p;2
(

Thus as long as pt. > p, system behaves as gpen -queue system and lor
%
pc < p , system behaves as an
closed queue system.

4.5.5 Comparison of Memory Models

|
« Each model is valid for a particular type of processor memory' interaction .
• Hellerman's model represents simplest type of processor. Since processor can not skip over
conflicting requests and has no buffer, it achieves lowest bandwidth .
• Strecker's model anticipates out of order requests but no queues. Its applicable to multiple simple
un buffered processors.
• /D/1 open (Flores) Model has limited accuracy still it is useful for initial estimates or in mixed
Af
queue models.
• Closed Queue Mg/ D/ l model represent a processor memory in equilibrium, where queue length
including the item in service equals n / m on a per module basis.
Simple binomial model is suitable only for processors making n requests per Tc.
^'The 6 binomial model is suitable for simple pipelined processors where n requests per Tc are
each made with probability 5.

Review and Selection of Queuing Models

• There are basically three dimensions to simple (single) server queuing models.
• These three represent the statistical characterization of arrival Rate, service rate and arqount of
buffering present before systerrTsaturates.
arrival rate, if the source always requests service during a service interval, use or simple
binomial model.
• If the particular requestor has diminishingly small probability of making a request during a
particular service interval , use poisson arrival.
Q.

• For service rate if service time is fixed, use constant ( D ) service distribution.
Q • If service time varies but variance is unknown, (choose d 2 - 1 for ease of analysis) use exponential
3 (A/) service distribution.
co
* • If variance is known and d2 can be calculated use M /G/ i model.
to
• The third parameter determining the simple queuing model is amount of buffering available to
the requestor to hold pending requests.

Arrival parameter Service parameter Waiting Time Mean Queue Size

dm* M /M / 1 Poisson arrival J2 =\ ( p 2/( 2( l - p ) ) ) » l /X ( p 2 /( I - p ))
distribution
icy
oM M / D/ 1 P=0 d' =0 Cpz/( 2( 1 - p)))* 1/X ( P2 /( 2( 1 - P ))
MB /D/ 1 P = him [ ( p 2 - pp /( 2( l
* \\
/
- p )) ] ( p2
— Pp/( 2( 1 - p ))

P is the probability that an item entering system requests service during unit service time.
d 2 is the coefficient of variance of service distribution .
 c«rrwi>W8'>' x«i»v w»K*5K* ir>rn
- '

1443§ ..-Advanced
. .
Computer Architecture
.ozv r -
,31 - - .--
7J . /f . - - • *
'
•
-
* -
• jv'fftV'* vti; ' * » * t‘ ~

lltf - >
'

4.6 CONCLUSIONS /

Queuing theory deals with problems which involve waiting.

formed because resources are limited,, • i
Queuing are a common every day experience. Queues arc
systems we need
e .g . , we have to wait in the banks, super markets to be serviced In designing queuing
.
to aim for a balance between sendee to customer . There are three queuing models exists : Open, closed
present in the fi
and mixed queue models. Arrival rate , service rate , and the amount of buffering that is
<
system server queuing model depending on the service distribution , measured by coefficient of vai iance R
( d 2 ) we have M/M/ 1 , MB/D/1 models coefficient of variance c op. =
if C = 1 => M : Poisson/Exponential distribution t
C = 1 => MB \ BinomiaJ distribution :•
C = 0 => D : Constant distribution
it
C < 1 => E : Erlangian distribution
C > 1 => H : hyperexponential distribution
C = any random value => G : General distribution

REVIEW QUESTIONS
Q. l . What do you understand by the term arrival process
, service mechanism and queue properties?
Q.2. Explain little’s theorem .
Q.3. Explain open, closed 4 mixed queue models.
Q'4' Wi h ,
^ ^ waiting time and
Q.5. Explain Binomial Model. Why it is used ?

CO
o)
Q
3
13
(D
Q.
-
r f
\
O
0)
3
C/3
O
CD
3
=
CD
3