CS 215

Data Analysis and Interpretation

Random Variables
Suyash P. Awate
Random Variable
• Definition:
Random variable X is a function defined on a probability space {Ω,ẞ,P}.
Function X:Ω→R, maps each element in sample space Ω to a single
numerical value belonging to the set of real numbers
• The name “random variable” is a misnomer X(.)
• Random variables are functions
s x = X(s)
• Random variable is an abstraction
• Used when we are more interested in analyzing the value associated with an
outcome, rather than the outcome itself
• In this way, the range of function X becomes the “sample space” of interest,
induced by X(.)
• X(.) also induces a probability function, say, PX(.),
associated with the range of values that X takes
Random Variable
• How can we design sample space Ω, function X(.) for the following ?
• Example
• Coin toss

• Example
• Roll two dice.
We want to analyse if the sum of the numbers is greater than 5.
• Example
• In an image, randomly select a pixel location (say, (i,j)).
We want to analyse if the pixel intensity (in a grayscale/black-and-white image) is 20.
Discrete Random Variable
• The definition will come soon. First, we build intuition.
• Values taken by the random variable X form a discrete set
• Cardinality of the set of possible values taken by X is
finite or countably infinite
• Sample space is usually a discrete (countable) set, but not necessarily
• Example
• Roll a die. Take number on top face (finite sample space).
• Example
• Number of packets transmitted by a router on a network, in unit time
• Example
• Number of photons hitting a detector/sensor in a digital camera
Continuous Random Variable
• The definition will come soon. First, we build intuition.
• Random variable X takes values anywhere within intervals on real line
• Cardinality of the set of possible values is
uncountably infinite
• Cardinality of the sample space must be uncountably infinite
• Example
• Heights of children in school
• Example
• Widths of leaves of a mango tree
• Example
• Real-valued measurements in a medical MRI scan
Events via Random Variables
• Notation (legacy)
• Upper case (e.g., X): random variable
• So, X has an associated distribution of values
• Lower case (e.g., x): a value taken by the random variable
• So, x doesn’t have an associated distribution
• Example events:

• Example events
• Consider a grayscale image with integer-valued intensities from 0 to 255
• Event A = “pixel intensity is 100”
• Event B = “pixel intensity between 100 and 200”
Event Probabilities via Random Variables
• Example: PX({a<X<b}) :=
• For simplicity of notation, we can refer to the induced PX(.) by P(.)
• Example:

• Example
• Consider a grayscale image with integer-valued intensities from 0 to 255
• P(“pixel intensity between 100 and 200”)
Cumulative Distribution Function (CDF)
• Definition:
For a real-valued random variable X, the CDF fX(x) := PX(X ≤ x)
• For all x, value fX(x) ∈ [0,1]. We will omit the subscript, at times, for simplicity.
• Properties of CDFs:

• Note: A right-continuous function is where, if we approach a limit point from the right,
then there isn’t any jump discontinuity.
This concept is especially relevant for discrete random variables.
• P(a<X≤b) = fX(b) – fX(a)
• Proof follows from {-∞<X≤b} = {-∞<X≤a} U {a<X≤b}
• P(X=c) = fX(c) – fX(c-)
• For proof, see Hogg-McKean-Craig
Discrete Random Variable. Probability Mass Function.
• Definition:
A random variable X is “discrete” if
the cardinality of the set of values X takes is countable,
i.e., either finite or countably infinite
• Definition: For a discrete random variable the
probability mass function (PMF) is the function PX(.),
where PX(a) = PX(X=a)
• Recall: PX(.) is defined in terms of the probability function/measure P(.)
that is a part of the probability-space triplet
• Properties of PMFs:
• For all x, 0 ≤ PX(x) ≤ 1
• ∑x PX(x) = 1
Continuous Random Variable. Probability Density Function.
• Definition:
A random variable X is “continuous” if
its CDF fX(x) is a continuous function (i.e., without jumps) for all x ∈ R
• Properties
• For a continuous RV, P(X=c) = fX(c) – fX(c-) = 0, for all c
Continuous Random Variable. Probability Density Function.
• We can write the CDF as an integral of another function PX(.) called the
probability density function (PDF)
$
• 𝑓! 𝑐 = ∫"# 𝑃!
𝑡 𝑑𝑡
• We assume ‘absolute’ continuity of CDF
• No weird cases like Cantor function (devil’s staircase), that isn’t integral of its derivative
&
• Thus, P(a < X ≤ b) = ∫% 𝑃! 𝑡 𝑑𝑡 = PX(a < X < b) = PX(a ≤ X ≤ b) = PX(a ≤ X < b)
• Because the CDF is non-decreasing,
the PDF PX(x) ≥ 0, for all x
• Because the CDF evaluates to 1 at +∞,
the PDF integrates to 1 over (-∞,+∞)
• Support of a discrete/continuous random variable X comprises all
points x for which PMF/PDF PX(x) > 0
“Distribution”
• “Distribution” is an over-used/over-loaded term
• It usually means PMF/PDF, as per:
• https://en.wikipedia.org/wiki/Probability_distribution,
• Usually, “a probability distribution is the mathematical function that gives the probabilities of
occurrence of different possible outcomes for an experiment.”
• https://ocw.mit.edu/courses/mathematics/18-443-statistics-for-applications-fall-
2006/lecture-notes/lecture1.pdf
• https://www.itl.nist.gov/div898/handbook/eda/section3/eda361.htm
• But then the word “distribution” is part of the phrase “CDF”
Random Variables
• Discrete and Continuous
Discrete Random Variable: Examples
• Bernoulli Distribution
• Random variable X can model success/failure at a task
• P(X=1; α) = α, where 0 ≤ α ≤ 1
• Notation:
g(y; θ) means g(.; .) is a function of variable y, and
g(.; .) is parametrized by constant θ.
θ is called the parameter.
• P(X=0; α) = 1-α
• e.g.,
(X=1) can model success in getting a head on a coin toss
(X=0) can model failure in getting a head on a coin toss
Bernoulli
• Jacob Bernoulli
• Swiss mathematician
• Law of large numbers
• Discovered ‘e’
• While studying compound interest

• Interest of R% per year,

when compounded continuously,
yields exp(R/100) after 1 year
Bernoulli
• Bernoulli family
• Among the greatest
in the history of
mathematics
• https://www.jstor.org/
stable/27958158
Discrete Random Variable: Examples
• Binomial Distribution: Repeated Bernoulli Trials
• Random variable X can model number of successes observed in multiple tries
• Probability of getting k successes from n trials is P(X=k;α,n) = nCk αk (1-α)(n-k)
• Left: PMF. Right: CDF.

α
Discrete Random Variable: Examples
• Binomial Distribution: Repeated Bernoulli Trials
• Check that the sum of all values in the PMF is 1
• We can apply the binomial theorem:

with x:=α and y:=1-α

to the binomial PMF P(X=k;α,n) = nCk αk (1-α)(n-k)

Discrete Random Variable: Examples
• Geometric Distribution: Repeated Bernoulli Trials
• Definition 1:
Random variable X models number of trials made to get the first success
• P(X=k) := P(first success in trial k) := (1–α)(k–1)α, for k = 1,2,3,…,∞
• Definition 2 (NOT identical to definition 1):
Random variable Y models number of failures before first success
• P(Y=k) := P(k failures before first success) := (1–α)kα, for k = 0,1,2,…,∞
• Note
• P(X=1) = P(Y=0). In general, P(X=n) = P(Y=n–1), for n = 1,2,3,…,∞
• ∑k=1,2,3,…,∞ P(X=k) = 1
• ∑k=0,1,2,…,∞ P(Y=k) = 1
Discrete Random Variable: Examples
• Geometric Distribution: Repeated Bernoulli Trials
• Cumulative distribution function
• Definition 1
• CDF f(k) = P(X ≤ k)
• P(first success in trial 1) + P(first success in trial 2) + … + P(first success in trial k)
!"# #" #"$ ! !
• 𝑃 𝑋 ≤ 𝑘 = 𝛼 + 𝛼 1 − 𝛼 + ⋯+ 𝛼 1 − 𝛼 = 𝛼 #" #"$ =1− 1−𝛼
• Then, CDF f(k) = 1 – (1–α)k
• Definition 2
• CDF f’(k) = P(Y ≤ k)
• P(0 failures before first success) + … + P(k failures before first success)
! #" #"$ !"# !%#
• 𝑃 𝑌 ≤ 𝑘 = 𝛼 + 𝛼 1 − 𝛼 + ⋯+ 𝛼 1 − 𝛼 = 𝛼 #" #"$ =1− 1−𝛼
• Then, CDF f’(k) = 1 – (1–α)k+1
Discrete Random Variable: Examples
• Geometric Distribution: Repeated Bernoulli Trials
• Left column:
Definition 1 α α

• Right column:
Definition 2
• Top row: PMFs
• Bottom row: CDFs

α α
Discrete Random Variable: Examples
• Geometric Distribution: Repeated Bernoulli Trials
• Definition 1: P(X=k) := P(first success in trial k)
• CDF: f(k) = P(X ≤ k) = 1 – P(X > k) = 1 – (1–α)k
• Memoryless Property: P(X > k+m | X > k) = P(X > m)
• Intuition
• Suppose, for person A, first k trials haven’t given success
• Then, for person A, probability of m subsequent trials also not giving success
is the same as
probability, for person B (who started just now), performing first m trials and not getting success
• Proof:
• P(X > k+m | X > k)
= P(X > k+m, X > k) / P(X > k)
= P(X > k+m) / P(X > k)
= (1–α)k+m / (1–α)k
= (1–α)m
= P(X > m)
Discrete Random Variable: Examples
• Geometric Distribution: Repeated Bernoulli Trials
• Definition 2: P(Y=k) := P(k failures before first success)
• CDF: f’(k) = P(Y ≤ k) = 1 – P(Y > k) = 1 – (1–α)k+1
• Memoryless Property: P(Y ≥ k+m | Y ≥ k) = P(Y ≥ m)
• Intuition
• Suppose, for person A, at least first k trials were failures
• Then, for person A, probability that next m trials are also failures
is the same as
probability, for person B (who started just now), probability of at least first m trials are failures
• Proof:
• P(Y ≥ k+m | Y ≥ k)
= P(Y ≥ k+m, Y ≥ k) / P(Y ≥ k)
= P(Y > k+m–1) / P(Y > k–1)
= (1–α)k+m / (1–α)k
= (1–α)m
= P(Y > m-1) = P(Y ≥ m)
Discrete Random Variable: Examples
• Poisson Distribution
• Consider random events/arrivals/hits occurring at constant average rate λ>0,
i.e., λ arrivals/hits (typically) per unit time
• Examples
• Vehicles passing an intersection
• Photons hitting a sensor inside a photographic camera
• The arrivals/hits occur independently of each other,
i.e., occurrence of new arrivals/hits doesn’t depend on time of previous one
• Poisson random variable X models
number of arrivals/hits occurring
in unit time
• P(X=k; λ) = λk e-λ / k!,
for k=0,1,2,…
Discrete Random Variable: Examples
• Poisson Distribution
Discrete Random Variable: Examples
• Poisson Distribution
Discrete Random Variable: Examples
• Poisson Distribution
• A limiting case of the Binomial distribution
• Limitn→∞ PBinomial(X=k; α=λ/n, n)
= λk e-λ / k! = PPoisson(X=k; λ)
• Proof:
!!
• PBinomial(X=k; α, n) = 𝛼 $ 1 − 𝛼 !#$
!#$ !$!
!! %! % ! % #$
• = !#$ !$! !! 1 − ! 1−!
%! !! % ! % #$
• = ( ) 1− 1−
$! !#$ !!! ! !
%!
• → 1 𝑒 #% (1) when n→∞
$!
• Counts “successes” in a unit-length interval by modeling the interval as
infinitely-many (n→∞) infinitesimal-length (1/n→0) intervals that each have
an infinitesimal success probability α=λ/n underlying a PoissonBernoulli PMF
• This gives the interpretation for parameter λ=nα as
n*(probability of success, or “hits”, in an interval). More analysis on this later.
Discrete Random Variable: Examples
• What is the effect/distribution when we combine photon guns ?
Random Variables
• Sum of Random Variables
• Consider two random variables X and Y,
and a joint probability space {Ω12,ẞ12,P12}
and an associated joint distribution PXY(x,y)
• Defining Z := X + Y means the following:
• Conduct joint random experiment, underlying the pair (X,Y),
leading to outcome (s1,s2) ∈ Ω12
• Get x := X(s1,s2) and y := Y(s1,s2)
• Define z := x + y
• So, Z is a random variable, taking values z,
with an associated distribution PZ(z)
and a probability space {Ω12,ẞ12,P12}
• Note that PZ(t) isn’t necessarily equal to PX(t)+PY(t),
i.e., adding random variables isn’t same as superposing or averaging their PDFs/PMFs
Discrete Random Variable: Examples
• Poisson Distribution
• Sums of Independent Poisson Random Variables
• Let X~Poisson(λ) and Y~Poisson(μ) be independent
• Define random variable Z := X + Y
• Then, P(Z) = PPoisson(Z; λ+μ)
• Proof:
• P(Z=k) = ∑$&'( 𝑃(𝑋 = 𝑗, 𝑌 = 𝑘 − 𝑗)
Discrete Random Variable: Examples
• Poisson Distribution
• Poisson Thinning
• Let X ~ Poisson(λ)
• Consider joint PMF P(X,Y), where P(Y|X=j) = PBinomial(Y;j,p)
• This is a “thinning” process that selects a subset of the j arrivals
• Selecting each of the j arrivals/hits (stochastically) with (Bernoulli) probability p
• Then, P(Y) = PPoisson(Y; λp)
• Proof:
• P(Y=k) = ∑∞
&'$ 𝑃 𝑋 = 𝑗, 𝑌 = 𝑘 = ∞
• ∑∞&'$ 𝑃 𝑦 = 𝑘 𝑋 = 𝑗)𝑃 𝑋 = 𝑗 =
∞
Poisson
• Siméon Denis Poisson
• French mathematician, engineer, physicist
• Advisors
• Lagrange, Laplace
• Students (non-PhD)
• Dirichlet
• Carnot (father of thermodynamics)
• One of 72 names inscribed on Eiffel Tower
Poisson
Random Variables
• Sum of Random Variables
• Consider two independent random variables X and Y,
and a joint probability space {Ω12,ẞ12,P12}
and an associated joint distribution PXY(X=x,Y=y)
• Define Z := X + Y. What is PZ(Z) ?
• Case 1: Discrete random variables
• PZ(Z = z)
= PXY( { (t,z–t): X = t, Y = z–t } )
= ∑t PXY(X = t, Y = z–t)
= ∑t PX(X = t) PY(Y = z–t)
• This operation, h(a) := ∑t f(t) g(a–t) is the
convolution of functions f(t) and g(t), producing the function h(t).
This is typically written in the notation h := f∗g
• Thus, addition of two independent discrete random variables X and Y leads to
a third random variable Z whose PMF PZ(.) is the convolution of the PMFs PX(.) and PY(.)
Random Variables
• Sum of Random Variables
• Consider two independent random variables X and Y, with PDFs A(.) and B(.)
• Consider and a joint probability space {Ω12,ẞ12,P12}
and an associated joint distribution PXY(x,y)
• Define Z := X + Y. What is its PDF C(.) ?
• Case 2: Continuous random variables
p(x,y) dy dx
= dp(x,y)
• CDF P(X+Y≤z) = = probability measure at (x,y)

• PDF C(z) = d/dz (P(X+Y≤z)) =

• Thus, C(z) = ∫ 𝑥 𝐴 𝑥 𝐵 𝑧 − 𝑥 𝑑𝑥 (because of independence of X and Y)

• This is the convolution operation for functions on the continuous domain (C := A*B)
Random Variables
• Leibniz’s
integral
rule
Joint and Marginal Distributions
• Consider a collection of random variables X1, …, Xn
• Definition: The marginal distribution of a subset of
a collection of random variables {X1, …, Xn}
is the probability distribution of the variables contained in the subset
• e.g., when n=2, P(X1) and P(X2) are the marginal distributions
• Term “marginal” comes from legacy where summation was written in
“margins” of sheet
• Specifying a joint PMF/PDMF automatically
specifies all underlying marginal PMFs/PDFs
via the associated summation/integration
Discrete Random Variable: Examples
• Uniform distribution
• Sample space has finite cardinality (say, n < ∞)
• Within sample space, each outcome has equal probability (1/n)
• Examples: Coin toss. Die roll.
Continuous Random Variable: Examples
• Uniform Distribution
• Random variable X takes values within a
finite interval that can be
closed, i.e., [a,b], or open, i.e., (a,b)
where -∞ < a < b < ∞
• Within [a,b] or (a,b), all sub-intervals of
same length have equal probability measure
Continuous Random Variable: Examples
• Exponential Distribution
• Consider random events/arrivals/hits occurring at a constant average rate λ>0,
i.e., λ arrivals/hits (typically) per unit time
• Example: packets arriving at a router
• The arrivals/hits occur independently of each other
• Exponential random variable models the probability density of the
time elapsed (or waiting time) between two consecutive arrivals/hits
• PDF
• P(x) = 0, for all x < 0
• P(x) = 𝜆 exp −𝜆𝑥 , ∀𝑥 ≥ 0
• CDF
• f(x) = 0, for all x < 0
• f(x) = 1 − exp −𝜆𝑥 , ∀𝑥 ≥ 0
Continuous Random Variable: Examples PDF
P(x) = 0, for all x < 0
P(x) = 𝜆 exp −𝜆𝑥 , ∀𝑥 ≥ 0
• Exponential Distribution CDF
f(x) = 0, for all x < 0
f(x) = 1 − exp −𝜆𝑥 , ∀𝑥 ≥ 0
Continuous Random Variable: Examples
• Exponential Distribution relates to Poisson Distribution
• Let average arrival/hit-occurrence rate be λ per unit time
• Let random variable Y model the number of arrivals/hits occurring in unit time
• So, PPoisson(Y=k; λ) = λk e-λ / k!
• Also, in time T units, average arrival/hit-occurrence rate will be λT
• For time interval [0,t], let Nt := number of arrivals occurred (last one at time t)
• Let random variable Xt model time taken for next arrival/hit to occur
• Consider the event where the next occurrence Xt is at any time after ‘x’
• Pexponential(Xt > x) = P(Nt = Nt+x)
• So, Pexponential(Xt ≤ x) = 1 – P(Nt = Nt+x)
• By assumptions underlying the Poisson distribution with parameter λx,
P(Nt = Nt+x) = PPoisson(Y=0; λx) = e-λx
• Thus, Pexponential (Xt ≤ x) = 1 – e-λx (this is the CDF)
• Thus, Pexponential (x) = λ e-λx (this is the PDF)
Continuous Random Variable: Examples
• Exponential Distribution satisfies the memoryless property
• Proof:
• Let X be an exponential random variable, with parameter λ
• Its CDF is: Pexponential (Xt ≤ x) = 1 – e-λx
• Then, we want to show that Pexponential(X > x+t | X > x) = Pexponential(X > t)
• “X > x” means waiting time for next arrival/hit is more than x
• Pexponential(X > x+t | X > x)
= Pexponential(X > x+t, X > x ) / Pexponential(X > x)
= Pexponential(X > x+t) / Pexponential(X > x)
= e-λ(x+t) / e-λx
= e-λt
= Pexponential(X > t)
Continuous Random Variable: Examples
• Exponential Distribution
• The only non-negative-valued continuous random variable satisfying the
memoryless property is the exponential random variable
• Proof:
• Let non-negative-valued continuous random variable X have PDF P(.)
• Define “survival” function S(x) := P(X > x)
• P(x)dx
≈ P(x < X < x+dx)
= P(X > x) P(X < x+dx | X > x)
= P(X > x) (1 – P(X > x+dx | X > x))
= P(X > x) (1 – P(X > dx)) assuming memoryless property
= S(x) (1 – S(dx))
• By Taylor-series expansion around x=0, we get S(dx) ≈ S(0) + S’(0)dx = 1 + S’(0)dx
• Because S(0) = P(X > 0) = 1 (because X is non-negative valued and continuous)
• So, P(x)dx ≈ S(x) (–S’(0)dx)
Continuous Random Variable: Examples
• Exponential Distribution
• The only non-negative-valued continuous random variable satisfying the
memoryless property is the exponential random variable
• Proof:
• Let non-negative-valued continuous random variable X have PDF P(.)
• Define survival function S(x) := P(X > x)
• P(x)dx ≈ S(x) (–S’(0)dx)
• Because 1–CDFX(x) = S(x), we get: –P(x) = S’(x)
• Thus, substituting –S’(x) on the left hand side, we get: S’(x)dx ≈ S(x) S’(0) dx
• So, S’(x) ≈ S(x) S’(0)
• So, d/dx [S(x)] ≈ S’(0) S(x)
• Given S(0) = 1, the solution is given by: S(x) = exp (S’(0) x)
• Defining –λ := S’(0), we get 1–CDFX(x) = exp(–λx), or P(x) = λ exp(–λx)
• λ is non-negative, but must also be strictly positive for P(x) to be a valid PDF
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• PDF is P(x)

• Why called “normal” ?

• https://condor.depaul.edu/ntiourir/NormalOrigin.htm
• PDF shape is called the “bell curve”
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• PDF is P(x)

• This PDF does integrate to 1

• Proof:
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• Parameter 𝜇 is called the “location parameter”
• Changing 𝜇 simply shifts the curve left/right
• Areas under curve
• Most of the area (around 99.8%) is within a distance of ±3𝜎 around location 𝜇
• Parameter 𝜎>0 is called the “scale parameter”
• Increasing 𝜎 causes the curve to flatten/spread
Gauss
• Carl Friedrich Gauss
• German mathematician and physicist
• “Greatest mathematician since antiquity”
• Child prodigy
• Refusing to publish work which he didn’t
consider complete and above criticism
• Declined to present intuition behind his
often very elegant proofs
• Mathematician Eric Temple Bell said that
if Gauss had published all his discoveries
in a timely manner, then he would have
advanced mathematics by 50 years
• Disliked teaching
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution in nature
• Velocities of molecules in the ideal gas
• Maxwell–Boltzmann distribution
in statistical mechanics
• Wikipedia:
“An ideal gas is a theoretical gas composed of a set of
randomly moving, non-interacting point particles.”
• “A gas behaves more like an ideal gas at higher temperature and lower pressure, as:
the work which is against intermolecular forces becomes less significant compared with the
particles’ kinetic energy, and
the size of the molecules becomes less significant compared to the empty space between them.”
• “More generally, velocities of the particles in any system in thermodynamic equilibrium will have a
Gaussian distribution, due to the maximum-entropy principle.”
• In the ideal case,
each component of the velocity vector will have a Gaussian distribution, and
each component will be independent of all other components
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution in nature
• Position of a particle undergoing diffusion
• If initially the particle is located at the origin,
then after time t,
its displacement (signed distance) traversed
is described by
a Gaussian distribution with σ2=t and μ=0
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• Central limit theorem
• Consider a large number of RVs X1, X2, …, Xn that are:
• Independent and Identically Distributed (i.i.d.) (with finite scale parameter)
• Let random variable 𝑋𝑛 := (X1 + X2 + … + Xn) / n
• Then, lim 𝑃(𝑋1 ) = Gaussian Distribution
1→3
• Examples
• Average of Bernoulli random variables à Gaussian
• Average of Binomial random variables à Gaussian
• Average of Geometric random variables à Gaussian
• Average of Poisson random variables à Gaussian
• Average of Uniform (discrete/continuous) random variables à Gaussian
• Average of Exponential random variables à Gaussian
• Average of Gaussian random variables à Gaussian
• Average of “M” random variables à Gaussian
• Not Cauchy random variables; PDF doesn’t have a finite scale parameter
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• Central limit theorem
• Example
• Average of “M” random variables
à Gaussian
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• Limiting case of a Binomial distribution
• De Moivre worked on this before Gauss:
De Moivre–Laplace theorem
• Imagine a “random walker” on the real line, starting at origin z=0 at time t=0
• Within each timeframe Δt, the walker takes either
a Δz step right with probability p, OR a Δz step left with probability q:=1-p
• Let total number of steps taken = n
• Consider an event where walker took ‘x’ steps right; ‘n-x’ steps left
• Random walker is located at z = Δz(2x-n)
• For (x+f) steps right, where 0<f<1, location can be linearly interpolated as z+Δz.2.f

• Probability of this event is P(x)

• We want to make the walker’s location a continuous (real-valued) random variable.
So, we want to make n, x, (n-x) (infinitely) large, and Δt and Δz (infinitesimally) small
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• Random walk on 1D real line
• Horizontal axis: location
• Vertical axis: time
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• Random walk in 2D
• Small steps,
numbering 25,000
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• Random walk in 2D
• Tiny steps,
numbering 2 million
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• Brownian motion in 2D
• https://www.youtube.com/watch?v=FAdxd2Iv-UA
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• Galton Board (also called bean machine)
• Device invented by Sir Francis Galton
• Simulates a random walk
• Shows that,
binomial distribution
can be visualized as
a Gaussian
• https://www.youtube.com/watch?v=g7A7o3KYa7o
• https://www.youtube.com/watch?v=Kq7e6cj2nDw
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• Limiting case of a Binomial distribution
• Factorial values can be approximated for large ‘n’ using Stirling’s formula

• So, P(x)
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• Limiting case of a Binomial distribution
• Let deviation of x from np be
• When n is large, then x is close to np by our design of “probability function”
• Success probability p is designed to be the ratio of number of successes ‘x’ to number of tries ‘n’.
) *
As 𝑛 → ∞, the ratio ! = ! − 𝑝 → 0 and 𝛿 ≪ 𝑛

• So

• Then,
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• Limiting case of a Binomial distribution
• Also,
?
# # # # # #
• Write = 14
and write = 16
14%5 #%%&
$ 16"5 #"%'
$

5
• As 𝑛 → ∞, the ratio 1 → 0. Also, probability p is fixed positive.
# 5
• We want to evaluate g t ≔ #%7
at a small displacement t = 14 around 0

8( 9 7"9 8(( 9 ;"9 )

• Taylor series: 𝑔 𝑎 + t ≔ 𝑔 𝑎 + #!
+ <!
+⋯
• Evaluate around a=0
# ;
• We get the Maclaurin series for g(t) := = 1 − + 𝑂(𝑡 <)
#%; <
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• Limiting case of a Binomial distribution
• So, for large ‘n’ and ‘x’, P(x) tends to:

• For this binomial distribution

• When δ is large, i.e., as x deviates more from np,
then P(x) goes closer to 0 (unlikely event)
• For P(x) to be not-close to 0, value x needs to be sufficiently close to np, i.e., δ << n

• Thus, functional form of P(x) →

Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• Limiting case of a Binomial distribution
• After x steps towards right (out of n steps) in time t = n.Δt ,
random walker’s location is at z = Δz(2x-n) that implies x = n/2 + z/(2.Δz)
• Consider: PMF P(X) on integer steps X leading to interpolated P'(X') on
continuous-valued steps X', PDF P(Z) on real coordinates Z
• As Δz→0 and n→∞, P'(.) better approximates P(.)
• Consider a mapping between P'(X') and P(Z)
• A change of 1 in x leads to a change of 2.Δz in z
• So, P(z) 2.Δz = P(x), where z corresponds to its associated x
• Let: for simplicity, p=q=0.5. “Diffusion coefficient” D := (Δz)2 / (2.Δt)
> > @C > A! @G > > A!
• Then, P(z;t) = exp − = exp − ;
?@A ?B D.?FC ? @A ! C ?B ?HC IHC
• A Gaussian with location parameter 0 (because p=q) and
scale-parameter2 ∝ t (spread increase with time)
De Moivre
• Abraham de Moivre
• French mathematician
• Persecution from France led to London
where he met Newton and
came across Principia Mathematica
• Used to work as private math tutor,
taking long walks to travel from one student
to another, carrying pages torn out of
Principia Mathematica to read during walks
• Was accepted into Royal Society for his work
in math, but couldn’t get a university job in
London due to his French origins
• Remained poor throughout his life
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
Continuous Random Variable: Examples
• Gaussian (Normal) Distribution
• “Everyone believes in the normal law,
the experimenters because they imagine that it is a mathematical theorem,
and the mathematicians because they think it is an experimental fact.”
• J.F. Gabriel Lippmann (1845 – 1921)
• Nobel laureate in physics (1908)
for inventing a method for capturing
color photographs
• Held positions as
Professor of Mathematical Physics,
Professor of Experimental Physics