Foundations of Data Science

(23ECE205)

Dr. Vivek Venugopal

Assistant Professor
Room no B-206, ECE department
Email id: [email protected]
 Notion of a Random Variable
• A random variable is defined as a function that assigns a numerical value to the outcome of the experiment

• The outcome of a random experiment need not be a number.

• However, we are usually interested not in the outcome itself, but rather in some measurement or numerical
attribute of the outcome.

• For example, in n tosses of a coin, we may be interested in the total number of heads and not in the specific
order in which heads and tails occur.
Random Variable
• In a randomly selected Web document, we may be interested only in the length of the document.

• In each of these examples, a measurement assigns a numerical value to the outcome of the random
experiment.

• Since the outcomes are random, the results of the measurements will also be random. Hence it makes
sense to talk about the probabilities of the resulting numerical values.

• The concept of a random variable formalizes this notion.

Random Variable
• A random variable is a mapping from the sample space to the set of real numbers.

• In other words, a random variable is an assignment of real numbers to the outcomes of a random
experiment.

• A random variable X is a function that assigns a real number, to each outcome in the sample space
of a random experiment.

• Recall that a function is simply a rule for assigning a numerical value to each element of a set, as shown
pictorially below
Random variable

• The specification of a measurement on the outcome of a random experiment defines a function on the
sample space, and hence a random variable.

• The sample space S is the domain of the random variable, and the set SX of all values taken on by X is the
range of the random variable.

• SX is a subset of the set of all real numbers.

• Notation: capital letters denote random variables, e.g., X or Y, and lower case letters denote possible values
of the random variables, e.g., x or y.
Numerical Example
Solution
Numerical Example
Solution

The above example shows that a function of a random variable produces another random variable.
Random Variable
• For random variables, the function or rule that assigns values to each outcome is fixed and deterministic,
as, for example, in the rule -count the total number of heads facing up in the toss of coin.

• The randomness in the experiment is complete as soon as the toss is done.

• The process of counting the dots facing up is deterministic.

• The randomness in the observed values of X is induced by the underlying random experiment, and we
should therefore be able to compute the probabilities of the observed values of X .

• Based on the type of values taken by random variable X, they can be categorized into discrete and
continuous random variables.
Technique for finding the probabilities of
events involving the random variable X.
• Let the underlying random experiment have sample space S and event class

• To find the probability of a subset B of R, we need to find the outcomes in S that are mapped to B, that is,

• If event A occurs then so event B occurs. Conversely, if event B occurs, then the value implies that is in A, so
event A occurs. We refer to A and B as equivalent events.

• The probability that X is in B is given by:

Numerical Example
Example
DISCRETE RANDOM VARIABLES AND
PROBABILITY MASS FUNCTION
• A discrete random variable X is defined as a random variable that assumes values from a countable set, that
is,

• A discrete random variable is said to be finite if its range is finite, that is,

• Since the sample space is discrete, we only need to obtain the probabilities for the events
in the underlying random experiment.

• The probabilities of all events involving X can be found from the probabilities of the

• The probability mass function (pmf) of a discrete random variable X is defined as:
Numerical Problem
Solution
Numerical Problem
Solution
• The events A1, A2,…Ak form a partition of S as illustrated in Figure

• Let the events be disjoint. Let then

since each is mapped into one and only one value in SX.

• we can show that S is the union of the Ak’s

• Every in S is mapped into some x k so that every belongs to an event Ak in the partition.

• Therefore

• All events involving the random variable X can be expressed as the union of events

• For example, suppose we are interested in the event X in B

 Probabilities of PMF

Property (i) is true because the pmf values are defined as a probability,
Property (ii) follows because the events form a partition of S.
property (iii). Any event B involving X is the union of elementary events, so by Axiom III we have:
Numerical Problem
Solution
Cumulative Distribution Function (CDF)
Numerical Problem
Solution
Solution
Numerical Problem
Solution
Solution
Standard Discrete Random Variables
• Bernoulli Random Variable

• Binomial Random Variable

• Geometric Random Variable

• Poisson Random Variable

• Uniform Random Variable

Bernoulli Random Variable
• Let A be an event related to the outcomes of some random experiment. The Bernoulli random variable IA
equals one if the event A occurs, and zero otherwise.

• It is a discrete random variable since it assigns a number to each outcome of S.

• It is a discrete random variable with range ={0,1} and its pmf is

where P[A]=p
Example
• Let A be an event of interest in some random experiment, e.g., a device is not defective which occurs with
probability p. We say that a “success” occurs if A occurs when we perform the experiment. The Bernoulli
random variable IA is equal to 1 if A occurs and zero otherwise, and is given by the indicator function for A:
Find the pmf of IA
Example
Application of Bernoulli random variable
• A Bernoulli random variable is a good model for a binary-data generator.
• When binary data is transmitted over a communication channel some bits are received in error.
• We can model an error by modulo-2 addition of a 1 to the input bit; thus, we change a 0 into a 1 and a 1
into a 0.
• Therefore, a Bernoulli random variable can also be employed to model the channel errors over a single bit
transmission.
Binomial random Variable
• Suppose that a random experiment is repeated n independent times. Let X be the number of times a certain event A
occurs in these n trials. X is then a random variable with a range Sx={0,1,2..n}

• For example, X could be the number of heads in n tosses of a coin.

• If we let Ij be the indicator function for the event A in the jth trial, then

X is the sum of the Bernoulli random variables associated with each of the n independent trials.

X is called the binomial random variable.

Let X be the number of heads in three independent tosses of a coin. Assuming that heads occur
with a probability p, Find the pmf of X. Also calculate the pmf if p=0.5.
Numerical Problem
• A player receives $1 if the number of heads in three coin tosses is 2, $8 if the number is 3, but
nothing otherwise. Find the pmf of the reward Y.
Solution
Numerical Example
Assume 10,000 bits are transmitted over a channel in which the error probability is 10-3. What is the probability
that the total number of errors is less than 3?
9/19/2024 Ms.Sunitha.R ,Dept.Of ECE 42
Numerical Example
• A pair of dice is thrown 4 times. If getting a doublet is considered a success, find the probability of getting
two succeses
Solution
Solution
 Geometric Random Variable
• The geometric random variable is used when one is modelling a series of experiments that have one of two possible
outcomes - success or failure.
• The geometric random variable tells you the number of experiments that were performed before obtaining a success.
• The geometric random variable arises when we count the number M of independent Bernoulli trials until the first
occurrence of a success.
• M is called the geometric random variable and it takes on values from the set {1, 2, 3, ... }
• The probability mass function pmf for the geometric random variable is given by:

where p=P[A] is the probability of “success” in each Bernoulli trial.

• Interpretation: Flip a coin until you get a head.

• Note that P[M=k] decays geometrically with k, and that the ratio of consecutive terms is

• As p increases, the pmf decays more rapidly.

• In a geometric distribution, each event is called a Bernoulli trial.
• Each Bernoulli trial has two outcomes—success or failure—with a fixed probability.
• The probability of success in a trial does not change across trials. Each trial is thus independent of past or future trials.
• This makes a geometric distribution memoryless—in other words, an individual trial that is a failure will not improve or
reduce the probability of the next trial being a success.
• If these conditions are not met, then a geometric distribution cannot be used to determine the probability of success.
Geometric Distribution
• The geometric distribution is similar to the binomial distribution in that each trial can have only two
outcomes: success or failure, with the probability of success being the same across trials.

• The geometric distribution is concerned with only the first instance of success, whereas the binomial
distribution seeks to determine the total number of successes over a fixed set of trials.

• The geometric random variable arises in applications where one is interested in the time (i.e., number of
trials) that elapses between the occurrence of events in a sequence of independent experiments
Numerical Example
• A die is cast until 6 appears. What is the probability that the die must be cast more than 5 times
Solution
Solution
• P(X>5)=0.4018
 Poisson Random Variable
• In many applications, we are interested in counting the number of occurrences of an event in a certain time
period or in a certain region in space.
• The Poisson random variable arises in situations where the events occur “completely at random” in time or
space.
• For example, the Poisson random variable arises in counts of emissions from radioactive substances, in
counts of demands for telephone connections, and in counts of defects in a semiconductor chip.
• The pmf for the Poisson random variable is given by

where α is the average number of event occurrences in a specified time interval or region in space.
• For is maximum at k=0

• For is maximum at [α ]

• if α is a positive integer, is maximum at k

= α and at k = α -1
• The pmf of the Poisson random variable sums to one, since

where we used the fact that the second summation is the infinite series expansion for
 Applications of Poisson RV
• One of the applications of the Poisson probabilities is to approximate the binomial probabilities in the case where p is very
small and n is very large,

• The Poisson random variable appears in numerous physical situations because many models are very large in scale and
involve very rare events.
• For example, the Poisson pmf gives an accurate prediction for the relative frequencies of the number of particles emitted
by a radioactive mass during a fixed time period.
• This correspondence can be explained as follows.
• A radioactive mass is composed of a large number of atoms, say n. In a fixed time interval each atom has a very small
probability p of disintegrating and emitting a radioactive particle. If atoms disintegrate independently of other atoms, then
the number of emissions in a time interval can be viewed as the number of successes in n trials.
• For example, one microgram of radium contains about n=10 16 atoms, and the probability that a single atom will
disintegrate during a one millisecond time interval is p=10 -15 .
• Hence, n is so large and p so small that one could argue that the limit n →∞ has been carried out and that the number of
emissions is exactly a Poisson random variable.
Numerical example
Solution
Numerical Example
Solution
Numerical Example
Solution
Uniform Random Variable
• The discrete uniform random variable Y takes on values in a set of consecutive integers
with equal probability:

• This random variable occurs whenever outcomes are equally likely,

• e.g., toss of a fair coin or a fair die, spinning of an arrow in a wheel divided into equal segments, selection
of numbers from an urn.

The uniform random variable occurs whenever outcomes are equally likely. It plays a key role in the
generation of random numbers.
Numerical Example
Solution
Expected Values of Discrete RV
• In order to completely describe the behavior of a discrete random variable, an entire function, namely pX(x)
must be given.

• In some situations we are interested in a few parameters that summarize the information provided by the
pmf.

• Mean is one of the parameter to quantify certain properties.

• Figure shows the results of many
repetitions of an experiment that produces
two random variables.
• The random variable Y varies about the
value 0, whereas the random variable X
varies around the value 5.
• It is also clear that X is more spread out
than Y.
Variance of a Random Variable
• The expected value E[X], by itself, provides us with limited information about X.

• For example, if we know that E[X]=0, then it could be that X is zero all the time.

• However, it is also possible that X can take on extremely large positive and negative values.

• We are therefore interested not only in the mean of a random variable, but also in the extent of the random
variable’s variation about its mean.

• Let the deviation of the random variable X about its mean be X-E[X] which can take on positive and
negative values.
Variance of a Random Variable
• Since we are interested in the magnitude of the variations only, it is convenient to work with the square of
the deviation, which is always positive,

• The expected value is a constant, so we will denote it by

• The variance of the random variable X is defined as the expected value of D:

Standard Deviation
• The standard deviation of the random variable X is defined by:
Numerical Example

Find the expected value and the variance of the Bernoulli Random Variable
Solution
Expected value of Discrete RV
• The use of the term “expected value” does not mean that we expect to observe E[X] when we perform the
experiment that generates X.

• For example, the expected value of a Bernoulli trial is p, but its outcomes are always either 0 or 1.

• E[X] corresponds to the “average of X” in a large number of observations of X.

• We expect the sample mean to converge to E[X] as n becomes large.

Numerical Example
• Compute the mean and variance of binomial Random Variable
Mean of a Binomial Random Variable
Variance
Numerical Example
Let X be the number of heads in three independent tosses of a coin. Find the mean and variance of
X.
 Solution

E[X]=np
Var[X]=np(1-p)=npq

with p=1/2 and n=3

Properties of Expectation operation

Adding a constant to a RV does not affect the variance

Properties of variance

Mean and Variance for a Geometric Random Variable

Mean and Variance for Poisson Random
variable
• The pmf for the Poisson random variable is given by

where α is the average number of event occurrences in a specified time interval or region in space.
Mean and variance of a Poisson random variable is given by:
Function of Random Variable
Numerical Example
Solution
Expected Value of Functions of a Random
Variable
• Let X be a discrete random variable, and let Z=g(x)
• Since X is discrete, Z=g(x) will assume a countable set of values of the form g(xk) where xk is an element of
Sx.
• Denote the set of values assumed by g(X) by {z1,z2…}
• One way to find the expected value of Z is to find the pmf of Z.
Numerical Example
Solution
Solution
Numerical Example
Solution
Conditional Probability Mass Function
• In many situations we have partial information about a random variable X or about the outcome of its
underlying random experiment.
• We are interested in how this information changes the probability of events involving the random variable.
• The conditional probability mass function addresses this question for discrete random variables.
Conditional Expected Value
• Let X be a discrete random variable, and suppose that we know that event B has occurred.
• The conditional expected value of X given B is defined as:

• where we apply the absolute convergence requirement on the summation.

• The conditional variance of X given B is defined as:
Useful results
• Find E[Z]
Useful results
Numerical Example
Solution