Department of Mathematics

UNIT-IV
SAMPLING AND ESTIMATION

SAMPLING THEORY
In a statistical investigation the interest usually lies in the assessment of the general magnitude
and the study of variation with respect to one or more characteristics relating to individuals
belonging to a group. This group of individuals under study is called population
or universe. Thus in statistics, population is an aggregate of objects, animate or inanimate,
under study. The population may be finite or infinite.
It is obvious that, for any statistical investigation complete enumeration of the population is
rather impracticable. For example, if we want to have an idea of the average per capita
(monthly) income of the people in India, we will have to enumerate all the earning individuals
in the country which is rather a very difficult task.
If the population is infinite, complete enumeration is not possible. Also if the units are
destroyed in the course of inspection, (e.g., inspection of crackers, explosive materials, etc.),
100% inspection, though possible, is not at all desirable. But even if the population is finite or
the inspection is not destructive, 100% inspection is not taken recourse to because of
multiplicity of causes, viz. ,administrative and financial implications, time factor, etc., and we
take the help of sampling.
Size of the population is the number of objects or observations in the population and is denoted
by 𝑁. A finite subset of statistical individuals in a population is called a sample and the number
of individuals in a sample is called the sample size. Size of the sample is denoted by 𝑛. If 𝑛 ≥
30, the sampling is said to be large sampling. If 𝑛 < 30, the sampling is said to be small
sampling.
For the purpose of determining population characteristics, instead of enumerating the entire
population, the individuals in the sample only are observed. Then the sample characteristics
are utilized to approximately determine or estimate the population. This method is called the
statistical inference. For example, on examining the sample of a particular stuff we arrive at a
decision of purchasing or rejecting that stuff. The error involved in such approximation is
known sampling error and is inherent and unavoidable in any and every sampling scheme. But
sampling results in considerable gains, especially in time and cost not only in respect of making
observations of characteristics but also in the subsequent handling of the data.

Fourth Semester 1 21MA41

 Department of Mathematics
Sampling is quite often used in our day to day practical life. For example, in a shop we assess
the quality of sugar, wheat or any other commodity by taking a handful of it from the bag and
then decide to purchase it or not. A housewife normally tests the cooked products to find if
they are properly cooked and contain the proper quantity of salt. Statistical measures or
constants obtained from the population such as population mean, variance etc., are called the
Parameters. Similarly, statistical quantities computed from sample such as sample mean,
sample variance etc., are known as statistics.
Types or Sampling: Some of the commonly known and frequently used types of sampling are:
(i) Purposive sampling (ii) Random sampling (iii) Stratified sampling, (iv) Systematic
sampling.
Consider the following example.
Suppose the following are the marks of 30 students in a test carrying 10 marks; the marks are
arranged, say, according to the roll number of the students.
2, 4, 0, 5, 8, 6, 4, 1, 3, 5, 3, 3, 2, 4, 7, 7, 3, 2, 0, 4, 6, 8, 7, 1, 8, 1, 4, 5, 6, 7.
An information of this type is called a raw (or an unclassified) statistical data. The individual
numbers present in the data are called the items or the observations in the data. Denote them
by 𝑥1 , 𝑥2 , 𝑥3 , …. The information can be put in the form of a table called the table of discrete
frequency distribution.
𝑥𝑖 𝑓𝑖
0 2
1 3
2 3
3 4
4 5
5 3
6 3
7 4
8 3

The entries in the first column are called the variables 𝑥𝑖 and the entries in the second column
are called the frequencies 𝑓𝑖 .
Further the data can be grouped as below.

Fourth Semester 2 21MA41

 Department of Mathematics
Marks No of students
class-intervals 𝑓𝑖
0-2 8
3-5 12
6-8 10

The table of the above type is called a table of grouped frequency distribution. The entries in
the first column are called the class-intervals (or classes) and the entries in the second column
are the frequencies.
While analyzing statistical data, it is generally observed that the items or the frequencies cluster
around some central value of the variable. Such a central value is called a measure of central
tendency of the data. The mean (or average) is one such measure.

Mean:
1. For a raw data consisting of ‘n’ items 𝑥1 , 𝑥2 , 𝑥3 , … 𝑥𝑛 , the arithmetic mean or mean is defined
by the formula
𝑥1 + 𝑥2 + 𝑥3 + ⋯ +𝑥𝑛 ∑ 𝑥𝑖
Mean = =
𝑛 𝑛
2. For a frequency distribution (𝑥𝑖 , 𝑓𝑖 ) , the mean is defined by the formula
𝑓1 𝑥1 + 𝑓2 𝑥2 + 𝑓3 𝑥3 + ⋯ +𝑓𝑛 𝑥𝑛 ∑ 𝑓𝑖 𝑥𝑖
Mean = =
𝑓1 + 𝑓2 + 𝑓3 + ⋯ + 𝑓𝑛 ∑ 𝑓𝑖
Variance:
1
1. For a raw data, the variance is defined by Variance = 𝑛 ∑(𝑥𝑖 − mean)2 .

2. For a frequency distribution the variance is defined by

∑ 𝑓𝑖 (𝑥𝑖 − 𝑥̅ )2 ∑ 𝑥𝑖2 𝑓𝑖
Variance = = − 𝑥̅ 2 .
∑ 𝑓𝑖 ∑ 𝑓𝑖
Sampling Distributions:
Given a population, suppose we consider a set of samples of a certain size drawn from the
population. For each sample, suppose we compute a statistic (such as the mean, standard
deviation, etc). These statistics will vary from one sample to the other sample. Suppose, we
group these different statistics according to their frequencies and form a frequency distribution.
The frequency distribution so formed is called a sampling distribution. The standard deviation
of a sampling distribution is called its standard error. The standard error is used to assess the
difference between the expected values and observed values.

Fourth Semester 3 21MA41

 Department of Mathematics
Sampling Distribution of Means:
Consider a population for which the mean is 𝜇 and the standard deviation is 𝜎. Suppose we
draw a set of samples of a certain size 𝑛 from this population and find the mean 𝑥̅ of each of
these samples. The frequency distribution of these means is called a sample distribution of
means.
Suppose the population is finite with size 𝑁. Then 𝜇𝑥̅ and 𝜎𝑥̅ are related to 𝜇 and 𝜎through the
following formulae:
𝜎 √𝑁−𝑛
𝜇𝑥̅ = 𝜇 , 𝜎𝑥̅ = ∗
√𝑛 √𝑁−1

If the population is infinite (or if the sampling is finite with replacement), the formula is given
as;
𝜎
𝜇𝑥̅ = 𝜇 , 𝜎𝑥̅ =
√𝑛

It can be proved that for samples of large size or for samples with replacement, the sampling
distribution of means is approximately a normal distribution for which the sample mean 𝑥̅ is
the random variable. If the population itself is normally distributed, then the sampling
distribution of means is a binomial distribution even for samples of small size. Accordingly,
the standard normal variate for the sampling distribution of means is given by
𝑥̅ −𝜇𝑥̅ 𝑥̅ −𝜇
𝑍= = .
𝜎𝑥̅ 𝜎𝑥̅

Problems:
1. A population consists of four numbers 3, 7, 11, 15. Consider all possible samples of size 2
which can be drawn from this population with and without replacement. Find the mean and
standard deviation in the population, and in the sampling distribution of means verify the
formulas 𝜇𝑋̅ and 𝜎𝑋̅ .
Solution: Given 𝑁 = 4
(3+7+11+15)
Mean = 𝜇 = =9
4
1
Variance = σ2 = 4 {(3 − 9)2 + (7 − 9)2 + (11 − 9)2 + (15 − 9)2 } = 20

Standard deviation = σ = √20

Case i) Sampling with replacement:
Possible samples of size two which can be drawn with replacement are (3, 3), (3, 7), (3,
11), (3, 15), (7, 3), (7, 7), (7, 11), (7, 15), (11, 3), (11, 7), (11, 11), (11, 15), (15, 3), (15,
7), (15, 11), (15, 15). The means of these 16 samples are 3, 5, 7, 9, 5, 7, 9, 11, 7, 9, 11,
13, 9, 11, 13, 15 respectively. The corresponding frequency distribution is

Fourth Semester 4 21MA41

 Department of Mathematics
𝑥̅𝑖 3 5 7 9 11 13 15
𝑓𝑖 1 2 3 4 3 2 1
∑𝑓𝑖 𝑥̅ 𝑖 3+10+21+36+33+26+15
Mean= 𝜇𝑥̅ = = =9
∑𝑓𝑖 16

∑𝑓𝑖 (𝑥̅ 𝑖 −𝜇𝑥 2

̅) 1
Variance= 𝜎𝑥̅2 = = 16 [(3 − 9)2 + (7 − 9)2 + (11 − 9)2 + (15 − 9)2 ]
∑𝑓𝑖

=5
Standard variation= 𝜎𝑥̅ = √5
⇒ 𝜇𝑥̅ = 𝜇
and
𝜎 √20
= = √5 = 𝜎𝑥̅
√𝑛 √4
Case ii) Sampling without replacement:
Possible samples of size two which can be drawn without replacement is (3, 7),
(3, 11), (3, 15), (7, 11), (7, 15), (11, 15). The mean of these 6 samples are 5, 7, 9,
9, 11, 13 respectively. For this distribution,
(5+7+9+9+11+13)
Mean = 𝜇𝑥̅ = = 9.
6

Variance = σ2x̅
1
= 6 {(5 − 9)2 + (7 − 9)2 + (9 − 9)2 + (9 − 9)2 + (11 − 9)2 +
20
(13 − 9)2 } = .
3

√20
Standard deviation = 𝜎𝑥̅ = .
√3
𝜎 √𝑁−𝑛 √20 √4−2 √20
∗ = ∗ = = 𝜎𝑥̅ . Also, 𝜇𝑥̅ = 𝜇 .
√𝑛 √𝑁−1 √2 √4−1 √3

2. The daily wages of 3000 workers in a factory are normally distributed with mean equal to Rs
68 and standard deviation equal to Rs 3. If 80 samples consisting of 25 workers each are
obtained, what would be the mean and standard deviation of the sampling distribution of means
if sampling were done (a) with replacement (b) without replacement? In how many samples
will the mean is likely to be (i) between Rs 66.8 & Rs 68.3 and (ii) less than Rs 66.4?
Solution: Given 𝑁 = 3000 , 𝜇 = 68 , 𝜎 = 3 , 𝑛 = 25 .
In case of sampling with replacement
𝜎 3
𝜇𝑥̅ = 𝜇 = 68 and 𝜎𝑥̅ = = = 0.6 .
√𝑛 √25

In case of sampling without replacement

𝜎 √𝑁−𝑛 3 √3000−25
𝜇𝑥̅ = 𝜇 = 68 and 𝜎𝑥̅ = ∗ = ∗ = 0.5976 ≈ 0.6 .
√𝑛 √𝑁−1 √25 √3000−1

Fourth Semester 5 21MA41

 Department of Mathematics
Since the population is normally distributed, the sampling distribution of means is
also taken as normally distributed. The standard normal variate associated with the
sample mean 𝑥̅ is
𝑥̅ −𝜇𝑥̅ 𝑥̅ −68
𝑍= =
𝜎𝑥̅ 0.6
66.8−68 68.3−68
𝑃(66.8 < 𝑥̅ < 68.3) = 𝑃 ( <𝑧< ) = 𝑃(−2 < 𝑧 < 0.5)
0.6 0.6

𝐹(0.5) − 𝐹(−2) = 𝐹(0.5) − 1 + 𝐹(2) = 0.6915 − 1 + 0.9773 = 0.6688 .

In a sample of 80; = 0.6687 ∗ 80 ≈ 53 .
66.4−68
𝑃(𝑋̅ < 66.4) = 𝑃 (𝑧 < 0.6 ) = 𝑃(𝑧 < −2.67)

= 1 − 𝐹(2.67) = 1 − 0.9962 = 0.0038 .

Thus, in a sample of 80 = 0.0038 * 80 = 0.3040.

3. Let 𝑥 be the mean of a random sample of size 50 drawn from a population with mean
112 and standard deviation 40. Find (a) the mean and standard deviation of 𝑥 , (b) the
probability that 𝑥 assumes a value between 110 and 114, (c) the probability that 𝑥
assumes a value greater than 113.
Solution:
𝑛 = 50, 𝜇 = 112, 𝜎 = 40
𝜎 40 𝑥̅ −𝜇𝑥̅
𝜇𝑥 = 𝜇 = 112, 𝜎𝑥 = 𝑛 = = 5.6569 and 𝑧 =
√ √50 𝜎𝑥̅

𝑃(110 < 𝑥 < 114) = 𝑃(−0.35 < 𝑧 < 0.35) = 0.6368 − 0.3632 = 0.2736
𝑃(𝑥 > 113) = 𝑃(𝑧 > 0.18) = 1 − 𝑃(𝑧 ≤ 0.18) = 1 − 0.5714 = 0.4286

4. An automobile battery manufacturer claims that its midgrade battery has a mean life
of 50 months with a standard deviation of 6 months. Suppose the distribution of battery
lives of this particular brand is approximately normal. On the assumption that the
manufacturer’s claims are true, find (a) the probability that a randomly selected battery
of this type will last less than 48 months, (b) the probability that the mean of a random
sample of 36 such batteries will be less than 48 months.
Solution:
𝑛 = 36, 𝜇 = 50, 𝜎 = 6
𝜎 6
𝜇𝑥 = 𝜇 = 50, 𝜎𝑥 = 𝑛 = =1
√ √36
𝑥−𝜇
Using 𝑧 = 𝑃(𝑥 < 48) = 𝑃(𝑧 < −0.33) = 0.3707
𝜎

Fourth Semester 6 21MA41

 Department of Mathematics
𝑋−𝜇𝑋
Using 𝑧 = 𝑃(𝑋 < 48) = 𝑃(𝑧 < −2) = 0.0228
𝜎𝑋

Exercise:
1. A population consists of four numbers 1, 5, 6, 8. Consider all possible samples of size
2 which can be drawn from this population with and without replacement. Find the
mean and standard deviation in the population, and in the sampling distribution of
means verify the formulas 𝜇𝑋̅ and 𝜎𝑋̅ .
2. The mean and the standard deviation of a normally distributed population of size 250
are 100 and 16 respectively. What are the mean and the standard deviation of the
sampling distribution of means for random samples of size 4 drawn with replacement
and without replacement?
3. With reference to the above Problem No2, what is the probability that the sample mean
lies between 95 and 105 for a sample of size 4 drawn with and without replacement?
4. The mean of a certain normal infinite population is equal to the standard error of the
distribution of means of samples of size 100 drawn from that population. Find the
probability that the mean of a sample of size 25 drawn from the population will be
negative.
5. If the mean of an infinite population is 575 with standard deviation 8.3 how large a
sample must be used in order that there be one chance in 100 that the mean of the sample
is less than 572?
6. Suppose that the number of customers entering a grocery shop each day over a five-
year period is a random variable with mean 100 and standard deviation of 10. Then
what is the probability that randomly selected 30-day period is between 95 and 105?
√13
Answers: 1) With replacement: 𝜇𝑥̅ = 5 and 𝜎𝑥̅ = , Without replacement: 𝜇𝑥̅ = 5
2

13
and 𝜎𝑥̅ = √ 6 , 2) With replacement: Mean= 100, SD= 8, Without replacement:

mean=100, SD 7.95, 3) With replacement: P=0.46, Without replacement: P=0.47, 4)

0.3085, 5) 𝑛 = 43. 6) 0.9946

Sampling distribution of mean (𝝈 unknown): t – distribution

Fourth Semester 7 21MA41

 Department of Mathematics
Sampling distribution on the assumption that they are normal or approximately normal is valid
when the sample size 𝑛 is large. However, if the sample size 𝑛 is small, the distribution of
various statistics are far from the normality.
For small samples (𝑛 < 30), we consider the t-distribution.
Let 𝑛 be the sample size, 𝑥̅ and 𝜇 be respectively the sample mean and the population mean,
and 𝑠 be the sample standard deviation.
(𝑥̅ −𝜇)
Consider the statistic 𝑡 defined by 𝑡 = √𝑛 .
𝑠

Suppose we obtain a frequency distribution of 𝑡 by computing the value of 𝑡 for each of a set
of samples of size 𝑛 drawn from a normal or a nearly normal population. The sampling
distribution so obtained is called the Student’s t-distribution with probability density function
−(𝛾+1)
𝑡2 2
𝑌(𝑡) = 𝑌0 (1 + 𝛾 ) with 𝛾 = 𝑛 − 1 𝑑. 𝑓. where 𝑌0 is a constant and −∞ < 𝑡 < ∞. The

curve 𝑌(𝑡) is called the t-curve.

The constant 𝑌0 is generally chosen in such a way that the total area under the curve is equal to
unity.
For large values of 𝑛 the function 𝑌 (𝑡) reduces to the standard normal distribution density
−𝑡2
1
function 𝜑(𝑡) = ( )𝑒 2 and the 𝑡-curve becomes the standard normal curve.
√2𝜋

The 𝑡-distribution curve is symmetric about the

mean 0, unimodal and bell shaped and asymptotic
on both sides of 𝑡 −axis. Thus the 𝑡-distribution
curve is similar to normal curve. While the variance
for normal distribution is 1, the variance for the 𝑡-
distribution is more than 1 since it depends on the
parameter 𝛾. So the 𝑡-distribution is more variable. As 𝑛 → ∞, variance of 𝑡-distribution
approaches 1. Thus as 𝛾 = 𝑛 − 1 → ∞, 𝑡-distribution approaches the standard normal
distribution.
Note: The n-1 degrees of freedom in the t-distribution account for the fact that one degree of
freedom is used to estimate the sample mean when calculating the sample variance. This
adjustment allows the t-distribution to provide a better approximation to the actual distribution
of sample means, accounting for the uncertainty introduced by estimating population
parameters from sample data.

Fourth Semester 8 21MA41

 Department of Mathematics
Applications of t-distribution:
• To estimate the population mean from a sample mean when the population standard
deviation is unknown and estimated using the sample standard deviation.
• To test the hypothesis of the population mean.
• To test the hypothesis of the difference of two population means of two normal
population.
Problems:
1. Find the students ‘t’ for the following values in a sample of eight: -4, -2, -2, 0, 2, 2, 3,
3 taking the mean of the population to be zero.
1
Solution: 𝑥̅ = 8 (−4 − 2 − 2 + 0 + 2 + 2 + 3 + 3) = 0.25
1
s2 = {(−4 − 0.25)2 + (−2 − 0.25)2 + (−2 − 0.25)2 + (0 − 0.25)2
7
+ (2 − 0.25)2 + (3 − 0.25)2 + (3 − 0.25)2 }
s 2 = 6.6339. Given 𝜇 = 0.
𝑋̅ −𝜇 0.25−0
Then; 𝑡 = √𝑛 = √6.6339 √8 = 0.2745 .
𝑠

2. The following are the I Qs of a randomly chosen sample of 10 boys;

70, 120, 110, 101, 88, 83, 95, 98, 107, 100. Find the students ‘t’ for the given sample
taking the mean of population to be 100.
Solution: n = 10
1
x̅ = (70 + 120 + 110 + 101 + 88 + 83 + 95 + 98 + 107 + 100)
10
= 97.2
1
s 2 = 9 {(70 − 97.2)2 + (120 − 97.2)2 + (110 − 97.2)2 + (101 − 97.2)2 +

(88 − 97.2) + (83 − 97.2)2 + (95 − 97.2)2 + (98 − 97.2)2 + (107 −

97.2)2 + (100 − 97.2)}
s 2 = 203.73
s = √203.73 = 14.27
μ = 100 given
𝑥̅ −𝜇 97.2−100
𝑡= √𝑛 = √10 = −0.62
𝑠 14.27

Fourth Semester 9 21MA41

 Department of Mathematics
3. A certain stimulus administered to each of 12 patients resulted in the following change
in blood pressure: 5, 2, 8, -1, 3, 0, 6, -2, 1, 5, 0, 4 (in appropriate units). Find the students
‘t’ for the given sample assuming 𝜇 = 0.
Solution: 𝑁 = 12, 𝑛 = 11
1
x̅ = (5 + 2 + 8 − 1 + 3 + 0 + 6 − 2 + 1 + 5 + 0 + 4)
12
= 2.58
1
s2 = {(5 − 2.58)2 + (2 − 2.58)2 + (8 − 2.58)2 + (−1 − 2.58)2 + (3 − 2.58)2
11
+ (0 − 2.58)2 + (6 − 2.58)2 + (−2 − 2.58)2 + (1 − 2.58)2
+ (5 − 2.58)2 + (0 − 2.58)2 + (4 − 2.58)2 }
s 2 = 9.538
s = √9.538 = 3.088
given
𝑥̅ −𝜇 (2.58−0)
μ = 0 then 𝑡 = √𝑛 = √12 = 2.89
𝑠 3.088

4. Eleven school boys were given a test in mathematics carrying a maximum of 25 marks.
They were given a month’s extra coaching and a second test of equal difficulty was
held thereafter. The following table gives the marks in the two tests.
Boy 1 2 3 4 5 6 7 8 9 10 11
I Test 23 20 19 21 18 20 18 17 23 16 19
Marks
II Test 24 19 22 18 20 22 20 20 23 20 17
Marks

Find the 𝑡 for difference in marks assuming the mean 𝜇 = 0.

Solution: The difference in the marks is given by 1, -1, 3, -3, 2, 2, 2, 3, 0, 4, -2.
1
x̅ = 11 (1 − 1 + 3 − 3 + 2 + 2 + 2 + 3 + 0 + 4 − 2) = 1
1
s2 = {(1 − 1)2 + (−1 − 1)2 + (3 − 1)2 + (−3 − 1)2 + (2 − 1)2 + (2 − 1)2
10
+ (2 − 1)2 + (3 − 1)2 + (0 − 1)2 + (4 − 1)2 + (−2 − 1)2 }
s2 = 5
s = √5 = 2.2361
Given

Fourth Semester 10 21MA41

 Department of Mathematics
𝑥̅ −𝜇 (1−0)
μ = 0 then 𝑡 = √𝑛 = 2.2361 √10 = 1.4142
𝑠

Problems:
1. A certain stimulus administered to each of 12 patients resulted in the following change
in blood pressure: 5, 2, 8, -1, 3, 0, 6, -2, 1, 5, 0, 4 (in appropriate units). Find the students
‘t’ for the given sample taking the mean of the population to be 0.
2. Nine items of a sample have the following values: 45, 47, 50, 52, 48, 47, 49, 53, 51.
Find the students ‘t’ for the given sample taking the mean of the population to be 47.5.
Answers: 1) 2.89, 2) 1.83

Sampling distribution of variance:

The sampling distribution of the variance is the distribution of sample variances, with all
samples having same sample size 𝑁 taken from the same population.

Chi – Square Distribution:

Sampling distribution of some important statistics allow us to learn information about the
parameters. Generally, the parameters are the counterpart to the statistics in questions. For
example, if an engineer is interested in the population mean resistance of a certain type of
resistor, the sampling distribution of mean will be exploited once the sample information is
gathered. On the other hand, if the variability in the resistance is to be studied clearly the
sampling distribution of variance will be used to know about the parameter, the population
variance.
If 𝑠 2 is the variance of a random sample of size 𝑛 taken from a normal population having
variance 𝜎 2 , then the statistic
𝑛
2
(𝑛 − 1)𝑠 2 (𝑥𝑖 − 𝑥̅ )2
𝜒 = =∑
𝜎2 𝜎2
𝑖=1

has a chi-squared distribution with 𝜈 = 𝑛 − 1 degrees of freedom.

The probability density function for 𝜒 2 -distribution with 𝜈 d.f. is given by
𝜒2 𝜈
1
𝑓(𝜒 2)
= 𝜈
𝜈
𝑒 −
2 (𝜒 2 )(2)−1 ; 0 < 𝜒2 < ∞
22 Γ( )
2

where, 𝜈 = 𝑛 − 1.
Suppose that, in a random experiment, a set of events 𝐸1 , 𝐸2 , 𝐸3 … … . 𝐸𝑛 are observed to occur
with frequencies 𝑓1 , 𝑓2 , 𝑓3 , … … 𝑓𝑛 . According a theory based on probability rules, suppose the
same event are expected to occur with the frequencies 𝑒1 , 𝑒2 , 𝑒3 … … 𝑒𝑛 . Then

Fourth Semester 11 21MA41

 Department of Mathematics
𝑓1 , 𝑓2 , 𝑓3 , … … 𝑓𝑛 are called the observed frequencies and 𝑒1 , 𝑒2 , 𝑒3 … … 𝑒𝑛 are called the
expected or theoretical frequencies. The statistic 𝜒 2 is defined through the following formula:
𝑛
2
(𝑓1 − 𝑒1 )2 (𝑓2 − 𝑒2 )2 (𝑓𝑛 − 𝑒𝑛 )2 (𝑓𝑖 − 𝑒𝑖 )2
𝜒 = + + ⋯………+ =∑
𝑒1 𝑒2 𝑒𝑛 𝑒𝑖
𝑖=1

If 𝑁 is the total frequency, we should have

𝑛 𝑛

𝑁 = ∑ 𝑓𝑖 = ∑ 𝑒𝑖
𝑖=1 𝑖=1

If the expected frequencies are at least equal to 5, then it can be proved that the sampling
distribution of statistic 𝜒 2 is approximately identical with the probability distribution of the
𝜒2
2
variate 𝜒 whose density function is given by 𝑃(𝜒 2)
= 𝑃0 𝜒 𝜈−2 𝑒 − 2 , where 𝜈 is a positive
constant, called the number of degrees of freedom, 𝑃0 is such that the total area under the
corresponding probability curve is one.

Applications of 𝜒 2 -distribution:
The applications of 𝜒 2 -distribution are very wide in statistics. It is used:
• To test the hypothetical value of population variance.
• To test the goodness of fit, that is, to judge whether there is a discrepancy between
theoretical and experimental observations.
• To test the independence of two attributes, that is, to judge whether the two attributes
are independent.

Problems:
1. A manufacturer of car batteries guarantees that the batteries will last, on average, 3 years
with a standard deviation of 1 year. Assuming the battery lifetime follows a normal
distribution, find 𝜒 2 for the life time 1.9, 2.4, 3.0, 3.5 and 4.2 years of five of these batteries.
Solution:
Given 𝜇 = 3 and 𝜎 = 1.
𝑛
2
(𝑛 − 1)𝑠 2 (𝑥𝑖 − 𝑥̅ )2
𝜒 = = ∑
𝜎2 𝜎2
𝑖=1

1.9 + 2.4 + 3.0 + 3.5 + 4.2

𝑥̅ = =3
5
2
(1.9 − 3)2 + (2.4 − 3)2 + (3.0 − 3)2 + (3.5 − 3)2 + (4.2 − 3)2
𝑠 = = 0.815
5−1

Fourth Semester 12 21MA41

 Department of Mathematics
4 × 0.815
𝜒2 = = 3.26
1
2. Pull-strength tests on 5 soldered leads for a semiconductor device yield the following
results, in pounds of force required to rupture the bond: 19.8, 12.7, 13.2, 16.9 and 10.6. If
the pull-strength of the population follows normal distribution with mean 15 pounds of
force and standard deviation of 0.5.
Solution:
Given 𝜇 = 15 and 𝜎 = 4.
𝑛
2
(𝑛 − 1)𝑠 2 (𝑥𝑖 − 𝑥̅ )2
𝜒 = = ∑
𝜎2 𝜎2
𝑖=1
19.8 + 12.7 + 13.2 + 16.9 + 10.6
𝑥̅ = = 14.64
5
53.892
𝑠2 = = 13.473
5−1
4 × 13.473
𝜒2 = = 3.368
16
3. In 200 tosses of a coin, 118 heads and 82 tails were observed. Find the 𝜒 2 .
Solution:
Here the observed frequencies of heads and tails are 𝑓1 = 118 and 𝑓2 = 82 respectively.
Also 𝑁 = ∑𝑓𝑖 = 𝑓1 + 𝑓2 = 200 =number of trials(tosses).
1
Expected frequencies of heads and tails in 200 trials are 𝑒1 = 200 × 2 = 100 and 𝑒2 =
1
200 × 2 = 100 respectively. Also we note that ∑𝑒𝑖 = 200.
(𝑓1 −𝑒1 )2 (𝑓2 −𝑒2 )2
We have 𝜒 2 = +
𝑒1 𝑒2

(118 − 100)2 (82 − 100)2

= + = 6.48
100 100
4. A die is thrown 60 times and the frequency distribution for number appearing on the face
𝑥 is given by the following table:
𝑥 1 2 3 4 5 6
frequency 15 6 4 7 11 17
Find the statistic 𝜒 2 .
Solution:
The observed frequencies from the table are
𝑓1 = 15, 𝑓2 = 6, 𝑓3 = 4, 𝑓4 = 7, 𝑓5 = 11, 𝑓6 = 17.

Fourth Semester 13 21MA41

 Department of Mathematics
1
The expected frequencies are 𝑒1 = 60 × 6 = 10 = 𝑒2 = 𝑒3 = 𝑒4 = 𝑒5 = 𝑒6 .
(𝑓𝑖 −𝑒𝑖 )2 1
Using the formula 𝜒 2 = ∑𝑛𝑖=1 = 10 {(15 − 10)2 + (6 − 10)2 + (4 − 10)2 + (7 −
𝑒𝑖

10)2 + (11 − 10)2 + (17 − 10)2 }=13.6

5. Fit a binomial distribution to the following data:
𝑥𝑖 0 1 2 3 4 5
𝑓𝑖 2 14 20 34 22 8
Find the corresponding theoretical estimates for 𝑓𝑖 . Also find the statistic 𝜒 2 .
Solution:
Here ∑𝑓𝑖 = 100
Mean of the given distribution is
∑𝑥𝑖 𝑓𝑖 284
Mean = = 100 = 2.84
∑𝑓𝑖

For binomial distribution Mean=𝑛𝑝

𝑀𝑒𝑎𝑛 2.84
⇒𝑝= = = 0.568
𝑛 5
𝑃(𝑥) =nCx 𝑝 𝑥 𝑞1−𝑥 = 5Cx(0.568)𝑥 (0.432)1−𝑥
Substituting 𝑥 = 0,1,2,3,4,5 in the above, we get
𝑃(0) = 1.505, 𝑃(1) = 10.4512, 𝑃(2) = 26.01, 𝑃(3) = 34.199, 𝑃(4) = 22.483,
𝑃(5) = 5.912
Hence the expected frequencies are
𝑒1 = 1.505, 𝑒2 = 10.4512, 𝑒3 = 26.01, 𝑒4 = 34.199, 𝑒5 = 22.483,
𝑒6 = 5.912
Sum of the observed frequencies is ∑𝑓𝑖 = 100 whereas the sum of the expected frequencies
is ∑𝑒𝑖 = 100.5602. In order to have ∑𝑓𝑖 = ∑𝑒𝑖 , we modify the theoretical frequencies by
subtracting 0.5602 from 𝑒3 where the discrepancy between the theoretical and observed
frequencies is largest. Thus we take the modified
𝑒3 = 26.01 − 0.5602 = 25.4498
We further note that the first of the expected frequencies, namely 𝑒1 = 1.505 is less than
5. Therefore we club with 𝑒2 to get 𝑒1 + 𝑒2 = 11.9562. Correspondingly we club the
observed frequencies 𝑓1 and 𝑓2 to get 𝑓1 + 𝑓2 = 16
(𝑓𝑖 −𝑒𝑖 )2 (16−11.9562)2 (20−25.4498)2 (34−34.199)2
Now we find 𝜒 2 = ∑𝑛𝑖=1 = + + +
𝑒𝑖 11.9562 25.4498 34.199

(22−22.483)2 (8−5.912)2
+ = 3.28
22.483 5.912

Fourth Semester 14 21MA41

 Department of Mathematics
Problems:
1. A maker of a certain brand of low-fat cereal bars claims that the average saturated fat content
is 0.5 gram and standard deviation 0.05. Find 𝜒 2 of a random sample of 8 cereal bars of this
brand with the saturated fat content 0.6, 0.7, 0.7, 0.3, 0.4, 0.5 0.4 and 0.2.
2. Fit a Poisson distribution to the following data, obtain the theoretical frequencies and find
the 𝜒 2 .
𝑥𝑖 0 1 2 3 4
𝑓𝑖 419 352 154 56 19
3. The following table gives the number of road accidents that occurred in a large city during
the various days of a week. Calculate 𝜒 2 for the above data.
Day Sun Mon Tue Wed Thu Fri Sat Total
No. of accidents 14 16 8 12 11 9 14 84

4. Fit a normal distribution to the following data of weights of 100 students of a certain college,
obtain the theoretical frequencies and hence find 𝜒 2 for the above data.
Weights (Kgs) 60-62 63-65 66-68 69-71 72-74
Frequency 5 18 42 27 8
5. Fit a binomial distribution to the following data:
𝑥𝑖 0 1 2 3 4 5
𝑓𝑖 38 144 342 287 164 25
Find the corresponding theoretical estimates for 𝑓𝑖 . Also find the statistic 𝜒 2 .

Answers: 1) 2.52, 2)5.8494, 3) 4.17, 4) 0.262

Estimation
Point estimation of a parameter is a statistical estimation where the parameter is estimated by
a single number (or value) from sample data.
Method of Maximum Likelihood
One of the best methods of obtaining a point estimation of a parameter is the method of
maximum likelihood. This technique was developed in the 1920s by famous British statistician
Sir R. A. Fisher. As the name implies, the estimator will be the value of the parameter that
maximizes the likelihood function.
In the case of a discrete random variable, the interpretation of the likelihood function is simple.
The likelihood function of the sample 𝐿(𝜃 ) is just the probability.

Fourth Semester 15 21MA41

 Department of Mathematics
𝑃(𝑋1 = 𝑥1 , 𝑋2 = 𝑥2 , … … … … … . . 𝑋𝑛 = 𝑥𝑛 )
That is, 𝐿(𝜃 ) is just the probability of obtaining the sample values 𝑥1 , 𝑥2 , … … 𝑥𝑛 . Therefore,
in the discrete case, the maximum likelihood estimator is an estimator that maximizes the
probability of occurrence of the sample values.

Example1: Bernoulli Distribution MLE

Let 𝑋 be a Bernoulli random variable. The probability mass function is
𝑝 𝑥 (1 − 𝑝)1−𝑥 , 𝑥 = 0, 1
𝑓(𝑥; 𝑝) = {
0, otherwise
where 𝑝 is the parameter to be estimated. The likelihood function of a random sample of size
𝑛 is
𝐿(𝑝) = 𝑝 𝑥1 (1 − 𝑝)1−𝑥1 𝑝 𝑥2 (1 − 𝑝)1−𝑥2 … … … … … … … 𝑝 𝑥𝑛 (1 − 𝑝)1−𝑥𝑛
𝑛 𝑛
= ∏𝑛𝑖=1 𝑝 𝑥𝑖 (1 − 𝑝)1−𝑥𝑖 = 𝑝∑𝑖=1 𝑥𝑖 (1 − 𝑝)𝑛−∑𝑖=1 𝑥𝑖
We observe that if 𝑝̂ maximizes 𝐿( 𝑝), 𝑝̂ also maximizes ln 𝐿( 𝑝). Therefore.
ln 𝐿(𝑝) = (∑𝑛𝑖=1 𝑥𝑖 ) ln 𝑝 + (𝑛 − ∑𝑛𝑖=1 𝑥𝑖 ) ln(1 − 𝑝)
Now,
𝑑 ln 𝐿(𝑝) ∑𝑛𝑖=1 𝑥𝑖 𝑛 − ∑𝑛𝑖=1 𝑥𝑖
= −
𝑑𝑝 𝑝 1−𝑝
1
Equating this to zero and solving for 𝑝 yields 𝑝̂ = (𝑛) ∑𝑛𝑖=1 𝑥𝑖 . Therefore, the maximum

likelihood estimator of 𝑝 is
1
𝑝̂ = 𝑛 ∑𝑛𝑖=1 𝑋𝑖 .

Example 2: Poisson Distribution MLE

Let 𝑋 be a Poisson variate with probability mass function
𝑒 −𝜆 𝜆𝑥
𝑓(𝑥; 𝜆) = , 𝑥 = 0,1,2 …
𝑥!

distributed with unknown 𝜇 and known variance 𝜎 2 . The likelihood function of a random
sample of size 𝑛, say 𝑋1 , 𝑋2 , … … 𝑋𝑛 , 𝑖𝑠
𝑛 𝑛
𝑒 −𝑛𝜆 𝜆∑𝑖=1 𝑥𝑖
𝐿(𝑥1 , 𝑥2 , 𝑥3 … 𝑥𝑛 ; 𝜆) = ∏ 𝑓(𝑥: 𝜆) =
∏𝑛𝑖=1 𝑥!
𝑖=1

Now

Fourth Semester 16 21MA41

 Department of Mathematics
𝑛 𝑛

ln 𝐿(𝑥1 , 𝑥2 , 𝑥3 … 𝑥𝑛 ; 𝜆) = −𝑛𝜆 + 𝑙𝑛𝜆 ∑ 𝑥𝑖 − 𝑙𝑛 ∏ 𝑥!

𝑖=1 𝑖=1

𝜕 ln 𝐿(𝑥1 ,𝑥2 ,𝑥3 …𝑥𝑛 ;𝜆) 𝑥

And = −𝑛 + ∑𝑛𝑖=1( 𝜆𝑖 )
𝜕𝜆

Equating this last result to zero and solving for 𝜆 yields

∑𝑛𝑖=1 𝑥𝑖
𝜆̂ = = 𝑥̅
𝑛

Conclusion: The sample mean is the maximum likelihood estimator of 𝜆.

Example 3: Exponential Distribution MLE

Let 𝑋 be exponentially distributed with parameter 𝜆. The likelihood function of a random

sample of size 𝑛, say, 𝑋1 , 𝑋2 , … … 𝑋𝑛 , 𝑖𝑠

𝑛
𝐿(𝜆) = ∏𝑛𝑖=1 𝜆𝑒 −𝜆𝑥𝑖 = 𝜆𝑛 𝑒 −𝜆 ∑𝑖=1 𝑥𝑖

The log likelihood is.

ln 𝐿(𝜆) = 𝑛 ln 𝜆 − 𝜆 ∑𝑛𝑖=1 𝑥𝑖

Now

𝑑 ln 𝐿(𝜆) 𝑛
= 𝜆 − ∑𝑛𝑖=1 𝑥𝑖
𝑑𝜆

and upon equating this last result to zero we obtain

𝑛 1
𝜆̂ = ∑𝑛 𝑥 = 𝑥̅
𝑖=1 𝑖

Conclusion: Thus, the maximum likelihood estimator of 𝜆 is the reciprocal of the sample mean.

Example 4: Normal Distribution MLEs for 𝝁 and 𝝈𝟐

Let 𝑋 be normally distributed with mean 𝜇 and variance 𝜎 2 , where both 𝜇 and 𝜎 2 are
unknown. The likelihood function for a random sample of size 𝑛 is.

Fourth Semester 17 21MA41

 Department of Mathematics
2
(𝑥𝑖 −𝜇) 1
1 − 1 − ∑𝑛 (𝑥 −𝜇)2
𝐿(𝜇, 𝜎 2)
= ∏𝑛𝑖=1 𝑒 2𝜎2 = 𝑛 𝑒 2𝜎2 𝑖=1 𝑖
𝜎√2𝜋 (2𝜋𝜎2 ) 2

And

𝑛 1
ln 𝐿(𝜇, 𝜎 2 ) = − ( 2) ln(2𝜋𝜎 2 ) − (2𝜎2 ) ∑𝑛𝑖=1(𝑥𝑖 − 𝜇)2

Now

𝜕 ln 𝐿(𝜇,𝜎2 ) 1
= 𝜎2 ∑𝑛𝑖=1(𝑥𝑖 − 𝜇) = 0
𝜕𝜇

𝜕 ln 𝐿(𝜇,𝜎2 ) 𝑛 1
= − 2𝜎2 + 2𝜎4 ∑𝑛𝑖=1(𝑥𝑖 − 𝜇)2 = 0.
𝜕𝜎2

The solutions to the above equations yield the maximum likelihood estimators.

𝜇̂ = 𝑥̅ ̂2 = 1 ∑(𝑥𝑖 − 𝑥̅ )2
𝜎
𝑛
𝑖=1

Video Links:

https://www.youtube.com/watch?v=zK70Fc_HHmg
https://www.youtube.com/watch?v=mlQRIoBErso

Disclaimer: The content provided is prepared by department of Mathematics for the specified
syllabus by using reference books mentioned in the syllabus. This material is specifically for the
use of RVCE students and for education purpose only.

Fourth Semester 18 21MA41