Interval Estimation

Definition 4.1
Given a sample X1, . . . , Xn from a distribution with pdf f (x | θ),
an interval estimator of the parameter θ is the random interval
I(X ) = [L(X ), U (X )]. The end-points of the interval are two
statistics such that L(x) ⩽ U (x) for all possible samples x in the
sample space.

4-1
Coverage Probability
Definition 4.2
The coverage probability of an interval estimator I(X ) for a
parameter θ is the probability that the random interval contains
the true parameter value Pθ (θ ∈ I(X )).

Definition 4.3
The confidence coefficient of an interval estimator I(X ) for a
parameter θ ∈ Θ is the infimum of the coverage probabilities

inf Pθ (θ ∈ I(X )).

θ∈Θ
An interval estimator I(X ) with confidence coefficient 1 − α for
some fixed α ∈ (0, 1) is called a 100(1−α)% confidence interval
for θ.
4-2
Inversion of Tests

∗ An interval estimator may be thought of as a set of “plausi-

ble” values for the parameter θ.

∗ One definition of a plausible value of the parameter is a value

θ0 for which we would not reject H0 : θ = θ0.

∗ This suggests that we could construct an interval estimator

as a set of all those values of θ0 such that we do not reject

H0 : θ = θ0 V H1 : θ ̸= θ0

∗ This process is known as inversion of a test.

4-3
Inversion of Tests
Theorem 4.1
For each θ0 ∈ Θ consider testing H0 : θ = θ0 with a level α test.
Let A(θ0) be the set of sample values for which we do not reject
the null hypothesis. That is we have the rule

reject H0 : θ = θ0 ⇐⇒ x∈
/ A(θ0).
Now, for a given sample point x, define the set

C(x) = {θ0 : x ∈ A(θ0)} .

Then the random set C(X ) is a 100(1 − α)% confidence set for
θ.

4-4
Pivotal Quantities

∗ A general way of finding confidence intervals is based on the

concept of a pivotal quantity.

Definition 4.4
Suppose X = (X1, . . . , Xn) is a sample from a population with cdf
depending on some parameters θ . A quantity R(X , θ ) which is a
function of the data and the parameters is called a pivot if the
sampling distribution of R does not depend on the parameters
θ.

∗ Suppose θ = (ω, ψ ) and interest is in estimation of ω. Then

we need a pivot R(X , ω) whose distribution is free of (ω, ψ ).

4-5
Pivots in Location-Scale Families
Theorem 4.2
Suppose that X1, . . . , Xn is a random sample from one of the
following distributions. Then pivotal quantities for the parameter
of interest ω exist and are given in the table.

Family pdf ω pivot

Location f (x − µ) µ X −µ

1 x X
 
Scale f σ
σ σ σ
1 x−µ X −µ
 
Location-scale f µ
σ σ S
1 x−µ S
 
Location-scale f σ
σ σ σ

where X and S are the sample mean and standard deviation.

4-6
General Pivots
Theorem 4.3
Suppose that T = T (X ) is a statistic whose sampling distribution
can be written as
∂
fT (t; θ) = g(R(t, θ)) R(t, θ)
∂t
for some function g and for R(t, θ) monotone in t for each fixed
θ. Then R(T (X ), θ) is a pivot.

4-7
Pivotal Interval Estimation

∗ Since the distribution of a pivot does not depend on any un-

known parameters, we can use the quantiles of its distribution
to determine an interval estimate.

∗ Given a pivot R(X , θ) a confidence region for θ is of the form

C(X ) = {θ ∈ Θ : a < R(X , θ) < b}
where a and b are such that
 
P a < R(X , θ) < b = 1 − α
for fixed confidence level 1 − α.

∗ If R(X , θ) is a monotone function of θ then C(X ) will be an

interval, otherwise we will get a confidence set.

4-8
Pivoting The CDF

∗ Recall the Probability Integral Transform:

If T is a random variable with continuous cdf FT (t | θ) then
the random variable

U = FT (T | θ) ∼ Uniform(0, 1)

∗ Suppose that we have a sample X1, . . . , Xn and T (X1, . . . , Xn)

is a statistic with continuous cdf FT (t | θ) then
   
R T (X ), θ = FT T (X ) | θ ∼ Uniform(0, 1).
 
∗ Thus FT T (X ) | θ is a pivot and so we can use it to find
confidence regions.

4-9
Pivoting a Continuous CDF
Theorem 4.4
Suppose that T is a statistic with continuous cdf FT (t | θ) which
is monotone in θ. Let α ∈ (0, 1), be fixed and choose α1 ⩾ 0 and
α2 ⩾ 0 such that α1 + α2 = α. Then we can define the statistics
θL(t) and θU (t) as the solution to the following equations

If FT (t | θ) is an increasing function of θ

FT (t | θL(t)) = α1 FT (t | θU (t)) = 1 − α2
If FT (t | θ) is a decreasing function of θ

FT (t | θL(t)) = 1 − α2 FT (t | θU (t)) = α1

The random interval I(X ) = (θL(T ), θU (T )) is a 100(1 − α)%

confidence interval for θ.
4-10
Pivoting a Discrete CDF
Theorem 4.5
Let T be a statistic with discrete cdf FT (t | θ) which is monotone
in θ. For a fixed α ∈ (0, 1), choose α1 ⩾ 0 and α2 ⩾ 0 such that
α1 + α2 = α. Then we can define the statistics θL(t) and θU (t)
as the solution to the following equations

If FT (t | θ) is an increasing function of θ

P(T ⩽ t | θL(t)) = α1 P(T ⩾ t | θU (t)) = α2

If FT (t | θ) is a decreasing function of θ

P(T ⩾ t | θL(t)) = α2 P(T ⩽ t | θU (t)) = α1

The random interval I(X ) = (θL(T ), θU (T )) is a 100(1 − α)%

confidence interval for θ.
4-11
Evaluation of Intervals

∗ For a fixed confidence coefficient 1 − α a better interval usu-

ally has smaller expected length.

Eθ [U (X ) − L(X )]

∗ In general, this is a function of θ.

∗ It is standard to compare interval lengths in terms of their

ratios.

4-12
Minimum Length Intervals
Theorem 4.6
Suppose that f (x) is a unimodal pdf. If the interval [a, b] satisfies

Z b
1. f (x) dx = 1 − α.
a

2. f (a) = f (b) > 0.

3. a < x∗ < b where x∗ is a mode of f (x), that is f (x∗) = max f (x).

then [a, b] is the shortest of all intervals having probability 1 − α.

4-13
Uniformly Most Accurate Intervals

∗ The Coverage Probability of an interval estimator I(X ) is

the function of θ given by Pθ (θ ∈ I(X )). This gives the
probability that the true value of θ is in the interval.

∗ Another measure of accuracy concerns the probability that a

false value of θ is in the interval.

∗ This is known as the False Coverage Probability.

4-14
False Coverage Probability
Definition 4.5
Suppose that I(X ) is an interval estimator of a parameter θ.
Then the False Coverage Probability of I(X ) is the function of
θ and θ′ give by
  h i
Pθ θ ∈ I(X ) , θ′ ̸= θ
′ if I(X ) = L(X ), U (X )
  h 
Pθ θ ∈ I(X ) , θ′ < θ
′ if I(X ) = L(X ), ∞
   i
Pθ θ′ ∈ I(X ) , θ′ > θ if I(X ) = − ∞, U (X )

4-15
False Coverage Probability
∗ Note that this is a function of both θ (the true parameter
value) and θ′ (the false parameter value).

∗ The set of possible false values depends on whether the in-

terval is one-sided or two-sided and the direction if it is one-
sided.

∗ An 1 − α interval which minimizes the probability of false

coverage over all possible θ and θ′ is called a Uniformly Most
Accurate interval.

∗ Uniformly most accurate intervals are generally found by in-

verting uniformly most powerful tests.

∗ Since UMP tests are generally one-sided tests, UMA intervals

are generally one-sided also.

4-16
UMP Tests and UMA Intervals
Theorem 4.7
Suppose that X ∼ f (x | θ) where θ is a real-valued parameter.
For every θ0 ∈ Θ, let A(θ0) be the UMP level α acceptance region
for testing
H0 : θ = θ0 V H1 : θ > θ0.
Let I ∗(X ) be the 1 − α confidence set formed by inverting the
UMP acceptance regions. Then for any other 1 − α confidence
set I(X ),

Pθ θ ′ ∈ I ∗ (X ) ⩽ Pθ θ′ ∈ I(X ) for all θ′ < θ.

   

4-17
False Coverage and Expected Length
Theorem 4.8
Let X ∼ f (x | θ) be a real-valued random variable where θ is a
real-valued parameter. Let I(x) = [L(x), U (x)] be a confidence
interval for θ. If L(x) and U (x) are both increasing functions of
x then, for any value θ∗,
  Z  
Eθ∗ U (X) − L(X) = Pθ∗ θ ∈ I(X) dθ.
θ̸=θ∗

4-18