Scribd
Upload
46 views

Contol of Quairyststae

This document provides solutions to exercises from a classical and Bayesian inference course. It includes solutions involving maximum likelihood estimation for negative binomial, binomial, gamma, Poisson, normal and exponential distributions. Confidence intervals are derived for the normal mean and methods like Newton-Raphson are described for estimating parameters of Cauchy distributions.

Uploaded by

askazy007
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
46 views

Contol of Quairyststae

This document provides solutions to exercises from a classical and Bayesian inference course. It includes solutions involving maximum likelihood estimation for negative binomial, binomial, gamma, Poisson, normal and exponential distributions. Confidence intervals are derived for the normal mean and methods like Newton-Raphson are described for estimating parameters of Cauchy distributions.

Uploaded by

askazy007
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

AMS 206: Classical and Bayesian Inference (Winter 2015)

Homework 2 solutions

1. Exercise 7.6.14
Solution: The first sampling scenario specifies a negative binomial distribution, with parameters r and p. That is, X N B(r, p), with X representing the number of failures before the
rth success. Therefore, r = 5, and we observe x = 38. Here, the MLE of p is pb = 5/43. In the
second case, we use the Binomial distribution and find pb = 3/58. Note that, in each case, the
sampling plan does not affect the calculation of pb, which is just the proportion of butterflies in
the sample that have the special type of marking.
2. Exercise 7.6.16
Solution: The likelihood observed by statistician A is proportional to a gamma distribution
with = 3 and = , that is, proportional to 3 exp(x). Statistician B observes a Poisson
random variable with parameter 2, which has likelihood proportional to y exp(2). When
x = 2 and y = 3, these are each proportional to 3 exp(2). The common MLE for can
b = y for the mean of the
be shown to be b = 3/2, either by differentiation, or recalling that
Poisson with 1 observation y, and therefore, 2b = 3.
3. Exercise 7.10.10
Solution: By the invariance property of MLEs, we know that b = 3b 1. For 0 1, we
require 1/3 2/3. Therefore, provided 1/3 x
n 2/3, the MLE for can be derived as
n , using the approach to finding the MLE for the probability of success based on i.i.d.
b = X
Bernoulli random variables. However, if x
n < 1/3, then b = 1/3, and if x
n > 2/3, then b = 2/3.
b
b
b
Therefore, = 3
xn 1 if 1/3 x
n 2/3; = 0 if x
n < 1/3; and = 1 if x
n > 2/3.
4. Exercise 8.8.3
Solution: For the Poisson distribution with mean , we have log f (x | ) = x log() log(x!).
The second derivative of log f (x | ) with respect to is given by x/2 , and since E(X) = ,
we obtain I() = 1/ for the (expected) Fisher information. Note that is the variance of X,
and thus the larger the variance, the less information X contains about .
5. Exercises 8.8.4 and 8.8.5
Solution:
For the normal distribution with mean 0 and standard deviation ,
we have log f (x | ) log() x2 /(2 2 ). The second derivative of log f (x | ) with respect to is given by 2 3x2 4 . Therefore, I() = 2/ 2 , using the fact that E(X 2 ) =
Var(X) + (E(X))2 = 2 .
Following the same approach, but now treating 2 as the parameter of the mean-zero normal
distribution, we obtain I( 2 ) = 1/{2( 2 )2 }.
6. Exercise 8.5.4
Solution: As derived in class, the confidence interval for the mean, , of a normal distribution

with known variance 2 is given by (


xn 1 ((1 + )/2)/ n, x
n + 1 ((1 + )/2)/ n).
For = 0.95, we have 1 (0.975) = 1.96, and therefore the length of the interval becomes

3.92/ n. For this to be less than 0.01, we must have n > 3922 , that is, n = 153665 or larger.

7. Exercise 8.5.6
n , and therefore X
n or Pn Xi are natural statistics to conSolution: The MLE for is X
i=1
sider for building a pivotal to derive a confidence interval for . The exponential distribution
with mean is a special case of the gamma distribution with parameters = 1 and = 1/.
Therefore, using the approach based on moment generating functions (refer to the corresponding
class
Pn notes), we obtain
Pna gamma distribution with parameters n and 1/ as the distribution for
1
i=1 Xi . Thus,
i=1 Xi follows a gamma distribution with parameters n and 1, providing
the pivotal quantity included in the hint for the exercise. Now, the gamma(n, 1) distribution is
not symmetric as the normal and t distributions (although the amount of skewness decreases
with increasing n), and therefore, in principle, there is an infinite number of c1 and c2 values
that could be used. In practice (especially for moderate to large n), c1 and c2 can be taken as
the respective (1 )/2 and (1 + )/2 percentiles of the gamma(n, 1) distribution.
8. Exercise 8.5.7

1
Solution: The confidence interval for the mean has the form (
xn 0 Tn1
(0.95)/ n, x
n +

1
0 Tn1
(0.95)/ n). Here, n = 20, and the 0.95 quantile of the t19 distribution is 1.729. The obP
n )2 /(n 1))1/2 = 22.64,
served average number of calories is x
n = 156.85, and 0 = ( ni=1 (xi x
yielding (148.1, 165.6) as the 90% confidence interval for the mean number of calories.
9. Consider the Cauchy distribution with p.d.f. given by
f (x | ) =

1
[1 + (x )2 ]

for < x <

where is the location parameter of the distribution ( < < ).


(a) Suppose that X = (X1 , ..., Xn ) is a random sample from the Cauchy distribution above
with unknown parameter . Develop the Newton-Raphson method to compute the maximum
likelihood estimate of based on the observed sample x.
(b) Consider a data set, x = (0.774, 0.597, 7.575, 0.397, 0.865, 0.318, 0.125, 0.961, 1.039),
assumed to arise from the Cauchy distribution above. Apply the Newton-Raphson method from
part (a) to estimate . To check your results, try different starting values for the NewtonRaphson algorithm and also plot the likelihood function for .
(c) Consider another data set, x = (0, 5, 9), again, assumed to arise from the same Cauchy
distribution. Apply again the Newton-Raphson method from part (a) to estimate , using
different starting values, including for instance, 0 = 1, 0 = 4.67, 0 = 10. As in part (b),
plot the likelihood function, and comment on the results.
Solution: The iteration for the Newton-Raphson method is given by
Pn
(xi k )/{1 + (xi k )2 }
k+1
k

= + Pn i=1
k 2
k 2 2
i=1 {1 (xi ) }/{1 + (xi ) }
where k denotes the current iterate of the algorithm, and k+1 the value at the next iteration.
For the data set with n = 9 observations, the likelihood function is unimodal, and the N-R
algorithm converges to b = 0.17919 within 3-4 iterations, provided the starting value is in the
interval (1, 1.5). For the data set with n = 3 observations, the likelihood function is trimodal.
The N-R method converges to either the global maximum or one of the local maxima for starting
values close enough to the corresponding maximum point. The method is sensitive to the starting
value, and, in particular, it diverges for 0 = 1 and 0 = 10.

You might also like