Statistical Tests

of
the Estimated Variance Factor
in

Different Least-Squares Survey Adjustment Packages

by

Henri B. Ayers, C.L.S., Q.L.S

February 2015

ABSTRACT

Similarities and differences between the implementation of testing the estimated variance
factor in different Least-Squares adjustment packages are presented. A review of the
characteristics of the Chi-Square (χ2) and Fisher (F) distributions indicate that both
distributions have similar shapes for low degrees of freedom and their shape approaches a
Normal distribution as the degrees of freedom increase. The F distribution becomes a χ2
distribution when the second variance is associated with the reference variance of the
whole population. Different formulations of testing the estimated variance factor from a
2-side and a 1-side test interval are derived and illustrated to provide background for their
implementation in the different software packages.

Results obtained from the different packages indicate the same outcome and equivalent
statistics especially for the large degree of freedom data set example despite the fact that
the χ2 and F distributions are mathematically different and that StarNet and GeoLab
packages use a 2-side χ2 test formulation compared with a 1-side F test formulation using
the B-method for the MOVE3 package.

Statistical tests of the estimated variance factor should always be examined in

conjunction with the degree of freedom, standardized residuals, measurement noise after
having removed significant systematic errors affecting the observations, mathematical
and stochastic models used in the adjustment.
1.0 Purpose of the Statistical Test on the Estimated Variance Factor:

The main purpose of statistical testing in Least-Squares adjustment is to verify if the

mathematical and stochastic models provide proper values for the determination of a
unique set of unknown parameters with their associated error estimates from a redundant
(extra) set of observations assumed to have random errors.

One of the quantities being tested in a Least-Squares adjustment of survey data is the
estimated variance factor ( ). The estimated variance factor is a unitless quantity
corresponding to the weighted sum of the square of residuals divided by the degrees of
freedom which is given by the following expression:

or Σ (wi .ri 2) / (n – u) for i = 1 to n

Where: is the estimated variance factor obtained after adjustment
is the residual vector obtained after adjustment
W is the weight matrix before adjustment containing (σ02/σ2i,i) in the diagonal
(n – u) is the degree of freedom (number of extra measurements)
The weight matrix is usually given by the inverse of the square of the initial measurement
errors assumed to be uncorrelated between each other from a unit (σ02 = 1) population
variance.

The statistical test of the estimated variance factor consists of verifying if its value ( ) is
within an interval computed from a χ2 or Fisher F distribution based on the degree of
freedom at a given error probability level.

The statistical testing of the estimated variance factor can be seen as way to confirm the
initial assumptions made about the measurements errors to be mostly random,
uncorrelated and coming from a unit variance population and by how much it deviates
from these premises.

Before examining the statistical testing of the estimated variance factor, let’s review the
characteristics of the χ2and F distributions.

2.0 Characteristics of the χ2 and F Distributions:

2.1 Characteristics of the χ2 Distribution:

The χ2 distribution is the mathematical representation of a random variable χ2ν defined by

the sum of the square values (Σỹi2) of normally distributed N(0,1) random variables (ỹ)
for a given number of samples called degree of freedom (ν). It is used to describe the
variance of random variables.

2
It has the following probability density function (pdf):

Φ(x) = x(ν/2 –1) . ℮ (-x/2) for x > 0 and ν = degree of freedom

χ2 ν 2(ν/2) . Γ(ν/2) where ℮ = 2.7182818…(natural exponent)
Γ(ν/2) = 0 ∫∞ ℮-t . t (ν/2 –1) dt (Gamma Function for ν/2)

The Chi-Square distribution is a special case of the Gamma distribution. It can be used to
make statistical inference about the data.

Its shape has the following characteristics for different degrees of freedom:

- Skew (non-symmetrical) shape for small degrees of freedom (ν <= 10)

- A value of 0 at x = 0
- A maximum value between 0 and +∞
- 2 points of inflection each side of a maximum value
0- Positive x-axis as asymptote

As the degree of freedom increases (ν > 10), its shape approaches a Normal Distribution
with mean and variance of ν and 2ν respectively such as χ2 (ν = large) ---> N(ν, 2ν) based on
the Central Limit theorem.

For large degrees of freedom, the χ2 value can be approximated by:

χ2ν, P ≈ ½ (zP + (2ν - 1)1/2 )2

where zP is the percentile (abscissa value) of a Cumulative (accumulated from -∞)

Normal Distribution for a given level of Probability P.

2.2 Characteristics of the F Distribution:

The F distribution is the mathematical representation of a random variable Fν1, ν2 defined

by the ratio between 2 Chi-Square variables χ2ν1 and χ2ν2 divided by their degree of
freedom ν1 and ν2 obtained from 2 sets of normally distributed N(0,1) random variables.
It used to compare the variance of 2 sets of random variables given by the following ratio:

Fν1, ν2 = (χ2ν1 / ν1)/( χ2ν2 / ν2)

3
It has the following probability density function (pdf) :

Φ(x) = Γ((ν1+ν2)/2) . (ν1/ν2) (ν1/2) . x(ν1/2 - 1) for x > 0, ν1 > 0 and ν2 > 0
Fν1, ν2 Γ(ν1/2) Γ(ν2/2) . (1 + x ν1/ ν2)(ν1+ν2)/2
where Γ(ν1/2) = 0 ∫∞ ℮-t . t (ν1/2 – 1) dt
Γ(ν2/2) = 0 ∫∞ ℮-t . t (ν2/2 – 1) dt
Γ((ν1+ν2)/2) = 0 ∫∞ ℮-t. t ((ν1+ν2)/2–1) dt

Its shape has the following characteristics for different degrees of freedom:

- Skew (non-symmetrical) shape for small degrees of freedom (ν1 and ν2 <= 10)
- A value of 0 at x = 0
- A maximum value between 0 and +∞
- 2 points of inflection each side of a maximum value
- Positive x-axis as asymptote

As both degrees of freedom increase (ν1 and ν2 > 10), its shape approaches a Normal
Distribution such as F (ν1, ν2 = large) ---> N(ν2/(ν2 - 2), 2ν22(ν1 + ν2 -2)/(ν1(ν2 - 2)2(ν2 - 4)))
for ν1 > 2 and ν2 > 4 .

Moreover, the F distribution becomes a χ2 distribution when the degree of freedom of the
second variance approaches infinity ν2 ---> ∞ such that χ2ν2 /ν2 = 1, thus F ν, ∞ = χ2 ν /ν .

3.0 Implementation of Statistical Test on the Estimated Variance Factor:

The statistical test of the Estimated Variance Factor consists of verifying if its value ( )
is within a confidence interval computed from χ2 or F distribution percentile values based
on a given degree of freedom at a given error probability level.

3.1 Chi-Square Test of the Estimated Variance Factor:

The Chi-Square test of the estimated variance factor is usually conducted by verifying if
its value is within an interval defined by the upper and lower limits obtained from 2
percentile values of a Chi-Square distribution for a given degree of freedom (n-u) at a
given error probability (α in %) divided by the adjustment degree of freedom.

4
The 2-side Chi-Square statistical test interval can be expressed as:

Probability that [ χ2(n-u), (1 - α/2) /(n-u) < < χ2(n-u), α/2 /(n-u) ] = (1 – α) % for σ02 = 1

or in terms of the initial observation unit variance (σ02 = 1)

Probability that [(n-u) . / χ2(n-u), α/2 < (σ02 = 1) < (n-u) . / χ2(n-u), (1 - α/2)] = (1 – α) %

where is the estimated variance factor and σ02 is the reference observation variance
χ (n-u), (1 - α/2) and χ2(n-u), (α/2) are the percentiles obtained from a χ2 distribution for
2

a degree of freedom (n-u) at (1 – α/2) % and α/2 % probability levels

α is the error probability (significance) level of the statistical test
(1 – α) is the confidence probability of the statistical test

The 2-Side Chi-Square test of the estimated variance factor can be illustrated as follow:

3.2 F Test of the Estimated Variance Factor:

The F test of the Estimated variance factor is usually conducted by verifying if its value
is within an interval defined by the upper limit obtained from the percentile value of
an F distribution for a given degree of freedom (n-u) for the first variance and +∞ for the
second variance assumed to be the reference variance of the whole population at a given
error probability (α in %).

5
Since the estimated variance factor is expected to be greater than the reference variance of
the whole population > 1.0, it is not necessary to set a lower limit to the F test in these
conditions.

The 1-side F statistical test interval can be expressed as:

Probability that [ < (F(n-u), ∞, α or χ2(n-u), α /(n-u) ) ] = (1 – α) %

where is the estimated variance factor

F (n-u), ∞, α is the percentile obtained from a F distribution for a
degree of freedom (n-u) at α % probability level.
α is the error probability (significance) level of the statistical test
(1 – α) is the confidence probability level of the statistical test

The 1-Side F test of the estimated variance factor can be illustrated as follow:

4.0 Excerpts from different Least-Squares Adjustment Packages:

Following are some numerical examples showing the implementations of the statistical
tests conducted on the estimated variance factor using the χ2 and the F distributions from
different Least-Squares adjustment packages used in Surveying and Geodesy.

Minimally constrained adjustments using 1 known control station from GNSS baseline
vectors are presented for small and large degree of freedom data sets processed under the
same conditions in three adjustment packages namely: StarNet, MOVE3 and GeoLab.

6
4.1 Small Degree Of Freedom Data Set:

The example for the small degree of freedom data set consists of adjusting 3 control
points from 5 single occupied GPS baseline vectors based on 1 known control station as
shown in Figure 1.

Figure 1: Small Degree Of Freedom Data Set Adjustment Layout

Excerpts from StarNet adjustment report: Excerpts from MOVE3 adjustment report:

Adjustment Statistical Summary OBSERVATIONS:

============================== GPS coordinate differences:
15 (5 baselines)
Number of Stations = 4 Number of Stations: 4
Number of Observations = 15
Number of Unknowns = 9 UNKNOWNS:
Number of Redundant Obs = 6
Number of Unknown Coordinates:
9 (3 stations)
Observation Count Sum Squares Error
of StdRes Factor
GPS Deltas 15 47.639 2.818 Degrees of freedom: 6
Sqrt( =7.94)----^ STATISTICAL TESTING:
Warning:The Chi-Square Test at 5.00%Level
Alfa error probability: 0.0500
Exceeded Upper Bound Limit of
Alfa multi-dimension: 0.2088 ------ |
Lower/Upper Bounds (0.454/1.552)
Beta (Power of Test) : 0.70 |
Sqrt(χ26,2.5%/6)--------^ ^
Critical value F-test: 1.40 <-- F6, ∞, 21%
Sqrt(χ26,97.5%/6)-------------|
F-test: =7.846 (rejected)
For χ26,2.5% = 1.24 and χ26,97.5% = 14.45

7
Excerpts from GeoLab adjustment report:
-----------------------------------------------------------------
| S T A T I S T I C S S U M M A R Y
-----------------------------------------------------------------
| Residual Critical Value Type | Tau Max
| Residual Critical Value | 2.3015
| Number of Flagged Residuals | 0
| Convergence Criterion | 0.0010
| Final Iteration Counter Value | 2
| Confidence Level Used | 95.0000
| Estimated Variance Factor | 7.9398
| Number of Degrees of Freedom | 6
|
| Chi-Square Test on the Variance Factor:
| 3.2968e+00 < 1.0000 < 3.8418e+01 ?
| (7.9398x6/χ26,97.5%)--^ ^--(7.9398x6/χ26,2.5%)
2
| for χ 6,97.5% = 14.45 THE TEST FAILS for χ26,2.5% = 1.24
-----------------------------------------------------------------

4.2 Large Degree Of Freedom Data Set:

The example for the large degree of freedom data set consists of adjusting 51 stations
from 4896 GPS baseline vectors occupied several times based on 1 known control station
as shown in Figure 2.

Figure 2: Large Degree Of Freedom Data Set Adjustment Layout

8
Excerpts from StarNet adjustment report: Excerpts from MOVE3 adjustment report:

Adjustment Statistical Summary OBSERVATIONS:

============================== GPS coordinate differences:
4896 (1632 baselines)
Number of Stations = 52 Number of Stations: 52
Number of Observations = 4896
Number of Unknowns = 153 UNKNOWNS:
Number of Redundant Obs = 4743
Number of Unknown Coordinates:
153 (51 stations)
Observation Count Sum Squares Error
of StdRes Factor
GPS Deltas 15 4615.337 0.986 Degrees of freedom: 4743
Sqrt( =0.973)---^ STATISTICAL TESTING:
The Chi-Square Test at 5.00% Level passed
Alfa error probability: 0.0500
Lower/Upper Bounds (0.980/1.020)
Alfa multi-dimension: 0.6777 ------- |
Sqrt(χ24743,2.5%/4743)-----^ ^ Beta (Power of Test) : 0.70 |
Sqrt(χ24743,97.5%/4743)-----------| Critical value F-test: 0.99 <-- F4743, ∞, 68%
For: F-test: =0.97 (accepted)
χ24743,2.5% = 4553.53 and χ24743,97.5% = 4935.31

Excerpts from GeoLab adjustment report:

-----------------------------------------------------------------
| S T A T I S T I C S S U M M A R Y
-----------------------------------------------------------------
| Residual Critical Value Type | Tau Max
| Residual Critical Value | 4.5561
| Number of Flagged Residuals | 0
| Convergence Criterion | 0.0010
| Final Iteration Counter Value | 2
| Confidence Level Used | 95.0000
| Estimated Variance Factor | 0.9731
| Number of Degrees of Freedom | 4743
|
| Chi-Square Test on the Variance Factor:
| 9.3518e-01 < 1.0000 < 1.0136e+00 ?
| (0.9731x4743/χ24743,97.5%)--^ ^--(0.9731x4743/χ24743,2.5%)
2
|for χ 4743,97.5% = 4935.31 THE TEST PASSES for χ24743,2.5% = 4553.53
-----------------------------------------------------------------

5.0 Summary and Conclusion:

Statistical tests of the estimated variance factor have been described in terms of the χ2 and
F distributions used in the calculation of the test interval limits for implementations used
in three different Least-Squares adjustment packages.

Review of the characteristics of the χ2 and F distributions indicates that as the degrees of
freedom increase, both distributions approach a Normal distribution as per the Central
Limit theorem of statistics.

9
Moreover, the F distribution converts to a χ2 distribution when the second degree of
freedom becomes very large which is used to test the estimated variance factor with
respect to the whole population variance.

Different formulations of testing the estimated variance factor from a 2-side and a 1-side
test interval are illustrated to provide background for their implementation in the different
software packages.

Excerpts from StarNet adjustment reports are presented in terms of the square-root
(positive) values of the estimated variance factor and the associated 2-side χ2 distribution
statistics over the degree of freedom whereas excerpts from GeoLab χ2 tests are presented
as 2-side χ2 tests in terms of the initial observation unit variance of 1.0 with the weighted
sum of the residuals over the χ2 statistics.

The MOVE3 adjustment package reports the statistical tests of the estimated variance
factor from a 1-side test interval (upper limit) obtained from an F distribution having its
second degree of freedom set to ∞ for the second variance describing the reference unit
variance of the whole population with additional statistics based on the power β (Beta) of
the test set at 70% and α multi-dimensional values computed by the B-method (Baarda)
of testing which slightly push the upper limits inside the interval.

Results obtained from the different adjustment packages indicate the same outcome and
equivalent statistics especially for the large degree of freedom example despite the fact
that the χ2and F distributions are mathematically different and that StarNet and GeoLab
use a 2-side χ2 test formulation compared with a 1-side F test formulation for the MOVE3
package.

Statistical test of the estimated variance factor should always be examined in conjunction
with the degree of freedom, standardized residuals, measurement noise, mathematical and
stochastic models used in the adjustment.

6.0 References:

Gagnon, P. (1979): Compensations Géodésiques. TPG-14278, Notes de Cours,

Département de Géodésie et de Cartographie, Université Laval.

Mikhail, E. (1976): Observations and Least-Squares. IEP – Dun-Donnelley, Harper &

Row Publishers, New York

R. Steeves and C. Fraser (1983): Statistical Post Analysis of Least-Squares Adjustment

Results. Papers from The CIS Adjustment and Analysis Seminars, ISBN 0-919088-15-5

D.Wells and E. Krakiwsky (1971): The Method of Least-Squares. Lectures Notes 18,
Department of Survey Engineering, University of New Brunswick

GeoLab V6, MOVE3 V4 and StarNet V8 User’s Manuals/On-Line Help Files

10