Statistical inferance

Anova, Monova, Moncova

Submitted By :
Ans Muhammad
Submitted To :
Sir Adnan Ali CH.

Anova Test :
Analysis of variance, or ANOVA, is strong statistical technique
that is used to show difference between two or more means or
components through significance tests. It also shows us a way
to make multiple comparisons of several population means.
The Anova test is performed by comparing two types of
variation, the variation between the sample means, as well as
the variation within each of the samples. Below mentioned
formula represents one way Anova test statistics:
Where,
F = Anova Coefficient
MST = Mean sum of squares due to treatment
MSE = Mean sum of squares due to error.

for MST is given below:

Where,
SST = Sum of squares due to treatment
p = Total number of populations
n = The total number of samples in a population.
Formula for MSE is given below:
Where,
SSE = Sum of squares due to error
S = Standard deviation of the samples
N = Total number of observations.
An ANOVA test is a way to find out if survey or experiment
results are significant. In other words, they help you to figure
out if you need to reject the null hypothesis or accept
the alternate hypothesis. Basically, you’re testing groups to see
if there’s a difference between them. Examples of when you
might want to test different groups:
 A group of psychiatric patients are trying three different
therapies: counseling, medication and biofeedback. You want
to see if one therapy is better than the others.
 A manufacturer has two different processes to make light
bulbs. They want to know if one process is better than the
other.
 Students from different colleges take the same exam. You
want to see if one college outperforms the other

ANOVA is used in the analysis of comparative experiments,

those in which only the difference in outcomes is of interest.
The statistical significance of the experiment is determined by
a ratio of two variances. This ratio is independent of several
possible alterations to the experimental observations: Adding
a constant to all observations does not alter significance.
Multiplying all observations by a constant does not alter
 significance. So ANOVA statistical significance result is
independent of constant bias and scaling errors as well as
the units used in expressing observations. In the era of
mechanical calculation it was common to subtract a constant
from all observations (when equivalent to dropping leading
digits) to simplify data entry
Types of Tests :
There are two main types: one-way and two-
way. Two-way tests can be with or without replication.
 One-way ANOVA between groups: used when you want
to test two groups to see if there’s a difference between
them.
 Two way ANOVA without replication: used when you have
one group and you’re double-testing that same group. For
example, you’re testing one set of individuals before and after
they take a medication to see if it works or not.
 Two way ANOVA with replication: Two groups, and the
members of those groups are doing more than one thing. For
example, two groups of patients from different hospitals trying
two different therapies.

One-way or two-way refers to the number of independent

variables (IVs) in your Analysis of Variance test. One-way has
one independent variable (with 2 levels) and two-way has two
independent variables (can have multiple levels). For example,
a one-way Analysis of Variance could have one IV (brand of
cereal) and a two-way Analysis of Variance has two IVs (brand
of cereal, calories).

MANOVA :
Multivariate analysis of variance (MANOVA) is
simply an ANOVA with several dependent variables. That is to
say, ANOVA tests for the difference in means between two or
more groups, while MANOVA tests for the difference in two or
more vectors of means.
Assumptions
Normal Distribution: -
The dependent variable should be normally
distributed within groups. Overall, the F test is robust to non-
normality, if the non-normality is caused by skewness rather
than by outliers. Tests for outliers should be run before
performing a MANOVA, and outliers should be transformed or
removed.
Linearity -
MANOVA assumes that there are linear relationships
among all pairs of dependent variables, all pairs of covariates,
and all dependent variable-covariate pairs in each cell.
Therefore, when the relationship deviates from linearity, the
power of the analysis will be compromised.
Homogeneity of Variances: -
Homogeneity of variances assumes
that the dependent variables exhibit equal levels of variance
across the range of predictor variables. Remember that the
error variance is computed (SS error) by adding up the sums of
squares within each group. If the variances in the two groups
are different from each other, then adding the two together is
not appropriate, and will not yield an estimate of the common
within-group variance. Homoscedasticity can be examined
graphically or by means of a number of statistical tests.
Homogeneity of Variances and Covariances: -
In multivariate
designs, with multiple dependent measures, the homogeneity
of variances assumption described earlier also applies.
However, since there are multiple dependent variables, it is
also required that their intercorrelations (covariances) are
homogeneous across the cells of the design. There are various
specific tests of this assumption.
Special Cases
Two special cases arise in MANOVA, the
inclusion of within-subjects independent variables and unequal
sample sizes in cells.
Unequal sample sizes -
As in ANOVA, when cells in a
factorial MANOVA have different sample sizes, the sum of
squares for effect plus error does not equal the total sum of
squares. This causes tests of main effects and interactions to
be correlated. SPSS offers and adjustment for unequal sample
sizes in MANOVA.
Within-subjects design -
Problems arise if the researcher
measures several different dependent variables on different
occasions. This situation can be viewed as a within-subject
independent variable with as many levels as occasions, or it
can be viewed as separate dependent variables for each
occasion. Tabachnick and Fidell (1996) provide examples and
solutions for each situation. This situation often lends itself to
the use of profile analysis, which is explained below.
Additional Limitations
Outliers -
Like ANOVA, MANOVA is extremely sensitive to
outliers. Outliers may produce either a Type I or Type II error
and give no indication as to which type of error is occurring in
the analysis. There are several programs available to test for
univariate and multivariate outliers.
Multicollinearity and Singularity -
When there is high correlation
between dependent variables, one dependent variable
becomes a near-linear combination of the other dependent
variables. Under such circumstances, it would become
statistically redundant and suspect to include both
combinations

The obvious difference between ANOVA and a "Multivariate

Analysis of Variance" (MANOVA) is the “M”, which stands for
multivariate. In basic terms, A MANOVA is an ANOVA with two
or more continuous response variables. Like ANOVA,
MANOVA has both a one-way flavor and a two-way flavor. The
number of factor variables involved distinguish a one-way
MANOVA from a two-way MANOVA.

One-way MANOVA compares two or more continuous

response variables (e.g. Test Score and Annual Income) by a
single factor variable (e.g. Level of Education).

Two-way MANOVA compares two or more continuous

response variables (e.g. Test Score and Annual Income) by two
or more factor variables (e.g. Level of Education and Zodiac
Sign).
When comparing two or more continuous response variables
by a single factor, a one-way MANOVA is appropriate (e.g.
comparing ‘test score’ and ‘annual income’ together by ‘level of
education’). A two-way MANOVA also entails two or more
continuous response variables, but compares them by at least
two factors (e.g. comparing ‘test score’ and ‘annual income’
together by both ‘level of education’ and ‘zodiac sign’).

One-way ANOVA investigates the effects of a categorical

variable (the groups, i.e. independent variables) on a
continuous outcome (the dependent variable). In one-way
ANOVA, we have m random variables x1, …, xm (also called
groups or treatments). For each group we have a sample,
where we denote the jth group sample as {, …, }. Group j is
said to have nj subjects in its sample. We also define .
Our objective is to test the null hypothesis H0: μ1 = μ2 = ⋯
= μm.
We use the following definitions for the total (T), between
groups (B) and within groups (W) sum of squares (SS),
degrees of freedom (df) and mean square (MS):

The test statistic F is defined as follows and has an F

distribution with dfB, dfW degrees of freedom:

We reject the null hypothesis if F > Fcrit.

MANCOVA
(Multivariate Analysis of Covariance) is the
multivariate counterpart of ANCOVA. MANCOVA tells you if
there are statistically significant mean differences among
groups. In other words, it tells you if group differences probably
happened by random chance, or if there is a repeatable trend.

Covariates are controled for in MANCOVA.

Image: Makingstats|Wikimedia Commons
MANCOVA removes the effects of one or more covariates
from your model; This allows you to see the true effect of
your independent variables on your dependent
variables without unwanted interference. This supercharged
test comes with a cost: a much larger sample size is needed.
This may not be worth the extra time and expense; more often
than not, a MANOVA (i.e. the same test without looking at
covariates) can be more powerful.
Types
 A one way MANCOVA needs at least four variables. One
independent variable plus one or more dependent variables
and one or more covariates.
 A two way MANCOVA includes two independent
variables.
Covariates / Covariance Defined
Covariance is a measure of how much two random
variables vary together. It’s similar to variance, but
where variance tells you how a single variable varies, co-
variance tells you how two variables vary together.
A covariate can be one of these two variables. It is any
variable that effects how your independent variables act
upon your dependent variables. For example, confounding
variables are covariates.
Assumptions
The assumptions for this test are the same as the assumptions
for MANOVA, with the addition of a couple more for covariance.
As you would expect with a complex test (compared to a much
simpler test like a z-test), these assumptions are lengthy and
somewhat complex. This is one reason why MANCOVAs are
nearly always performed using software, as most statistical
software will test for these assumptions before running the
test.
 Covariates and dependent variables
are continuous and ratio/ ordinal.
 Covariance matrices should be equal (reduces Type I
error).
 Independent variables are categorical.
 Independence of variables: the variables do not influence
each other.
 Random sampling: the data was collected using a random
selection method.
 Normality: the dependent variables follow a (multinomial)
normal distribution for each group.
 Absence of multicollinearity — the dependent variables
shouldn’t be significantly correlated.
 Homogeneity of variance between groups.
The covariates you choose should be correlated with the
dependent variables. This can be tested prior to their inclusion
in MANCOVA with correlation analysis. The best dependent
variables on the other hand are not correlated with each other.
Differences Between MANCOVA and Other Similar Tests
MANCOVA, MANOVA, ANOVA, ANCOVA: it can all get a little
confusing to remember which is which. However, all of the tests
can be thought of as variants of the MANCOVA, if you
remember that the “M” in MANCOVA stands for Multiple and
the “C” stands for Covariates.
 ANOVA: a MANCOVA without multiple dependent
variables and covariates (hence the missing M and C).
 ANCOVA: a MANCOVA without multiple dependent
variables (hence the missing M).
 MANOVA: A MANCOVA without covariates (hence the
missing C).