Anova Test

ANOVA (Analysis of Variance) is a statistical method used to determine whether there are significant differences between the means of three or more independent groups by analyzing the variability within each group and between the groups. It helps in testing the null hypothesis that all group means are equal.

It does this by comparing two types of variation: (F-statistics)

1. Differences BETWEEN groups (how much group averages differ from each other)
2. Differences WITHIN groups (how much individuals in the same group vary naturally).

If the between-group differences are significantly larger than within-group variation, ANOVA tells us: At least one group is truly different. Otherwise, it concludes: The differences are likely due to random chance.

For example:

Compare test scores of students taught with 3 methods (Traditional, Online, Hybrid). ANOVA is used to determine if at least one teaching method yields significantly different average scores.

ANOVA Formula

The ANOVA formula is made up of numerous parts. The best way to tackle an ANOVA test problem is to organize the formulae inside an ANOVA table.  

Here's a general structure of an ANOVA table:

where,

• F = ANOVA Coefficient
• MSB = Mean of the total of squares between groupings
• MSW = Mean total of squares within groupings
• MSE = Mean sum of squares due to error
• SST = total Sum of squares
• p = Total number of populations
• n = The total number of samples in a population
• SSW = Sum of squares within the groups
• SSB = Sum of squares between the groups
• SSE = Sum of squares due to error
• s = Standard deviation of the samples
• N = Total number of observations

Assumptions of ANOVA

These must be validated before analysis:

1. Independence: Observations are randomly sampled, and groups are independent.
2. Normality: Residuals (errors) are approximately normally distributed (checked via Q-Q plots or Shapiro-Wilk test).
3. Homoscedasticity: Equal variances across groups (verified using Levene’s or Bartlett’s test).

ANOVA is robust to minor violations of normality and homoscedasticity with balanced sample sizes.

Calculating ANOVA

Let's explore calculating ANOVA for the scenario:

Compare plant growth under 3 fertilizers (A, B, C):

• Fertilizer A: [10, 11, 12]
• Fertilizer B: [7, 8, 9]
• Fertilizer C: [4, 5, 6]

1. State Hypothesis

• Null Hypothesis (H₀): μ_A = μ_B = μ_C
• Alternative Hypothesis (H_a​): At least one μ differs.

2. Calculate Group means and Grand mean.

• Group Means: \bar X_A, \bar X_B, and \bar X_C
• Grand Mean: \overline{X}_{\text{grand}}

\overline{X}_{A}=\frac{10+11+12}{3}=11 \\\overline{X}_{B}=\frac{7+8+9}{3}=8 \\\overline{X}_{C}=\frac{4+5+6}{3}=5

\overline{X}_{\text{grand}}=\frac{10+11+12+7+8+9+4+5+6}{9}=\frac{72}{9}=8

3. Compute Sum of Squares (SS):

SSB (Sum of Squares Between Groups): Accounts for variation due to the treatment or independent variable.
SSB = \sum n_i(\bar{X}_i - \bar{X}_{\text{grand}})^2

SSE (Sum of Squares Error or Within Groups): Accounts for variation within groups (random error or residuals).
SSE = \sum ({x}_i - \bar{X})^2

SST (Total Sum of Squares): Accounts for total variation from overall mean.
SST = SSB + SSW

SSB = 3(11 − 8)² + 3(8 − 8)² + 3(5 − 8)² = 3(9) + 3(0) + 3(9) = 54

SSE:

• Fertilizer A: (10 − 11)² + (11−11)² + (12−11)² = 1 + 0 + 1 = 2
• Fertilizer B: (7 − 8)² + (8 − 8)² + (9 − 8)² = 1 + 0 + 1 = 2
• Fertilizer C: (4 − 5)² + (5 − 5)² + (6 − 5)² = 1 + 0 + 1 = 2

SSW = 2 + 2 + 2 = 6
SST = 54 + 6 = 60

4. Calculate Degrees of Freedom (df):

df1 (Between Groups) = k - 1, where k is number of groups.
df2 (Within Groups) = N - k, where N is the total observations.
df3 (Total) = N - 1.

• df1 = 3 - 1 = 2
• df2 = 9 - 3 = 6
• df3 = 9 - 1 = 8

5. Calculate Mean Squares (MS):

MSB (Mean Square Between Groups) = SSB / df1.

MSE (Mean Square Error) = SSE / df2.

• MSB = \frac{SSB}{df1} = \frac{54}{2} = 27
• MSW = \frac{SSW}{df2} = \frac{6}{6} = 1

6. F-statistic:

The F-statistic is calculated as the ratio of MSB to MSE:

F = \frac{MSB}{MSE}

• F = \frac{27}{1} = 27

7. P-value:

The p-value is used to decide whether differences among groups are statistically significant. When the p-value is smaller than the significance level (α), the null hypothesis is rejected.
If F > F_critical → p < 0.05 : Null Hypothesis Rejected

Use the F-distribution table or software with: Numerator df1 = 2 , Denominator df2 = 6, α=0.05
Critical F-value, F_critical: 5.14 (From F-distribution table)
F > F_critical : 27 > 5.14 → p < 0.05; Reject null hypothesis

Types of ANOVA

An ANOVA test can be classified as either one-way or two-way based on the number of independent variables involved.

One-Way ANOVA

This test is used to see if there is a variation in the mean values of three or more groups. Such a test is used where the data set has only one independent variable. If the test statistic exceeds the critical value, the null hypothesis is rejected, and the averages of at least two different groups are statistically significant.

Two-Way ANOVA

Two independent variables are used in the two-way ANOVA. A two-way ANOVA test is used to determine the main effect of each independent variable and whether there is an interaction effect. Each factor is examined independently to determine the main effect, as in a one-way ANOVA. Furthermore, all components are analyzed at the same time to test the interaction impact.

Solved Examples on ANOVA

Example 1: Three different kinds of food are tested on three groups of rats for 5 weeks. The objective is to check the difference in mean weight(in grams) of the rats per week. Apply one-way ANOVA using a 0.05 significance level to the following data:

Solution:

H₀: μ₁= μ₂=μ₃
H₁: The means are not equal

Since, X̄₁ = 5, X̄₂ = 9, X̄₃ = 10

Total mean = X̄ = 8

• SSB = 6(5 - 8)² + 6(9 - 8)² + 6(10 - 8)² = 84
• SSE = 68
• MSB = SSB/df₁ = 42
• MSE = SSE/df₂ = 4.53
• f = MSB/MSE = 42/4.53 = 9.33

Since f > F, the null hypothesis stands rejected.

Example 2: Calculate the ANOVA coefficient for the following data:

Solution:

Plant	n	x	s	s2
Hibiscus	5	12	2	4
Marigold	5	16	1	1
Rose	5	20	4	16

p = 3
n = 5
N = 15
x̄ = 16

SST = Σn(x−x̄)²

• SST= 5(12 − 16)² + 5(16 − 16)² + 11(20 − 16)² = 160
• MST = SST/p-1 = 160/3-1 = 80
• SSE = ∑ (n−1) = 4 (4 + 1) + 4(16) = 84
• MSE = 7
• F = MST/MSE = 80/7
• F = 11.429

Example 3: The following data show the number of worms quarantined from the GI areas of four groups of muskrats in a carbon tetrachloride anthelmintic study. Conduct a two-way ANOVA test.

Solution:

Source of Variation	Sum of Squares	Degrees of Freedom	Mean Square
Between the groups	62111.6	8	9078.067
Within the groups	98787.8	16	4567.89
Total	167771.4	24

Since F = MST / MSE
= 9.4062 / 3.66
F = 2.57

Example 4: Enlist the results in APA format after performing ANOVA on the following data set:

\begin{bmatrix}  \textbf{n} & \textbf{mean} & \textbf{sd} \\  30 & 50.26 & 10.45 \\  30 & 45.32 & 12.76 \\  30 & 53.67 & 11.47 \\ \end{bmatrix}

Solution:

• Variance of first set = (10.45)² = 109.2
• Variance of second set = (12.76)² = 162.82
• Variance of third set = (11.47)² = 131.56

MS_error = {109.2 + 162.82 + 131.56} / {3}
= 134.53

MS_between = (17.62)(30) = 528.75
F = MS_between  /  MS_error 
  = 528.75 / 134.53
F = 4.86

APA writeup: F(2, 87)=3.93, p >=0.01, η²=0.08.

Practice Problem Based On ANOVA

Question 1. Method A = {80, 85, 90, 87}, Method B = {75, 78, 72, 74}, and Method C = {88, 85, 90, 92} are given. State the null and alternative hypotheses for performing a One-Way ANOVA test.
Question 2. Calculate the F-statistic for the given data using One-Way ANOVA. Group 1 = {5, 6, 7, 8}, Group 2 = {4, 5, 6, 5}, and Group 3 = {7, 7, 6, 8}.
Question 3. Interpret the significance of the p-value for the interaction effect in a Two-Way ANOVA, where the p-value for the interaction effect is 0.02 and the significance level (α\alphaα) is 0.05.
Question 4. Group A = {10, 12, 14, 13}, Group B = {15, 17, 16, 18}, and Group C = {20, 22, 21, 23} are given. Interpret the p-value of the ANOVA test and explain whether the null hypothesis is rejected, where F-statistic = 4.86 and p-value = 0.01.

Answer:-

1. Null Hypothesis (H₀): μ₁ = μ₂ = μ₃ (The means of all groups are equal). Alternative Hypothesis (H₁): At least one mean is different.
2. F-statistic = 4.58.
3. If the p-value (0.02) is less than the significance level (0.05), reject the null hypothesis and conclude that there is a significant interaction effect.
4. Since the p-value (0.01) is less than 0.05 and the F-statistic is significant, we reject the null hypothesis, indicating a significant difference between the group means.