One-Way ANOVA vs. Two-Way ANOVA

In statistics, the Analysis of Variance (ANOVA) is a powerful tool used to analyze differences among group means and their associated procedures. ANOVA is essential for students and professionals in fields such as psychology, biology, education, and business, as it helps in understanding how different factors influence a particular outcome. This article aims to provide a comprehensive overview of two common types of ANOVA: one-way and two-way ANOVA.

One-Way ANOVA

One-way ANOVA is a statistical test used to determine whether there are any statistically significant differences between the means of three or more independent (unrelated) groups. It compares the means of the groups to see if at least one of them is significantly different from the others.

When to Use One-Way ANOVA?

One-way ANOVA is used when you have:

• One independent variable (factor) with three or more levels (groups).
• A continuous dependent variable.

For example, suppose a researcher wants to test the effect of three different diets on weight loss. The diets are labeled as Diet A, Diet B, and Diet C. The weight loss (in pounds) of participants on each diet is recorded, and one-way ANOVA is used to determine if there is a significant difference in weight loss among the diets.

Assumptions of One-Way ANOVA

1. Independence of Observations: The data collected from the groups should be independent of each other.
2. Normality: The data in each group should be approximately normally distributed.
3. Homogeneity: The variances among the groups should be approximately equal.

How to Perform One-Way ANOVA

Step 1: State the Hypothesis

Null Hypothesis(\Eta₀) : All group means are equal.

Alternative Hypothesis (\Eta₁) : At least one group mean is different.

Step 2: Calculate the ANOVA table

Assume, there are k classes and each class k_i contains n_i number of elements, mean \mu_i and n = n₁ + n₂ + ... is the total number of elements.

The mean of all elements ie.,Global mean is given by

\mu = \frac{\sum_{i=1}^{k}\mu_{i}}{k}

Now, Calculate the following,

Sum of squares between the Groups (SSB) = \sum_{i=1}^{k}n_{i}(\mu_{i}-\mu)^2

SSB measures the variation among the means of the different groups (or levels) of the independent variable. It calculates how much each group mean differs from the overall mean of all observations combined.

Sum of squares within the Groups (SSW) = \sum_{i=1}^{k} \sum_{j=1}^{n_i} (x_{ij} - \mu_{i})^2

where x_{ij} is the jth element in ith group/class.

SSW measures the variation within each group (or level) of the independent variable. It calculates how much each individual observation deviates from its group mean.

Total sum of squares (SST) = SSB +SSW

Mean sum of squares between the Groups (MSSB) = \frac{\text{SSB}}{\text{k-1}}

Mean sum of squares within the Groups (MSSW) = \frac{\text{SSW}}{\text{n-k}}

Now, Find the F-ratio which is given by,

\text{f-ratio} = \frac{\text{Varianve Between (MSSB)}}{\text{Varianve Within (MSSW)}}

Step 3: Find Critical f-value

Use the F-table to find the critical value for f_{(k-1,n-k)}. Refer this article to know more about f-test.

Step 4: Make the Decision

Compare the calculated and critical value of f-ratio and make the decision as,

• If critical f-value > calculated f-value, then Accept the null hypothesis which means all group means are equal.
• If critical f-value < calculated f-value, then Reject the null hypothesis which means at least one group mean is different.

One way ANOVA Example

Let's take the example of effect of three different diets on weight loss. The diets are labeled as Diet A, Diet B, and Diet C. The weight loss (in pounds) of participants on each diet is recorded.

Step 1: State the Hypothesis

Null Hypothesis(\Eta_0) : \mu_1 = \mu_2 = \mu_3

Alternate Hypothesis(\Eta_1) : \mu_1 \neq \mu_2 \neq \mu_3

Step 2: Calculate the ANOVA table

For Diet A, mean(\mu_1) = \frac{25+30+36+38+31}{5} = \frac{160}{5} = 32

For Diet B, mean(\mu_2) = \frac{31+39+38+42+35}{5} = \frac{185}{5} = 37

For Diet C, mean(\mu_3) = \frac{24+30+28+25+28}{5} = \frac{135}{5} = 27

Therefore the Global mean(\mu) = \frac{32+37+27}{3} = \frac{96}{3} = 32

Now, Calculate:

SSB = \sum_{i=1}^{k}n_{i}(\mu_{i}-\mu)^2 = 5(32-32)^2 + 5(37-32)^2 + 5(27-32)^2 = 250

SSW = \sum_{i=1}^{k} \sum_{j=1}^{n_i} (x_{ij} - \mu_{i})^2

= (25-32)^2 + (30-32)^2+(36-32)^2+(38-32)^2+(31-32)^2+(31-37)^2+(39-37)^2+(38-37)^2+(42-37)^2+(35-37)^2+(24-27)^2+(30-27)^2+(28-27)^2+(25-27)^2+(28-27)^2

= 200

Create the ANOVA table.

The value of f-ratio = \frac{MSSB}{MSSW} = \frac{125}{16.667}=7.4998

Step 3: Find Critical f-value

Find the critical f-ratio from the f-table where df1=2, df2=12 for \alpha = 0.05 (If not given consider 5%).

Therefore Critical f-ratio = 3.89

Step 4: Make the Decision

Compare the calculated and critical value of f-ratio.

Calculated f-ratio > Critical f-ratio (7.4998 > 3.89)

Hence, we reject the Null Hypothesis stating that the group means are not equal and there is a significant difference in weight loss among the diets.

Two-way ANOVA

Two-way ANOVA is used to examine the influence of two different categorical independent variables on one continuous dependent variable. It also helps in understanding if there is an interaction between the two independent variables on the dependent variable.

When to Use Two-Way ANOVA

One-way ANOVA is used when you have:

• Two independent variables (factors), each with two or more levels (groups).
• A continuous dependent variable.

For example, Consider a study evaluating the effects of two different fertilizers and two different watering frequencies on plant growth. Here, the two fertilizers and the watering frequencies are the independent variables, and the plant growth is the dependent variable. Two-way ANOVA can determine if there are significant effects of fertilizers, watering frequencies, and their interaction on plant growth.

Assumptions of Two-Way ANOVA

1. Independence of Observations: The data collected from the groups should be independent of each other.
2. Normality: The data in each group should be approximately normally distributed.
3. Homogeneity: The variances among the groups should be approximately equal.
4. Additivity and Linearity: The combined effect of the two independent variables should be equal to the sum of their individual effects plus any interaction effect.
5. Interaction Effect: The effect of one independent variable on the dependent variable might depend on the level of the other independent variable.

How to Perform Two-Way ANOVA

Step 1: State the Hypothesis

Null Hypothesis (\Eta_0) :

1. There is no significant difference between the groups of the first variable.
2. There is no significant difference between the groups of the second variable.
3. There is no interaction between both variables.

Alternate Hypothesis (\Eta_1) :

1. There is a significant difference between the groups of the first variable.
2. There is a significant difference between the groups of the second variable.
3. There is an interaction between both variables.

Step 2: Calculate the ANOVA table

Calculate Correction Factor (CF), which is given by,

\text{CF} = \frac{(\Sigma{x})^2}{n}

Where n is the total number of observations in the given data.

Now, Calculate the following,

Total Sum of Squares (SST) = \sum_{i}^{} \sum_{j}^{} x_{ij}^2 - CF

For variation between groups of first variable,

Sum of Squares of Column (SSB_c) = \frac{\sum_{i=1}^{p}n_i^2}{a}-CF

• p is the number of groups in first independent variable.
• n_i is the sum of observation in p_i group of first independent variable.
• a is the number of observation in p_i group of first independent variable.

For variation between groups of second variable,

Sum of Squares of Row (SSB_r) = \frac{\sum_{j=1}^{q}m_j^2}{b}-CF

• q is the number of groups in first independent variable.
• m_i is the sum of observation in q_i group of first independent variable.
• b is the number of observation in q_i group of first independent variable.

For the error between the Groups,

Sum of Squares within the Groups (SSE) = \sum_{i=1}^{p} \sum_{j=1}^{q} \sum_{k=1}^{n_{ij}} (X_{ijk} - \bar{X}_{ij})^2

• p is the number of levels (groups) in the first independent variable.
• q is the number of levels (groups) in the second independent variable.
• n_{ij} is the number of observations in the cell corresponding to the i-th group of the first variable and the j-th group of the second variable.
• X_{ijk} is the k-th observation in i,j-th cell.
• \bar{X}_{ij} is the mean of observations in i,j-th cell.

For Interaction between Both variables
Sum of Squares Interaction (SSI) = SST - SSB_c - SSB_r - SSE

The table will be formed as follows:

Here,

• p is the number of groups in first independent variable.
• q is the number of groups in second independent variable.
• n is the number of observations in one group.
• N is the total number of observations.

Step 3: Find Critical f-value

Now find Critical f-value for Column, Row and Interaction from the F table at \alpha = 0.05.

• f_{c(p-1,vdf)}
• f_{r(q-1,vdf)}
• f_{i((p-1)(q-1),vdf)}

Step 4: Make the Decision

Compare the calculated and critical value of f-ratio for all sources.

• If f_{c(p-1,vdf)} > f_{c \ cal.}, then accept the Null Hypothesis, indicating no significant difference between the groups of the first variable, or vice versa.
• If f_{r((q-1),vdf)} > f_{r \ cal.}, then accept the Null Hypothesis, indicating no significant difference between the groups of the second variable, or vice versa.
• If f_{i((p-1)(q-1),vdf)} > f_{i \ cal.}, then accept the Null Hypothesis, indicating no significant interaction between the variables, or vice versa.

Two-way ANOVA Example

Let's take the example of effect of three different drugs on weight loss of different group of people.

Step 1: State the Hypothesis

Null Hypothesis (\Eta_0) :

• There is no significant difference between the effect of drugs.
• There is no significant difference of weight loss between the group of people.
• There is no interaction between both variables.

Alternate Hypothesis (\Eta_1) :

• There is a significant difference between the effect of drugs.
• There is a significant difference of weight loss between the group of people.
• There is an interaction between both variables.

Step 2: Calculate the ANOVA table

Calculate Correction Factor (CF) as,

\text{CF} = \frac{(\Sigma{x})^2}{n} \newline = \frac{(25+27+7+8+13+18+21+24+16+11+19+14+29+31+19+21+30+27)^2}{18}\newline =\frac{(360)^2}{18}\newline =7200

Now calculate the below values:

\text{SST}=\sum_{i}^{} \sum_{j}^{} x_{ij}^2 - CF\newline = (25^2+27^2+21^2+24^2+...)-7200\newline = 8144-7200\newline =944

\text{SSB}_c=\frac{\sum_{i=1}^{p}n_i^2}{a}-CF\newline = (\frac{(25+27+21+24+29+31)^2}{6}+\frac{(7+8+11+16+19+21)^2}{6}+\frac{(13+18+19+14+30+27)^2}{6})-7200\newline =469

\text{SSB}_r=\frac{\sum_{j=1}^{q}m_j^2}{a}-CF\newline = (\frac{(25+27+7+8+13+18)^2}{6}+\frac{(21+24+11+16+19+14)^2}{6}+\frac{(29+31+19+21+30+27)^2}{6})-7200\newline =346.333

For Residual, find mean of each cell. For example, \frac{25+27}{2}=26

\text{SSE}=\sum_{i=1}^{p} \sum_{j=1}^{q} \sum_{k=1}^{n_{ij}} (X_{ijk} - \bar{X}_{ij})^2\newline = (25-26)^2+(27-26)^2+(7-7.5)^2+(8-7.5)^2+(13-15.5)^2+(18-15.5)^2+...+(30-28.5)^2+(27-28.5)^2\newline =53

The Interaction will be,

\text{SSI}=SST - SSB_c - SSB_r - SSE\newline =944-469-346.333-53\newline =75.667

The table will be formed as follows:

Step 3: Find Critical f-values

Find Critical values of f-ratio from F-table for,

• f_{c(p-1,vdf)} = f_{c(2,9)}=4.26
• f_{r(q-1,vdf)}=f_{r(2,9)}=4.26
• f_{i((p-1)(q-1),vdf)}=f_{i(4,9)}=3.63

Step 4: Make the Decision

Compare all the calculated and critical f-values.

• f_{c(p-1,vdf)} < f_{c \ cal.}(4.26<39.82), therefore the Null Hypothesis is rejected and there is a significant difference between the effect of drugs.
• f_{r((q-1),vdf)} < f_{r \ cal.}(4.26<29.405), therefore the Null Hypothesis is rejected and there is a significant difference of weight loss between the group of people.
• f_{i((p-1)(q-1),vdf)} > f_{i \ cal.}(3.63>3.2122), there the Null Hypothesis is accepted and there is no interaction between both variables.

Difference between One-Way ANOVA and Two-Way ANOVA

Conclusion - One-Way ANOVA vs. Two-Way ANOVA

Understanding the differences between one-way and two-way ANOVA is crucial for conducting effective statistical analyses in research and data interpretation. One-way ANOVA is suitable for comparing the means of multiple groups defined by a single factor, while two-way ANOVA expands this capability by assessing the effects of two independent variables and their interaction. Each method has specific applications and considerations, such as interpreting interaction effects in two-way ANOVA and addressing assumptions like normality and homogeneity of variances. By choosing the appropriate ANOVA method based on your research design and hypotheses, you can effectively analyze and draw meaningful conclusions from your data.