Parametric Test: ANOVA

1
 ANOVA
Purpose: Analysis of Variance (ANOVA) is a statistical method used to
compare means between three or more groups to determine if there are
significant differences.

Key Characteristics:
1. Compares variability between groups to variability within groups.
2. Useful when comparing means across multiple groups simultaneously.

Types:
3. One-Way ANOVA: Compares means across one categorical independent
variable with three or more levels/groups.
4. Two-Way ANOVA: Examines the interaction between two independent
variables on one dependent variable.
Types of ANOVA
1. One-Way ANOVA:
- Compares means across three or more independent groups on one factor.
- Example: Comparing teaching methods on student performance.
2. Two-Way ANOVA:
- Compares means across groups defined by two factors simultaneously.
- Examines main effects and interaction effects.
- Example: Studying diet and exercise on weight loss.
3. Factorial ANOVA:
- Handles multiple independent variables with multiple levels.
- Analyzes main effects and interactions.
- Example: Investigating temperature, humidity, and soil type on plant growth.
4. Repeated Measures ANOVA:
- Compares means across groups with repeated measurements.
- Considers correlations between measurements.
- Example: Evaluating drug treatment on patients' pain levels.
5. Mixed ANOVA:
- Combines between-subjects and within-subjects factors.
- Suitable for designs with both independent groups and repeated measures.
- Example: Assessing training program effects on performance across age groups.
Steps:

1. Define null (no difference) and alternative (difference exists) hypotheses.

2. Collect data from multiple groups.

3. Check assumptions, including normality and homogeneity of variances.

4. Calculate the F-statistic (The F-statistic is a ratio of two variances, typically used in analysis of variance

(ANOVA) and regression analysis, to test the null hypothesis that the variances of the populations from which

the samples are drawn are equal.) using the ratio of between-group variance to within-group variance.

5. Determine degrees of freedom for both between-group and within-group variability.

6. Find the critical value or calculate the p-value associated with the F-statistic.

7. Interpret results: Reject the null hypothesis if the p-value is less than the significance level.
Assumptions of ANOVA

1. Independence: Observations within each group are independent of each other.

2. Normality: Data within each group follows a normal distribution.

3. Homogeneity of Variances (Homoscedasticity): The variances of the

populations from which the groups are sampled are equal.

4. Random Sampling: Samples are selected randomly from the population or are
representative of it.
Explain the difference between one-way ANOVA and two-way ANOVA.
Sr. No Basis for Comparison One-Way ANOVA Two-Way ANOVA

1 Definition One-way ANOVA compares means Two-way ANOVA compares means across
across three or more independent two factors simultaneously.
groups on one factor (independent
variable).

2 Independent Variable One Two

3 Compares Three or more levels of one factor Effect of multiple level of two factors
4 Design of Experiments Need to satisfy only two samples All the principles need to be satisfied.
5 Number of Need to be the same in each group Need to be equal in each group
observations
6 Example Comparing the mean test scores of Examining the effect of both Teaching
students across different teaching Method (Factor 1) and Gender (Factor 2)
methods (Teaching Method A, B, C). on test scores.
7. Factors It involves only one factor (e.g., It involves two factors (e.g.,
Teaching Method) with multiple Teaching Method and Gender) and
levels (e.g., Method A, B, C). their interactions.
Null and alternative hypotheses for ANOVA:

Null Hypothesis (H0):There is no significant difference in means among the

groups.

Alternative Hypothesis (H1): At least one group mean is significantly

different from the others.

In symbols, these hypotheses can be represented as:

1. Null Hypothesis (H0): μ1 = μ2 = μ3 = ... = μk (where μ represents the

population mean of each group).
2. Alternative Hypothesis (H1): At least one μi (where i = 1, 2, ..., k) is
different from the others.
 What does it mean if the p-value in ANOVA is less than the significance
level?
If the p-value in ANOVA is less than the significance level (usually denoted by
α, typically set at 0.05), it means that there is sufficient evidence to reject the
null hypothesis.
In other words:
1. The observed differences in means among the groups are unlikely to have
occurred by random chance alone.
2. At least one group mean is significantly different from the others.
Therefore, a p-value less than the significance level indicates that there are
statistically significant differences among the groups being compared.
One-way ANOVA Observation table

Sum of Degrees of Mean

Source Squares (SS) Freedom (df) Square (MS) F-ratio (F) p-value (p)
Between
Groups SSB dfBetween MSB F p

Within
Groups SSW dfWithin MSW

Total SST dfTotal

Between Groups: Represents the variability explained by differences between the group means.
Within Groups: Represents the variability within each group, which is not explained by differences between the
group means.
Total: Represents the total variability in the data.
You would need to calculate the values for SSB, SSW, and SST based on your data and experimental design. Then,
you can calculate the degrees of freedom (dfBetween, dfWithin, dfTotal), mean squares (MSB, MSW), and F-ratio
(F) to assess the significance of the between-group differences. Finally, you would compare the obtained F-ratio
to the critical value from the F-distribution and determine the statistical significance of the between-group
differences.
One-Way ANOVA:
Merits:
1. Simple to understand and implement.
2. Efficient for comparing means across multiple groups on a single factor.
3. Useful for analyzing the main effect of one independent variable on the
dependent variable.
4. Suitable for experimental designs with one categorical independent variable.
Demerits:
5. Limited in assessing interactions between multiple factors.
6. Does not account for the combined effects of multiple independent variables.
7. Requires assumptions of equal variances and normality to be met.
Two-Way ANOVA
Sum of Squares Degrees of Mean Square
Source (SS) Freedom (df) (MS) F-ratio (F) p-value (p)
Factor A SSA dfA MSA F(A) p(A)

Factor B SSB dfB MSB F(B) p(B)

Interaction (AB) SSAB dfAB MSAB F(AB) p(AB)

Within Groups SSW dfWithin MSW

Total SST dfTotal

Factor A: Represents the variability explained by the levels of Factor A.
Factor B: Represents the variability explained by the levels of Factor B.
Interaction (AB): Represents the combined effect of Factors A and B.
Within Groups: Represents the variability within each combination of Factor A and Factor B, not explained by the factors
themselves.
Total: Represents the total variability in the data.
You would need to calculate the values for SSA, SSB, SSAB, SSW, and SST based on your data and experimental design. Then,
you can calculate the degrees of freedom (dfA, dfB, dfAB, dfWithin, dfTotal), mean squares (MSA, MSB, MSAB, MSW), and F-
ratios (F(A), F(B), F(AB)) to assess the significance of each factor and their interaction. Finally, you would compare the
obtained F-ratios to critical values from the F-distribution and determine the statistical significance of each factor and
interaction.
1. What is the primary purpose of Analysis of Variance (ANOVA) in statistics?
Ans: ANOVA is primarily used to compare means among three or more groups to
determine whether they are significantly different from each other.
2. What are the key assumptions of ANOVA?
Ans: The key assumptions of ANOVA include:
1. Normality: The data within each group are normally distributed.
2. Homogeneity of variances: The variances of the groups are equal.
3. Independence: Observations within each group are independent of each other.
If these assumptions are violated, it may affect the validity of the ANOVA results.
3. What is the purpose of conducting a post-hoc test after performing an ANOVA?
Ans: The purpose
Q: of conducting a post-hoc test after ANOVA is to determine which
specific groups differ significantly from each other. ANOVA only tells us whether
there are overall differences between groups, but post-hoc tests provide detailed
comparisons to identify where those differences lie.