Drawing and testing assumptions
Parametric and non-parametric hypothesis tests
Please do not copy without permission. © ExploreAI 2023.
Drawing and testing assumptions
Statistical tests overview
| In hypothesis testing, statistical tests are used to decide whether we reject or fail to reject the null hypothesis.
We use either parametric or non-parametric tests, depending on some factors of the underlying data.
Parametric tests
● Assume that the data follow a specific distribution, such as a normal distribution.
● More precise estimates of the population parameters when the assumptions are met.
● When the data violate the assumptions, these tests may produce inaccurate or unreliable
results.
Non-parametric tests
● No assumptions about the distribution of the data.
● More robust to violations of the assumptions but less powerful when the assumptions of the
parametric test are met.
● Often preferred when the sample size is small.
2
Drawing and testing assumptions
Parametric tests overview
| Parametric tests are a group of statistical tests that are used to test hypotheses on population parameters,
such as the mean or variance, by making certain assumptions about the underlying distribution of the data.
T-test Z-test
Used to test hypotheses on the mean of a single Similar to the t-test but the z-test assumes that the
population, or the difference between the means of population standard deviation is known and the
two populations with small sample sizes. sample is large.
F-test Analysis of Variance (ANOVA)
Used to test hypotheses on the difference between Used to test hypotheses on the means of three or
the variances of two or more populations with large more populations.
sample sizes.
3
Drawing and testing assumptions
The t-test
| The t-test is a parametric test based on the t-distribution and is used to test hypotheses on the mean of a
single population, or the difference between the means of two samples, when the sample size is smaller.
The t-distribution, or Student’s t-distribution, is used to calculate the
probability of obtaining a sample mean that is different from the
population mean. It has heavier tails to account for the increased t-distribution
variability in smaller sample sizes.
The t-test uses the t-distribution to calculate the critical value that is
used to determine if the difference between is statistically
significant.
normal
distribution
|t-score| ≥ critical value -> Reject the null hypothesis
|t-score| < critical value -> Fail to reject the null hypothesis
4
Drawing and testing assumptions
One-sample t-test
| To compare a sample mean with the population mean, we use a one-sample t-test.
The test statistic t (t-score) is: Assumptions for one-sample t-test:
is the sample mean 01. Random sampling: The data are collected using a random
is the sample standard deviation sampling method to ensure that the sample is
is the sample size representative of the population.
is the population mean 02. Normality: The distribution of the sample means is
approximately normal.
03. Independence: The observations in the sample are
We can use the AVERAGE(), STDEV(), and COUNT() Google independent of each other. In other words, the value of one
Sheet functions to calculate the t-score. observation is not related to the value of another
observation.
Note: We also need to determine the critical value using the 04. Homogeneity of variance (homoscedasticity): The
degrees of freedom and level of significance either from a variance of the sample is approximately equal to the
statistical table, online calculator, or using Google Sheets. variance of the population.
5
Drawing and testing assumptions
Independent two-sample t-test
| To compare the means of two independent samples, when there is no link between the two groups, we use an
independent two-sample t-test.
The t-score is: Assumptions for independent samples t-test:
is sample one’s mean
is sample two’s mean 01. Normality: The data in each group are normally distributed.
is pooled standard deviation 02. Homoscedasticity: The variance of the data in each group is
is sample one’s size equal.
is sample two’s size 03. Independence: The observations within each group are
independent of each other, and the two groups are
independent of each other.
The pooled standard deviation is:
Note: To find the p-value from the statistical table, we need to
use the degrees of freedom as (n1 + n2 - 2) for the independent
two-sample test.
6
Drawing and testing assumptions
Paired two-sample t-test
| To compare the means of two related samples, i.e. two sets from the same group, we use a paired two-sample
t-test.
The t-score is: Assumptions for paired samples t-test:
01. Normality: The differences between the paired observations
are normally distributed.
02. Independence: The paired observations are independent of
each other.
is the mean of the differences between the samples
is the standard deviation of the differences
is the sample size
Note: To find the p-value from the statistical table, we need to
use the degrees of freedom as (n - 1) for the paired two-sample
test.
7
Drawing and testing assumptions
T-tests in Google Sheets
We can either use AVERAGE(), STDEV(), and COUNT() to calculate the t-score using Google Sheets and a p-value
table to determine the p-value, or we can use the built-in function T.TEST() to calculate the p-value.
=T.TEST(range1, range2, tails, type)
● range1 – The first sample of data or group of cells to ● type – Specifies the type of t-test.
consider for the t-test. ○ If 1: a paired test is performed.
● range2 – The second sample of data or group of cells to ○ If 2: a two-sample equal variance (homoscedastic) test
consider for the t-test. is performed.
● tails – Specify the number of distribution tails. ○ If 3: a two-sample unequal variance (heteroscedastic)
○ If 1: uses a one-tailed distribution. test is performed.
○ If 2: uses a two-tailed distribution.
If the populations being compared have equal variance, then we
can use the two-sample equal variance test. If the variances
differ, we use the two-sample unequal variance test.
Note: The T.TEST() function output is the probability
one-tailed two-tailed associated with the t-test, i.e. the p-value.
8
Drawing and testing assumptions
The z-test
| The z-test is a parametric test based on the normal distribution (a.k.a. the z-distribution) and is similar to the
t-test. However, the z-test is used when the sample is large and the population standard deviation is known.
In the z-test, the difference between the means of two populations
is expressed in terms of the number of standard deviations (𝝈).
The z-score (test statistic z) therefore represents the number of
standard deviations between the sample mean and population mean,
assuming the null hypothesis is true.
|z-score| ≥ critical value -> Reject the null hypothesis
|z-score| < critical value -> Fail to reject the null hypothesis
-2𝝈 -1𝝈 𝜇 +1𝝈 +2𝝈
9
Drawing and testing assumptions
One-sample z-test
| To compare a sample mean with the population mean when the sample size is large and standard deviation
known, we use a one-sample z-test.
The test statistic z is: Assumptions for one-sample z-test:
is the sample mean 01. Random sampling: The sample is selected randomly from
is the population standard deviation the population.
is the sample size 02. Normal distribution: The population from which the sample
is the population mean is drawn is normally distributed.
03. Large sample size: The sample size is sufficiently large,
typically at least 30, so that the central limit theorem can
Note: The only difference between the test statistic t and z for be applied.
a one-sample test is using the sample standard deviation for 04. Independence: The observations in the sample are
the t statistic and the population standard deviation for the z independent of each other.
statistic. 05. Known population standard deviation: The standard
deviation of the population is known.
10
Drawing and testing assumptions
Two-sample z-test
| To compare the means of two different samples when the sample size is large and standard deviation known,
we use a two-sample z-test.
The test statistic z for independent groups: Assumptions for two-sample z-test:
is sample one’s mean 01. Normality: The data in each group are normally distributed.
is sample two’s mean 02. Homoscedasticity: The variance of the data in each group is
is population one’s standard deviation equal.
is population two’s standard deviation 03. Independence: The two samples are independent of each
is sample one’s size other.
is sample two’s size 04. Known population standard deviations: The standard
deviation of the populations are known.
The test statistic z for related groups:
is the mean of the differences between the samples
is the hypothesised mean of the differences (usually equal to zero)
is the standard deviation of the differences
is the sample size
11
Drawing and testing assumptions
Z-tests in Google Sheets
We can either use AVERAGE(), STDEV(), and COUNT() to calculate the z-score using Google Sheets and a
statistical table to determine the p-value, or we can use the built-in function Z.TEST() to calculate the p-value.
=Z.TEST(data, value, [standard_deviation])
● data – The sample of data to consider for the z-test.
Note: The Z.TEST() function output is the probability
● value – The test statistic to use in the z-test, most often
associated with the z-test, i.e. the p-value.
the population mean.
● [standard deviation] – The standard deviation to
assume for the z-test, often the standard deviation of the
population.
It is important to note how the Google Sheet hypothesis testing functions differ based on the test we need to
use, whether it is a one or two-sample test, and whether our samples are independent or paired.
12
Drawing and testing assumptions
How to choose a parametric test
If our data follow
One-sample z-test
a normal
distribution 1
Number of
then: No Two-sample
samples
2 independent z-test
Yes
Samples
unrelated
No Two-sample paired
z-test
Sample size <
30
Yes
Yes One-sample t-test
No 1
Population Number of
Two-sample
stdev known samples
Yes independent t-test
2 Samples
unrelated
No Two-sample paired
t-test
13
Drawing and testing assumptions
Non-parametric tests overview
| Non-parametric tests are a group of statistical tests that are used to test hypotheses when the
underlying distribution of the data is unknown.
Several non-parametric tests are available to use in
hypothesis testing.
Continuous
Which test to apply usually depends Type of
Ordinal
on: data Categorical
Nominal
Difference
Test purpose
Correlation
Independent
Number of
Relationship
groups
Related
14
Drawing and testing assumptions
Non-parametric tests overview
| Rather than relying on estimates of population parameters as parametric tests do, non-parametric tests
are based on ranks or the number of times certain events occur in the data.
Some of the most commonly used non-parametric tests include:
● Kolmogorov-Smirnov test: Compares the distribution of a sample with a theoretical distribution.
● Mann-Whitney U-test: Determines if two independent groups come from populations with the same
distribution.
● Chi-square: Tests for independence between categorical variables in a contingency table.
● Spearman’s rank correlation coefficient: Measures the strength and direction of the relationship between two
variables using ranks.
● Wilcoxon signed rank test: Determines if there is a significant difference between two paired samples using
ranks.
● Friedman test: Tests for significant differences between three or more paired samples using ranks.
● Kruskal-Wallis H test: Tests for significant differences between three or more independent groups using
ranks.
15
Drawing and testing assumptions
Kolmogorov-Smirnov and ECDF
| Kolmogorov-Smirnov (KS) is a non-parametric test based on the empirical cumulative distribution function
(ECDF), which is a way to visually represent how data are distributed.
The ECDF maps each observation in a dataset to the proportion of
observations that are less than or equal to it.
It is a step function that increases by 1/n at each point, where n is the
sample size.
For example, considering the ECDF for the average annual salary
for women in Africa, we see that more than 50% of women earn
$2500 or less per year. We also see that 75% of women earn less
than $3750 per year.
16
Drawing and testing assumptions
Kolmogorov-Smirnov test overview
| Kolmogorov-Smirnov (KS) is used to test hypotheses on whether an underlying distribution observed in a
sample is similar to the hypothesised distribution, or whether two distributions are similar.
Considering that KS helps us examine underlying We will also need to specify the level of significance
distributions, it is a useful tool to test for normality, (𝛼), calculate a test statistic (denoted D for
which is a prerequisite of parametric tests. Kolmogorov-Smirnov), and determine the critical
value and p-value.
As with parametric tests, we need to state the null
and alternative hypotheses:
|D| ≥ critical value -> Reject the null hypothesis
● H0 is that the sample is drawn from a population |D| < critical value -> Fail to reject the null hypothesis
with a specific distribution, e.g. a normal
distribution.
p-value ≤ 𝛼 -> Reject the null hypothesis
● HA is that the sample is not drawn from a
p-value > 𝛼 -> Fail to reject the null hypothesis
population with the specified distribution.
17
Drawing and testing assumptions
Kolmogorov-Smirnov test statistic
The Kolmogorov-Smirnov test statistic is:
We need to calculate both difference terms to ensure that the
ECDF starts at 0 ((i-1)/n) and ends at 1 (i/n), i.e. the entire range.
where
is the index of the ordered sample Y1, Y2, …,Yn, i.e. the rank
is the sample size
is the ith ordered value in the sample
is the hypothesised cumulative distribution function (CDF)
evaluated at the ith ordered value of the sample data Yi
Both (i-1)/n and i/n represent the empirical cumulative
distribution function (ECDF)*.
The test statistic D is a single value which is the maximum
across both difference terms, |F(Yi) - (i-1)/n| and |i/n - F(Yi)|, for
all sample values, Yi.
*(i-1)/n represents the cumulative proportion observations that are expected to be strictly less than the ith ordered value, while i/n represents the proportion that is less than or equal to the ith. 18
Drawing and testing assumptions
Steps to the Kolmogorov-Smirnov test
The steps to performing KS: Although the p-value for a KS test can be calculated
01. State the null and alternative hypotheses: using statistical software or a programming language
a. H0 is that the sample is drawn from a like Python using built-in functions, it is much more
population with a specific distribution, e.g. a
involved and resource intensive in Google Sheets.
normal distribution.
b. HA is that the sample is not drawn from a In theory, the p-value can be calculated using either
population with the specified distribution.
the exact method, when n ≤ 35, or the approximate
02. Specify the level of significance (𝛼). (also known as asymptotic) method for larger sample
03. Calculate the test statistic, D, using the sizes (n).
Kolmogorov-Smirnov test statistic formula.
04. Determine the critical value using the KS table,
level of significance, and sample size.
05. Compare the test statistic (D) to the critical
value.
19
