Scribd
Upload
14 views

Session 6 Tests of Significance

Uploaded by

dohaivy06062016
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
14 views

Session 6 Tests of Significance

Uploaded by

dohaivy06062016
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 16

Tests of

Significance
The concept of statistical significance testing

The concept of statistical significance is used to decide


whether a coefficient is high enough, for a given sample
size, not to be considered as due to chance. We have the
opportunity to decide the level of certainty that a
coefficient is not due to chance by selecting different
significance levels. We normally choose the 5% (or 0.05)
significance level in educational and most other social
science research as the appropriate level.
In general terms this means that there is only a 5%
chance (or 1 chance in 20) that the coefficient obtained is
that high due to chance. Thus coefficients with
significance levels less than 0.05 are reported as being
statistically significant. The figure 0.05 is said to be the
probability (of the coefficient being significant), and
hence is often written as p=0.05.
The SPSS output for both the Pearson and the
Spearman coefficients gives you the significance level
in each case. You simply have to note whether the
level shown is above or below 0.05. When the
significance level (or probability) is above 0.05 we
cannot say that the correlation coefficient is
significantly different from zero.
Two issues related to statistical significance testing

Two issues associated with significance testing are the


probability level selected and the use of one or two
tailed tests. It is important that you have a basic level of
understanding of the issues when reporting the results of
any significance testing.
Probability level selected

As stated above, the 5% (or 0.05) probability level is the


SPSS default. This describes the probability that the
statistic (eg, a correlation coefficient of that size) could
have arisen by chance. In most social science research
we normally accept a 5% chance of being wrong as
acceptable.

However, there are some studies where a 5% chance of


being wrong would not be acceptable. A decision on the
appropriate probability level comes down to a
judgement of the consequences of two opposing errors -
accepting a statistic as significant when it is not,
compared with treating a statistic as not significant
when it is significant.
One and two tailed tests

The significance tests shown above for correlations are


two-tailed tests, meaning that differences were being
tested on either side of zero. In other words, the
coefficient could have been positive or negative (ie, the
relationship could have been direct or inverse), only the
size of the coefficient was being tested. Use of two-tailed
tests is the usual and the safest option.
However, a one-tailed test may be used where the direction
of any likely difference is known beforehand. If you are
confident (from the literature or otherwise) that the
correlation could be in only one direction, you should use a
one-tailed test.

Remember that the use of one-tailed tests depends on


reliable prior knowledge.
Significance of other statistics

Similar methods for determining statistical significance


apply for other statistical parameters, such as chi-square.

A chi-square test of significance is used with a


crosstabulation (see Session 3) of two variables to
determine whether there is a relationship between the
variables.

In Session 3, the crosstabulation of responses to Item 9


and class of the student was shown in two tables (the
READ file was used in this example). This information is
repeated below, with the addition of a table with
statistical testing results including the chi- square test
statistic. To obtain the third table, you simply need to
click on the Statistics box within Crosstabs, and then
select the Chi-square option at the top of the next screen.
Crosstabs
This test gives the chi-square value (2.484) and the two-
tailed significance level (p=0.289). The level is greater than
0.05, so the chi-square value is not significant. Thus for this
group the relationship between class membership and being
correct on Item 9 in the test is not significant.
To assist you in knowing how to report statistical
significance tests for correlation coefficients and chi-square,
you should look through journal articles in your chosen
area to see how others have done it. There is no fixed
way, so choose a method you prefer from those you read.

Reporting in tabular form is often helpful for the reader, but


where one test only is being reported it is common to use
the following format for correlations (where p stands for the
probability or significance level).

Look at the example in Session 6.


... the correlation between age and ability was significant,
with younger students having higher ability (r=-0.524,
n=15, p<0.05).

An alternative is to report the actual probability level:

... the correlation between age and ability was significant,


with younger students having higher ability (r=-0.524,
n=15, p=0.045).
If you were using the chi-square test, the reporting could be:
(File RREAD, See Session 3)

... gender was not significantly linked with correctness on


Item 3 (chi-sq=3.28, n=15, p=0.07).
When do you use a correlation and when should you
choose a crosstabulation with a chi-square test when
considering relationships?

In some cases the methods are interchangeable, but not


always.

You cannot normally use a crosstabulation and chi-


square when you have interval variables because there
are too many values. The table would be huge if values
from, say, a test with scores from 1 to 100 was one of the
variables. In this case you would normally use a
correlation, unless the other variable was nominal.
Alternatively you could use a crosstabulation after
reducing the number of categories (using RECODE).
The correlation would normally be the more efficient
procedure.

You must not use a correlation where either variable is


nominal (or categorical) with three or more categories,
but use only a crosstabulation in that case. If the variable
is nominal but has only two categories (eg, gender), you
can use either a correlation or a crosstabulation to
describe the degree of relationship.

You might also like