Types of Reliability
Reliability of Consistency of What?
Observers or raters
Tests over time
Different versions of the same test
A test at one point in time
Inter-Rater or Inter-Observer Reliability
Object or
phenomenon
Inter-Rater or Inter-Observer Reliability
Object or
phenomenon
Observer 1
Inter-Rater or Inter-Observer Reliability
Object or
phenomenon
Observer 1 Observer 2
Inter-Rater or Inter-Observer Reliability
Object or
phenomenon
?
=
Observer 1 Observer 2
Inter-Rater or Inter-Observer Reliability
Are different observers consistent?
Can establish this outside of your study
in a pilot study.
Test-Retest Reliability
Time 1 Time 2
Test-Retest Reliability
Test = Test
Time 1 Time 2
Test-Retest Reliability
Stability over time
Test = Test
Time 1 Time 2
Test-Retest Reliability
Measure instrument at two times for
multiple persons.
Compute correlation between the two
measures.
Assumes there is no change in the
underlying trait between time 1 and time
2.
Parallel-Forms Reliability
Time 1 Time 2
Parallel-Forms Reliability
Form A
=
Form B
Time 1 Time 2
Parallel-Forms Reliability
Form A Stability across forms
=
Form B
Time 1 Time 2
Parallel-Forms Reliability
Administer both forms to the same
people.
Get correlation between the two forms.
Internal Consistency Reliability
Average inter-item correlation
Internal Consistency Reliability
Average Inter-Item correlation
Test
Internal Consistency Reliability
Item 1 Average Inter-Item correlation
Item 2
Item 3
Test
Item 4
Item 5
Item 6
Internal Consistency Reliability
Item 1 Average inter-item correlation
Item 2 I1 I2 I3 I4 I5 I6
I1 1.00
Item 3 I2 .89 1.00
Test I3 .91 .92 1.00
Item 4 I4 .88 .93 .95 1.00
.84 .86 .92 .85 1.00
I5
.88 .91 .95 .87 .85 1.00
Item 5 I6
Item 6
Internal Consistency
Reliability
Split-half correlations
Internal Consistency
Reliability
Split-half correlations
Test
Internal Consistency
Reliability
Split-half correlations
Item 1
Item 2
Item 3
Test
Item 4
Item 5
Item 6
Internal Consistency
Reliability
Split-half correlations
Item 1
Item 2 Item 1 Item 3 Item 4
Item 3
Test
Item 4
Item 5
Item 6
Internal Consistency
Reliability
Split-half correlations
Item 1
Item 2 Item 1 Item 3 Item 4
Item 3
Test
Item 4
Item 5
Item 2 Item 5 Item 6
Item 6
Internal Consistency
Reliability
Split-half correlations
Item 1
Item 2 Item 1 Item 3 Item 4
Item 3
Test
Item 4 .87
Item 5
Item 2 Item 5 Item 6
Item 6
Internal Consistency
Reliability
Cronbach’s alpha ()
Internal Consistency
Reliability
Cronbach’s alpha ()
Test
Internal Consistency
Reliability
Cronbach’s alpha ()
Item 1
Item 2
Item 3
Test
Item 4
Item 5
Item 6
Internal Consistency
Reliability
Cronbach’s alpha ()
Item 1 item 1 item 3 item 4 item 1 item 3 item 4 item 1 item 3 item 4
Item 2 .87 .87 .87
item 2 item 5 item 6 item 2 item 5 item 6 item 2 item 5 item 6
Item 3
Test
Item 4
Item 5
Item 6
Internal Consistency
Reliability
Cronbach’s alpha ()
Item 1 item 1 item 3 item 4 item 1 item 3 item 4 item 1 item 3 item 4
Item 2 .87 .87 .87
item 2 item 5 item 6 item 2 item 5 item 6 item 2 item 5 item 6
Item 3 SH1 .87
Test SH2 .85
Item 4 SH3 .91
SH4 .83
Item 5 SH5 .86
...
Item 6 SHn .85
Internal Consistency
Reliability
Cronbach’s alpha ()
Item 1 item 1 item 3 item 4 item 1 item 3 item 4 item 1 item 3 item 4
Item 2 .87 .87 .87
item 2 item 5 item 6 item 2 item 5 item 6 item 2 item 5 item 6
Item 3 SH1 .87
Test SH2 .85
Item 4 SH3 .91
SH4 .83
Item 5 SH5 .86
...
Item 6 SHn .85
= .85
Internal Consistency
Reliability
Cronbach’s alpha ()
Item 1 item 1 item 3 item 4 item 1 item 3 item 4 item 1 item 3 item 4
Item 2 .87 .87 .87
item 2 item 5 item 6 item 2 item 5 item 6 item 2 item 5 item 6
Item 3 SH1 .87
Test SH2 .85
Item 4 SH3 .91 Like the average
SH4 .83 of all possible
Item 5 SH5 .86 split-half
... correlations
Item 6 SHn .85
= .85
Internal Consistency Reliability - Summary
Average inter-item correlation
Average item-total correlation
Split-half reliability
Cronbach’s alpha ()
THANKS
