2007; 29: e108–e116

WEB PAPER

Perceived academic quality and approaches

to studying in the health professions
JOHN T. E. RICHARDSON1, LESLEY DAWSON2, GAYNOR SADLO2, VIRGINIA JENKINS2 &
JANET MCINNES2
1
Institute of Educational Technology, The Open University, Milton Keynes, United Kingdom, 2School of Health Professions,
University of Brighton, Eastbourne, United Kingdom

Abstract
Background: Students in higher education may adopt different approaches to studying, depending upon their perceptions of
the academic quality of their courses and programmes, and both are likely to depend upon the nature of the curricula to which
they are exposed.
Med Teach Downloaded from informahealthcare.com by University of Mississippi on 10/29/14

Aims: Perceptions of quality and approaches to studying were investigated in students taking pre-registration programmes in
a school of health professions. Two of the programmes were 3-year undergraduate programmes with subject-based curricula,
and two were 2-year entry-level masters programmes with problem-based curricula.
Method: The Course Experience Questionnaire (CEQ) and the Revised Approaches to Studying Inventory (RASI) were
administered to the students within a single survey. Their teachers were also surveyed with regard to their beliefs and intentions
about teaching.
Results: The teachers on the two kinds of programme exhibited similar beliefs and intentions about teaching. However,
the students on the masters programmes produced higher ratings than did the students on the undergraduate programmes with
regard to the appropriateness of their assessment, the acquisition of generic skills and the emphasis on student independence.
For personal use only.

The students on the masters programmes were also more likely to show a deep approach to studying and less likely to show
a surface approach to studying than were the students on the undergraduate programmes.
Conclusions: The CEQ and the RASI provide complementary evidence for use in research, in quality assurance and in quality
enhancement. In comparison with subject-based curricula, problem-based curricula seem to enhance students’ perceptions of their
programmes and the quality of their learning.

Introduction Practice points

It is well established that students in higher education may
. Evaluations of academic programmes in the health
demonstrate different approaches to studying across different
professions need to take into account students’ per-
programmes (Richardson 2000). These variations seem to ceptions of academic quality and the approaches to
depend upon the students’ perceptions of their academic studying that they adopt.
context and in particular their perceptions of the quality of . The CEQ and the RASI can be recommended for use as
their courses (Richardson 2005). Accordingly, when evaluating research instruments and together provide complemen-
programmes of study, it is desirable to obtain students’ tary evidence for use in quality assurance and quality
accounts both of the perceived academic quality of those enhancement.
programmes and of the approaches to studying that they adopt . The introduction of problem-based curricula in the
on those programmes. The former will complement the latter health professions can enhance students’ perceptions
in illuminating the nature of the student experience. of the quality of their programmes and the quality of
The Course Experience Questionnaire (CEQ) was devel- their learning.
oped by Ramsden (1991) as an indicator of the quality of
degree programmes in Australia. It consists of 30 statements
in five scales that correspond to various aspects of effective
instruction. Students indicate their level of agreement with of the five scales are shown in Table 1. Sadlo (1997) found that
each statement on a scale from 1 to 5. Half of the items are students studying occupational therapy at institutions of higher
consistent with the meaning of the relevant scale, and the education in six different countries were significantly different
actual response is scored for those items. The other half of the in their scores on the 30-item CEQ.
items have a meaning that is opposite to that of the relevant Since 1993, an adapted version of the CEQ (containing only
scale, and these items are scored in reverse. The defining items 17 out of the original 30 items) has been administered annually

Correspondence: Professor John T. E. Richardson, Institute of Educational Technology, The Open University, Walton Hall, Milton Keynes MK7 6AA,
United Kingdom. Tel: þ44 1908 858014; fax: þ44 1908 654173; email: [email protected]

e108 ISSN 0142–159X print/ISSN 1466–187X online/07/050108–9 ß 2007 Informa UK Ltd.

DOI: 10.1080/01421590701529389
 Perceived quality and approaches to studying

Table 1. Defining items of the original scales in the Course Table 2. Subscales contained in the Revised Approaches to
Experience Questionnaire. Studying Inventory.

Subscale Defining item Deep approach Strategic approach Surface approach

Good teaching Teaching staff here normally give helpful Seeking meaning Organised studying Lack of purpose
feedback on how you are going.
Clear goals and You usually have a clear idea of where you’re Relating ideas Time management Unrelated memorising
standards going and what’s expected of you in this Use of evidence Alertness to assessment Syllabus-boundness
course. demands
Appropriate workload The sheer volume of work to be got through in Interest in ideas Achieving Fear of failure
this course means you can’t comprehend
it all thoroughly.* Monitoring effectiveness
Appropriate assessment Staff here seem more interested in testing
what we have memorised than what we
have understood.*
Emphasis on Students here are given a lot of choice in the
independence work they have to do.

Note. The defining items are taken from the results obtained by Ramsden
(1991). Items with asterisks are intended to be scored in reverse.
learning and by Sadlo & Richardson (2003) and Richardson
et al. (2005) in students who were taking campus-based
Med Teach Downloaded from informahealthcare.com by University of Mississippi on 10/29/14

programmes in occupational therapy. From both a theoretical

perspective and a practical perspective, therefore, it was
considered to be appropriate to combine the CEQ and the
to all new graduates from Australian universities. This includes RASI into a single survey to compare perceptions of academic
a sixth scale concerned with the fostering of generic skills quality and approaches to studying among students who were
and is supplemented by an item in which students rate their taking programmes in the health professions.
general level of satisfaction with their programmes. Wilson
et al. (1997) proposed that the original 30-item version of the
CEQ should be augmented with the Generic Skills scale to Context
yield a 36-item questionnaire, and they presented results
For personal use only.

from Australian students to demonstrate the reliability and This study considered the four pre-registration programmes
validity of this instrument. Richardson et al. (2005) found that taught in the same school at an English university. All were
this questionnaire was highly robust when used to compare full-time programmes involving periods of clinical experience
students studying at seven Danish schools of occupational and accredited by the relevant professional bodies. The two
therapy. undergraduate programmes (in physiotherapy and podiatry)
The Revised Approaches to Studying Inventory (RASI) followed the normal English model and lasted 3 academic
was devised by Entwistle et al. (2000). In its present version, years. The two masters programmes (in occupational therapy
it contains 52 statements in 13 subscales that are subsumed and physiotherapy) followed an accelerated model (of the sort
under three broad approaches to studying (see Table 2). that has been introduced in the UK during the last decade)
over 2 calendar years. The terms of Margetson (1991), the
. A deep approach involves a focus on the underlying
smaller masters programmes adopted explicitly problem-based
meaning of the course materials and would generally be
curricula, but the larger undergraduate programmes adopted
regarded as a desirable way of studying in higher education.
broadly subject-based curricula, albeit involving considerable
. A strategic approach involves a focus on achieving the best
interactive learning activities and a similar emphasis on the
results, regardless of whether this involves attention to the
development of practical clinical skills.
meaning of the course materials.
A recent evaluation of the programmes carried out by the
. A surface approach involves a focus on memorizing course
Quality Assurance Agency for Higher Education was unequi-
materials for the purposes of assessment and would be
vocally positive, concluding in each case that the reviewers
regarded as an undesirable way of studying in higher
had confidence in the academic and practitioner standards that
education.
were achieved by the relevant programmes. To obtain more
Again, students indicate their level of agreement with each information about the academic context, all of the teaching
statement on a scale from 1 to 5. Reid et al. (2005) found staff responsible for these programmes were asked to
that the RASI was broadly satisfactory when it was used to complete a questionnaire devised by Norton et al. (2005)
monitor students’ approaches to learning at a Scottish medical concerning various aspects of their underlying beliefs and their
school. actual intentions in teaching. The questionnaire contained
If students are asked about their perceptions of academic 34 items concerned with learning facilitation (a student-
quality and approaches to studying within a single survey, then centred and learning-orientated conception of teaching) or
on theoretical grounds one would expect higher ratings of knowledge transmission (a teacher-centred and content-
academic quality to be linked to the use of a deep approach to orientated conception of teaching). Once again, the respon-
studying and lower ratings of perceived academic quality to be dents indicated their level of agreement or disagreement with
linked to the use of a surface approach to studying. This was each item along a scale from 1 to 5.
confirmed by Lawless & Richardson (2002) and Richardson Not counting two of the authors who distributed the survey,
(2005) in students who were taking courses by distance completed questionnaires were received from 34 (or 74%)
e109
 J. T. E. Richardson et al.

Table 3. Mean (and SD) scores for beliefs and intentions in two significant trend for the masters teachers to obtain higher
groups of teachers.
scores on learning facilitation than did the undergraduate
Beliefs Intentions teachers, F (1, 32) ¼ 3.36, p ¼ 0.08. In other words, the teachers
who were delivering problem-based curricula to masters
Mean SD Mean SD students were rather more student-centred than were the
teachers who were delivering subject-based curricula to
Learning facilitation undergraduate students. This is of interest as it is known that
students whose teachers adopt a student-centred approach to
Undergraduate teachers 4.48 0.34 4.33 0.39
teaching are more likely to show a deep approach to studying
Masters teachers 4.69 0.21 4.61 0.26
and are less likely to show a surface approach to studying than
Overall 4.52 0.32 4.39 0.38 are students whose teachers adopt a subject-centred approach
Knowledge transmission to teaching (Trigwell et al. 1999). (It should, however, be
borne in mind that some staff taught on both the under-
Undergraduate teachers 3.08 0.67 3.88 0.53
graduate and the masters physiotherapy programmes.)
Masters teachers 2.91 0.36 3.79 0.22

Overall 3.05 0.62 3.86 0.48

Methods
Med Teach Downloaded from informahealthcare.com by University of Mississippi on 10/29/14

Note. The scores on each scale vary from 1 (low) to 5 (high).

In this quantitative investigation, a questionnaire was adminis-
tered to students in each of the four programmes, both as
a research study and to obtain student feedback in order
of the 46 teachers; 27 stated that they taught mainly on either to enhance the quality of the four programmes. Institutional
of the two undergraduate programmes, and 7 stated that they ethics approval was obtained in advance for the student survey
taught mainly on one of the two Masters programmes, and for the staff survey described above. Publication approval
although some physiotherapists taught on both the under- was granted on the basis that results would be reported in
graduate and the Masters pre-registration programmes. Their terms of level of study, not programme of study.
mean scores on the questionnaire are shown in Table 3. Both
For personal use only.

groups showed a very high concern with learning facilitation

Study population
and less concern with knowledge transmission. In comparison
with the 638 teachers who were surveyed by Norton et al. Each programme was structured into three stages which for the
(2005) and who were teaching a range of disciplines at undergraduate programmes followed the 3 academic years.
four different institutions of higher education, the teachers The target population consisted of all students who were in
in the present study obtained somewhat higher scores on Stages 1 and 2 of each of the four programmes during the
learning facilitation and somewhat lower scores on knowledge 2004–2005 academic year, a total of 351 students.
transmission: that is, they exhibited a commitment to student-
centred rather than subject-centred teaching.
Materials
A multivariate analysis of variance using a doubly multi-
variate design found that there was a significant overall The CEQ was piloted with the previous cohort of students on
difference between the teachers’ beliefs and their intentions, the programme in occupational therapy since it had been
F (2, 31) ¼ 27.80, p < 0.01. Univariate tests showed that for claimed to be inappropriate for evaluating problem-based
knowledge transmission they obtained higher scores on curricula (Lyon & Hendry 2002). As a result, a few items were
intentions than on beliefs, F (1, 32) ¼ 56.76, p < 0.01, but for amended slightly, and the instructions were extended so that
learning facilitation there was a marginally significant trend a response of 1 was to be used if an item was never true and
for them to obtain higher scores on beliefs than on intentions, a response of 5 was to be used if an item was always true.
F (1, 32) ¼ 3.41, p ¼ 0.07. In other words, the teachers’ inten- Following the previous studies by Richardson (2005) and
tions in practice were more orientated towards knowledge Richardson et al. (2005), the CEQ and the RASI were then
transmission and rather less orientated towards learning combined into a single questionnaire and supplemented by
facilitation than were their underlying pedagogical beliefs. questions about the participants’ age and gender. Otherwise,
A similar pattern was obtained by Norton et al. (2005), who their responses were entirely anonymous. Finally, they were
suggested that the academic and social context of higher asked whether they had any other comments on their
education compromised teachers in practising their underlying programme in general or about the questionnaire itself.
beliefs about teaching.
There was no significant difference between the scores
Data collection
obtained by the undergraduate teachers and the scores
obtained by the masters teachers, F (2, 31) ¼ 1.97, p ¼ 0.16, The questionnaire was administered to each of the eight
and no significant interaction with the difference between cohorts of students during regular classroom activities by the
their beliefs and intentions, F (2, 31) ¼ 0.40, p ¼ 0.67. There first author (who described himself as a researcher indepen-
was no significant difference between the undergraduate dent of the institution) and a research fellow. The students
teachers and the masters teachers on knowledge transmission, were advised both of the purposes of the study and that their
F (1, 32) ¼ 0.41, p ¼ 0.53. However, there was a marginally participation was entirely voluntary. They provided written
e110
 Perceived quality and approaches to studying

consent on a separate form that was retained by the institution. The probability level of 0.05 was employed as the criterion
They were assured that their individual responses would be of statistical significance. Comparisons may be statistically
kept wholly confidential by the first author and that only the significant and yet of little practical importance, especially
aggregate data from each cohort would be provided by way when there are large numbers of participants. This can be
of feedback to their institution. addressed by deriving a measure of the relevant effect (see
Richardson 1996). When two different groups are being
compared, the most common measure of effect size is derived
Data analysis by standardising the difference between their two means by
dividing it by the pooled within-group standard deviation;
The students responded to each of the 36 items in the CEQ by thus, an effect size of 0.5 means that the two groups differ on
indicating their agreement or disagreement with a particular average by an amount equal to half of their common standard
statement along a 5-point scale from 5 for ‘definitely agree’ deviation. Cohen (1988, pp. 24–27) proposed that effect sizes
to 1 for ‘definitely disagree’. As noted earlier, 15 items are of 0.2, 0.5 and 0.8 should be described as ‘small’, ‘medium’ and
opposite in meaning to the scale to which they belong, and ‘large’, respectively.
these are scored in reverse, so that 5 is scored as 1 and vice
versa. The students were assigned scores on the six scales in
the usual way by calculating the mean score across the Results
Med Teach Downloaded from informahealthcare.com by University of Mississippi on 10/29/14

constituent items. This yields scores between 1 and 5, where

Completed copies of the questionnaire were returned by 269
high scores represent favourable perceptions. An overall
students, which represents an overall response rate of 77%.
measure of perceived quality was calculated by taking the
Completed copies were returned by 182 (or 77%) of the 236
mean of the six scale scores. As in previous research, the
students on the undergraduate programmes and by 87
37th item (‘Overall, I am satisfied with the quality of this
(or 76%) of the 115 students on the masters programmes.
programme’) was used to assess the criterion validity of The difference between the two proportions was not
the CEQ. statistically significant, X2(1) ¼ 0.09, p ¼ 0.76. In most cases,
Similarly, the students responded to each of the 52 items the failure to achieve a 100% response rate was due to the
in the RASI by indicating their agreement or disagreement with students’ absence from the relevant class session rather than to
a particular statement along a 5-point scale from 5 for
For personal use only.

the students’ non-compliance with the request to participate

‘definitely agree’ to 1 for ‘definitely disagree’. All the items in the survey.
are consistent in meaning with the scale to which they belong. One respondent failed to indicate their age or gender.
The students were assigned scores on the 13 subscales in Of the remaining respondents, 220 (or 82%) were women and
the usual manner by calculating the totals of the scores on 48 (or 18%) were men. There was no significant difference
the constituent items. This then yields scores between 4 and 20 between the proportion of men on the undergraduate
on each subscale, where higher scores indicate that the programmes (16%) and the proportion of men on the masters
respondent has a greater disposition to adopt the relevant programmes (22%), X2(1) ¼ 1.35, p ¼ 0.25. The ages of the
approach to studying. Scores on the three main scales were undergraduate students ranged from 18 to 60 with a mean of
calculated by calculating the total scores across the relevant 26.1 years, and those of the masters students ranged from 20 to
subscales. 48 with a mean of 26.3 years. The mean ages of the two groups
Cronbach’s (1951) coefficient alpha was used as a measure were not significantly different, F (1, 266) ¼ 0.04, p ¼ 0.84,
of the reliability of the scales in each of the instruments. Their but the standard deviation of the undergraduate
construct validity was assessed by exploratory factor analysis students (9.06 years) was significantly greater than the
using principal axis factoring. In each case, the number of standard deviation of the masters students (4.26 years),
factors to be extracted was determined by the number of F (1, 266) ¼ 56.27, p < 0.001.
principal components whose eigenvalues were greater than
one, by Cattell’s (1966) scree test, and by O’Connor’s (2000)
procedure based upon the parallel analysis of random Course Experience Questionnaire
correlation matrices. Squared multiple correlations were used Table 4 shows the overall mean and standard deviation on
as the initial estimates of communality. Where appropriate, the each of the six scales in the CEQ, together with the values of
extracted factor solution was submitted to an oblique rotation coefficient alpha. The latter values were generally satisfactory
using a quartimin method. on conventional research-based criteria (Robinson et al. 1991).
Since the CEQ and the RASI had been administered within a A principal components analysis on the CEQ scores identified
single survey, it was then feasible to evaluate the relationship two components with eigenvalues greater than one. However,
between the students’ scores on the two instruments by means the eigenvalues-one rule tends to overestimate the true
of a multivariate analysis of variance and by examining number of components because of sampling effects
the correlation coefficients among the various scale scores. (Cliff 1988). Cattell’s (1966) scree test and the parallel analysis
Finally, comparisons between the scores obtained by the of 1000 random correlation matrices using O’Connor’s (2000)
undergraduate students and the Masters students were carried program implied that only one factor should be extracted.
out using multivariate analyses of variance, and univariate Table 4 shows that the scores on this factor defined a single
analyses of variance were used to identify the scales and underlying dimension that could plausibly be interpreted as an
subscales on which there was statistically significant variation. overall measure of perceived academic quality. According to
e111
 J. T. E. Richardson et al.

Table 4. Descriptive statistics for the Course Experience Questionnaire.

Scale Mean SD Coefficient alpha Factor loadings Correlation with general satisfaction
Appropriate assessment 3.80 0.70 0.67 0.59 0.39

Appropriate workload 2.73 0.78 0.74 0.27 0.26

Clear goals and standards 3.49 0.71 0.76 0.47 0.50

Generic skills 4.01 0.64 0.82 0.58 0.39

Good teaching 3.77 0.63 0.84 0.85 0.62

Emphasis on independence 2.86 0.66 0.63 0.57 0.38

Overall perceived quality 3.44 0.44 0.71 0.65

General satisfaction 4.13 0.76

Note. The scores on each of the six scales vary from 1 (low) to 5 (high). The overall measure of perceived quality is the mean of the six subscale scores. The ratings
of general satisfaction also vary from 1 to 5 and are taken from the 37th item in the CEQ.
Med Teach Downloaded from informahealthcare.com by University of Mississippi on 10/29/14

the factor loadings, this factor was most closely associated with Relationships between CEQ scores and RASI scores
scores on good teaching, appropriate assessment, generic
A multivariate analysis of variance showed that the amount of
skills and emphasis on independence but less closely related
variation shared between the respondents’ scores on the six
with scores on clear goals and standards and appropriate
scales of the CEQ and their scores on the 13 subscales of the
workload.
RASI was 75.7% (Wilks’ lambda ¼ 0.243), F (78, 1385) ¼ 5.19,
Table 4 also shows the correlation coefficients between the
p < 0.01. In particular, the overall measure of perceived quality
students’ scale scores and their ratings of general satisfaction.
on the CEQ showed a strong negative relationship with their
The latter were significantly correlated with their scores on all
For personal use only.

scores on surface approach (r ¼
0.46) but weaker positive

six scales of the CEQ. However, they were more strongly
relationships with their scores on deep approach (r ¼ þ0.30)
associated with their scores on good teaching, clear goals and
and strategic approach (r ¼ þ0.29). In short, perceptions
standards, appropriate assessment, generic skills and emphasis
of academic quality varied inversely with the adoption of an
on independence than with their scores on appropriate
undesirable approach to studying and were directly if less
workload. This is similar to the pattern obtained from the
strongly related to the adoption of more desirable approaches.
loadings in the factor analysis mentioned above, and it
provides evidence for the criterion validity of the CEQ as
a measure of perceived academic quality. Comparisons between the CEQ scores of
undergraduate and masters students
Table 6 shows the mean scores on the six scales of the CEQ
Revised Approaches to Studying Inventory
for the undergraduate students and the Masters students.
Table 5 shows the overall means and standard deviations on A multivariate analysis of variance showed that the difference
each of the 13 subscales of the RASI, together with the values between the two groups explained 40.4% of the variation in
of coefficient alpha. Some of the latter are unsatisfactory on their scale scores (Wilks’ lambda ¼ 0.596), F (6, 262) ¼ 29.60,
conventional research-based criteria (Robinson et al. 1991). p < 0.01. Univariate tests showed that the two groups produced
Nevertheless, the values for the three main RASI scales, significantly different scores on appropriate assessment,
calculated as the total scores across the relevant subscales, F (1, 267) ¼ 88.92, p < 0.01, clear goals and standards,
are more satisfactory. Overall, the students tended to F (1, 267) ¼ 15.26, p < 0.01, generic skills, F (1, 267) ¼ 15.89,
obtain higher scores on the subscales measuring deep p < 0.01, and emphasis on independence, F (1, 267) ¼ 32.96,
approach and strategic approach than on the subscales p < 0.01. They also produced significantly different
measuring surface approach. A principal components analysis overall scores on perceived academic quality,
identified four components with eigenvalues greater than F (1, 267) ¼ 16.77, p < 0.01, but they did not produce
one. Nevertheless, both Cattell’s (1966) scree test and the significantly different ratings of general satisfaction with their
parallel analysis of 1000 random correlation matrices using programmes, F (1, 267) ¼ 0.33, p ¼ 0.56.
O’Connor’s (2000) program implied that just three factors Inspection of Table 6 shows that the masters students
should be extracted. Table 5 shows that these factors were produced higher scores than did the undergraduate students
associated with the three main scales of the RASI. However, on appropriate assessment, generic skills, emphasis on
the lack of purpose subscale did not show a clear loading on independence and perceived academic quality. However,
the surface approach scale. As in previous studies, the ‘deep’ the undergraduate students produced higher scores than did
and ‘strategic’ factors were positively correlated with each the masters students on clear goals and standards. Table 6
other but were essentially uncorrelated with the ‘surface’ also shows the relevant effect sizes. In Cohen’s (1988) terms,
factor. all the effects were between medium and large; in other
e112
 Perceived quality and approaches to studying

Table 5. Descriptive statistics for the Revised Approaches to Studying Inventory.

Factor loadings

Subscale Mean SD Coefficient alpha Factor 1 Factor 2 Factor 3

Deep Approach

Seeking meaning 14.61 2.65 0.70 0.73 0.11 0.00

Relating ideas 14.25 2.38 0.58 0.86
0.11
0.02

Use of evidence 14.76 2.52 0.70 0.83
0.02 0.06

Interest in ideas 15.21 2.59 0.69 0.52 0.18
0.07

Total 58.83 8.28 0.83

Strategic Approach

Organised studying 12.87 2.90 0.55
0.04 0.78
0.10

Time management 13.50 3.77 0.85
0.11 0.87
0.06

Med Teach Downloaded from informahealthcare.com by University of Mississippi on 10/29/14

Alertness to assessment demands 14.42 2.57 0.53
0.02 0.43 0.15

Achieving 15.29 2.60 0.64 0.22 0.63
0.16

Monitoring effectiveness 15.93 2.46 0.66 0.25 0.56 0.03

Total 72.00 10.92 0.81

Surface Approach

Lack of purpose 7.44 2.93 0.70
0.15
0.29 0.27

Unrelated memorizing 10.09 2.77 0.54
0.19 0.03 0.74

Syllabus-boundness 12.56 3.31 0.68
0.23
0.10 0.42

For personal use only.

Fear of failure 13.93 3.70 0.77 0.25 0.04 0.69

Total 44.02 8.70 0.61

Factor correlations

Factor 1 1.00 0.44
0.21

Factor 2 0.44 1.00
0.17

Factor 3
0.21
0.17 1.00

Note. The scores on each subscale vary from 4 (low) to 20 (high). The three factors broadly correspond to a deep approach, a strategic approach and a surface
approach, respectively.

words, they are likely to be of both theoretical and practical

Table 6. Mean scores on the Course Experience Questionnaire.
importance.
Undergraduate Masters
Effect
Scale Mean SD Mean SD size
Comparisons between the RASI scores of
undergraduate and masters students
Appropriate assessment 3.56 0.66 4.31 0.48 1.23*

Appropriate workload 2.70 0.80 2.79 0.72 0.11

Table 7 shows the mean scores on the subscales of the RASI
for the undergraduate students and the Masters students.
Clear goals and standards 3.61 0.67 3.26 0.73
0.51*
A multivariate analysis of variance showed that the difference
Generic skills 3.91 0.66 4.23 0.54 0.52* between the two groups explained 22.4% of the variation
Good teaching 3.74 0.62 3.83 0.65 0.15 in their subscale scores (Wilks’ lambda ¼ 0.776),
Emphasis on independence 2.71 0.63 3.18 0.61 0.75* F (13, 255) ¼ 5.67, p < 0.01. Univariate tests showed that the
two groups obtained significantly different scores on seeking
Overall perceived quality 3.37 0.44 3.60 0.40 0.53*
meaning, F (1, 267) ¼ 24.55, p < 0.01, relating ideas,
General satisfaction 4.12 0.73 4.17 0.81 0.08
F (1, 267) ¼ 20.84, p < 0.01, use of evidence, F (1, 267) ¼ 33.18,
Note. The scores on each of the six scales vary from 1 (low) to 5 (high).
p < 0.01, alertness to assessment demands, F (1, 267) ¼ 4.22,
The overall measure of perceived quality is the mean of the six subscale scores. p ¼ 0.04, monitoring effectiveness, F (1, 267) ¼ 7.04, p < 0.01,
The ratings of general satisfaction also vary from 1 to 5 and are taken from the lack of purpose, F (1, 267) ¼ 4.45, p ¼ 0.04, unrelated memor-
37th item in the CEQ. Effect sizes of 0.2, 0.5 and 0.8 can be regarded
as ‘small’, ‘medium’ and ‘large’, respectively (Cohen 1988). Effects that are izing, F (1, 267) ¼ 27.73, p < 0.01, and syllabus-boundness,
significant ( ¼ 0.05) are indicated by asterisks. F (1, 267) ¼ 24.23, p < 0.01.
e113
 J. T. E. Richardson et al.

Table 7. Mean scores on the Revised Approaches to Studying Discussion

Inventory.
The CEQ once again turned out to be reasonably robust: each
Undergraduate Masters
Effect of the six scales demonstrated satisfactory internal consistency,
Scale Mean SD Mean SD size and a factor analysis confirmed their intended constituent
structure. As in previous research (Lawless & Richardson 2002;
Sadlo & Richardson 2003), students’ perceptions of academic
Deep approach
quality depended on various aspects of their programmes, as
Seeking meaning 14.08 2.77 15.72 1.97 0.65* reflected in the different scales of the CEQ. The students’
Relating ideas 13.81 2.35 15.17 2.17 0.59* ratings of their programmes were broadly positive, but they
Use of evidence 14.18 2.59 15.97 1.85 0.75* were highest on generic skills, appropriate assessment and
good teaching and lowest on appropriate workload and
Interest in ideas 15.00 2.55 15.64 2.62 0.25
emphasis on independence. (The heavy workload of
Total 57.07 8.41 62.51 6.69 0.69*
programmes in the health professions is often evident from
Strategic approach student feedback.) Nevertheless, the students’ ratings
Organised studying 12.76 2.73 13.09 3.23 0.11 of general satisfaction were also positive, with a mean rating
of 4.13 and a modal rating of 4 out of a maximum of 5. In other
Med Teach Downloaded from informahealthcare.com by University of Mississippi on 10/29/14

Time management 13.37 3.62 13.77 4.09 0.11

words, collectively the students expressed a high level
Alertness to assessment 14.64 2.60 13.95 2.43
0.27*
demands
of satisfaction with the quality of their programmes.
The undergraduate students produced higher scores than
Achieving 15.12 2.70 15.64 2.37 0.20
did the masters students on the scale concerned with clear
Monitoring effectiveness 15.65 2.43 16.49 2.43 0.35* goals and standards. This is perhaps unsurprising, because the
Total 71.55 10.63 72.95 11.50 0.13 latter students were responsible for setting their own learning
Surface approach goals for each problem that they encountered. Lyon & Hendry
(2000) argued that goals and standards in problem-based
Lack of purpose 7.70 3.06 6.90 2.59
0.27*
curricula could be seen as intrinsically ambiguous
Unrelated memorizing 10.68 2.60 8.86 2.74
0.69* if the students’ choice of topic was given priority.
For personal use only.

Syllabus-boundness 13.22 3.08 11.18 3.35
0.64* Nevertheless, they themselves found no difference in the
Fear of failure 13.97 3.63 13.86 3.86
0.03 scores on clear goals and standards produced by students who
were taking problem-based and traditional curricula, a finding
Total 45.56 8.13 40.80 9.00
0.56*
confirmed by Sadlo & Richardson (2003). Moreover, it should
Note. The scores on each subscale vary from 4 (low) to 20 (high). Effect sizes of be noted that both the undergraduate students and the masters
0.2, 0.5 and 0.8 can be regarded as ‘small’, ‘medium’ and ‘large’, respectively students in this study obtained mean scores on clear goals and
(Cohen 1988). Effects that are significant ( ¼ 0.05) are indicated by asterisks.
standards that were above the midpoint of the response scale,
reflecting broadly positive judgements. The programme teams
responsible for the masters students might usefully consider
how to make their programme goals and standards even more
A separate multivariate analysis of variance showed explicit, but it would clearly be misleading to describe the
that the difference between the undergraduate and the goals and standards as ‘ambiguous’.
masters students explained 14.1% of the variation in their Nevertheless, the masters students produced higher overall
scores on the three major scales (Wilks’ lambda ¼ 0.859), ratings of perceived quality than did the undergraduate
F (3, 265) ¼ 14.48, p < 0.01. Univariate tests showed that students. This was associated with higher scores on the
the two groups obtained significantly different scores scales concerned with appropriate assessment, with generic
on deep approach, F (1, 267) ¼ 27.88, p < 0.01, and on surface skills and with emphasis on Independence. In principle, these
approach, F (1, 267) ¼18.78, p < 0.01, but not on strategic differences might simply reflect the fact that Masters students
approach, F (1, 267) ¼ 0.97, p ¼ 0.32. have had more experience of higher education than under-
Inspection of Table 7 shows that the masters students graduate students. In the annual surveys of recent graduates
obtained higher scores than did the undergraduate students in Australia, masters students do tend to give more positive
on seeking meaning, relating ideas, use of evidence and ratings of their programmes than undergraduate students.
deep approach. However, the undergraduate students However, the differences in question tend to be small in
obtained higher scores than did the Masters students magnitude and achieve statistical significance only because of
on unrelated memorizing, syllabus-boundness and surface the very large sample size (Ainley & Long 1994, 1995; Johnson
approach. In Cohen’s (1988) terms, these effects were between et al. 1996). In the present study, the relevant differences were
medium and large and hence likely to be of both theoretical between medium and large on Cohen’s (1988) criteria and thus
and practical importance. The undergraduate students also more likely to be due to the different curricula that the students
obtained higher scores than the masters students on alertness had experienced.
to assessment demands, monitoring effectiveness and lack In fact, Sadlo (1997) (see also Sadlo & Richardson 2003)
of purpose; however, these effects would be regarded only obtained a similar pattern when comparing undergraduate
as small. programmes with problem-based and subject-based curricula
e114
 Perceived quality and approaches to studying

across six schools of occupational therapy. As Sadlo and have had more experience of higher education and thus have
Richardson remarked, ‘The results indicate that problem-based developed appropriate approaches to studying. However,
curricula are perceived as fostering student autonomy through Richardson (1998) found that undergraduate and postgraduate
the use of assessment methods that are consistent with the students taking the same courses obtained similar scores on a
intended learning outcomes’ (p. 266). It is also worth noting predecessor to the RASI, and this would imply that there are
that students taking the accelerated masters programmes did no intrinsic differences between undergraduate and masters
not produce significantly different ratings of either the quality students in their approaches to studying.
of the teaching that they received or the appropriateness of Of course, masters students are typically older than
their workload. This could mean that the problem-based undergraduate students, and it is well established that older
curricula did not involve a heavier overall workload, or that students are more likely to adopt a deep approach and are less
the students calibrated their ratings against their initial likely to adopt a surface approach than younger students
expectation that the workload would be heavier on an (Richardson 1994). Even so, in the present study, the under-
accelerated programme. graduate students and the masters students had very similar
The RASI was rather less satisfactory in this context. Four mean ages. Instead, the present results confirm the findings
subscales (relating ideas, organized studying, alertness to of previous investigations that, in comparison with subject-
assessment demands and unrelated memorising) did not based curricula, problem-based curricula tend to enhance
exhibit satisfactory internal consistency, and one subscale the use of a deep approach to studying and to discourage the
Med Teach Downloaded from informahealthcare.com by University of Mississippi on 10/29/14

(lack of purpose) failed to show a clear loading on any of the use of a surface approach to studying (Coles 1985; Newble
extracted factors. In other respects, however, the factor & Clarke 1986; Sadlo & Richardson 2003). Insofar as problem-
solution reflected the intended structure of the 13 subscales, based curricula are designed around real-life issues, this may
and the three main scales did demonstrate satisfactory internal help the students to create personal meaning by integrating the
consistency. Moreover, students’ scores on the six scales of the various topics that they have studied.
CEQ and the 13 subscales of the RASI shared three-quarters of
their respective variation. This confirms previous findings of an
Acknowledgements
intimate relationship between students’ perceptions of the
academic quality of their courses and the approaches to The authors are most grateful to Lynne Caladine for permission
For personal use only.

studying they adopt on those courses (Lawless & Richardson to carry out this study and to publish the findings, to Mark
2002; Sadlo & Richardson 2003; Richardson 2005; Richardson Cage for his assistance in the administration of the survey
et al. 2005). This is, of course, a purely correlational relation- and to the staff of the Survey Office of the Open University for
ship, and the nature of the underlying causal mechanisms is their assistance in the design and processing of the survey
currently a matter of debate (see Richardson 2006). questionnaires.
As has already been noted, the students tended to obtain
higher scores on subscales that measured the use of a deep
approach or a strategic approach than on the subscales that
Notes on contributors
measured the use of a surface approach. In general, then, JOHN T. E. RICHARDSON is Professor of Student Learning and Assessment
all the programmes were fostering desirable approaches to in the Institute of Educational Technology at the Open University.
studying rather than undesirable ones. Nevertheless, in LESLEY DAWSON is course leader of the M.Sc. Rehabilitation Science
programme in the School of Health Professions at the University of
contrast to previous studies, the present students tended to
Brighton.
obtain relatively high scores on syllabus-boundness (‘relying
GAYNOR SADLO is Head of the Division of Occupational Therapy in the
on staff to define learning tasks’) and fear of failure School of Health Professions at the University of Brighton.
(‘pessimism and anxiety about academic outcomes’):
VIRGINIA JENKINS is course leader of the B.Sc. (Hons.) Physiotherapy
(Ramsden & Entwistle 1981, p. 371). This is perhaps not programme in the School of Health Professions at the University of
surprising on professional training programmes where the Brighton.
syllabi are defined by professional bodies and where the JANET MCINNES is Head of the Division of Podiatry in the School of Health
students’ future employability depends crucially upon their Professions at the University of Brighton.
satisfactory academic performance. A similar pattern is evident
in the results that were obtained by Reid et al. (2005) in the
case of Scottish medical students. References
The students on the masters programmes obtained Ainley J, Long M. 1994. The Course Experience Survey 1992 Graduates
significantly higher scores on deep approach than did the (Canberra, Australian Government Publishing Service).
students on the undergraduate programmes; this was asso- Ainley J, Long M. 1995. The 1994 Course Experience Questionnaire:
A Report Prepared for the Graduate Careers Council of Australia
ciated specifically with higher scores on the subscales
(Parkville, Victoria, Graduate Careers Council of Australia).
measuring seeking meaning, relating ideas and use of Cattell RB. 1966. The scree test for the number of factors. Multivar Behav
evidence. In contrast, the students on the undergraduate Res 1:245–276.
programmes obtained significantly higher scores on surface Cliff N. 1988. The eigenvalues-greater-than-one rule and the reliability of
components. Psychol Bull 103:276–279.
approach than did the students on the masters programmes;
Cohen J. 1988. Statistical Power Analysis for the Behavioral Sciences, 2nd
this was mainly associated with higher scores on unrelated edn (NJ, Erlbaum, Hillsdale).
memorizing and syllabus-boundness. Again, in principle these Coles CR. 1985. Differences between conventional and problem-based
differences might simply reflect the fact that masters students curricula in their students’ approaches to studying. Med Educ 19:308–309.
e115
 J. T. E. Richardson et al.

Cronbach LJ. 1951. Coefficient alpha and the internal structure of tests. Richardson JTE. 1994. Mature students in higher education: I. A literature
Psychometrika 16:297–334. survey on approaches to studying. Stud High Educ 19:309–325.
Entwistle N, Tait H, McCune V. 2000. Patterns of response to an approaches Richardson JTE. 1996. Measures of effect size. Behav Res Methods Instrum
to studying inventory across contrasting groups and contexts. Euro J Comput 28:12–22.
Psychol Educ 15:33–48. Richardson JTE. 1998. Approaches to studying in undergraduate and
Johnson T, Ainley J, Long M. 1996. The 1995 Course Experience postgraduate students. Stud High Educ 23:217–220.
Questionnaire: A Report Prepared for the Graduate Careers Council Richardson JTE. 2000. Researching Student Learning: Approaches to
of Australia (Parkville, Victoria, Graduate Careers Council of Australia). Studying in Campus-Based and Distance Education (Buckingham,
Lawless CJ, Richardson JTE. 2002. Approaches to studying and perceptions SRHE & Open University Press).
of academic quality in distance education. High Educ 44:257–282. Richardson JTE. 2005. Students’ perceptions of academic quality and
Lyon PM, Hendry GD. 2002. The use of the Course Experience approaches to studying in distance education. Brit Educ Res J
Questionnaire as a monitoring evaluation tool in a problem-based 31:7–27.
medical programme. Assessment Evalu High Educ 27:339–352. Richardson JTE. 2006. Investigating the relationship between variations in
Margetson DB. 1991. Problem-focused education and the question of students’ perceptions of their academic environment and variations in
theory and practice, with special reference to some university courses. study behaviour in distance education. Brit J Educ Psychol 76:867–893.
Unpublished doctoral dissertation (Hobart, Tasmania, University of Richardson JTE, Gamborg G, Hammerberg G. 2005. Perceived academic
Tasmania). quality and approaches to studying at Danish schools of occupational
Newble DJ, Clarke RM. 1986. The approaches to learning of students in therapy. Scand J Occup Ther 12:110–117.
a traditional and an innovative problem-based medical school. Med Robinson JP, Shaver PR, Wrightsman LS. 1991. Criteria for scale selection
Educ 20:267–273. and evaluation, in: J.P Robinson, P.R Shaver & L.S Wrightsman (Eds),
Med Teach Downloaded from informahealthcare.com by University of Mississippi on 10/29/14

Norton L, Richardson JTE, Hartley J, Newstead S, Mayes J. 2005. Teachers’ Measures of Personality and Social Psychological Attitudes, pp. 1–16
intentions and beliefs concerning teaching in higher education. High (San Diego, CA, Academic Press).
Educ 50:537–571. Sadlo G. 1997. Problem-based learning enhances the educational
O’Connor BP. 2000. SPSS and SAS programs for determining the number of experiences of occupational therapy students. Educ Health 10:101–114.
components using parallel analysis and Velicer’s MAP test. Behav Res Sadlo G, Richardson JTE. 2003. Approaches to studying and perceptions of
Methods Instrum Comput 32:396–402. the academic environment in students following problem-based and
Ramsden P. 1991. A performance indicator of teaching quality in higher subject-based curricula. High Educ Res Devel 22:253–274.
education: the Course Experience Questionnaire. Stud High Educ Trigwell K, Prosser M, Waterhouse F. 1999. Relations between teachers’
16:129–150. approaches to teaching and students’ approaches to learning. High
Ramsden P, Entwistle NJ. 1981. Effects of academic departments on Educ 37:57–70.
students’ approaches to studying. Brit J Educ Psychol 51:368–383. Wilson KL, Lizzio A, Ramsden P. 1997. The development, validation
Reid WA, Duvall E, Evans P. 2005. Can we influence medical students’ and application of the Course Experience Questionnaire. Stud High
For personal use only.

approaches to learning? Med Teach 27:401–407. Educ 22:33–53.

e116