C2

C1

Improving Feedback for

Young Learners: Introducing
B2 the New Statement of
Results for Pre A1 Starters,
A1 Movers and A2 Flyers
B1
Maggie Dunlop, Mark Elliott
and Lynne Stevenson

A2

A1

Pre A1

Improving Feedback for Young Learners: Introducing the New Statement of Results for Pre A1 Starters, A1 Movers and A2 Flyers 1
INTRODUCTION
This article describes why and how changes were made to Pre A1 Starters, A1 Movers and A2 Flyers reporting
practices. It outlines the studies undertaken to develop both additional feedback, and its presentation.

THE REPORTING OF RESULTS FOR YOUNG LEARNERS

A central philosophy of the Pre A1 Starters, A1 Movers and A2 Flyers tests is celebrating children’s achievement
in English and providing encouragement for their learning journey. Therefore, all Pre A1 Starters, A1 Movers and
A2 Flyers candidates receive a certificate – there is no minimum ‘pass’ mark required. In addition, all certificates
display at least one shield for each paper (listening, reading/writing, speaking), recognising that participation is
valued as highly as test performance. A maximum of five shields is awarded for each paper, for a possible total
of 15 shields. Figure 1 shows an example of a certificate.

Figure 1: Example of certificate

LEARNERS AND TEACHERS’ NEEDS

Research was carried out at the start of the revision process with schools, teachers, parents and learners across
the world, covering all aspects of the Pre A1 Starters, A1 Movers and A2 Flyers tests, including the certificates. The
research showed that users were consistently satisfied with the existing certificates, where the shields recognise
achievement in the different skills. Respondents felt that the shields provide both encouragement to learners and
are easy to understand. However, a majority of respondents also indicated that they would like more detailed
feedback. Therefore, it was decided to keep certificates and use of shields unchanged, but to investigate options
for providing more information, as outlined below.

2
FOLLOWING BEST PRACTICE
From the perspective of best practices in learning and assessment, providing more information to candidates was
an imperative. Multiple approaches to validating tests and assessments make clear that considering and designing
according to the intended and/or likely effects, consequences and impact of tests are a key component of validity
arguments (Bachman 2005, Fulcher and Davidson 2007, Weir 2005). A test needs to be fit for purpose, and one
way it does this is by facilitating intended outcomes.
For the Pre A1 Starters, A1 Movers and A2 Flyers tests, the intended outcomes are an increased joy of learning and
a sense of achievement among children learning English, and, for teachers and parents, a sense that they are being
supported in their endeavours to teach practical English skills to children. In order to fully meet these intended
outcomes – particularly the third – candidates, their teachers and their families needed more information.
Research shows the central role that feedback has within learning processes (Clark 2012, Hattie and Timperley
2007, Jones and Saville 2016, Pintrich 2004, Shute 2008, Zimmerman 2002). Only with information about
one’s performance, can one know what they should do to progress. More importantly for this age group, with
appropriate information and with the support of adults, children will know where they are on their English
learning journey, where they are going, and how they are going to get there (Hattie and Timperley 2007). They
and their teachers and families can use that information to make learning choices that result in ability to use
English in real life.
Note however, that in order for feedback to enhance learning, it needs to be easy to use and offer an appropriate
level of control to the learner (Dunlop 2017, Fernández-Toro and Hurd 2014). It was these questions – what
amount of detail would be appropriate, and what format would be most easy to use – that were the two main
areas of research for this project.

ARRIVING AT AN APPROACH
Several options were considered to improve feedback quality. The options offered different types of information
at different levels of detail. In addition to shields, the use of Common European Framework of Reference
for Languages (CEFR) levels (Council of Europe 2001), the Cambridge English Scale, and detailed task-level
information were all available options. Stakeholder responses were clear – the more information, the better.
Some stakeholders expressed enthusiasm for numeric scores that enabled ranking of children via their overall
achievement. However, a very large majority stressed that enhanced feedback should uphold the key features of
the Pre A1 Starters, A1 Movers and A2 Flyers tests; ‘no fail’, fun, child-friendliness, and positive encouragement
were absolutely essential.
As the primary function of Pre A1 Starters, A1 Movers and A2 Flyers tests is to support learning, a learning-first
approach to feedback was adopted. Any new information to be provided would not consist of more scores, but
rather substantive information about what learners showed they can do and are still mastering, and practical
suggestions on how to move forward. This information would be personal to individual candidates and based on
their performance on their test.
At the same time, the relationship between shields, CEFR levels and Cambridge English Scale scores was to be
shown visually in a figure, so that parents and teachers would easily know where their child was both on an
international scale (the CEFR) and on their journey through Cambridge English Qualifications. In this way, new
‘big picture’ information would be available for interested adults, and it would be presented in a way that serves
learning and development purposes.

Improving Feedback for Young Learners: Introducing the New Statement of Results for Pre A1 Starters, A1 Movers and A2 Flyers 3
DEVELOPMENT OF SUBSKILLS AND DESCRIPTORS
To provide the additional information desired by learners, their families and teachers, a review of the tests was
conducted. The review aimed to identify what information was available over and above a diagnosis of overall
achievement in a skill, which is already provided in the shields. A key issue was to identify subskills that were
tested often enough to provide a reasonably confident diagnosis of learners’ abilities in those subskills.
The review found that subskills were tested at task level, and that tasks focused on functional, communicative
subskills such as making and understanding descriptions, holding and understanding conversations, and
understanding instructions. In addition, some tasks require candidates to demonstrate their ability to use level-
appropriate vocabulary and grammar knowledge accurately.
Two or more tasks typically assessed subskills at each level. These tasks often appeared on different papers,
showing how the subskills were tested for both productive and receptive skills and print and oral mediums.
The initial list of subskills and associated test tasks are listed in Table 1.

Level Subskill (testing focus) Tasks*

Descriptions RW2, RW4
Conversation Speaking test
Pre A1 Starters
Using words RW1, L1, L4
Writing words RW3, RW5, L2
Descriptions RW1, RW6, L3
Story RW3, RW5
A1 Movers
Conversation RW2, L2, L4
Using language knowledge RW4, L1
Understand descriptions by reading and listening to information RW1, L1
Understand and tell stories by reading and writing RW3, RW5, RW6
Use your grammar and vocabulary knowledge accurately RW4, RW6
A2 Flyers
Understand conversations L2, L3, L4
Describe things and events using speaking Speaking test
Pronunciation Speaking test

*RW = Reading/Writing paper; L = Listening paper; Numbers 1-6 = part number, e.g. RW2 = Reading/Writing part 2.

Table 1. Initial identified subskills: summary

Five assessment specialists, who were also experienced teachers of young learners, then separately developed
descriptors for each of the subskills. The five submissions were combined to create the first full draft of
the descriptors.
Next, these descriptors were sent to one researcher and five specialist teachers of young learners based in Latin
America, South Asia, Southeast Asia, Europe and the Middle East. These instructional specialists reviewed the
descriptors for clarity to children and their families, and for representativeness of the construct to which each
descriptor referred. The specialists also wrote a selection of practical suggestions that children could use to
practise and improve English.
Based on the suggestions and comments from teachers, the descriptors were reorganised and their wording
refined through a further four rounds of revision. At the same time, the teachers’ suggestions for learning
were organised according to the subskills that they were most associated with (see Table 2). At this time, the
suggestions were also revised for readability and appropriateness to as many children as possible.

4
 Level Subskill (testing focus) Tasks*
Understanding conversation L3
Understanding basic descriptions RW2, RW4
Pre A1 Starters Writing words/spelling RW3, RW5, L2
Using vocabulary knowledge RW1, L1, L4
Conversation Speaking test
Understanding conversation RW2, L2, L4
Understanding spoken descriptions L3, L5
Using written descriptions RW1, RW6
A1 Movers
Reading and understanding stories RW3, RW5
Using grammar and vocabulary knowledge accurately RW4, L1
Conversation Speaking test
Understanding conversation RW2, L2, L3, L4
Understanding descriptions RW1, L1, L5
A2 Flyers Understanding and telling stories by reading and writing RW3, RW5, RW7
Using grammar and vocabulary knowledge accurately RW4, RW6
Describing things and events using speaking Speaking test

*RW = Reading/Writing paper; L = Listening paper; Numbers 1-7 = part number. E.g., L3 = Listening part 3.

Table 2. Final identified subskills: summary

Three further areas of work remained. The first was what information to provide to candidates who had not yet
mastered any of the subskills, and to those who had mastered them all. This was an important issue, because
children not only need to feel a sense of achievement, but also be provided with sufficient challenge and
pathways to move on to the next phases of learning.
As a result, basic and advanced descriptors were developed for each of Pre A1 Starters, A1 Movers and A2 Flyers.
Analyses of pretesting data indicated that children who were still in the process of mastering all the descriptors
could typically, as a minimum, use English in several ways, so these ways were described in the basic descriptors.
For the advanced descriptors, candidates were encouraged to develop skills represented in the exam the next level
up. For example, the A2 Flyers advanced descriptor encourages candidates to use English at an A2 Key for Schools
level of proficiency.
The second issue was to decide when candidates could be considered to have mastered each subskill. Data from
pretesting was gathered and analysed to determine a ‘mastery’ threshold – the point at which a subskill becomes
classified as a strength. As is always the case in educational testing, different versions of tests contain items of
somewhat varying difficulty. Therefore, applying a fixed raw score mastery threshold would result in variation of
the standard from version to version. To deal with this issue, a fixed Rasch ability threshold was agreed, thereby
allowing variations in difficulty between test versions to be accounted for.
Finally, it was crucial to develop reporting that was as accessible as possible to candidates and their families.
One way to do this was to provide the Statements of Results in languages other than English where desired and
possible. Therefore, Statements of Results will be provided in some languages in addition to English.

Improving Feedback for Young Learners: Introducing the New Statement of Results for Pre A1 Starters, A1 Movers and A2 Flyers 5
DEVELOPMENT OF LAYOUT
While development of descriptors that represented detailed candidate mastery was underway, the other key piece
of the Statement of Results was also in development: the layout of the information. This issue was important
because feedback must always be as intuitive and easy to understand as possible, otherwise people cannot use
it effectively (Dunlop 2017). Three options were initially discussed, two of which were chosen for international
trialling based on their clarity. The two options are presented in Figure 2.
Version 1 Version 2

Figure 2: Trialled options

Ten countries representing Latin America, Europe, South Asia and Southeast Asia participated in the trial, consisting
of over 200 teachers, school staff, parents, and children. It was made clear to the respondents that the artwork
used was indicative and not final. With nearly three quarters of the vote, the preferred version was Version 2, where
the information was presented in categories.
Looking at the feedback on the versions, there were a number of factors which had led to why Version 2 was more
popular – the information architecture was much more intuitive and easy to understand. The strength noted for
Version 1 was that the lines with the bikes clearly showed candidates’ progress/abilities. However, the weaknesses
were that the lines require interpreting, and it was difficult to understand at a glance. In contrast, several strengths
were noted for Version 2. These included the clean appearance, its visual appeal, the well organised presentation of
information, that it was easy to understand (if in a known language), the presence of the Pre A1 Starters, A1 Movers
and A2 Flyers monkey, and the clear presentation of strengths and areas to improve. Weaknesses noted for Version
2 addressed only the choice of graphics, and not the core information design. A few respondents preferred using
boys and girls to the monkey, and a handful felt that A2 Flyers candidates might find the monkey a little ‘young’.

CONCLUSION
In summary, the Pre A1 Starters, A1 Movers and A2 Flyers philosophy of celebrating achievement and encouraging
learning is highly valued by teachers, parents and children. The certificates and shields are recognised as a key part
of this philosophy and remain unchanged in the revised tests which were introduced in January 2018. However,
users of these tests recognised the need for more information that would serve children’s learning. Second
language learning experts have already conducted substantial research into what kinds of assessment information
best serve learning purposes and these principles were applied to developing additional feedback for the Pre A1
Starters, A1 Movers and A2 Flyers tests.
Additional feedback will be provided in a new Statement of Results that will complement candidates’ certificates.
The Statement of Results will be provided in local languages where possible, so that parents can read them. The
Statement of Results will show what kinds of communicative activities children have shown they can do, and
which kinds of activities they still need to work on. The Statement of Results will also provide some suggestions for
how children can improve their English skills.
These more detailed, descriptive and personalised reporting practices are designed specifically to support children,
teachers and families in learning English. It is hoped that test users will find them useful, and that children will be
even more enthusiastic and confident in their English studies.

6
REFERENCES
Bachman, L F (2005) Building and supporting a case for test use, Language Assessment Quarterly 2 (1), 1–34.
Clark, I (2012) Formative assessment: Assessment is for self-regulated learning, Educational Psychology Review
24 (2), 205–249.
Council of Europe (2001) Common European Framework of Reference for Languages: Learning, Teaching,
Assessment, Cambridge: Cambridge University Press.
Dunlop, M (2017) Maximizing feedback for language learning: English language learners’ attention, affect, cognition
and usage of computer-delivered feedback from an English language reading proficiency assessment, Unpublished
doctoral dissertation, University of Toronto.
Fernández-Toro, M and Hurd, S (2014) A model of factors affecting independent learners’ engagement with
feedback on language learning tasks, Distance Education 35 (1), 106–125.
Fulcher, G and Davidson, F (2007) Language Testing and Assessment: An Advanced Resource Book,
London: Routledge.
Jones, N and Saville, N (2016) Learning Oriented Assessment: A Systemic Approach, Studies in Language Testing
volume 45, Cambridge: UCLES/Cambridge University Press.
Hattie, J and Timperley, H (2007) The power of feedback, Review of Educational Research 77 (1), 81–112.
Pintrich, P R (2004) A conceptual framework for assessing motivation and self-regulated learning in college
students, Educational Psychology Review 16 (4), 385–407.
Shute, V J (2008) Focus on formative feedback, Review of Educational Research 78 (1), 153–189.
Weir, C J (2005) Language Testing and Validation: An Evidence-based Approach, Basingstoke: Palgrave Macmillan.
Zimmerman, B J (2002) Becoming a self-regulated learner: An overview, Theory into Practice 41 (2), 64–70.

Improving Feedback for Young Learners: Introducing the New Statement of Results for Pre A1 Starters, A1 Movers and A2 Flyers 7
Contact us
Cambridge Assessment English cambridgeenglish.org/younglearners
The Triangle Building /cambridgeenglish
Shaftesbury Road
Cambridge /cambridgeenglishtv
CB2 8EA /cambridgeeng
United Kingdom
/cambridgeenglish
All details are correct at the time of going to print in November 2020.
/cambridge-assessment-english
Cambridge Assessment is committed to making our documents accessible in accordance with
the WCAG 2.1 Standard. We’re always looking to improve the accessibility of our documents.
If you find any problems or you think we’re not meeting accessibility requirements, contact
our team: [email protected].

If you need this document in a different format contact us telling us your name, email
address and requirements and we will respond within 15 working days. Copyright © UCLES 2020 | CER/6668/V1/NOV20