Item Writing for Language Assessment
Item writing glossary
A glossary of words and phrases relevant to item writing
[Link]
Glossary
This document contains words and phrases that are relevant to item writing and does not include
more general terms related to language testing and education. It is intended as a practical guide
to help in writing test items. Most of the words and phrases occur more than once in the test
specifications1 you will use on this course.
1
This course will use test specifications from the Aptis General test throughout.
Abstract
An abstract idea “is based on general ideas rather than on real things and events.” (see
concrete). Existing in thought or as an idea but not having a physical or concrete existence. For
example, describing thoughts on the importance of the media or contrasting entertainment
today with that of 50 years ago.
Authentic
Genuine, based on observation or data taken from the real world, outside of the classroom or
other setting in which speech or other text is produced for the purposes of education, testing,
etc. A quality of the text that means the language used is typical of a particular context or genre
and is “not designed or recorded for non-native speakers, or for language learning purposes”.
example usage from specs: “Although not intended to be authentic, they should reflect features
of relevant texts from the TLU domain.”
BYU-BNC resource
[Link]
Cognitive processing
The mental work that is needed in order to understand an idea or piece of language. The type
(difficulty) of cognitive processing required in different language tasks/tests will vary according
to how much information needs to be accessed (recalled and/or synthesised) by test takers.
Cohesion
“The use of linguistic devices to join sentences together” example usage from specs: “the
cohesion/links between information will utilise referential links.”
Cohesive devices
Used to describe the way a text is structured to form a uniform whole. Cohesive devices include
pronouns, synonyms and conjunctions.
Collocation
The habitual juxtaposition of a particular word with another word or words with a frequency
greater than chance.
Comprehension
The ability to understand something such as written or spoken language.
Concrete
Used to describe information that is real and perceptible (e.g. colour, size, cost, age etc. - see
abstract). A familiar topic, for example asking the candidate to describe what they are wearing,
talk about their last holiday and so on.
Construct (Test construct)
What is being measured by the test; those aspects of the candidate’s underlying knowledge or
skill which are the target of measurement in a test.
Construct irrelevant variance
Differences in the performance of candidates on a test which are unrelated to the ability or skill
being measured.
Construct under representation
“The failure of a test to adequately capture the complexity of the communicative demands of the
criterion [the domain of behaviour relevant to test design]; oversimplification of the construct.”
Context setter
The sentence that gives the candidate a context for listening (it comes before the stem).
Cultural specificity
The quality of belonging or relating uniquely to a particular culture. It can be tagged as either
specific (clearly bound to a particular culture) or neutral (not associated with any culture - see
neutral).
Cumulative token percentage
A measurement of the lexical profile of a text in terms of K lists. You reach the cumulative token
percentage by adding the percentage of individual K list words (tokens) in a text. A text that
comprises the following lexical profile:
K1 = 70% K2 = 20% K3 = 5%.
will have a cumulative token percentage of 95% at K3.
Decoding (listening)
The process of trying to understand the meaning of a word, phrase, or sentence; meaning
construction based on sound perception and word and phrase recognition.
Discourse
A connected series of utterances that make up a text or a conversation (or other spoken
contexts).
Domain
A real-world context or field where a particular type of language may be encountered
(educational, occupational etc.) An area of knowledge or skill. Example from the specs: “target
language use domain”
Elicit
A stimulus, instruction or other prompt elicits (produces in a test taker) examples of the
language output that is to be tested. Example usage from specs: “The task is designed to elicit
expeditious global reading of longer expository and argumentative texts.”
Ellipsis
“The omission of parts of structures that would usually be expected”
Explicit
Definite, clear and detailed, without any hidden meaning; not requiring the imagination or
interpretation by the listener/reader (see implicit).
Exponents
Individual parts of grammar that have been listed, numbered and separated according to CEFR
level.
Genre
A style or category of text or speech. For example, telephone message, magazine article, radio
interview.
Gist
The main idea(s) contained in a text that can be understood by reading/listening to and
comprehending the text’s meaning [aka expeditious global reading or listening].
Guidelines on Adhering to Lexical Level
Words outside the specified level may be considered for use provided all of the following criteria
are met: 1) Less than 5% of text and are not key or targeted words;
2) There is a clear, logical argument supporting the use;
3) Quantitative evidence to justify the use at this level is obtained from 1 of the subsidiary
vocabulary level resources.
Item writers must demonstrate in writing all 3 criteria above when submitting an item with
vocabulary outside the lexical level.
Guidelines on Adhering to Grammatical Level
A grammatical exponent outside the level may be considered provided all of the following
criteria are met:
1) from an adjacent level (e.g. A2 for an A1 item);
2) an isolated usage;
3) the other features of the text support the usage and ensure the accessibility of the text for
test takers at this level;
4) the exponent is not used for a key piece of information;
5) rewriting would result in unnatural/awkward usage.
Item writers must indicate such cases clearly and attach a written justification for the usage.
Idiomatic
An expression which functions as a single unit and whose meaning cannot be worked out from
its separate parts. For example, ‘pass an exam with flying colours’ - it means do very well in an
exam but cannot easily be guessed from a literal understanding of the words in the expression.
Implicit
Not directly expressed; requiring the imagination or interpretation of the listener/reader (see
explicit).
Inferencing
A conclusion or judgment that is formed based on evidence and reasoning rather than on the
basis of direct observation.
A: Would you like another piece of pie?
B: No, thanks.
A: But I thought you were hungry
B: I am, but….
From the above exchange, we can infer that B did not like the pie. [Note. This is based on the
information available. This conclusion may, in fact, be incorrect, but such conclusions can only
be made on the information available.]
B: …. but I have to go. I’m late
A: Would you like to take some pie with you?
B: I’d love to. With this additional information, we can infer that B did, in fact, like the pie.
Input
Used to describe the stimulus presented to a test taker to which they are required to respond.
For example, in a listening test the input is the audio material that the test taker listens to. (see
also response format).
Integrate
To put together parts and combine them into a whole.
Lexical Levels
To check words by K list, visit, [Link]
1. Click on “Vocabprofile” in the second column from the left. A new page will open;
2. Click on the words BNC-20 in the yellow box under the heading “VP-Compleat”. A new page
will open; 3. Select BNC 1-20k from “framework” options on the top right side;
4. Delete the instructions text in the window;
5. Copy and paste the words you are checking into the window;
6. Click on the Submit button;
7. The profile of the words will appear in a table format. The words are colour coded (e.g. blue =
K1, green = K2 etc).
Lexical matching
Identifying a word, phrase, sentence or passage of text that is directly equivalent to or refers
explicitly to a separate word, phrase, sentence or passage of text, without requiring inference.
Linear sequence
In the reading task 2 there has to be a clear linear sequence that the different parts of the story
must fit into. It clearly has a beginning (the example) and then everything that follows must do
so based on meaning so for this task the linear sequence is recognised by understanding how
the different stages of the story fit together, as well as by cohesive devices
Macro-propositions
Longer texts may contain more than one main idea, instead having two or more central ideas
that are spread throughout the text. Example usage from specs: “...macro-propositions of each
paragraph”
Meaning
What is intended by a word, text, concept, or action (see also decoding).
Mode
Types of discourse (e.g. Argumentative; Descriptive; Expository; Narrative; Instructive).
Neutral
Having no strongly marked characteristics or features. For example, an item that deals with
child-parent relationships is likely to be culturally neutral as these relationships occur in all
cultures; however, an item about going to a pub is not culturally neutral as pubs don’t exist in all
countries and not all cultures have (or approve of) the custom of socialising and drinking alcohol
publicly.
Overlap
The appearance of information (or lexis) in more than one place in a text (e.g. in a heading and
in a paragraph or across paragraphs).
Paraphrase
An expression of the meaning of a word or phrase using other words or phrases.
Plausible
The possibility that an option (or distractor), based on the context and other lexico-grammatical
clues, could reasonably be chosen by a candidate of the level for whom the item is intended.
Example usage from specs: “predict the correct answer (or a range of plausible alternatives)”
Pretesting
All pre-tests that take place before an exam is launched (‘goes live’).
Prompt
The stimulus to which the candidate responds in speaking or writing.
Proposition(s)
A statement or assertion that expresses a judgement or opinion. The underlying meaning(s),
argument(s) or concept(s) that a reader/listener infers by reading or listening to a sentence,
paragraph or longer text. Example from the specifications: “textual inferencing to integrate
propositions across utterances.”
Resources section
[Link]
[Link]
Response format
Used to describe the form in which the test taker can give their response to an item (e.g. a
multiple-choice question) (see also input).
Rubric
The instructions to candidates before each question or task. The instructions in tests which
indicate to the student what he or she has to do to complete a task or activity. In the USA this
word has a completely different meaning and relates to how the test is scored.
Semantic/ lexical fields
A grouping of words into a ‘family’ with related meanings and/or uses. Example usage from
specs: “choose/avoid choosing words from the same semantic/lexical fields”
Stem
This provides a context sentence in gap fill questions (grammar and vocabulary) and a short
background for listening questions.
Syntactic
Used to describe the manner in which an utterance conforms to the expected rules of structure
(i.e. word order) in a language.
Syntactic parsing
What listeners and readers do as they make sense of an extended period of text (spoken or
written) in order to give meanings to (decode) sentences and utterances.
Task(s)
An activity that involves individuals in using language for the purpose of achieving a particular
goal or objective in a particular situation.
Utterance
An uninterrupted chain of spoken or written language e.g. a spoken or written clause or
sentence. A sequence of words within a single person’s turn at talking.
Verbatim
In exactly the same words as were used originally. Sound perception and word and phrase
recognition
