ASSESSING READING Even as we are bombarded with an unending supply of visual and auditory media, the written word continues in its function to convey information, to amuse and entertain us, to codify our social, economic, and legal conventions, and to fulfill a host of other functions. In literate societies, most “normal” children learn to read by the age of five or six, and some even earlier. With the exception of a small number of people with learning disabilities, reading is a skill that is taken for granted. In foreign language learning, reading is likewise a skill that teachers simply expect learners to acquire. Basic, beginning-level textbooks in a foreign language presuppose a student's reading ability if only because it’s a book that is the medium. Most formal tests use the written word as a stimulus for testtaker response; even oral interviews may require reading performance for certain tasks. Reading, arguably the most essential skill for success in all educational contexts, remains a skill of para- mount importance as we create assessments of general language abi Is reading so natural «ind normal that learners should simply be exposed to written texts with no particular instruction? Will they just absorb the skills neces- sary to convert their perception of a handful of letters into meaningful chunks of information? Not necessarily. For learners of English, rwo primary hurdles must be cleared in order to become efficient readers. First, they need to be able to master fundamental bottom-up strategies for processing separate letters, words, and phrases, as well as top-down, conceptually driven strategies for comprehension. Second, as part of that top-down approach, second language readers must develop appropriate content and formal schemata—background information and cultural experience—to carry out those interpretations effectively. The assessment of reading ability does not end with the measurement of com- prehension. Strategic pathways to full understanding are often important factors to include in assessing learners, especially in the case of most classroom assessments that are formative in nature. An inability to comprehend may thus be traced to a need to enhance a test-taker's strategies for achieving ultimate comprehension. For example, an academic technical report may be comprehensible to a student at the sentence level, but if the learner has not exercised certain strategies for noting the discourse conventions of that genre, misunderstanding may occur. 185186 TYPES (GENRES) OF READING CHAPTER B Assessing Reading As we consider a number of different types or genres of written texts, the com- ponents of reading ability, and specific tasks that are commonly used in the assess- ment of reading, let's not forget the unobservable nature of reading. Like listening, one cannot see the process of reading, nor can one observe 4 specific product of reading, Other than observing 2 reader's eye movements and page turning, there is no technology that enables us to "see" sequences of graphic symbols traveling from the pages of a book into compartments of the brain (in a possible bottom-up process). Even more outlandish is the notion that one might be able to watch infor- mation from the brain make its way down onto the page (in typical top-down strate- gies). Further, once something is read—information from the written text is stored—no technology allows us to empirically measure exactly what is lodged in the brain. All assessment of reading must be carried out by inference. Each type or genre of written text has its own set of governing rules and conven- tions. A reader must be able to anticipate those conventions in order to process meaning efficiently. With an extraordinary number of genres present in any literate culture, the reader's ability to process texts must be very sophisticated. Consider the following abridged list of common genres, which ultimately form part of the speci- fications for assessments of reading ability. Genres of reading 1. Academic reading general interest articles (in magazines, newspapers, etc.) technical reports (e.g,, lab reports), professional journal articles reference material (dictionaries, etc.) textbooks, theses essays, papers test directions editorials and opinion writing 2. job-related reading messages (e.g., phone messages) letters/emails memos (e.g,, interoffice) reports (e.g., job evaluations, project reports) schedules, labels, signs, announcements forms, applications, questionnaires financial documents (bills, invoices, etc.) directories (telephone, office, etc.) manuals, directions‘CHAPTER 8 Assessing Reading 187 =com- assess: 3. Personal reading ening, newspapers and magazines \uct of letters, emails, greeting cards, invitations here is messages, notes, lists g from schedules (train, bus, plane, etc.) omup recipes, menus, maps, calendars 1 infor- advertisements (commercials, want ads) strate- novels, short stories, jokes, drama, poetry text is financial documents (e.g., checks, tax forms, loan applications) ged in forms, questionnaires, medical reports, immigration documents comic strips, cartoons When we realize that this list is only the beginning, it is easy to see how over- whelming it is to learn to read in a foreign language! The genre of a text enables conven readers to apply certain schemata that will assist them in extracting appropriate process meaning. If, for example, readers know that a text is a recipe, they will expect a cer literate tain arrangement of information (ingredients) and will know to search for a sequen- ider the tial order of directions. Efficient readers also have to know what their purpose is in e speci reading a text, the strategies for accomplishing that purpose, and how to retain the information The content validity of an assessment procedure is largely established through the genre of a text. For example, if learners in a program of English for tourism have been learning how to deal with customers needing to arrange bus tours, then assess- ments of their ability should include guidebooks, maps, transportation schedules, calendars, and other relevant texts. MICROSKILLS, MACROSKILLS, AND STRATEGIES FOR READING Aside from attending to genres of text, the skills and strategies for accom reading emerge as a crucial consideration in the assessment of reading abi micro- and macroskills below represent the spectrum of possibilities for objectives in the assessment of reading comprehension. Micro- and macroskills for reading comprehension Microskills 1. Discriminate among the distinctive graphemes and orthographic patterns of English. 2, Retain chunks of language of different lengths in short-term memory. 3. Process writing at an efficient rate of speed to suit the purpose.188 cunpren 8 Assessing Reading 4. Recognize a core of wards, and interpret word order patterns and their significance. 5. Recognize grammatical word classes (nouns, verbs, etc.), systems (e.g., tense, agreement, pluralization), patterns, rules, and elliptical forms. 6. Recognize that a particular meaning may be expressed in different grammatical forms. 7. Recognize cohesive devices in written discourse and their role in signaling the relationship between and among clauses. Macroskills 8. Recognize the rhetorical forms of written discourse and their significance for interpretation, 9. Recognize the communicative functions of written texts, according to form and purpose. | 10. Infer context that is not explicit by using background knowledge. 11. From described events, ideas, etc,, infer links and connections between events, deduce causes and effects, and detect such relations as main idea, supporting idea, new information, given information, generalization, and exemplification. 12. Distinguish between literal and implied meanings. 13. Detect culturally specific references and interpret them in a context of the appropriate cultural schemata. 14. Develop and use a battery of reading strategies, such as scanning and skimming, detecting discourse markers, guessing the meaning of words from context, and activating schemata for the interpretation of texts. ‘The assessment of reading can imply the assessment of a storehouse of rea strategies, as indicated in item #14. Aside from simply testing the ultimate achieve. ment of comprehension of a written text, it may be important in some contexts assess one or more of a storchouse of classic reading strategies, The brief taxon of strategies below is a list of possible assessment criteria. Some principal strategies for reading comprehension 1. Identify your purpose in reading a text, 2. Apply spelling rules and conventions for bottom-up decoding. 3. Use lexical analysis (prefixes, roots, suffixes, etc.) to determine meaning. 4. Guess at meaning (of words, idioms, etc.) when you aren’ certain. 5. Skim the text for the gist and for main ideas, 6. Scan the text for specific information (names, dates, key words). 7. Use silent reading techniques for rapid processing,CHAPTER 8 Assessing Reading 189 8. Use marginal notes, outlines, charts, or semantic maps for understanding, and retaining information. 9. Distinguish between literal and implied meanings. 10. Capitalize on discourse markers to process relationships. TYPES OF READING In the previous chapters we saw that both listening and speaking could be subdivided into at least five different types of listening and speaking performance. In the case of reading, variety of performance is derived more from the multiplicity of types of texts (the genres listed above) than from the variety of overt types of performance. Never- theless, for considering assessment procedures, several types of reading performance are typically identified, and these will serve as onganizers of various assessment tasks. 1. Perceptive. In keeping with the set of categories specified for listening com- prehension, similar specifications are offered here, except with some differing ter- minology to capture the uniqueness of reading. Perceptive reading tasks involve attending to the components of larger stretches of discourse: letters, words, punctu- ation, and other graphemic symbols. Bottom-up processing is implied. 2. Selective. This category is largely an artifact of assessment formats. In order to ascertain one’s reading recognition of lexical, grammatical, or discourse features of language within a very short stretch of language, certain typical tasks are used: picture-cued tasks, matching, true/false, multiple-choice, etc. Stimuli include sen- tences, brief paragraphs, and simple charts and graphs, Brief responses are intended as well. A combination of bottom-up and top-down processing may be used reading 3. Interactive, Included among interactive reading types are stretches of lan- achieve- guage of several paragraphs to one page or more in which the reader must, in a psy- bits ta cholinguistic sense, interact with the text. That is, reading is a process of negotiating aa meaning; the reader brings to the text a set of schemata for understanding it, and in- take is the product of that interaction. Typical genres that lend themselves to inter- active reading are anecdotes, short narratives and descriptions, excerpts from longer texts, questionnaires, memos, announcements, directions, recipes, and the like. The focus of an interactive task is to identify relevant features (lexical, symbolic, gram- matical, and discourse) within texts of moderately short length with the objective of retaining the information that is processed. Top-down processing is typical of such tasks, although some instances of bottom-up performance may be necessary. ing 4. Extensive. Extensive reading, as discussed in this book, applies to texts of more than a page, up to and including professional articles, essays, technical re- ports, short stories, and books, (It should be noted that reading research commonly refers to “extensive reading” as longer stretches of discourse, such as long articles and books that are usually read outside a classroom hour. Here that definition is190 carmen 8 Assessing Reading massaged a little in order to encompass any text longer than a page.) The purposes of assessment usually are to tap into a learner's global understanding of a text, as opposed to asking test-takers to“zoom in" on small details. Top-down processing is assumed for most extensive tasks. ‘The four types of reading are demonstrated in Figure 8.1, which shows the rela- tionships of length, focus, and processing mode among the four types. Length Focus Process Short | Medium | Long | Form | Meaning | Bottom-Up | Top-Down Perceptive Selective + Interactive Extensive ‘+= strong emphasis += moderate emphasis Figure 8.1. Types of reading by length, focus, and process DESIGNING ASSESSMENT TASKS: PERCEPTIVE READING At the beginning level of reading a second language lies a set of tasks that are fae damental and basic: recognition of alphabetic symbols, capitalized and | letters, punctuation, words, and grapheme-phoneme correspondences. Such 12% of perception are ofien referred to as literacy tasks, implying that the learner is the early stages of becoming ‘literate.’ Some learners are already literate in their c native language, but in other cases the second language may be the first lang’ that they have ever learned to read, This latter context poses cognitive and s times age-related issues that need to be considered carefully. Assessment of li is no casy assignment, and if you are interested in this particular challenging = further reading beyond this book is advised (Harp, 1991; Farr &Tone, 1994; Gen 1994; Cooper, 1997). Assessment of basic reading skills may be carried out = number of different ways. Reading Aloud The testtaker sees separate letters, words, and/or short sentences and reads © aloud, one by one, in the presence of an administrator. Since the assessment = reading comprehension, any recognizable oral approximation of the response is considered correct.CHAPTER 6 Assessing Reading 191 Written Response ‘The same stimuli are presented, and the testtaker's task is to reproduce the probe in writing. Because of the transfer across different skills here, evaluation of the test- taker’s response must be carefully treated. If an error occurs, make sure you deter- mine its source; what might be assumed to be a writing error, for example, may actually be a reading error, and vice versa, Multiple-Choice Multiple-choice responses are not only a matter of choosing one of four or five pos sible answers. Other formats, some of which are especially useful at the low levels of reading, include same/different, circle the answer, true/false, choose the letter,and matching. Here are some possibilities Minimal pair distinction Test-takers read:* Gircle “S" for same or *D" for different. 4. led et s 0D 2. bit bit s D 3, seat set s oD 4. too to 5s D *In the case of very low level learners, the teacher/administrator reads directions. Grapheme recognition task Test-takers read:* Circle the “odd!” item, the one that doesn't “belong.” 1. piece Peace piece 2. book book boot “In the case of very low level learners, the teacher/administrator reads directions. Picture-Cued Items Testtakers are shown a picture, such as the one on the next page, along with a written text and are given one of a number of possible tasks to perform.192 cuapree 8 Assessing Reading Picture-cued word identification (Brown & Sahni, 1994, p. 124) Test-takers hear: Point to the word that you read here. cat clock chair Picture-cued sentence identification With the same picture, the test-taker might read sentences and then point to the rect part of the picture: Test-takers hear: Point to the part of the picture that you read about here. The man is reading a book. Test-takers see the picture and read each sentence written on a separate card. The cat is under the tablro the cor- card. CHAPTER 8 Assessing Reading 193 Ora true/false procedure might be presented with the same picture cue: Picture-cued true/false sentence identification Test-takers read: 1. The pencils are under the table. cy 2. The cat is on the table, T 3. The picture is over the couch. G a7 Matching can be an effective method of assessing reading at this level. With objects labeled A, B, C, D, E in the picture, the testtaker reads words and writes the appropriate letter beside the word: Picture-cued matching word identification Test-takers read: clock chair books cat table GTO s Finally, test-takers might see a word or phrase and then be directed to choose one of four pictures that is being described, thus requiring the test-taker to transfer from a verbal to a nonverbal mode. In the following item, test-takers choose the cor rect letter: ‘Multiple-choice picture-cued word identification Test-iakers read: Rectangle Test-takers see, and choose the correct item: & ORY194 coarite 8 Assessing Reading DESIGNING ASSESSMENT TASKS: SELECTIVE READING Just above the rudimentary skill level of perception of letters and words is a cate- gory in which the test designer focuses on formal aspects of language (lexical, gram- matical,and a few discourse features). This category includes what many incorrectly think of as testing “vocabulary and grammar” How many textbooks provide little tests and quizzes labeled “vocabulary and grammar” and never feature any other skill besides reading? Lexical and grammatical aspects of language are simply the forms ‘we use to perform all four of the skills of listening, speaking, reading, and writing. (Notice that in all of these chapters on the four skills, formal features of language have become a potential focus for assessment.) Here are some of the possible tasks you can use to assess lexical and grammat- ical aspects of reading ability. Multiple-Choice (for Form-Focused Criteria) By far the most popular method of testing a reading knowledge of vocabulary and grammar is the multiple-choice format, mainly for reasons of practicality:it is easy to administet and can be scored quickly. The most straightforward. multiple-choice items may have little context, but might serve as a vocabulary or grammar check. Multiple-choice vocabulary/grammar tasks 1. He's not married. He's ‘A. young B. single C. first D. a husband 2. If there's no doorbell, please ‘on the door. A. kneel B. type C. knock D. shout 3. The mouse is the bed, A. under 4, The bank robbery occurred | was in the restroom, that during while which