Major 5: Introduction to the Assessment of Language Prepared by: JOMEL B.

MANUEL, PhD
Associate Professor 4

Lesson 4

PRINCIPLES OF TESTING

Introduction
Language testing at any level is a highly complex undertaking that must be based on theory
as well as practice. Although trends in testing, as in other fields, change over time, some
principles of testing/assessment are permanent and are not overly affected by current
fashions. They need to be held in mind by assessors whenever they construct a test, whether
this be a class quiz, a class essay or an end-of-year examination. Thus, this lesson explores
how principles of language assessment/testing can should be applied to format tests.

Learning Outcomes
At the end of the lesson, you are expected to:

a. Discuss the basic principles of testing and

b. Evaluate practices in testing by analyzing a case study.

Discussion
Principles/Cornerstones of Testing

1.Usefulness

- the most important quality or cornerstone of testing.

-Bachman and Palmer’s model of test usefulness requires that any language test
must be developed with a specific purpose, a particular group of test-takers, and
a specific language use in mind.

2.Validity

- refers to the extent to which a test measures what it purports to measure

- Types of validity: content, construct, face validity.

a) Content validity means that the test assesses the course content and
outcomes using formats familiar to the students.

1| Bachelor of Arts in English Language Studies, College of Arts and Sciences, CSU-Carig Campus, Tuguegarao City
No part of this module may be reproduced in any form or by any means without permission from the author.
Major 5: Introduction to the Assessment of Language Prepared by: JOMEL B. MANUEL, PhD
Associate Professor 4

b) Construct validity refers to the “fit” between the underlying theories and
methodology of language learning and the type of assessment.

c) Face validity means that the test looks as though it measures what it is
supposed to measure.
-It is purely a factor of the “eye of beholder”- how the test-taker or possibly
the test giver, intuitively perceives the instrument.

3.Reliability

- refers to the consistency of test scores, which simply means that a test would
offer similar results if it were given at another time.

- 3 important factors affect test reliability: test factors, administrative factors,

affective factors

a) Test factors- formats and content of the questions and the time
given for students to take the exam must be consistent.
-longer exams produce more reliable results.

b) Administrative factors- classroom setting (seating arrangement,

lighting) and how the teacher manages the administration of the
exam.
c) Affective factors in response of individual students can also affect
reliability, as can fatigue, personality type, and learning style.
-test anxiety should be reduced.

-Henning(1987) describes threats to test reliability:

a) Fluctuations in the Learner. A variety of changes may take place
within the earner that may change a learner’s true score from test to
test.
-fatigue, sickness, emotional problem, and practice effect may cause
the learner’s score to deviate from the score that reflects his or her
actual ability.
-Practice effect means that a students’ score could improve because
he or she has taken the test so many times that the content is familiar.

b) Fluctuations in Scoring. Subjectivity in scoring or mechanical errors in

the scoring process may introduce error into scores and affect the
reliability of the test’s results.
- These kinds of errors usually occur withing (intra-rater/or between (inter-
rater) the raters themselves.
2| Bachelor of Arts in English Language Studies, College of Arts and Sciences, CSU-Carig Campus, Tuguegarao City
No part of this module may be reproduced in any form or by any means without permission from the author.
Major 5: Introduction to the Assessment of Language Prepared by: JOMEL B. MANUEL, PhD
Associate Professor 4

c)Fluctuations in Test Administration- inconsistent administrative

procedures and testing conditions.
-This is most common in institutions where different groups of students
are tested in different locations on different days.

Sources of unreliability:

a. Features of test- unclear instructions, ambiguous questions

b. Scoring as a source of unreliability- if same test is given totally different
score by different markers, or different score by same marker on
different occasions

4.Practicality
- A good classroom test should be “teacher-friendly.”
- A teacher should be able to develop, administer, and mark it within the
available time and with available resources.

5.Washback
- refers to the effect of testing on teaching and learning.
- is generally said to be positive or negative.

6.Authenticity
- Language learners are motivated to perform when they are faced with
tasks that reflect real-world situations and contexts.

-Authentic materials at the appropriate language level provides additional

exposure to concepts and vocabulary as students will encounter them in
real-life situations.

7.Transparency
-refers to the availability of clear, accurate information to students about testing.

-Such information should include outcomes to be evaluated, formats used,

weighting of items and sections, time allowed to complete the test, and grading
criteria.

8. Security
- If a teacher invests time and energy in developing good tests that accurately
reflect the course outcomes, then it is desirable to be able to recycle the test
materials.
3| Bachelor of Arts in English Language Studies, College of Arts and Sciences, CSU-Carig Campus, Tuguegarao City
No part of this module may be reproduced in any form or by any means without permission from the author.
Major 5: Introduction to the Assessment of Language Prepared by: JOMEL B. MANUEL, PhD
Associate Professor 4

Ten Things to Remember!

1. Test what has been taught and how it has been taught.
2. Set tasks in context whenever possible.
3. Choose formats that are authentic for tasks and skills.
4. Specify the material to be tested.
5. Acquaint students with techniques and formats prior to testing.
6. Administer the test in uniform, non-distracting conditions.
7. Provide timely feedback.
8. Reflect on the exam without delay.
9. Make changes based on analyses and feedback from colleagues and
students.
10. Employ multiple measures assessment in your classes.

Time to Answer:
Read this case study about Mr. Knott, a colleague of Ms. Wright’s, and try to spot
the cornerstones violations. What could be done to solve these problems?

Background Information

Mr. Knott is a high school ESL and Spanish teacher. His current teaching load is two
ESL classes. His students come from many language backgrounds and cultures. in
his classes, he uses an integrated-skills textbook that espouses a communicative
methodology.

His Test

Mr. Knott firmly believes in the KISS philosophy of “keeping it short and simple.”
Most recently he has covered modal verbs in his classes. He decides to give his
students only one question to test their knowledge about modal verbs: “Write a
300-word essay on the meanings of modal verbs and their stylistic uses. Give
examples and be specific.” Because he was short of time, he distributed a
handwritten prompt on unlined paper. Incidentally, he gave this same test last
year.

4| Bachelor of Arts in English Language Studies, College of Arts and Sciences, CSU-Carig Campus, Tuguegarao City
No part of this module may be reproduced in any form or by any means without permission from the author.
Major 5: Introduction to the Assessment of Language Prepared by: JOMEL B. MANUEL, PhD
Associate Professor 4

Information Given to Students

To keep students on their toes and to increase attendance, he told them that the
test could occur anytime during the week. Of his two classes, Mr. Knott has a
preference for his morning class because they are much more well behaved and
hard working so he hinted during the class that modal verbs might be the focus of
the test. His afternoon class received no information on the topic of the test.

Test Administration Procedures

Mr. Knott administered his test to his afternoon class on Monday and to his morning
class on Thursday. Always wanting to practice his Spanish, he clarified the
directions for his Spanish-speaking students in Spanish. During the Monday
administration, his test was interrupted by a fire drill. Since this was the first time a
fire drill had happened, he did not have any back-up plan for collecting test
papers. Consequently, some students took their papers with them. In the
confusion, several test papers were mislaid.

Grading Procedures

Mr. Knott didn’t tell his students when to expect their results. Due to his busy
schedule, he graded tests over several days during the next week. Students finally
got their tests back ten days later. Because the test grades were extremely low,
Mr. Knott added ten points to everyone’s paper to achieve a good curve.

Post-Exam Follow-Up Procedures

Mr. Knott entered grades in his grade book but didn’t annotate or analyze them.
Although Mr. Knott announced in class that the exam was worth 15 percent of
the students’ grade, he downgraded it to five percent. Next year he plans to
recycle the same test but will require students to write 400 words.

5| Bachelor of Arts in English Language Studies, College of Arts and Sciences, CSU-Carig Campus, Tuguegarao City
No part of this module may be reproduced in any form or by any means without permission from the author.