GUID 203 ADVANCED PSYCHOLOGICAL TESTING
Chapter 7
Test Administration
Learning
Objectives:
> Discuss how the relationship between the examiner and the test
taker can affect test scores
> Explain how an expectancy effect might affect a test score
> Examine the relationship between test performance and reinforcing
particular responses
> identify some of the advantages of computer-assisted test
administration
> List what characteristics of the state of the subject should be
considered when a test score is evaluated
Test Administration
Test administration is the process of administering tests to individuals or groups
of people. A test score should help us predict how the client will feel and act
outside the clinic, how the student will achieve in his studies, how a colleague
will perform in the job.
Test administration is an important process that ensures that tests are
administered in a consistent and standardized manner, so that the results of the
test are accurate and meaningful
It is an important process that helps to ensure that tests are used effectively
and responsibly.
In Guidance and Counseling, testing is done to identify the strength and
weaknesses in areas of personality, aptitudes, interests, needs and jobs skills
are assessed through the use of standardized tests. The results will be
interpreted to the clients for self-awareness, growth and development.
The Relationship Between the Examiner and Test Taker:
The relationship between the examiner and the test taker is an important factor in the test
administration process. The nature of this relationship can impact the test-taker's performance on
the test and their experience of the testing process. Here are some ways in which the relationship
between the examiner and the test-taker can influence the testing process:
Trust and rapport: A positive relationship between the examiner and the test-taker can help to
establish trust and rapport. This can help to reduce anxiety and increase the test-taker's confidence
in their ability to perform well on the test.
In one older study, first- through seventh-grade children were given the Wechsler Intelligence Scale
for Children under one of two conditions. Half of the children were given the test under an
enhanced rapport condition in which the examiner used friendly conversation and verbal
reinforcement during the test administration. The other children took the test under a neutral
rapport condition in which the examiner neither initiated conversation nor used reinforcement
(Feldman & Sullivan, 1960). The examiner’s rapport had little effect on the scores of the younger
children (through third grade).
However, average IQ scores for the fifth-grade through ninth-grade students were higher for those
who had received the test under the enhanced rapport condition (mean IQ = 122) than for those
with a neutral administrator (mean IQ = 109). This difference (122 − 109) is almost a full standard
deviation.
The Relationship Between the Examiner and Test Taker:
Another study compared scores obtained by examiners who made approving comments (such as “good” or “fine”)
with scores obtained by examiners who used disapproving comments (“I thought you could do better than that”)
or neutral comments. Children who took the test under a disapproving examiner received lower scores than did
children exposed to a neutral or an approving examiner (Witmer, Bernstein, & Dunham, 1971). For younger
children, a familiar examiner may make a difference. In one study, 137 children took a reading test, half with a
familiar proctor, half with an unfamiliar proctor. Reading scores were significantly lower when the proctor was
unfamiliar (DeRosa & Patalano, 1991).
If the examiner and test taker have a good rapport, it can make the test taker more comfortable
and confident during the test. This can result in higher scores. A good rapport and trust between
the examiner and the test-taker can lead to open communication and a better understanding of
the test instructions, resulting in higher-quality responses.
In most testing situations, examiners should be aware that their rapport with test takers can
influence the results. They should also keep in mind that rapport might be influenced by
subtle processes such as the level of performance expected by the examiner.
Overall, a positive and respectful relationship between the examiner and the test-
taker can create a supportive environment that helps the test-taker perform to the
best of their abilities. A negative or strained relationship, on the other hand, can
negatively impact the test-taker's motivation, anxiety levels, and perception of the
fairness of the testing process.
The Race of the Tester:
Because of concern about bias, the effects of the tester’s race have generated considerable
attention.
If the examiner has a bias towards or against the test taker, it can affect the way they score
the test. For example, if the examiner has a personal dislike for the test taker, they may give
lower scores than they deserve.
Some groups feel that their children should not be tested by anyone except a member of
their own race
For example, some people claim that African American children receive lower test scores
when tested by white examiners than by examiners of their own race. Although the effects
of racial bias in test administration are discussed frequently, relatively few experimental
studies have examined the exact impact of these effects. Sattler reviewed such effects on
several occasions (Sattler, 2002, 2004).
Even so, after a detailed review of the literature, Sattler (1988) concluded that the effects
of administrators’ race are negligible. Even though race effects in test administration may
be relatively small, efforts must be made to reduce all potential bias.
It is important that the test administration is conducted in a standardized and unbiased
manner to ensure that the results are accurate and reliable. This means that all test takers
should be given the same instructions, materials, and testing conditions to minimize any
potential sources of error or bias
Language of Test Taker:
The test taker's language can influence both the test administration process and the test
results. It is critical to ensure that the test taker understands the language in which the test is
given in order to accurately measure their knowledge or skills.
Some of the new standards concern testing individuals with different linguistic backgrounds.
The standards emphasize that some tests are inappropriate for people whose knowledge of
the language is questionable.
For example, the validity and reliability of tests for those who do not speak English is
suspect. Translating tests is difficult, and it cannot be assumed that the validity and
reliability of the translation are comparable to the English version. Concern about the
internal validity of research studies often compromises external validity (Okazaki & Sue,
2003; Sue, 2003).
External validity concerns the use of research findings in groups other than those who
participated in the original validation studies. The e standard is that, for test takers who are
proficient in two or more languages, the test should be given in the language that the test
takers feel is their best. Evidence for test comparability across languages should be available.
Furthermore, interpreters should be used only with great caution because test interpreters
can introduce bias into the testing situation [American Educational Research Association
(AERA), American Psychological Association (APA), & National Council on Measurement in
Education(NCME), 1999].
Training of Test Administrators:
Training test administrators is crucial for ensuring the validity and reliability of any testing process. Test
administrators are responsible for carrying out the test procedure, monitoring test-takers, and ensuring
that testing environment is conducive to fair and accurate testing.
In 1997, Spitzer et al., stated that, different assessment procedures require different levels of
training. Many behavioral assessment procedures require training and evaluation but not a
formal degree or diploma. Psychiatric diagnosis is sometimes obtained using the Structured
Clinical Interview for DSM-IV (SCID)
The Structured Clinical Interview for DSM-IV (SCID) is a semi structured interview created to make reliable
psychiatric diagnoses in adults according to the Diagnostic and Statistical Manual, fourth edition (DSM-IV).
Typical SCID users are licensed psychiatrists or psychologists with additional training on the
test. There are no standardized protocols for training people to administer complicated tests
such as the Wechsler Adult Intelligence Scale-Revised (WAIS-R;) although these tests are
usually administered by licensed psychologists
Expectancy Effects
Expectancy effects also known as experimenter effects or observer effects, refer to the phenomenon where
the expectations of the test administrator or examiner cam influence the performance or behavior of the
test takers.
A well-known line of research in psychology has shown that data sometimes can be affected
by what an experimenter expects to find. Robert Rosenthal and his colleagues at Harvard
University conducted many experiments on such expectancy effects, often called Rosenthal
effects (Rosenthal, 2002a).
The Rosenthal effect, also known as the “Pygmalion effect” is a type of expectancy effect that
occurs when an experimenter’s expectations about a participant’s behavior or performance
unintentionally influence that participant’s behavior or performance. High expectations lead
to improved performance and low expectation lead to worsened performance.
The Rosenthal effect was first described by psychologist Robert Rosenthal and his colleagues in
the 1960’s In a typical experiment, Rosenthal employed a large number of student
experimenters to help collect data on a task such as rating human faces for success or failure.
Half of the student experimenters were led to believe that the average response would fall
toward the success side of the scale, and the other half were told that the average response
would fall on the failure side. The results of these experiments have consistently demonstrated
that the subjects actually provide data that confirm the experimenter’s expectancies.
However, the magnitude of the effects is small—approximately a 1-point difference on a 20-
point scale (Rosenthal, 1966).
Expectancy Effects
Expectancies shape our judgments in many important ways (Kirsch, 1999). One of the most
important responsibilities for faculty in research-oriented universities is to apply for grant
funding. Grant reviewers are supposed to judge the quality of proposals independently of the
reputation of the applicant. However, studies suggest that reviewers’ expectancies about the
investigators do influence their judgment (Marsh & Bazeley, 1999).
There are two aspects of expectancy effect that relate to the use of standardized test.
A. The expectancy effects observed in Rosenthal’s experiments were obtained when all of the
experimenters followed a standardized script.
B. The expectancy effect has a small and subtle effect on the scores and occurs in some
situation but not in others.
The expectancy effect can impact intelligence testing in many ways, such as scoring. In a series
of experiments, graduate students with some training in intelligence testing were asked to score
ambiguous responses from intelligence tests.
Sometimes they were told that the responses had been given by “bright” people, and other
times they were told the responses were from “dull” people. The students tended to give more
credit to responses purportedly from bright test takers (Sattler, Hillix, & Neher, 1970; Sattler &
Winget, 1970). Other studies have demonstrated that the expectancy effect can occur even if
the responses are not ambiguous (Sattler, 1998).
Effects of Reinforcing Responses
Because reinforcement affects behavior, testers should always administer tests under
controlled conditions. Sattler and Theye (1967) reviewed the literature on procedural and
situational variables in testing and found that an inconsistent use of feedback can damage the
reliability and validity of test scores.
Reinforcement and feedback guide the examinee toward a preferred response. Another way to
demonstrate the potency of reinforcement involves misguiding the subject. A variety of studies
have demonstrated that random reinforcement destroys the accuracy of performance and
decreases the motivation to respond (Eisenberger & Cameron, 1998). Consider how you might
feel if the grades you received were totally random. The effects of random feedback are rather
severe, causing depression, low motivation for responding, and inability to solve problems.
This condition is known as learned helplessness (Abramson, Alloy, & Metalsky, 1995).
Learned helplessness is a psychological condition in which an individual feels unable to control
or change their circumstances even when they have the ability to do so. This was introduced by
a psychologist martin Seligman in the 1960’s following his research on dogs.
Effects of Reinforcing Responses
PSYCHOLOGICAL TESTING IN EVERYDAY LIFE 7.1
The Incentive Scoreboard
As noted in the text, because most psychologists agree that reinforcement can affect test
performance, methods are usually implemented to standardize reinforcement during
testing procedures.
However, as in most areas of psychology, there is some inconsistency in the literature.
Sattler (1988) reviewed 34 studies that evaluated the effect of incentives, which included
praise, candy, money, social reinforcement, and tokens.
The subjects in these experiments included normal and handicapped children of various
ethnic groups. By tallying the results of these studies, Sattler observed that 14 studies
found that incentives or feedback did not affect performance, 13 studies found mixed
results, and 7 studies found clear evidence that reinforcement either improved or
hindered performance.
There appeared to be no clear and consistent difference between the studies that
showed a positive effect and those that showed a negative effect of token and social
reinforcement.
Computer – Assisted Test Administration
Computer-assisted test administration refers to the use of computer technology to
administer and manage various types of tests. This can include standardized tests,
achievement tests, aptitude tests, and more.
With computer-assisted test administration, the tests are administered electronically, and
the computer software typically manages all aspects of the testing process, from test
delivery to scoring and reporting results.
This approach to testing can offer several advantages over traditional paper-and-pencil
testing. For example, it can be more efficient, as test-takers can complete the test more
quickly, and the results can be processed and reported more rapidly. Additionally,
computer-assisted testing can be more flexible and adaptable, allowing for customized
testing experiences that can be tailored to the needs of individual test-takers.
Interactive testing involves the presentation of test items on a computer terminal or personal
computer and the automatic recording of test responses. The computer can also be
programmed to instruct the test taker and to provide instruction when parts of the testing
procedure are not clear. As early as 1970, Cronbach recognized the value of computers as
test administrators
Computer – Assisted Test Administration
Not all observers endorse the rapid development of computerized test administration. For
example, J. D. Matarazzo (1986) suggested that computergenerated test reports in the hands
of an inexperienced psychologist cannot replace clinical judgment. In such cases,
computerized reports may actually cause harm if misinterpreted
Overall, computer-assisted test administration can provide a more streamlined, accurate, and
efficient way to administer tests, and is becoming increasingly popular in a wide range of
testing contexts, from educational assessments to employment screening and certification
testing.
Mode of Administration
The mode of administration in psychological testing refers to the method or format used to
administer a psychological test to a participant or test-taker.
There are several modes of administration commonly used in psychological testing, including:
>Paper-and-pencil: This mode involves providing the test-taker with a paper copy of the test
and having them complete it by filling in responses on the paper.
>Computer-based: This mode involves administering the test using a computer or other
electronic device. Test-takers may respond to questions using a keyboard, mouse, or touch
screen.
>Interview: This mode involves having a trained interviewer ask the test-taker a series of
questions, either in person or over the phone. The interviewer may record the test-taker's
responses manually or using a computer.
>Performance-based: This mode involves assessing the test-taker's performance on a task
or set of tasks, such as building a puzzle or solving a problem.
Mode of Administration
In educational testing, it is less clear that mode of test administration has a strong impact.
One study synthesized all published studies on mode of administration for reading tests
administered to K–12 students. Analysis compared performance of computer-administered
tests with those administered using traditional paper-and-pencil measures. The analysis did
not find significant differences between scores from computer-administered versus paper-
and-pencil modes (Wang, Jiao, Young, Brooks, & Olson, 2008).
The mode of administration can have important implications for the validity and reliability
of the test results. Different modes of administration may require different types of test
items and scoring methods, and may impact the test-taker's level of engagement and
motivation during the testing process. The choice of mode of administration should be
carefully considered based on the specific goals and requirements of the testing situation.
Subject Variables
A subject variable in psychological testing refers to a characteristic or trait of the participant
or subject that may influence their behavior or responses on the test. Subject variables are
typically measured or assessed prior to or in conjunction with the administration of the test.
Examples of subject variables might include age, gender, personality traits, cognitive
ability, or psychological disorders. These variables can impact how a test-taker responds
to the test items and can affect the validity and reliability of the test results.
A final variable that may be a serious source of error is the state of the subject. Motivation
and anxiety can greatly affect test scores. For example, many college students suffer from
a serious debilitating condition known as test anxiety.
It may seem obvious that illness affects test scores. When you have a cold or the flu, you
might not perform as well as when you are feeling well. Many variations in health status
affect performance in behavior and in thinking (Kaplan, 2004)
Overall, subject variables play an important role in psychological testing, and researchers
must carefully consider their effects and potential impact on test results in order to ensure
the validity and reliability of their findings.
Behavioral Assessment Methodology
Behavioral assessment is a methodology used in psychological testing that involves observing
and measuring an individual's behavior in order to make inferences about their psychological
functioning. This approach is often used to identify and diagnose psychological disorders or
other behavioral problems, as well as to evaluate the effectiveness of interventions or
treatments.
Behavioral observation raises some of the same problems faced in test administration. In
such observation, an observer records the responses of others, whereas in traditional test
taking the subject records his or her own behavior. A common problem in behavioral
observation is reactivity in which the observer is most accurate only when he or she thinks
someone is checking the work. A second problem is drift in which observers gradually come
to ignore the procedure they were taught and adopt their own observation method. A third
problem is expectancy, or the tendency for observations to be affected by what the
observer expects to observe (the Rosenthal effect).
Lack of standardization: Behavioral observation may lack standardization across different
observers or across different observation sessions, leading to inconsistencies in the data
collected and difficulties in comparing results across different contexts. Observer training
and reliability: Observers may require extensive training in order to accurately and reliably
record and interpret behavior. Differences in observer training or ability can lead to
variability in the results and can affect the validity and reliability of the data.
Overall, behavioral observation can be a valuable tool in psychological testing and research, but
it is important to be aware of these potential problems and to take steps to minimize their
impact on the results. This may include using multiple observers, standardizing the observation
process, and carefully training and monitoring observers to ensure reliability and accuracy.
That’s all…
