PRINCIPLES OF
HIGH QUALITY
ASSESSMENT
TAXONOMY OF LEARNING
DOMAINS
Instructional objectives, including behavioral
objectives, can be written for any of the domains
of instruction
• Cognitive
• Affective
• Psychomotor
The Cognitive Domain
Bloom's Taxonomy of the Cognitive Domain
(started in 1948 and completed in 1956) was one
of the most influential statements about levels of
knowing.
The official title of the book is Taxonomy of
educational objectives: The classification of
educational goals. Handbook I: Cognitive
domain with the text having 4 other authors
(M. Englehart, E. Furst, W. Hill, and D
Krathwohl).
The Cognitive Domain
the major idea of the taxonomy is that what
educators want students to know (and,
therefore, statements of educational objectives)
can be arranged in a hierarchy from less to
more complex.
The taxonomy contains six levels, with
sublevels identified for each.
The Cognitive Domain
A mnemonic device for remembering the six
levels:
Killing Knowledge
Cats Comprehension
Almost Application
Always Analysis
Seems Synthesis
Evil Evaluation
The Cognitive Domain
Student recalls or
recognizes information,
Knowledge
ideas, and principles in the
approximate form in which
they were learned.
The Cognitive Domain
Write
List
Knowledge
Label
Name
State
Define
The Cognitive Domain
The student will define the 6
Knowledge levels of Bloom's taxonomy
of the cognitive domain.
The Cognitive Domain
Student translates,
Comprehension comprehends, or interprets
information based on prior
learning.
The Cognitive Domain
Explain
Summarize
Comprehension Paraphrase
Describe
Illustrate
The Cognitive Domain
The student will explain
Comprehension the purpose of Bloom's
taxonomy of the cognitive
domain.
The Cognitive Domain
Student selects, transfers,
Application and uses data and
principles to complete a
problem or task with a
minimum of direction.
The Cognitive Domain
Use
Compute
Application Solve
Demonstrate
Apply
Construct
The Cognitive Domain
The student will write an
Application instructional objective for
each level of Bloom's
taxonomy.
The Cognitive Domain
Student distinguishes,
classifies, and relates the
Analysis assumptions, hypotheses,
evidence, or structure of a
statement or question.
The Cognitive Domain
Analyze
Categorize
Analysis Compare
Contrast
Separate
The Cognitive Domain
The student will compare
Analysis and contrast the cognitive
and affective domains.
The Cognitive Domain
Student originates,
integrates, and combines
Synthesis
ideas into a product, plan
or proposal that is new to
him or her.
The Cognitive Domain
Create
Design
Synthesis Hypothesize
Invent
Develop
The Cognitive Domain
The student will design a
classification scheme for
Synthesis writing educational
objectives that combines
the cognitive, affective, and
psychomotor domains.
The Cognitive Domain
Student appraises,
Evaluation assesses, or critiques on a
basis of specific standards
and criteria.
The Cognitive Domain
Judge
Evaluation Recommend
Critique
Justify
The Cognitive Domain
The student will judge the
Evaluation effectiveness of writing
objectives using Bloom's
taxonomy.
The Cognitive Domain
In general, research over the last 40 years has
confirmed the taxonomy as a hierarchy with the
exception of the last two levels.
It is uncertain at this time whether synthesis
and evaluation should be reversed (i.e.,
evaluation is less difficult to accomplish than
synthesis) or whether synthesis and evaluation
are at the same level of difficulty but use
different cognitive processes.
The Cognitive Domain
I believe the latter is more likely as it relates to the
differences between creative and critical thinking.
Creative Thinking Critical Thinking
Synthesis Evaluation
Analysis
Application
Comprehension
Knowledge
The Affective Domain
Being aware of or attending to
Receiving
something in the environment
Showing some new behaviors as
Responding
a result of experience
Showing some definite
Valuing
involvement or commitment
Krathwohl, D., Bloom, B., & Masia, B. (1956). Taxonomy of educational objectives.
Handbook II: Affective domain. New York: David McKay.
The Affective Domain
Integrating a new value into
Organization one's general set of values,
giving it some ranking among
one's general priorities.
Characterization Acting consistently with the new
by Value value; person is known by the
value.
The Psychomotor Domain
Process of becoming aware of
Perception objects, qualities, etc by way of
senses. Basic in situation-
interpretation-action chain
leading to motor activity.
Readiness for a particular kind
Set
of action or experience; may be
mental, physical or emotional.
Simpson, J. S. (1966). The classification of educational objectives, psychomotor
domain. Office of Education Project No. 5-85-104. Urbana, IL: University of Illinois.
The Psychomotor Domain
Overt behavioral act under
Guided guidance of an instructor, or
Response following model or set criteria.
Learned response becomes
Mechanism habitual; learner has achieved
certain confidence and
proficiency or performance.
The Psychomotor Domain
Complex Performance of motor act
Overt considered complex because of
Response movement pattern required.
Adaptation Altering motor activities to
meet demands of problematic
situations.
The Psychomotor Domain
Creating new motor acts or
ways of manipulating
Origination materials out of skills, abilities
and understandings developed
in the psychomotor area.
Writing Instructional Objectives
While it is possible to write instructional
objectives of all types for each of the three
domains, the vast majority are written for
the cognitive domain.
The major exceptions include preschool,
physical education, and perhaps fine arts
courses such as sculpturing and drama.
APPROPRIATENESS OF
ASSESSMENT METHODS
a. Written-Response Instruments
• Objective tests – appropriate for assessing the various
levels of hierarchy of educational objectives
• Essays – can test the students’ grasp of the higher
level cognitive skills
• Checklists – list of several characteristics or activities
presented to the subjects of a study, where they will
analyze and place a mark opposite to the
characteristics.
ASSESSMENT METHODS
b. Product Rating Scales
• Used to rate products like book reports, maps, charts,
diagrams, notebooks, creative endeavors
• Need to be developed to assess various products over the
years
c. Performance Tests - Performance checklist
• Consists of a list of behaviors that make up a certain type
of performance
• Used to determine whether or not an individual behaves
in a certain way when asked to complete a particular task
ASSESSMENT METHODS
d. Oral Questioning – appropriate assessment
method when the objectives are to:
• Assess the students’ stock knowledge and/or
• Determine the students’ ability to communicate
ideas in coherent verbal sentences.
e. Observation and Self Reports
• Useful supplementary methods when used in
conjunction with oral questioning and
performance tests
PROPERTIES OF ASSESSMENT
METHODS
• Validity
• Reliability
• Fairness
• Positive Consequences
• Practicality and Efficiency
• Ethics
PROPERTIES OF ASSESSMENT
METHODS
Validity &
Reliability
Defining Validity
Validity refers to the accuracy of
inferences drawn from an assessment.
It is the degree to which the assessment
measures what it is intended to measure.
Types of Validity
Construct validity- the assessment
actually measures what it is designed
to measure.
A actually is A
Types of Validity
Concurrent validity- the assessment
correlates with other assessments that
measure the same construct.
A correlates with B
Types of Validity
Predictive validity- the assessment
predicts performance on a future
assessment.
A predicts B
Valid Inferences
Validity is closely tied to the purpose or use
of an assessment.
DON’T ASK: “Is this assessment valid?”
ASK: “Are the inferences I’m making based
on this assessment valid for my purpose?”
Evidence-Centered Design
• Validity is about providing strong evidence
• Evidence-centered design boosts validity
– What do you want to know?
– How would you know?
– What should the assessment look like?
Defining Reliability
• Reliability refers to consistency and
repeatability.
• Reliability is the extent to which an
experiment, test, or any measuring procedure
yields the same result on repeated trials.
• A reliable assessment provides a consistent
picture of what students know, understand, and
are able to do.
RELIABILITY
Equivalency reliability is the extent to
which two items measure identical
concepts at an identical level of
difficulty. Equivalency reliability is
determined by relating two sets of test
scores to one another to highlight the
degree of relationship or association.
RELIABILITY
Stability reliability (sometimes called
test, re-test reliability) is the
agreement of measuring instruments
over time. To determine stability, a
measure or test is repeated on the
same subjects at a future date.
RELIABILITY
Internal consistency is the extent to
which tests or procedures assess the
same characteristic, skill or quality. It
is a measure of the precision between
the observers or of the measuring
instruments used in a study.
RELIABILITY
Interrater reliability is the extent to
which two or more individuals
(coders or raters) agree. Interrater
reliability addresses the
consistency of the implementation
of a rating system.
.
Remember!
An assessment that is highly
reliable is not necessarily
valid. However, for an
assessment to be valid, it must
also be reliable.
Improving Validity &
Reliability
• Ensure questions are based on taught curricula
• Ensure questions are based on standards
• Allow students to demonstrate knowledge/skills in
multiple ways
• Ensure a variety of item types (multiple-choice,
constructed response)
• Ask questions at varying Depth of Knowledge levels
• Ensure accurate test administration
• Include items that address the full range of standards
• Include multiple items that assess the same standard
• Review scorer reliability, when necessary
V&R : Student Learning
Objectives
What makes high-quality evidence for SLOs:
• Aligned to the content standards (construct
validity)
• Being used for the purpose for which it was
designed
• Administered properly
Questions?
THANK YOU
FOR
LISTENING
