See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/242479451

Critical Thinking Why Is It So Hard to Teach?

Article in Arts Education Policy Review · August 2010

DOI: 10.3200/AEPR.109.4.21-32

CITATIONS READS

694 8,350

1 author:

Daniel Willingham
University of Virginia
100 PUBLICATIONS 10,273 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Book on self-regulation of memory View project

All content following this page was uploaded by Daniel Willingham on 04 July 2020.

The user has requested enhancement of the downloaded file.

 Arts Education Policy Review

ISSN: 1063-2913 (Print) 1940-4395 (Online) Journal homepage: https://www.tandfonline.com/loi/vaep20

Critical Thinking: Why Is It So Hard to Teach?

Daniel T. Willingham

To cite this article: Daniel T. Willingham (2008) Critical Thinking: Why Is It So Hard to Teach?,
Arts Education Policy Review, 109:4, 21-32, DOI: 10.3200/AEPR.109.4.21-32

To link to this article: https://doi.org/10.3200/AEPR.109.4.21-32

Published online: 07 Aug 2010.

Submit your article to this journal

Article views: 1684

View related articles

Citing articles: 97 View citing articles

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=vaep20
Copyright retained by author

Critical Thinking:
Why Is It So Hard to Teach?
DANIEL T. WILLINGHAM

Reprinted with permission from the Sum- exhorted schools to do a better job of After more than 20 years of lamenta-
mer 2007 issue of the American Educa- teaching students to think critically. And tion, exhortation, and little improvement,
tor, the quarterly journal of the American they are not alone. Organizations and maybe it’s time to ask a fundamental
Federation of Teachers, AFL-CIO. initiatives involved in education reform, question: Can critical thinking actually
such as the National Center on Educa- be taught? Decades of cognitive research

V
tion and the Economy, the American point to a disappointing answer: not
irtually everyone would Diploma Project, and the Aspen Insti- really. People who have sought to teach
agree that a primary, yet tute, have pointed out the need for stu- critical thinking have assumed that it is a
insufficiently met, goal of dents to think and/or reason critically. skill, like riding a bicycle, and that, like
schooling is to enable stu- The College Board recently revamped other skills, once you learn it, you can
dents to think critically. In the SAT to better assess students’ criti- apply it in any situation. Research from
laypersons terms, critical thinking con- cal thinking. And ACT, Inc. offers a test cognitive science shows that thinking is
sists of seeing both sides of an issue, of critical thinking for college students. not that sort of skill. The processes of
being open to new evidence that discon- These calls are not new. In 1983, A thinking are intertwined with the content
firms your ideas, reasoning dispassion- Nation At Risk, a report by the National of thought (that is, domain knowledge).
ately, demanding that claims be backed Commission on Excellence in Education, Thus, if you remind a student to “look
by evidence, deducing and inferring found that many 17-year-olds did not at an issue from multiple perspectives”
conclusions from available facts, solv- possess the “’higher order’ intellectual often enough, he will learn that he ought
ing problems, and so forth. Then too, skills” this country needed. It claimed to do so, but if he doesn’t know much
there are specific types of critical think- that nearly 40 percent could not draw about an issue, he can’t think about it
ing that are characteristic of different inferences from written material and only from multiple perspectives. You can teach
subject matter: That’s what we mean one-fifth could write a persuasive essay. students maxims about how they ought to
when we refer to “thinking like a scien- Following the release of A Nation think, but without background knowledge
tist” or “thinking like a historian.” At Risk, programs designed to teach and practice, they probably will not be
This proper and commonsensical students to think critically across the able to implement the advice they memo-
goal has very often been translated into curriculum became extremely popu- rize. Just as it makes no sense to try to
calls to teach “critical thinking skills” lar. By 1990, most states had initia- teach factual content without giving stu-
and “higher-order thinking skills”—and tives designed to encourage educators dents opportunities to practice using it, it
into generic calls for teaching students to teach critical thinking, and one of also makes no sense to try to teach critical
to make better judgments, reason more the most widely used programs, Tac- thinking devoid of factual content.
logically, and so forth. In a recent sur- tics for Thinking, sold 70,000 teacher In this article, I will describe the
vey of human resource officials1 and in guides3. But, for reasons I’ll explain, the nature of critical thinking, explain why
testimony delivered just a few months programs were not very effective—and it is so hard to do and to teach, and
ago before the Senate Finance Commit- today we still lament students’ lack of explore how students acquire a spe-
tee2, business leaders have repeatedly critical thinking. cific type of critical thinking: thinking

Vol. 109, No. 4, March/April 2008 21

scientifically. Along the way, we’ll see Thinking Tends to Focus on a annual Homecoming Parade. First they
that critical thinking is not a set of skills Problem’s “Surface Structure” tried marching in rows of twelve, but
Andrew was left by himself to bring up
that can be deployed at any time, in any
To understand why the surface struc- the rear. Then the director told the band
context. It is a type of thought that even members to march in columns of eight,
3-year-olds can engage in—and even ture of a problem is so distracting and,
but Andrew was still left to march alone.
trained scientists can fail in. And it is as a result, why it’s so hard to apply Even when the band marched in rows of
very much dependent on domain knowl- familiar solutions to problems that three, Andrew was left out. Finally, in
edge and practice. appear new, let’s first consider how you exasperation, Andrew told the band direc-
understand what’s being asked when tor that they should march in rows of five
in order to have all the rows filled. He was
Why Is Thinking Critically So Hard? you are given a problem. Anything you
right. Given that there were at least 45
Educators have long noted that school hear or read is automatically interpreted musicians on the field but fewer than 200
attendance and even academic success in light of what you already know about musicians, how many students were there
are no guarantee that a student will be similar subjects. For example, suppose in the West High School Band?
an effective thinker in all situations. you read these two sentences: “After
Earlier in the experiment, subjects had
There is an odd tendency for rigorous years of pressure from the film and tele-
read four problems along with detailed
thinking to cling to particular examples vision industry, the President has filed a
explanations of how to solve each one,
or types of problems. Thus, a student formal complaint with China over what
ostensibly to rate them for the clarity of
may have learned to estimate the answer U.S. firms say is copyright infringement.
the writing. One of the four problems
to a math problem before beginning These firms assert that the Chinese gov-
concerned the number of vegetables to
calculations as a way of checking the ernment sets stringent trade restrictions
buy for a garden, and it relied on the
accuracy of his answer, but in chem- for U.S. entertainment products, even
same type of solution necessary for the
istry lab, the same student calculates as it turns a blind eye to Chinese com-
band problem—calculation of the least
the components of a compound without panies that copy American movies and
common multiple. Yet few subjects—
noticing that his estimates sum to more television shows and sell them on the
just 19 percent—saw that the band
than 100 percent. And a student who black market.” Background knowledge
problem was similar and that they could
has learned to thoughtfully discuss the not only allows you to comprehend the
use the garden problem solution. Why?
causes of the American Revolution from sentences, it also has a powerful effect
When a student reads a word problem,
both the British and American perspec- as you continue to read because it nar-
her mind interprets the problem in light
tives doesn’t even think to question how rows the interpretations of new text that
of her prior knowledge, as happened
the Germans viewed World War II. Why you will entertain. For example, if you
when you read the two sentences about
are students able to think critically in later read the word “Bush,” it would not
copyrights and China. The difficulty
one situation, but not in another? The make you think of a small shrub, nor
is that the knowledge that seems rel-
brief answer is: Thought processes are would you wonder whether it referred
evant relates to the surface structure—
intertwined with what is being thought to the former President Bush, the rock
in this problem, the reader dredges up
about. Let’s explore this in depth by band, or a term for rural hinterlands.
knowledge about bands, high school,
looking at a particular kind of critical If you read “piracy” you would not
musicians, and so forth. The student is
thinking that has been studied exten- think of eye-patched swabbies shout-
unlikely to read the problem and think of
sively: problem solving. ing “shiver me timbers!” The cognitive
it in terms of its deep structure—using
Imagine a seventh-grade math class system gambles that incoming infor-
the least common multiple. The surface
immersed in word problems. How is it mation will be related to what you’ve
structure of the problem is overt, but
that students will be able to answer one just been thinking about. Thus, it sig-
the deep structure of the problem is not.
problem, but not the next, even though nificantly narrows the scope of possible
Thus, people fail to use the first problem
mathematically both word problems are interpretations of words, sentences, and
to help them solve the second: In their
the same, that is, they rely on the same ideas. The benefit is that comprehension
minds, the first was about vegetables in
mathematical knowledge? Typically, the proceeds faster and more smoothly; the
a garden and the second was about rows
students are focusing on the scenario cost is that the same deep structure of a
of band marchers.
that the word problem describes (its sur- problem is harder to recognize.
face structure) instead of on the math- The narrowing of ideas that occurs With Deep Knowledge, Thinking
ematics required to solve it (its deep while you read (or listen) means that Can Penetrate beyond Surface
structure). So even though students have you tend to focus on the surface struc- Structure
been taught how to solve a particular ture, rather than the underlying structure If knowledge of how to solve a prob-
type of word problem, when the teacher of the problem. For example, in one lem never transferred to problems with
or textbook changes the scenario, stu- experiment,4 subjects saw a problem new surface structures, schooling would
dents still struggle to apply the solution like this one: be inefficient or even futile—but of
because they don’t recognize that the Members of the West High School Band course, such transfer does occur. When
problems are mathematically the same. were hard at work practicing for the and why is complex,5 but two factors

22 Arts Education Policy Review

are especially relevant for educators: in terms of sand, caves, and treasure; the goals of the critical thinking pro-
familiarity with a problem’s deep struc- they thought of it in terms of finding grams that were popular 20 years ago.
ture and the knowledge that one should something with which to leave a trail. As appendix B explains, these programs
look for a deep structure. I’ll address The deep structure of the problem is are not very effective. Their modest
each in turn. so well represented in their memory, benefit is likely due to teaching students
When one is very familiar with a that they immediately saw that structure to effectively use metacognitive strate-
problem’s deep-structure, knowledge when they read the problem. gies. Students learn to avoid biases that
about how to solve it transfers well. That most of us are prey to when we think,
familiarity can come from long-term, Looking for a Deep Structure Helps, such as settling on the first conclusion
repeated experience with one problem, but It Only Takes You So Far that seems reasonable, only seeking evi-
or with various manifestations of one Now let’s turn to the second factor dence that confirms one’s beliefs, ignor-
type of problem (i.e., many problems that aids the transfer despite distracting ing countervailing evidence, overconfi-
that have different surface structures, but

C
the same deep structure). After repeated
exposure to either or both, the subject
simply perceives the deep structure as
part of the problem description. Here’s ritical thinking is not a set of skills
an example:
that can be deployed at any
A treasure hunter is going to explore a
cave up on a hill near a beach. He sus- time, in any context. It is a type of
pected there might be many paths inside
the cave so he was afraid he might get thought that even 3-year-olds can
lost. Obviously, he did not have a map
of the cave; all the he had with him were engage in—and even trained scientists
some common items such as a flashlight
and a bag. What could he do to make sure can fail in.
he did not get lost trying to get back out
of the cave later?
differences in surface structure—know- dence, and others.7 Thus, a student who
The solution is to carry some sand ing to look for a deep structure. Consider has been encouraged many times to see
with you in the bag, and leave a trail as what would happen if I said to a student both sides of an issue, for example, is
you go, so you can trace your path back working on the band problem “this one is probably more likely to spontaneously
when you’re ready to leave the cave. similar to the garden problem.” The stu- think “I should look at both sides of this
About 75 percent of American college dent would understand that the problems issue” when working on a problem.
students thought of this solution—but must share a deep structure and would Unfortunately, metacognitive strate-
only 25 percent of Chinese students try to figure out what it is. Students can gies can only take you so far. Although
solved it.6 The experimenters suggested do something similar without the hint. they suggest what you ought to do, they
that Americans solved it because most A student might think “I’m seeing this don’t supply the knowledge necessary
grew up hearing the story of Hansel and problem in a math class, so there must to implement the strategy. For example,
Gretel, which includes the idea of leav- be a math formula that will solve this when experimenters told subjects work-
ing a trail as you travel to an unknown problem.” Then he could scan his mem- ing on the band problem that it was
place in order to find your way back. The ory (or textbook) for candidates, and see similar to the garden problem, more
experimenters also gave subjects anoth- if one of them helps. This is an example subjects solved the problem (35 per-
er puzzle based on a common Chinese of what psychologists call metacogni- cent, compared to 19 percent without
folk tale, and the percentage of solvers tion, or regulating one’s thoughts. In the hint), but most subjects, even when
from each culture reversed. (To read the the introduction, I mentioned that you told what to do, weren’t able to do it.
puzzle based on the Chinese folk tale, can teach students maxims about how Likewise, you may know that you ought
and the tale itself, go to http://www.aft. they ought to think. Cognitive scientists not accept the first reasonable-sounding
org/pubs-reports/american_educator/ refer to these maxims as metacogni- solution to a problem, but that doesn’t
issues/summer07/folktale.htm.) tive strategies. They are little chunks of mean you know how to come up with
It takes a good deal of practice with knowledge—like “look for a problem’s alternative solutions or weigh how rea-
a problem type before students know it deep structure” or “ consider both sides sonable each is. That requires domain
well enough to immediately recognize of an issue”—that students can learn and knowledge and practice in putting that
its deep structure, irrespective of the then use to steer their thoughts in more knowledge to work.
surface structure, as Americans did for productive directions. Since critical thinking relies so heav-
the Hansel and Gretel problem. Ameri- Helping students become better at ily on domain knowledge, educators
can subjects didn’t think of the problem regulating their thoughts was one of may wonder if thinking critically in

Vol. 109, No. 4, March/April 2008 23

a particular domain is easier to learn. cation might increase the probability of was not, and for sequence 2, they chose
The quick answer is yes, it’s a little a headache only when you’ve had a cup equally between the two blocks.
easier. To understand why, let’s focus on of coffee. The relationship of the medi- This body of studies has been sum-
one domain, science, and examine the cation and headaches is conditional on marized simply: Children are not as
development of scientific thinking. the presence of coffee. dumb as you might think, and adults
Understanding and using conditional (even trained scientists) are not as smart
Is Thinking Like a Scientist Easier? probabilities is essential to scientific as you might think. What’s going on?
Teaching science has been the focus thinking because it is so important in One issue is that the common concep-
of intensive study for decades, and reasoning about what causes what. But tion of critical thinking or scientific
research can be usefully categorized people’s success in thinking this way thinking (or historical thinking) as a set
into two strands. The first examines depends on the particulars of how the of skills is not accurate. Critical think-
how children acquire scientific con- question is presented. Studies show ing does not have certain characteristics
cepts; for example, how they come that adults sometimes use conditional one normally associates with skills—in
to forgo naïve conceptions of motion probabilities successfully,9 but fail to particular, being able to use that skill
and replace them with an understand- do so with many problems that call for at any time. If I told you that I learned
ing of physics. The second strand is it.10 Even trained scientists are open to to read music, for example, you would
what we would call thinking scientifi- pitfalls in reasoning about conditional expect, correctly, that I could use my
cally, that is, the mental procedures by probabilities (as well as other types new skill (i.e., read music) whenever
which science is conducted: developing of reasoning). Physicians are known I wanted. But critical thinking is very
a model, deriving a hypothesis from to discount or misinterpret new patient different. As we saw in the discussion
the model, designing an experiment to data that conflict with a diagnosis they of conditional probabilities, people can
test the hypothesis, gathering data from have in mind,11 and Ph.D.-level scien- engage in some types of critical thinking
the experiment, interpreting the data in tists are prey to faulty reasoning when without training, but even with exten-
light of the model, and so forth.† Most faced with a problem embedded in an sive training, they will sometimes fail to
researchers believe that scientific think- unfamiliar context.12 think critically. This understanding that
ing is really a subset of reasoning that And yet, young children are some- critical thinking is not a skill is vital.††
is not different in kind from other types times able to reason about conditional It tells us that teaching students to think
of reasoning that children and adults probabilities. In one experiment,13 the critically probably lies in small part in
do.8 What makes it scientific thinking researchers showed 3-year-olds a box showing them new ways of thinking,
is knowing when to engage in such rea- and told them it was a “blicket detector” and in large part in enabling them to
soning, and having accumulated enough that would play music if a blicket were deploy the right type of thinking at the
relevant knowledge and spent enough placed on top. The child then saw one of right time.
time practicing to do so. the two sequences shown in figure 1 in Returning to our focus on science,
Recognizing when to engage in scien- which blocks are placed on the blicket we’re ready to address a key question:
tific reasoning is so important because detector. At the end of the sequence, Can students be taught when to engage
the evidence shows that being able the child was asked whether each block in scientific thinking? Sort of. It works
to reason is not enough; children and was a blicket. In other words, the child easier than trying to teach general criti-
adults use and fail to use the proper rea- was to use conditional reasoning to infer cal thinking, but not as easy as we
soning processes on problems that seem which block caused the music to play. would like. Recall that when we were
similar. For example, consider a type of Note that the relationship between discussing problem solving, we found
reasoning about cause and effect that is each individual block (yellow cube that students can learn metacognitive
very important in science: conditional and blue cylinder) and the music is the strategies that help them look past the
probabilities. If two things go together, same in sequences 1 and 2. In either surface structure of a problem and iden-
it’s possible that one causes the other. sequence, the child sees the yellow cube tify its deep structure, thereby getting
Suppose you start a new medicine and associated with music three times, and them a step closer to figuring out a solu-
notice that you seem to be getting head- the blue cylinder associated with the tion. Essentially the same thing can hap-
aches more often than usual. You would absence of music once and the presence pen with scientific thinking. Students
infer that the medication influenced of music twice. What differs between can learn certain metacognitive strate-
your chances of getting a headache. the first and second sequence is the gies that will cue them to think scientifi-
But it could also be that the medica- relationship between the blue and yel- cally. But, as with problem solving, the
tion increases your chances of getting a low blocks, and therefore, the condi- metacognitive strategies only tell the
headache only in certain circumstances tional probability of each block being a students what they should do—they do
or conditions. In conditional probability, blicket. Three-year-olds understood the not provide the knowledge that students
the relationship between two things (e.g., importance of conditional probabilities. needs to actually do it. The good news
medication and headaches) is dependent For sequence 1, they said the yellow is that within a content area like science,
on a third factor. For example, the medi- cube was a blicket, but the blue cylinder students have more context cues to help

24 Arts Education Policy Review

 Sequence 1:

Object A activates the Object B does not Both objects activate Children are asked if
detectpr by itself activate the detector the detector each one is a blicket
by itself (demonstrated twice)

Sequence 2:

Object A activates the Object B does not Object B activates the Children are asked
detectpr by itself activate the detector detector by itself if each one is a blicket
(demonstrated three by itself (demonstrated twice)
times) (demonstrated once)

FIGURE 1. Mechanisms of Theory Formation in Young Children.

Source. Gopnik, A. and Schulz, L. E. (2004). “Mechanisms of Theory formation in young children,” Trends in Cognative Sciences,
8, p 373, Elsevier.

them figure out which strategy to use, were designing an experiment and that comparison conditions, having a con-
teachers have a clearer idea of what cued them to recall the metacognitive trol group in addition to an experimental
domain knowledge they must teach to strategy “When I design experiments, group helps you focus on the variable
enable students to do what the strategy I should try to control variables.” Of you want to study. But knowing that you
calls for. course, succeeding in controlling all of need a control group is not the same as
For example, two researchers14 taught the relevant variables is another mat- being able to create one. Since it’s not
second-, third-, and fourth-graders the ter—that depends on knowing which always possible to have two groups that
scientific concept behind controlling variables may matter and how they are exactly alike, knowing which fac-
variables; that is, of keeping everything could vary. tors can vary between groups and which
in two comparison conditions the same, must not vary is one example of neces-
except for the one variable that is the Scientific Thinking Depends on sary background knowledge. In experi-
focus of investigation. The experiment- Scientific Knowledge ments measuring how quickly subjects
ers gave explicit instruction about this Experts in teaching science recom- can respond, for example, control groups
strategy for conducting sound experi- mend that scientific reasoning be taught must be matched for age, because age
ments and then had students practice in the context of rich subject matter affects response speed, but they need not
with a set of materials (e.g., springs) to knowledge. A committee of prominent be perfectly matched for gender.
answer a specific question (e.g., which science educators, brought together by More formal experimental work veri-
of these factors determine how far a the National Research Council15 put it fies this impression that background
spring will stretch: length, coil diam- plainly: “Teaching content alone is not knowledge is necessary to reason sci-
eter, wire diameter, or weight?). The likely to lead to proficiency in science, entifically. For example, consider devis-
experimenters found that students not nor is engaging in inquiry experiences ing a research hypothesis. One could
only understood the concept of control- devoid of meaningful science content.” generate multiple hypotheses for any
ling variables, they were able to apply The committee drew this conclu- given situation. Suppose you know that
it seven months later with different sion based on evidence that background car A gets better gas mileage than car B
materials and a different experiment- knowledge is necessary to engage in sci- and you’d like to know why. There are
er, although the older children showed entific thinking. For example, knowing many differences between the cars, so
more robust transfer than the younger. that one needs a control group in an which will you investigate first? Engine
In this case, the students knew that they experiment is important. Like having two size? Tire pressure? A key determinant

Vol. 109, No. 4, March/April 2008 25

of the hypothesis you select is plausibil- initial knowledge and how much sub- ability to think critically (to actually
ity. You won’t choose to investigate a jects learned in future sessions, in part do what the metacognitive strategies
difference between cars A and B that due to how the subjects interpreted the call for) depends on domain knowledge
you think is unlikely to contribute to gas data from the experiments that they had and practice. For teachers, the situa-
mileage (e.g., paint color), but if some- conducted. Subjects who started with tion is not hopeless, but no one should
one provides a reason to make this fac- more and better integrated knowledge underestimate the difficulty of teaching
tor more plausible (e.g., the way your planned more informative experiments students to be critical thinkers.
teenage son’s driving habits changed and made better use of experimental
after he painted his car red), you are outcomes. Notes
more likely to say that this now-plau- Other studies have found similar † These two strands are the most often
sible factor should be investigated.16 results, and have found that anoma- studied, but these two approaches—
One’s judgment about the plausibility lous, or unexpected, outcomes may be content and process of science—are incom-
of a factor being important is based on particularly important in creating new plete. Underemphasized in U.S. classrooms
are the many methods of scientific study, and
one’s knowledge of the domain. knowledge—and particularly dependent the role of theories and models in advancing
Other data indicate that familiar- upon prior knowledge.19 Data that seem scientific thought.
ity with the domain makes it easier to odd because they don’t fit one’s mental †† Although this is not highly relevant for
juggle different factors simultaneously, model of the phenomenon under investi- K–12 teachers, it is important to note that
which in turn allows you to construct gation are highly informative. They tell for people with extensive training, such as
Ph.D.-level scientists, critical thinking does
experiments that simultaneously control you that your understanding is incom- have some skill-like characteristics. In par-
for more factors. For example, in one plete, and they guide the development ticular, they are better able to deploy critical
experiment,17 eighth-graders completed of new hypotheses. But you can only reasoning with a wide variety of content,
two tasks. In one, they were to manipu- recognize the outcome of an experiment even that with which they are not very famil-
late conditions in a computer simulation as anomalous if you had some expecta- iar. But, of course, this does not mean that
to keep imaginary creatures alive. In the tion of how it would turn out. And that they will never make mistakes.
other, they were told that they had been expectation would be based on domain
Endnotes
hired by a swimming pool company to knowledge, as would your ability to
1
evaluate how the surface area of swim- create a new hypothesis that takes the Borja, R.R. (2006). “Work Skills of Grad-
ming pools was related to the cooling anomalous outcome into account. uates Seen Lacking,” Education Week, 26,
9, 10.
rate of its water. Students were more The idea that scientific thinking must
adept at designing experiments for the be taught hand in hand with scientific 2
Green, W.D. (2007). “Accenture Chairman
first task than the second, which the content is further supported by research and CEO William D. Green Addresses Sen-
researchers interpreted as being due to on scientific problem solving; that is, ate Finance Committee,” Accenture, www.
students’ familiarity with the relevant when students calculate an answer to accenture.com.
variables. Students are used to think- a textbook-like problem, rather than 3
Viadero, D. (1991). “Parents in S.C.
ing about factors that might influence design their own experiment. A meta- Attack Alleged ‘New Age’ Program.” Edu-
creatures’ health (e.g., food, predators), analysis20 of 40 experiments investi- cation Week, www.edweek.org.
but have less experience working with gating methods for teaching scientific 4
factors that might influence water tem- problem solving showed that effective Novick, L.R. and Holyoak, K.J. (1991).
“Mathematical problem-solving by analogy,”
perature (e.g., volume, surface area). approaches were those that focused on Journal of Experimental Psychology: Learn-
Hence, it is not the case that “control- building complex, integrated knowl- ing, Memory and Cognition, 17, 398–415.
ling variables in an experiment” is a edge bases as part of problem solving,
5
pure process that is not affected by sub- for example by including exercises like For reviews see: Reeves, L.M. and Weis-
jects’ knowledge of those variables. concept mapping. Ineffective approach- berg, R.W. (1994). “The role of content and
abstract information in analogical transfer,”
Prior knowledge and beliefs not only es focused exclusively on the strategies Psychological Bulletin, 115, 381–400; Bar-
influence which hypotheses one chooses to be used in problem solving while nett, S.M. and Ceci, S.J. (2002). “When
to test, they influence how one interprets ignoring the knowledge necessary for and where do we apply what we learn? A
data from an experiment. In one experi- the solution. taxonomy for far transfer,” Psychological
ment,18 undergraduates were evaluated What do all these studies boil down Bulletin, 128 (4), 612–637.
for their knowledge of electrical cir- to? First, critical thinking (as well as 6
Chen, Z., Mo, L., and Honomichl, R.
cuits. Then they participated in three scientific thinking and other domain- (2004). “Having the memory of an elephant:
weekly, 1.5-hour sessions during which based thinking) is not a skill. There is Long-term retrieval and the use of analogues
they designed and conducted experi- not a set of critical thinking skills that in problem solving,” Journal of Experimen-
ments using a computer simulation of can be acquired and deployed regardless tal Psychology: General, 133, 415–433.
circuitry, with the goal of learning how of context. Second, there are metacogni- 7
For a readable review see: Baron, J.
circuitry works. The results showed a tive strategies that, once learned, make (2000). Thinking and Deciding, Cambridge,
strong relationship between subjects’ critical thinking more likely. Third, the UK: Cambridge University Press.

26 Arts Education Policy Review

 8 19
For example see: Klahr, D. (2000). Explor- mechanisms in very young children: Two-, For example see: Dunbar, K. N. and
ing science: The cognition and development three-, and four-your-olds infer causal rela- Fugelsang, J. A. (2005). “Causal thinking in
of discovery processes, Cambridge, Mass.: tions from patterns of variation and covaria- science: How scientists and students inter-
MIT press. tion,” Developmental Psychology, 37(5), pret the unexpected,” in M. E. Gorman, R.
620–629. D. Tweney, D. C. Gooding, and A. P. Kin-
9
Spellman, B. A. (1996). “Acting as intuitive cannon (eds.) Scientific and Technological
14
scientists: Contingency judgments are made Chen, Z. and Klahr, D. (1999). “All Other Thinking, 57–79, Mahwah, N.J.: Erlbaum;
while controlling for alternative potential Things Being Equal: Acquisition and Trans- Echevarria, M. (2003), “Anomalies as a cat-
causes,” Psychological Science, 7, 337–342. fer of the Control of Variables Strategy,” alyst for middle school students’ knowledge
Child Development, 70 (5), 1098–1120. construction and scientific reasoning dur-
10
For example see: Kuhn, D., Garcia-Mila, ing science inquiry,” Journal of Educational
15
M., and Zohar, A. (1995). “Strategies of National Research Council (2007). Tak- Psychology, 95, 357–374.
knowledge acquisition,” Monographs of the ing Science to School. Washington, D.C.:
20
Society for Research in Child Development, National Academies Press. Taconis, R., Ferguson-Hessler, M.G.M.,
60, 1–128. and Broekkamp, H., (2001). “Teaching
16
Koslowski, B. (1996). Theory and Evi- science problem solving: An overview of
11
Groopman, J. (2007). How Doctors dence: The Development of Scientific Rea- experimental work,” Journal of Research in
Think, New York: Houghton Mifflin. soning, Cambridge, Mass.: MIT Press. Science Teaching, 38(4), 442–468.
12 17
Tweney, R. D. and Yachanin, S. A. (1985), Friedler, Y., Nachmias, R., and Linn, M.
“Can scientists rationally assess conditional C. (1990). “Learning scientific reasoning Daniel T. Willingham is a professor of
inferences?” Social Studies of Science, 15, skills in microcomputer-based laboratories,” cognitive psychology at the University of
155–173; Mahoney, M.J. and DeMonbreun, Journal of Research in Science Teaching, Virginia and author of Cognition: The Think-
B.G. (1981), “Problem-solving bias in scien- 27, 173–191. ing Animal as well as over 50 articles. With
tists,” in R. D. Tweney, M. E. Doherty, and C. Barbara Spellman, he edited Current Direc-
18
R. Mynatt (eds.) On Scientific Thinking, 139– Schauble, L., Glaser, R., Raghavan, K., tions in Cognitive Science. He regularly con-
144, New York: Columbia University Press. and Reiner, M. (1991). “Causal models and tributes to American Educator by writing the
experimentation strategies in scientific rea- “Ask the Cognitive Scientist” column. His
13
Gopnik, A., Sobel, D.M., Schulz, L.E., soning,” The Journal of Learning Sciences, research focuses on the role of conscious-
and Glymour, C. (2001). “Causal learning 1, 201–238. ness in learning.

APPENDIX A
How Do Cognitive Scientists Define Critical Thinking?

From the cognitive scientists point of view, the mental activities that are typically called critical thinking are actually a subset of three types
of thinking: reasoning, making judgments and decisions, and problem solving. I say that critical thinking is a subset of these because we
think in these ways all the time, but only sometimes in a critical way. Deciding to read this article, for example, is not critical thinking. But
carefully weighing the evidence it presents in order to decide whether or not to believe what it says is. Critical reasoning, decision making,
and problem solving—which, for brevity’s sake, I will refer to as critical thinking—have three key features: effectiveness, novelty, and
self-direction. Critical thinking is effective in that it avoids common pitfalls, such as seeing only one side of an issue, discounting new evi-
dence that disconfirms your ideas, reasoning from passion rather than logic, failing to support statements with evidence, and so on. Critical
thinking is novel in that you don’t simply remember a solution or a situation that is similar enough to guide you. For example, solving a
complex but familiar physics problem by applying a multi-step algorithm isn’t critical thinking because you are really drawing on memory
to solve the problem. But devising a new algorithm is critical thinking. Critical thinking is self-directed in that the thinker must be calling
the shots: We wouldn’t give a student much credit for critical thinking if the teacher were prompting each step he took.

Vol. 109, No. 4, March/April 2008 27

 APPENDIX B
Critical Thinking Programs: Lots of Time, Modest Benefit

Since the ability to think critically is a primary goal of education, it’s no surprise that people have tried to develop programs that could
directly teach students to think critically without immersing them in any particular academic content. But the evidence shows that such pro-
grams primarily improve students’ thinking with the sort of problems they practiced in the program—not with other types of problems. More
generally, it’s doubtful that a program that effectively teaches students to think critically in a variety of situations will ever be developed.
As the main article explains, the ability to think critically depends on having adequate content knowledge; you can’t think critically about
topics you know little about or solve problems that you don’t know well enough to recognize and execute the type of solutions they call for.
Nonetheless, these programs do help us better understand what can be taught, so they are worth reviewing briefly.
A large number of programs1 designed to make students better thinkers are available, and they have some features in common. They
are premised on the idea that there is a set of critical thinking skills that can be applied and practiced across content domains. They are
designed to supplement regular curricula, not to replace them, and so they are not tied to particular content areas such as language arts,
science, or social studies. Many programs are intended to last about three years, with several hours of instruction (delivered in one or two
lessons) per week. The programs vary in how they deliver this instruction and practice. Some use abstract problems such as finding patterns
in meaningless figures (Reuven Feuerstein’s Instrumental Enrichment), some use mystery stories (Martin Covington’s Productive Think-
ing), some use group discussion of interesting problems that one might encounter in daily life (Edward de Bono’s Cognitive Research Trust,
or CoRT), and so on. However it is implemented, each program introduces students to examples of critical thinking and then requires that
the students practice such thinking themselves.
How well do these programs work? Many researchers have tried to answer that question, but their studies tend to have methodological
problems.2 Four limitations of these studies are especially typical, and they make any effects suspect: 1) students are evaluated just once
after the program, so it’s not known whether any observed effects are enduring; 2) there is not a control group, leaving it unclear whether
gains are due to the thinking program, to other aspects of schooling, or to experiences outside the classroom; 3) the control group does not
have a comparison intervention, so any positive effects found may be due, for example, to the teacher’s enthusiasm for something new,
not the program itself; and 4) there is no measure of whether or not students can transfer their new thinking ability to materials that differ
from those used in the program. In addition, only a small fraction of the studies have undergone peer review (meaning that they have been
impartially evaluated by independent experts). Peer review is crucial because it is known that researchers unconsciously bias the design and
analysis of their research to favor the conclusions they hope to see.3
Studies of the Philosophy for Children program may be taken as typical. Two researchers4 identified eight studies that evaluated
academic outcomes and met minimal research-design criteria. (Of these eight, only one had been subjected to peer review.) Still, they con-
cluded that three of the eight had identifiable problems that clouded the researchers’ conclusions. Among the remaining five studies, three
measured reading ability, and one of these reported a significant gain. Three studies measured reasoning ability, and two reported significant
gains. And, two studies took more impressionistic measures of student’s participation in class (e.g., generating ideas, providing reasons),
and both reported a positive effect.
Despite the difficulties and general lack of rigor in evaluation, most researchers reviewing the literature conclude that some critical
thinking programs do have some positive effect.5 But these reviewers offer two important caveats. First, as with almost any educational
endeavor, the success of the program depends on the skill of the teacher. Second, thinking programs look good when the outcome measure
is quite similar to the material in the program. As one tests for transfer to more and more dissimilar material, the apparent effectiveness of
the program rapidly drops.
Both the conclusion and the caveats make sense from the cognitive scientist’s point of view. It is not surprising that the success of
the program depends on the skill of the teacher. The developers of the programs cannot anticipate all of the ideas—right or wrong—that
students will generate as they practice thinking critically, so it is up to the teacher to provide the all-important feedback to the students.
It is also reasonable that the programs should lead to gains in abilities that are measured with materials similar to those used in the
program. The programs that include puzzles like those found on IQ tests, for instance, report gains in IQ scores. In an earlier column,* I
described a bedrock principle of memory: You remember what you think about. The same goes for critical thinking: You learn to think
critically in the ways in which you practice thinking critically. If you practice logic puzzles with an effective teacher, you are likely to get
better at solving logic puzzles. But substantial improvement requires a great deal of practice. Unfortunately, because critical thinking cur-
ricula include many different types of problems, students typically don’t get enough practice with any one type of problem. As explained in
the main article, the modest benefits that these programs seem to produce are likely due to teaching students metacognitive strategies—like
“look at both sides of an issue”—that cue them to try to think critically. But knowing that one should think critically is not the same as
being able to do so. That requires domain knowledge and practice.
*
See “Students Remember . . . What They Think About” in the Summer 2003 issue of American Educator; online at www.aft.org/
pubs-reports/american_educator/summer2003/cogsci.html.

Endnotes
1
Adams, M. J. (1989), “Thinking skills curricula: Their promise and progress,” Educational Psychologist, 24, 25–77; Nickerson, R. S.,
Perkins, D. N., and Smith, E. E. (1985), The Teaching of Thinking, Hillsdale, N.J.: Erlbaum; Ritchart, R. and Perkins, D. N. (2005). “Learn-
ing to think: The challenges of teaching thinking,” in K. J. Holyoak and R. G. Morrison (eds.) The Cambridge Handbook of Thinking and
Reasoning, Cambridge, UK: Cambridge University Press.

appendix continues

28 Arts Education Policy Review

 APPENDIX B Continued

2
Sternberg, R. J. and Bhana, K. (1996). “Synthesis of research on the effectiveness of intellectual skills programs: Snake-oil remedies or
miracle cures?” Educational Leadership, 44, 60–67.
3
Mahoney, M. J. and DeMonbreun, B. G. (1981). Problem-solving bias in scientists. In R. D. Tweney, M. E. Doherty, and C. R. Mynatt
(Eds.) On Scientific Thinking (pp. 139–144). New York: Columbia University Press.
4
Trickey, S. and Topping, K. J. (2004). “Philosophy for Children: A Systematic Review,” Research Papers in Education 19, 365–380.
5
Adams, M. J. (1989). “Thinking skills curricula: Their promise and progress.” Educational Psychologist, 24, 25–77; Nickerson, R. S.,
Perkins, D. N., and Smith, E. E. (1985), The Teaching of Thinking, Hillsdale, N.J.: Erlbaum; Ritchart, R., and Perkins, D. N. (2005), “Learn-
ing to think: The challenges of teaching thinking,” in K. J. Holyoak and R. G. Morrison (eds.) The Cambridge Handbook of Thinking and
Reasoning, Cambridge, UK: Cambridge University Press.

APPENDIX C
Teaching Critical Thinking

Teaching students to think critically is high on any teacher’s to-do list. So what strategies are consistent with the research?
• Special programs aren’t worth it. In appendix B, I’ve mentioned a few of the better known programs. Despite their widespread
availability, the evidence that these programs succeed in teaching students to think critically, especially in novel situations, is very limited.
The modest boost that such programs may provide should be viewed, as should all claims of educational effectiveness, in light of their
opportunity costs. Every hour students spend on the program is an hour they won’t be learning something else.
• Thinking critically should be taught in the context of subject matter. The foregoing does not mean that teachers shouldn’t teach stu-
dents to think critically—it means that critical thinking shouldn’t be taught on its own. People do not spontaneously examine assumptions
that underlie their thinking, try to consider all sides of an issue, question what they know, etc. These things must be modeled for students,
and students must be given opportunities to practice—preferably in the context of normal classroom activity. This principle is true not only
for science (as discussed in the main article), but for other subject matter. For example, an important part of thinking like a historian is to
consider the source of a document—who wrote it, when, and why. But teaching students to ask that question, independent of subject matter
knowledge, won’t do much good. Knowing that a letter was written by a Confederate private to his wife in New Orleans just after the Battle
of Vicksburg won’t help the student interpret the letter unless he knows something of Civil War history.
• Critical thinking is not just for advanced students. I have sometimes heard teachers and administrators suggest that critical thinking
exercises make a good enrichment activity for the best students, but struggling students should just be expected to understand and master
more basic material. This argument sells short the less advanced students and conflicts with what cognitive scientists know about thinking.
Virtually everyone is capable of critical thinking and uses it all the time—and, as the conditional probabilities research demonstrated (see
figure 1), has been capable of doing so since they were very young. The difficulty lies not in critical thinking, but in recognizing when to
do so, and in knowing enough to do so successfully.
• Student experiences offer entrée to complex concepts. Although critical thinking needs to be nested in subject matter, when students
don’t have much subject matter knowledge, introducing a concept by drawing on student experiences can help. For example, the importance
of a source in evaluating a historical document is familiar to even young children; deepening their understanding is a matter of asking
questions that they have the knowledge to grapple with. Elementary school teachers could ask: Would a letter to a newspaper editor that
criticized the abolishment of recess be viewed differently if written by a school principal versus a third grader? Various concepts that are
central to scientific thinking can also be taught with examples that draw on students’ everyday knowledge and experience. For example,
“correlation does not imply causation” is often illustrated by the robust association between the consumption of ice cream and the number
of crimes committed on a given day. With a little prodding, students soon realize that ice cream consumption doesn’t cause crime, but high
temperatures might cause increases in both.
• To teach critical thinking strategies, make them explicit and practice them. Critical thinking strategies are abstractions. A plausible
approach to teaching them is to make them explicit, and to proceed in stages. The first time (or several times) the concept is introduced,
explain it with at least two different examples (possibly examples based on students’ experiences, as discussed above), label it so as to
identify it as a strategy that can be applied in various contexts, and show how it applies to the course content at hand. In future instances,
try naming the appropriate critical thinking strategy to see if students remember it and can figure out how it applies to the material under
discussion. With still more practice, students may see which strategy applies without a cue from you.

Vol. 109, No. 4, March/April 2008 29

���������
����������������������������
���������
�������������������������������������������������
�����������������������������������������������������������������������
��������������������������������������
����������������������������������������
���������
������������������������������������������������

�
�����������������������������������������������������
��������������������������������������
� ���������������������������������������������
������������������
���������
� ����������������������������������������� �������������������������������������������������
�����������������������������������������������������������������������
���������������������������������������������������� ��������������������������������������
����������������
������������������������������������������������� ���������
�������������������������������������������������
����������������������������������������������������� �����������������������������������������������������������������������
��������������������������������������
���������������������������������������������������� ���������������������������������
���������
������������������������������������������������ �������������������������������������������������
������������������������������������������������������������������������
�������������������������������� ����������������� ��������������������������������������
�����������������������������������
����������������������������������������������������� ���������
�������������������������������������������������
�������������������������������������������������������� �����������������������������������������������������������������������
��������������������������������������
�������������������������������������������������� ���������������������������������
���������
��������������������������������������������������� �������������������������������������������������
������������������������������������������������������������������������
��������������������������������� ��������������������������������������
��������������������������������������
���������
�������������������������������������������������
�����������������������������������������������������������������������
������������������� ��������������������������������������
������������������������������������������������������������� �������������������������������������
����������������������������������������������������������� ���������
�������������������������������������������������
�������������������� �����������������������������������������������������������������������
����������������������������� ��������������������������������������
���������������������������������������� �������������������������
�������������������������������������������������������������� ���������
�������������������������������������������������
������������������������������������������������������������� �����������������������������������������������������������������������
��������������������������������������������� ��������������������������������������
��������������������������������������
���������
��������������� ���� �������������������������������������������������
������������������������������������������������������������������������
��������������������������������������
��������������� ����
������������������
��������������������������� ���������
� ����������������������������� �����������
�������������������������������������������������
������������������������������������������������������������������������
���������� ������������������ ������������������ ��������� ��������������������������������������

����������

����������������

�������

��������������� � �������

���������������
����� ��������������������������������������������������������������������������������������������
 How is this publication thinking about the future?

By becoming part of the past.

This publication is available from
ProQuest Information and Learning
in one or more of the following ways:

• Online, via the ProQuest®

information service
• Microform
• CD-ROM
• Via database licensing

Electronic Databases Microform & Print Chadwyck-Healey Curriculum Products

For more information, call

1-800-521-0600, ext. 2888 (US) or 01-734-761-4700 (International)
www.il.proquest.com

From
View publication stats