Firestone 1987
Firestone 1987
The current debate about quantitative and qualitative methods& Cook, 1979). Quantitative studies are usually based on
focuses on whether there is a necessary connection between a positivist paradigm while qualitative research is often
method-type and research paradigm that makes the different based on a phenomenological one. There are problably
approaches incompatible. This paper argues that part of the several reasons for this association. A very important one
["connection is rhetorical. Quantitative methods express the has to do with means of expression rather than logic
ì assumptions of a posihvist paradigm which holds that behavior (Eisner, 1981). Essentially, qualitative and quantitative
λ can be explained through objective facts. Design and instrumenta- methods lend themselves to different kinds of rhetoric
'—- hon persuade by showing how bias_ and error are eliminated. (Gusfield, 1976; House, 1979). As a result, each method
CQuahtatwe methods express the assumptions of a phenomenologi- type uses different techniques of presentation to project
\ cal paradigm that there are multiple realities that are socially divergent assumptions about the world and different
means to persuade the reader of its conclusions. Yet, they
) defined. Rich description persuades by showing that the researcher
iwas immersed in the setting and giving the reader enough detail are not antithetical. They present the reader with different
i^to "make sense" of the situation. While rhetoncallyAifferenl, kinds of information and can be used to triangulate to gain
Γ~the results of the two methodologies can be conjµlem£utaĸy. greater confidence in one's conclusions. This argument
about the rhetorical connection between method type and
I Examples are drawn from two studies using different
paradigm is advanced first in general terms and then
Å methodologies to study the same problem.
illustrated by a comparison of two studies that use
qualitative and quantitative methods to address the same
issue.
OCTOBER Ì987 77
raised that issue again (Edmonds, 1979). for funds. This concern was an important barrier to efforts
Both studies described here examine the relative contri to increase services by seeking grants. While environmental
butions of leadership and environment to organizational constraints were similar, the orientations of agency leaders
performance. The quantitative study addresses this issue reflected the difference in approach. The director of the
by defining a specific set of variables and procedures for high-service entrepreneurial IU set the tone for his agency
measuring them. The environmental variable is the socio- by aggressively looking for new services to offer and new
economic status (SES) of the students as indicated by the sources of funds When his board objected to this
principal's report of the percentage of,students who receive approach, he found ways to better justify new programs,
free lunches. There are two leadership variables: centraliza but he did not give up the search. The leaders of the more
tion of influence in the principal as opposed to decentraliza laissez-faire IU were more defensive, spending relatively
tion to the teacher and the extent of principal support for more time justifying their budget than seeking funds. The
teaching. The outcome of interest is how much students director of the high service agency challenged environ
learn. The last four variables—centralization, support, mental constraints, but he did not simply "cause" the high-
teaching, and student learning—are all measured through service approach. In fact, ħe appeared to be chosen because
a survey of teachers in the school. The initial theory he reflected high-service orientations preferred by a key
assumes that the extent to which teachers work hard and constituency on the board when the IU was founded. His
try to teach all students in their class mediates between contribution was important but did not provide a complete
the two leadership variables and student learning. The explanation.
analysis is guided by a theoretical model which proposes
that student learning is influenced by teaching, support, The Studies Compared
centralization, and SES and that leadership is influenced An examination of the rhetorical elements of those
by SES Information comes from a national sample of 107 studies indicates that they use quite different strategies to
elementary and secondary schools (details of procedures persuade the reader of the validity of the analysis and that
are provided in Firestone & Wilson, 1986). they project different assumptions about organizational
Two statistical analyses of the model are presented. 2 The phenomena. An important by-product of these differences
first shows that SES has three times as much influence on is that they provide complementary information to the
student learning as the leadership variables. The second reader.
is a path analysis (Duncan, 1966) which verifies the
existence of hypothesized relationships between variables. Persuasion
It shows that SES has a strong direct effect on learning and The quantitative study persuades by de-emphasizing
that increasing control reduces learning. The effect of individual judgment and stressing the use of established
support is indirect. Increasing support improves teaching procedures. While the language of hypothesis testing is
which in turn increases student learning. Together the two avoided, the impression is given that the whole study is
analyses suggest that leadership does influence student a disciplined exploration of a preexisting conceptual frame
learning although not to the extent that the environment work. This is done in a variety of ways. At the most obvious
does. level, almost as much space is given to describing the
The qualitative study was part of a larger exploration of study's methods (four pages) as the results (six pages,
regional educational service agencies (RESAs), those including tables). The methods section describes the
agencies located midway between the state and local study's sample in a paragraph. The rest of the section is
district. This project examined their contribution to devoted to a detailed discussion of measurement pro
research use in schools through training and dissemina cedures. Another limitation to individual discretion is the
tion activities. The study focused on pairs of Intermediate use of a theoretical model to guide the analysis. This model
Units (IUs) in Pennsylvania and Education Improvement is provided in a pictorial form in the introductory theory
Centers (EICs) in New Jersey that were known to differ section, and criteria for determining when a hypothesized
in the amount of training they offered. Data were collected relationship is deemed to be supported are described in
using semi-structured interviews with agency directors and the results section and justified with reference to previous
administrators, the training staff, and representatives of methodological research Thus, the text [Link] strong
client districts. Interviews with state departments of educa impression that exploratory "data dredging" has been
tion clarified the larger political context in which these avoided.
agencies operated (see Firestone & Rossman, 1986 for pro In this case, the form hides part of the story because there
cedures). All the information on each agency was then was an exploratory element to the study. The study was
pulled together into a case study. By examining each case triggered by the finding of negative associations between
and comparing pairs of cases, a typology of agency centralization and student outcomes when validity
approaches to their task and explanations for why one analyses were done for a manual describing the survey
approach was selected over another were developed. This instrument. We viewed this finding as contradictory to the
use of a variety of materials and an inductive approach in effective schools research which argues that strong leader
a comparative case study design is typical of a fair amount ship promotes achievement. We reasoned further that if
of qualitative policy research (Herriott & Firestone, 1983). this rinding could be replicated when controlling for
Exploration of the pairs of cases showed that environ student background, it would be an important contribu
mental constraints were similar. In Pennsylvania, for tion. Reporting this personal aspect would undercut the
instance, state law which gave school boards control of IU impression the paper now, gives of being a detached "test"
budgets made those agencies responsive to district of a theory.
concerns. 3 However, the districts suffered from severe Less attention is given to describing procedures and how
financial constraints and saw those agencies as competitors individual judgment is disciplined in the qualitative study.
18 EDUCATIONAL RESEARCHER
The study is presented as frankly exploratory. The strategy dichotomies: variables versus actions, hydraulic deter-
is one of comparing pairs of agencies known to be different minism versus limits and opportunities, and randomness
in order to discover what might explain those differences. and error versus choice.
That search is not described as strongly controlled by The quantitative study portrays a world of variables and
preexisting theory. There is no preliminary model. Instead, static states. The text refers to levels of centralization or
reference to past research is incorporated into the presenta- principal support, but one can only infer from the question-
tion of findings. The methods section is only two pages naire items what a principal does to centralize or how he
long while the results take over 20. About half the methods or she provides support. By contrast the qualitative study
discussion describes the agencies selected and demon- describes people acting in events. A director tells how he
strates that they did in fact differ in services provided since uses hiring interviews to encourage staff to actively sell
the reader must be convinced of that fact to be persuaded services or an informant tells about the political battles that
of the value of what follows. led to the legislation governing IUs. Even the "state" of
While analysis procedures are not described, some con- entrepreneurialism is portrayed through a dispute between
trols stemming from the criteria of good qualitative analysis an agency's board and its top leadership over how actively
were used. One of these is the admonition to search for outside funding should be pursued.
competing explanation and negative evidence (Campbell, The quantitative study uses a hydraulic image of deter-
1979). This served well in the analysis of leader contribu- minism as if pressure from one variable changes another.
tions. The first analysis showed a remarkable similarity Regression coefficients indicate how much one might
between the values professed by the top leaders and the expect the dependent variable to increase for a given
organizations' approaches. This could have led to an over- change in an independent variable. The effect of the tables
estimate of leaders' influence on those approaches, is reinforced by language about the "percent of variance"
especially where the entrepreneurial IU maintained its a p - in one variable that is attributable to others. The implicit
proach in spite of external opposition. However, further imagery is of a system of interconnected variables where
exploration indicated that the director there had been pressure from one forces change in another. The abstrac-
selected because his values fit the interests of a strong tion process that directs attention from the total situation
constituency. Openness to this possibility helped to avoid in a school to a set of variables implies an almost physical
an overestimate of leader influence. connection between those variables. The qualitative study
That search is not described in the text, in large measure presents a more complex view of a world in which there
because it is less relevant for persuading the reader. In fact are limits and opportunities that individuals must take into
two very different strategies are used: rich depiction and account and use. These include the IUs' required budget
strategic comparison. The first is the hallmark of most approval procedure and external funding competitions.
qualitative research; the second is required by the effort These limits and opportunities shape action, but do not
to understand differences between pairs of agencies. Taken determine it.
together, these approaches pile up a series of significant, Finally, the two studies suggest different alternatives to
concrete details to give a convincing depiction of each causality. The quantitative study emphasizes randomness
agency and of the differences between them. For instance, and error. The study design, especially the sampling pro-
several different kinds of data were used to show that cedure and instrumentation, are intended to reduce the
Farmland IU was more entrepreneurial than Rural- amount of error in the study. They are described to help
Industrial. These included telling quotes from interviews, the reader assess how well that task was accomplished.
a description of agency staffing patterns, and excerpts from Once as much error is eliminated as possible, two alterna-
agency history illustrating the entrepreneurial orientation tives remain—randomness and the causal forces of the
in action. The details are convincing because they create measured variables—and statistical tests are used to choose
a gestalt that makes sense to the reader. This is a process between them. The alternative to causality in the qualitative
Polanyi and Pròsch (1975) describe as building up a focal study is choice. Constraints and opportunities are real but
impression out of a series of subsidiary details. It depends ambiguous. Leaders decide how to respond to them, for
upon the active effort of the reader and the reader's willing- instance, when they encourage or discourage fund seeking.
ness to check these details against personal experience. If their choice violates those constraints, there will be a
In sum, the persuasive strategies of the two kinds of response that requires a change of strategy (as in
research are very different. The quantitative study must Farmland's case) or even organizational demise (as in the
convince the reader that procedures have been followed New Jersey situation not discussed).
faithfully because very little concrete description of what These different assumptions do not stem from the
anyone does is provided. The qualitative study provides phenomenon studied. They come in large measure from
the reader with a depiction in enough detail to show that the way the researchers collect and process their informa-
the author's conclusion "makes sense." For that reason, tion. These steps shape the nature of the final text which
discussion of procedure is not emphasized. Too much then reinforces those assumptions stylistically.
attention to procedure can get in the way of the narrative
line which attempts to build a concrete impression of the Complementarity
phenomenon studied. The differences presented above give qualitative and
quantitative studies different descriptive strengths. The
Assumptions quantitative study assesses the magnitude of relationships
In addition to using different persuasive strategies, the more precisely. One can say rather clearly that 61% of the
two studies make different assumptions about the world. variance in student learning is explained. The qualitative
This distinction is highlighted when one focuses on how study concludes with more ambiguous statements like
each handles causation.4 It can be described through three "strong leadership is necessary, but not sufficient for
OCTOBER 1987 19
excellence." The other advantage of the quantitative study a pragmatic question of fitting research techniques to a
is that it shows a pattern that extends across a large number problem as the pragmatists suggest, although that does
of situations. In fact, the conclusion rests on showing the happen. On the other hand, one's method is not as
ļoint association of variables in many settings. The use of rigorously determined by the choice of paradigm as the
many sites increases confidence in the generalizability of purists suggest. There are in fact a number of reasons for
results although technically generalizability depends upon selecting a methodological approach, but one's decision
the randomness and representativeness of the sample often expresses values about what the world is like, how
selected. The trade-off of course is in abstraction. One's one ought to understand it, and what the most important
confidence in the conclusions depends on one's comfort threats to that understanding are. The method selected
with the way variables are measured and relate to the encourages one to adopt conventions of presentation that
issues of interest, the quality of the sample, and the general advance certain kinds of arguments for the credibility of
design of the study one's conclusions. These nonlogical methodological
The classical strengths of qualitative methods are concrete tendencies fit with individual stylistic predictions as well
depiction of detail, portrayal of process in an active mode, as the philosophical underpinnings of the positivist and
and attention to the perspectives of those studied (Patton, phenomenological paradigms of research.
1980). These strengths help to overcome the abstraction
inherent in quantitative studies. These advantages appear
in this qualitative study through quotations and descrip
tions that illustrate the perspectives of staff, leaders, and Notes
outside clients. However, the description is thinner than The work upon which this publication is based was conducted at
in more ethnographic studies. On the other hand, the use Research for Better Schools and funded by the Office of Educational
of four cases allows for some comparison in order to Research and Improvement (OERI), U S. Department of Education.
identify patterns across situations. It also gives greater The opinions expressed in this publication do not necessarily reflect
the position or policy of the OERI, and no official endorsement by
confidence that conclusions do not depend upon the the OERI should be inferred. Thanks are due to Bruce Wilson, Robert
idiosyncracies of the specific situation, so it is something Herπott, and three anonymous reviewers for their helpful comments
of a mixed case. 'Phillips (1983) has reviewed several versions of positivist and post-
Used separately, qualitative and quantitative studies pro positivist thought among philosophers. This discussion does not reflect
that variety so much as the more limited use of the term among those
vide different kinds of information. When focused on the who debate the value of combining or separating qualitative and
same issue, qualitative and quantitative studies can quantitative methods
triangulate—that is, use different methods to assess the ¾xamples are taken from the analysis of elementary schools A
robustness or stability of findings (Jick, 1979). Where parallel analysis of secondary schools yields substantially similar
studies using different methods have similar results, one results.
¾xamples are taken primarily from the comparison of IUS The
can be more certain that the findings are not influenced analysis of the ElCs in New Jersey yielded very different concrete
by the methodology Where the results diverge more events, but the conclusions of the analysis were very similar. Agency
research is needed; but a comparison of studies can often names are pseudonyms
4
suggest important lines of inquiry to pursue. In this case Some indication that the differences between methods are not as
the two studies generally corroborate each other. The sharp as the purists claim comes from the observation that qualitative
methods deal with causality at all.
quantitative analysis shows that SES has the strongest
impact on student learning, but the leadership variables
also have an undeniable effect. The qualitative study shows References
a strong congruence between leader values and organiza Boocock, S S. (1972) An introduction to the sociology of learning Boston
tional approach. Leaders can even challenge the environ Houghton-Mifflin
ment. Yet, it is also clear that leaders are partly chosen for Campbell, D.T (1979). "Degrees of freedom" and the case study In
their values and have their greatest effect at certain critical T D Cook & C S Reichardt (Eds ), Qualitative and quantitative methods
in evaluation research (pp 49-67) Beverly Hills, CA- Sage.
times so their influence is limited. Carnine, D , Gersten, R , & Green, S (1982) The principal as instruc-
tional leader. A second look. Educational Leadership, 40, 47-50.
Conclusion Cohen, A (1979) Political symbolism Annual Review of Anthropology,
8, 87-113.
Analysis of these two studies of the same topic suggests Cronbach, L.J (1975) Beyond the two disciplines of scientific
that there is a rhetorical connection between method types psychology American Psychologist, 30, 116-127
and paradigms. The methods and conventions of presenta Duncan, O Ď. (1966) Path analysis: Sociological examples. American
tion used in each express the values of the related paradigm journal of Sociology, 72, 1-6.
about what the world is like and how one must show the Durkheim, E (1938). The rules of sociological method New York Free
Press.
truth of an argument. What is not so clear is how tightly Edmonds, R (1979) Effective schools for the urban poor. Educational
or consistently these conventions link paradigm and Leadership, 37(1), 15-24
method. Such an analysis requires examination of more Eisner, E. (1981) On the differences between scientific and artistic
than two papers and probably a more historical perspec approaches to qualitative research..Educational Researcher, 30(4), 5-9
tive. In art and literature, the stylistic conventions of one Firestone, W A., & Rossman, G B (1986). Exploring organizational
approaches to dissemination and training. Knowledge Creation, Dif
generation are often made to be broken by the next. It fusion, Utilization, 7(3), 303-30
seems likely that as we become more self-conscious about Firestone, W.A , & Wilson B (1986) Management and organizational out
the rhetorical techniques'used in research, some indi comes. The effects of approach and environment in schools. Philadelphia
viduals will begin to test them and look for new ways to Research For Better Schools
break the mold. .r •••„ Frye, N (1957) Anatomy of criticism Princeton, NJ Princeton University
Press
Choosing methods then is not just a matter of coming Goodenough, W (1971) Culture, language, and society Reading, MA
at a single truth from different directions. Nor is it solely Addison Wesley
20 EDUCATIONAL RESEARCHER
Guba, E.G (1978) Toward a methodology of naturalistic inquiry m educa- Polanyi, M , & Prosch, H (1975). Meaning. Chicago University of
tional evaluation Los Angeles, CÀ: Center for the Study of Chicago Press
Evaluation Powdermaker, H (1966) Stranger and friend The liny of the anthropolo
Gusfield, J (1976) The literary rhetoric of science. Comedy and pathos gist New York W W Norton
in drinking driver research. American Sociological Review, 47(1), 16-34 Reichardt, C S , & Cook, T D (1979) Beyond qualitative versus
Herπott, R E , & Firestone, W A. (1983) Multisite qualitative policy quantitative methods In T D Cook & C S Reichardt (Eds ),
research Optimizing description and generalizabílity Educational Qualitative and quantitative methods m evaluation research (pp. 7-32)
Researcher, 72(2), 14-19 . Beverly Hills, CA Sage.
House, E (1979) Coherence and credibility The aesthetics of evalua- Rossman, G B , & Wilson, B L. (1985). Numbers and words Combin
tion Educational Evaluation and Policy Analysis, 7(5), 5-18 ing quantitative and qualitative methods in a single large-scale eval
uation study. Evaluation Review, 9(5), 627-43.
Jacob, E. (1987) Qualitative research traditions A review. Review of Sanday, P R (1979) The ethnographic paradigm(s) Administrative
Educational Research, 57(1), 1-50 Science Quarterly, 24(2), 577-38
Jick, T D (1979) Mixing qualitative and quantitative methods: Shulman, L S (1981) Disciplines of inquiry in education- An overview
Tπangulation in action Administrative Science Quarterly, 24(4), 602-11. Educational Researcher, 70(6), 5-12.
Kuhn, T.S (1970). The structure of scientific revolutions (2nd ed ) Smith, A ; G , & Louis, K S -{Eds ) (1982) Multimethod policy
Chicago University of Chicago Press. research Issues and applications. American Behavioral Scientist, 26(1),
Parelius, A.P & Parelius, R J. (1978) The sociology of education 1-144.
Englewood Cliffs, NJ Prentice-Hall. Smith, J K , & Heshusius, L. (1986). Closing down the conversation
Patton, M Q (1980) Qualitative evaluation methods Beverly Hills, CA The end of the quantitative-qualitative debate Educational Researcher,
Sage 15(1), 4-13
Phillips, D C (1983). After the wake. Post-positivistic educational Taylor, S J , & Bogdan, R (1984) Qualitative research methods. The search
thought Educational Researcher, 32(5), 4-12. for meanings (2nd ed.). New York. John Wiley
.,n ΐ .
\¦Φ'ÿíψiìiξK>¡fâļ,!-'' j. Λţ*\.
'*»•",
«*
6C
a^:.,eC*<><V
e
0tf ,es on-
¾@g¢>
1 0 ≠
'°° S
^
>vO ^ ^