324 DISCUSSION

which is the case here. To illustrate, consider the vacation than a 1 week vacation?' (AHP
matrix in (8) of the paper. If the criteria weights question);
under A1 (resp. A2 , A3 ) are correct, then the (b) `How many times greater is the attractiveness
conventional AHP criteria weights are (48/117, improvement from a 1 day vacation to a
33/117, 36/117) (resp. (72/235, 55/235, 108/235), 2 week vacation than from a 1 day vacation
(16/333, 308/333, 9/333)). Note that these three to a 1 week vacation?' (MAVT question).
sets of weights have different ordinal orderings.
The extreme inconsistency of the criteria weights is
the cause of the rank reversal, not the supermatrix
methodology. REFERENCES

Schenkerman, S., `Avoiding rank reversal in AHP

EASE OF APPLICATION decision-support models', Eur. J. Oper. Res., 74,
407±419 (1994).
If the ratio scale properties of the AHP are not Schoner, B., Choo, E. U. and Wedley, W. C., `A
needed in a decision, an evaluation of the linking comment on `Rank disagreement: a comparison of
pin AHP versus MAVT (including the method of multi-criteria methodologies', J. Multi-Crit. Decis.
S&H) should be based on ease and accuracy of use Anal., 6, 197±200 (1997).
by real decision makers. We leave it to the reader Schoner, B. and Wedley, W. C., `Ambiguous criteria
to draw a conclusion with respect to the relative weights in AHP: consequences and solutions', Decis.
ease of response and accuracy from the following Sci., 20, 462±475 (1989).
two typical questions: Schoner, B., Wedley, W. C. and Choo, E. U., `A unified
approach to AHP with linking pins', Eur. J. Oper.
(a) `How many times more attractive is a 2 week Res., 64, 384±392 (1993).

Discussion
That Is Not the Analytic Hierarchy Process: What the AHP Is and
What It Is Not
THOMAS L. SAATY
University of Pittsburgh, Pittsburgh, PA, U.S.A.

There are two basic principles that are totally general response. Their relevance will be clear in
missed by Salo and HaÈmaÈlaÈinen [R6] in their the ensuing discussion.
attempt to reformulate the AHP, which is based
on ratio scales, to fit multiattribute value theory, . A man, upon meeting another in the street, says
which is based on interval scales. A theorem due to to him `Hey, George, what have you done to
Vargas [R8] that is discussed later says that your hair, and look at that, you are also
interval scale measurement cannot be converted wearing glasses. What has happened to you
to ratio scale measurement without knowing what with all these changes?'. The man replies `I am
amounts to the entire ratio scale in the first place. not George!'. What, and you have changed
We also give a counter-example which shows that your name too?'
this approach gives the wrong numerical outcome. . Tehran radio broadcast that a certain
Thus in my response I must first remind the Armenian by the name of Humbersunian had
reader what these basic principles are and why won 1000 toumans in a state lottery. The
their absence has caused these authors to stray far switchboard was flooded with calls correcting
from the mark of making useful suggestions. The the broadcast. They said `It was not Mr.
following two anecdotes tell it all and hint at my Humbersunian in the first place, but Mr.

J. Multi-Crit. Decis. Anal. 6: 320±339 (1997) & 1997 John Wiley & Sons, Ltd.
 DISCUSSION 325

Hartunian. It was not a 1000 toumans, but only those who have committed themselves to assigning
a 100 toumans. It was not a state lottery, but a numbers directly to elements. But paired compar-
friendly poker game. And lastly he did not win. isons, previously looked down on by many of us
He lost'. (including me, brought up in the tradition of pure
mathematics), owing to their great simplicity and
The authors spend the first part of their paper to our lack of knowledge about how to use them to
suggesting that they do not like the AHP as it is create valid measurement, are essential. Let me
and propose radically changing it to an approach explain what I understand by paired comparisons.
that uses ratios of differences without a single The vernacular interpretation is that relative
example for the reader to see the problems with measurement is simply taking numbers from an
that idea, and then spend the remainder of their absolute or a ratio scale (the only kind of numbers
paper proposing doctored-up modifications of one can add) and dividing each by their sum. But
existing AHP practice. They want to be counted that is far from how relative measurement
in both ways. They are on the side of not A, but priorities are derived in the AHP. One first
just in case, and to be safe, they are also on the side makes the paired comparisons and then derives
of A. Apparently they are not sure about the the priorities from them. Paired comparisons are
validity of their first proposal. Nor am I. the engine for generating relative measurement. In
What is missing from these authors' thinking is the AHP one creates a hierarchy or network to
a clear understanding of the fundamental differ- represent a decision and establishes a matrix
ence between measurement in the form of ratio containing the pairwise comparison judgements
scales derived mathematically from paired compar- for the elements linked under a parent element.
isons, and measurement obtained by operating on One then derives a priority vector of relative
numbers assigned one at a time, their differences weights for these elements, the principal eigen-
taken and their ratios estimated. One has to have vector of the matrix. An inconsistency measure is
numbers in mind to form differences and compare derived as a by-product of this process. There is
them. Assigned numbers may be arrived at one such matrix for every parent element. All the
through a guess, a judgement, or by dividing an priority vectors are appropriately weighted and
interval of unit length into parts of appropriate summed to obtain the overall priorities for the
lengths and parcelling these ordinal parts to each alternatives of a decision.
of the elements being measured, or even by a well
worked-over value function. In this regard they
say in their conclusion section `Starting from the MEASUREMENT IS CREATED FROM
foundations of multiattribute value measurement, PAIRED COMPARISONS
we have demonstrated that pairwise comparisons
in ratio estimation should be interpreted in terms Paired comparisons are the intermediate fine-
of value differences between pairs of underlying structured process with which one uses one's best
alternatives'. Their proposal requires knowing the knowledge and understanding to determine how
numbers, e.g. for temperature because it is many times the dominant of two elements, on
measured on an interval scale. Then one takes some property, is a multiple of the less dominant
the difference of two such readings and divides it one taken as the unit of measurement. Thus one
by another difference. The object is to estimate this must answer the question `How many times more
ratio when one does not know the underlying is the second element than the unit element?'. The
interval scale. But the difference in temperatures unit then has the reciprocal relation to the second
and the strength of how we feel about this element. The unit, as it is the smaller element in
difference are not easily correlated in practice. In whatever pair is being compared, is different in
addition, the choice of minimum may not coincide every pairwise comparison. The number one uses
with the natural origin of the interval scale and one to express `how many' belongs to the fundamental
cannot make the transformation and obtain the scale of absolute numbers 1±9 of the AHP. Later
correct outcome. Incidentally, these authors ask us we explain why the range of the scale must be
to take the difference of non-existing measure- limited. This scale is unit-free, for if the elements
ments of intangibles and estimate their ratios! can be measured in pounds or in kilograms, for
The AHP is a new paradigm that some people example, the second element is the same multiple
find hard to accept on first exposure, particularly of the unit element in both scales. An absolute

& 1997 John Wiley & Sons, Ltd. J. Multi-Crit. Decis. Anal. 6: 320±339 (1997)
326 DISCUSSION

scale is invariant under the identity transforma- measurement that ties well with measurement in
tion, which means that no other number would do the physical sciences, attempted to move slowly
to express the comparison judgement. from ordinal scale ranking to something slightly
`The proof of the pudding is in the eating', the stronger. Their approach has yielded weaker scales
saying goes, and it equally applies to the validity of than one obtains by starting from the other end,
AHP applications. Paired comparisons make it with an absolute scale from which the very strong
possible to ask scientific questions that are ratio scales are derived. An advantage here is that
measurement-oriented. Making paired compari- ratio scales make it possible to incorporate greater
sons does not mean assigning numbers from an precision to capture people's understanding and
interval to each of a set of elements and then make it possible to deal with decisions involving
forming their pairwise ratios. It is from all the dependence and feedback because they can be
comparisons that one must first derive a set of multiplied and added.
relative weights or priorities that belong to the
same ratio scale for all the elements in that group.
To say how many times more one element is than HOMOGENEITY AND CLUSTERING
another, taken as the unit for that comparison,
when the elements are close entails some kind of The other missing point in Salo and HaÈmaÈlaÈinen's
analytical thought directed precisely to the ques- account is the absence of a recognition that people
tion. It is not the same as simply calling for a can only provide judgements about things that are
difference of two numbers out of one's head and fairly close or homogeneous. Using the scale 1±9
assigning it to one element, and then calling for means that one can only compare things that are
another such number sensing the magnitude of within an order of magnitude from each other.
their ratio and estimating its value. It never ceases Thus the homogeneity requirement of the AHP.
to amaze me how people have numbers waiting in Otherwise, if they are dealing with a widely
the wings of their heads to be assigned as disparate set of elements on some property, people
measurements or probabilities used to make are forced to guess wildly about how much to
decisions. Only where an individual has long assign to the small and how much to assign to the
familiarity with physical objects that have large. This problem does not disappear by using
measurement on some existing scale can that zero and one to represent the extreme contrast
individual accurately assign numbers directly because assignment would have to be made to the
from that scale. Usually only the grocery clerk next set of widely contrasting objects calling for a
can tell when a piece of cheese is close to one diversity of estimations and guesses. I know of no
pound of weight. mathematical or technical problem associated with
Besides, our goal here is to extend the concept of putting a bound on the values of a fundamental
deriving measurement from paired comparisons to scale to derive relative measurement from it.
cover intangibles for which there are no scales. If one makes paired comparisons of two
Paired comparisons relate one element to another alternatives, one determines which is the smaller
according to one's perception and experience. and estimates how many times (multiples of it) the
Paired comparisons should enable us to make larger one is. If they are too far apart to compare,
fairly accurate estimates and predictions of e.g. a grape with a watermelon according to size,
happenings in the real world. Even though the one would make such a bad mistake in judging
preponderance of humanity has not gone to school how many grapes are equal to a watermelon that it
to learn how to perform measurement according is not worthwhile to make the comparison directly.
to our sophisticated theories or even deal with How does the AHP measure widely separated
numbers in some depth, yet many make paired elements?
comparisons with alacrity and confidence and One first compares the grape with a plum and
select the best of several options. They possess also with a small apple in one set, and then again
some built-in ability that we suspect is some compares in a different set the small apple with a
version of paired comparisons using their feelings large apple and a grapefruit, and for a third time
and perceptions. separately compares the grapefruit with a melon
In passing, we observe that theories of measure- and the watermelon. One then has a comparison of
ment before the AHP, not having a convincing the grape with the watermelon. In each set of
way to create practical and meaningful cardinal comparisons one uses the 1±9 scale. In the second

J. Multi-Crit. Decis. Anal. 6: 320±339 (1997) & 1997 John Wiley & Sons, Ltd.
 DISCUSSION 327

comparison set one divides by the weight of the decisions, we deal with many criteria and must
small apple so it gets a `one' and multiplies by its combine their different measurements. How? The
weight in the first comparison so it has the same AHP is concerned with the measurement of
weight relative to the grape. The weights of the myriads of intangibles (things for which no
large apple and the grapefruit are also multiplied measurement scales exist) and how to bring that
by this weight of the small apple from the first measurement together with the measurement of
comparison. Again if one divides the weights in the the few tangibles there are.
third set of comparisons by the weight of the In their paper, Salo and HaÈmaÈlaÈinen give an
grapefruit and multiplies them by its weight from example of the need to combine tangibles that are
the second comparisons (adjusted as explained measured on the same scale proportionately by
above), one gets the relation of the watermelon to weighting according to the quantity each
the grape, by simply dividing to see how many possesses. This is well known in the literature of
grapes make a watermelon. One should not use the the AHP, and the Expert Choice software has a
scales 1±3 or 1±5 etc., because, as we have seen, command, called Transformation, to do it. Our
one would do much more work. In addition, purpose in any case is more often to relate tangible
people are observed to have the ability to compare and intangible criteria, criteria that do not have
homogeneous objects that are within the same the same scale, or finally, criteria that have the
order of magnitude spread out from 1 to 9. I same scale but in spite of that whose measure-
believe that this order of magnitude was not ments on this scale have different impacts on our
chosen arbitrarily but came from observing what value system in different settings.
people are able to do. I and numerous other people We will show with an example how tangibles are
have applied the 1±9 scale to compare many combined, first with like tangibles, then with other
physical phenomena (lengths, weights, distances, tangibles and with intangibles in the AHP. Assume
brightness, etc.) and got excellent results. In that a family is considering buying a house and
addition, I have carried out a number of experi- there are three houses to consider. Four factors
ments to use numbers from other scales instead of dominate their thinking: the price of the house, the
the 1±9 scale and invariably failed to get good remodelling costs, the size of the house as reflected
results. Here one has to use a verbal scale to by its footage, and the style of the house which is
express the judgements which are then replaced by an intangible. They have looked at three houses
the appropriate numbers. with data shown in Table I on the quantifiables.
If one is uncertain about the homogeneity of the Thus, in terms of dollars, price and remodelling
elements being compared, one performs the costs have weights of 1000/1300 and 300/1300
comparisons on a few elements at a time using it respectively (Table II). Of course, the importance
as a filtering process and observes the wide of these two factors can also depend on the
deviations in the resulting vector, and then rear- manner in which the payments are made, how
ranges them accordingly into homogeneous groups. much money is available at a certain time and so
In theory there should be no problem in doing this. on, and one may have to combine these tangible
In practice the total set of elements may have to be weights with other intangibles derived through
increased (possibly with hypothetical members as in paired comparisons. When one uses measurement
the case of the grape and the watermelon) to make a from an existing scale, the implicit assumption is
smooth transition from one set to another. Note that a unit has the same value at all levels of
that the range of human interests spans a very small measurement. Most people doubt this use of
number of orders of magnitude and this filtering counting numbers in measurement. Thus one
process would not be as cumbersome as it could in
physics and astronomy.
Table I. Choosing the best house

Price Remodelling Size

COMBINING TANGIBLES WITH THE SAME
($1000) costs ($300) (ft2) Style
UNIT AND WITH OTHER TANGIBLES AND
INTANGIBLES A 200 150 3000 Colonial
B 300 50 2000 Ranch
So far we have talked about measurement with C 500 100 5500 Split level
respect to a single attribute or criterion. In making

& 1997 John Wiley & Sons, Ltd. J. Multi-Crit. Decis. Anal. 6: 320±339 (1997)
328 DISCUSSION

Table II. Choosing the best house the units. We can, for example, compare the
intangible criterion style with the tangible criterion
Remodelling size to determine which is more important to serve
Price costs our higher objective of comfort. It is a truism to
1000/1300 300/1300 Size Style
say that people do this all the time and very
successfully. Since the total units (encounters and
A 200/1000 150/300 3000/10500 Colonial
B 300/1000 50/300 2000/10500 Ranch
knowledge indicating the degree of significance to
C 500/1000 100/300 5500/10500 Split level comfort) available for each criterion are different,
one compares the average or equivalently the total
of the units available for one against the average
Table III. Choosing the best house or total for the other [R5].

Economic factors
(combining price and THE AHP IS DESCRIPTIVE AND
remodelling costs) Size Style PREDICTIVE, NOT NORMATIVE
A 350/1300 3000/10500 Colonial At its foundation the AHP is intended to be a
B 350/1300 2000/10500 Ranch descriptive, not a normative, theory. Sometimes it
C 600/1300 5500/10500 Split level is used to respond to the question of what is
preferred and this is usually done at the level of the
alternatives. More often it is used to answer
needs to establish priorities for different ranges of questions about what is more important at the
measurements taken from existing scales, which level of the criteria and what is more likely at the
must be done in the particular context of the level of the alternatives, as in the presidential
decision problem and its objectives. Here we have elections. Its present thrust is to tell how things are
tried to be faithful to the arithmetic in combining rather than to prescribe how they should be, and
each set of measurements as if numbers are the to determine best choices in the face of current and
same at any level. A better way would be to projected conditions. The effectiveness of this
interpret the significance of number ranges by orientation has been borne out through successful
using paired comparison judgements. Numbers prediction in a large number of applications. A
provide information for analysis, they do not in good descriptive theory should be able to say what
themselves contain answers to problems. We are the current situation is now and how it will be in
the ones who decide what a number means to us. the future. In decision theory, when adequate
Then we must combine the factors measured structure is provided to analyse the decision, it
with respect to the same scale, in this case price should be possible to predict the best decision to
(Table III). survive the conditions of the present and the
To make relative and absolute measurement futureÐan optimal decision. The question before
compatible, we must first combine any factors us is: How can a theory claim scientific (not just
measured on the same existing scale into a single mathematical) validity if all it does is assign
factor under which the alternatives are measured. numbers to objects subject to theoretical assump-
We do this by multiplying and adding and not by tions that appear to be eminently reasonable,
raising to powers and multiplying as some people unless at the same time it has some predictive
would like to have us do. We can then proceed to content that can be tested in practice? The MIT
make paired comparisons among the different physicist Slater once said `Questions about a
factors and among the alternatives. theory which do not affect its ability to predict
We see here that we also need ratio scales in a experimental results correctly seem to me quibbles
multicriteria setting because we must trade off one about words'.
unit of one ratio scale against another unit of Here are some references to examples of
another ratio scale, and we can do this by making successful predictions made with the AHP [P12]
paired comparisons. We measure the relative and ANP [R4, P2, P9]. Many applications have
importance of the criteria, whether tangible or been made of the young and robust generalization
intangible, which these units represent, on equal of the AHP to dependence and feedback, known
footing, and use the resulting priorities to weight as the analytic network process (ANP). These

J. Multi-Crit. Decis. Anal. 6: 320±339 (1997) & 1997 John Wiley & Sons, Ltd.
 DISCUSSION 329

applications include rise and fall of stocks in the mental ratios based on such a theory, there would
stock market [P8], political candidacy [P5], oil be no civilization today. There has to be an easier
prices [P3, P4, P6], energy rationing [P7], inter- way accessible to every person.
national chess competition (predicting the winner According to Roberts [R2], suppose that K is a
and by how many games) [P10, P11], predicting set of consequences and L is a collection of
family size in rural India [P13], predicting foreign lotteries with consequences in K. Any function
exchange rates for the dollar versus the yen [P1], u : K ! Re will be called a value function on K and
turnaround of the U.S. economy and the strength it will be convenient to distinguish value functions
of its recovery [P2] and the outcomes of the play- on K from utility functions, which are value
off games and Superbowl in the 1995±1996 foot- functions on K with certain special properties,
ball season [P9]. e.g. the property of preserving certain observed
relations on K.
The definition of a value function does not say
GROUP DECISION MAKING how to construct such a function by assigning
numerical values to the elements being measured.
Prior to the development of the AHP it was According to how the assignment is made and the
considered that to develop a theory to aggregate assumptions it satisfies, the resulting mapping to
individuals' cardinal preferences was `chasing what the reals is a given type of scale. For example, if
cannot be caught' [R1]. It is precisely the paired the admissible transformations of this assignment
comparison process with its reciprocal property are monotone, the resulting scale is ordinal. On the
that makes this possible. In addition, we have the other hand, if the assignment is invariant under
recent proof that Arrow's so-called impossibility linear transformations, then it is an interval scale.
theorem is possible with cardinal numbers [R7]. If the elements are assigned likelihoods of occur-
Another problem with the difference method rences (one way to do this is through lotteries),
proposed by Salo and HaÈmaÈlaÈinen is how to then the result leads to an interval scale under the
generate and aggregate judgements of a group expected value hypothesis. If the assignment is
correctly. made through direct assessment by dividing an
interval from 0 to 100 heuristically, in pieces each
assigned to one alternative, then the result is an
VALUE FUNCTIONS ordinal scale.
The distance from Philadelphia example was
The approach suggested by Salo and HaÈmaÈlaÈinen intended to show that when numbers are available,
and what is done in the AHP reduce to the they can be compared as ratios by applying
following. In the AHP, if A ˆ 4C and B ˆ 2C, judgement to their magnitude, a special case of
then A=B ˆ 4=2 ˆ 2 while A ÿ B ˆ 2C. The latter paired comparisons and not the other way around
depends on C whereas the former is independent as the authors seem to think. If one compares
of C, and what are we going to do about it? Taking estimates of differences between the alternatives,
differences is not unit-free and resorting to and assuming that these comparisons satisfy the
complex arguments to make it so creates the new axioms of difference measurement, then the result
problem of how to get a measurement on C in the is an interval scale. The authors assume that the
first place. Again we see that the AHP is a ratio value function is an interval scale and take ratios
scale theory and not a `difference in magnitude' of the differences of two readings and claim that
theory. The object is not to collect large or small one gets a meaningful ratio scale. The axioms for
values but to have ratios derived from absolute such a theory must assume transitivity and weak
comparisons that provide the necessary contrast order. Even the first is not required by the AHP.
between elements of different sizes. Nor does transitivity hold in most human activ-
Salo and HaÈmaÈlaÈinen observe that one should ities.
not use ratios directly in making comparisons but Here is a fundamental observation. The authors
should use value functions constructed laboriously assume that one can go back and forth between
from interval scale readings. Value functions interval and ratio scales. It is true that one can go
should apply to recover measurements on a ratio from a ratio scale to an interval scale because ratio
scale, as Salo and HaÈmaÈlaÈinen propose. If, to scales are a special case of interval scales. Since
make a decision, human beings had to make their they are a special case, there must be an additional

& 1997 John Wiley & Sons, Ltd. J. Multi-Crit. Decis. Anal. 6: 320±339 (1997)
330 DISCUSSION

requirement to satisfy to convert an interval scale nothing about getting a meaningful result that
to a ratio scale. That requirement has been shown has something to do with known weights.
by Vargas [R8] to be the presence of one Nowhere do they mention that the eigenvalue
`anchoring' paired comparison that must come approach is the procedure through which one
from the ratio scale one is seeking. Such a paired solves equations to obtain the priorities when they
comparison requires that one define the meaning are not known at the start.
of comparing two interval scale values. Once that The authors cite other people versed in utility
is done, one is able to convert all the other interval theory about what constitutes acceptable proce-
readings. dure for preference elicitation. This reminds one of
Subtracting interval scales is not the answer. all the predictions made by pundits of the past
Here is a counter-example. Assume that we have about man's ability to fly, split the atom, give
three objects whose weights on a ratio scale are 6, 3 women the right to vote and so on. What is
and 1 units. The ratio of the first two is 6=3 ˆ 2. apparent in all this is that they are attempting to
Now let us convert them to an interval scale by convolute a new idea to fit an old one. There is a
using the transformation proposed by the authors, large community of scientific people who use the
x ÿ x †= x* ÿ x †, where x is the smallest value AHP style of preference elicitation and find it
and x* is the largest. The corresponding interval acceptable and effective. We need to focus on the
scale values are given by 1, 2/5, 0. It follows that manner of making paired comparisons and deriv-
the ratio of the first two scale values is given by ing relative measurement rather than on who says
1= 2=5† ˆ 2:5 and not 2. The only way one obtains what from their own old point of view. And,
the ratio 6/3 is if one assumes that the smallest I repeat, the authors provide no practical example
value is zero, which implies that one already whatsoever to illustrate the cumbersome proce-
knows the scale with which one is dealing and in dure they propose for replacing paired
this case it is a ratio scale. comparisons.
They ask the question `how many times greater
is the quality improvement' in making compar-
isons by relating a good-quality and a bad-quality OH, WHY CAN'T RATIOS BE LIKE
car. It is obvious that the answer to how many DIFFERENCES?
times is not as they say, but requires that one
alternative be given as a multiple of the other (an The authors replace the judgement values 1 by 2
absolute number as the quotient of two ratio scale and 8 by 9 on two-element comparisons and
numbers). It makes no sense to ask such a question observe that the ratio of the difference in the
by taking differences because the two numbers resulting priorities of the first component of
must be known individually to form their differ- ‰1= 1 ‡ r†, r= 1 ‡ r†Š by changing r from 1 to 2
ence. This difference is not an absolute number but and again from 8 to 9 is 15, to which they attach
is scale-dependent because the numbers themselves great significance. In fact, in paired comparisons
belong to a particular scale. As we have seen in the one cannot ignore what also happens to the
example above, one does not get meaningful priorities of the second element. In that case the
numbers with the proposed approach, and even judgements in the matrix go from 1 to 12 and from 18
if one did, their difference is taken automatically to 19. If one carries through the same type of
and does not need asking the question `how many analysis, one again gets 15. The difference of the
times greater'. In the AHP one does so directly by two results is equal to zero, or if we take the ratio
comparing the two cars using the smaller as the of this 15 to the other 15, we get one. Along the
unit without intervening a third car as a reference. line of thinking of the authors, we conclude that
It is a bizarre and haphazard procedure to get a one need not be any more concerned that people
number for an alternative get another number for may change their judgement from 1 to 2 than from
a second alternative, get a third number for a very 8 to 9. Because judgements are never made on one
unfavourable alternative, subtract it from each of element at a time, adding a unit to a judgement in
the others and then form the ratio of the resulting relative measurement of homogeneous elements
two differences. Does this process lead to anything seems to have the same effect at whatever level it
meaningful? The authors refrain from showing us is added. Again the lack of understanding of
how to use differences in an intelligent way in a paired comparisons has led the authors on a wild
real-life decision with intangibles and prove goose chase to next propose new scales they think

J. Multi-Crit. Decis. Anal. 6: 320±339 (1997) & 1997 John Wiley & Sons, Ltd.
 DISCUSSION 331

are better than absolute numbers to express the Yet a third example of the optics experiment in
judgement that A is so many times larger than B! physics about the relative brightness of four chairs
placed at distances of 9, 15, 21 and 28 ft from a
light source yields the results in Table V.
The words intended to capture three times more,
PROPOSING NEW SCALES TO REPLACE 1±9
five times more and so on have been tested
numerous times in practice and found in general
Consider four line segments Si , i ˆ 1, . . ., 4, of to give good results. There is no intrinsic number
lengths L Si † ˆ i inches. When normalized, these behind a word until people have used it sufficiently
become the relative lengths 0.i. The paired often to associate it with their mental experience.
comparison judgements from the fundamental It is not difficult to argue that in making
scale 1±9 in this case involve the values 1, 12, 13, 14 comparisons, the mind is much better trained
in the first row of the matrix and their reciprocals through experience to estimate integer multiples of
in the first column. It is known that in the a given unit (the smaller of two elements) than to
consistent case the eigenvector is given by any of assign a number directly from a conveniently
the normalized columns and hence the relative chosen scale. If we have several elements to
lengths are precisely given by 0.i, i ˆ 1, . . ., 4, and compare, we would find that the derived scale
once more we see that the scale 1±9 gives back the still captures some of the lost accuracy that occurs
desired values. If instead we use the values 1, 1.22, because integer values from the fundamental scale
1.50, 1.86 proposed by the authors in the first must be used in verbal comparisons. One can show
column and normalize them, we obtain for the mathematically that reciprocal matrices are the
relative lengths 0.179, 0.219, 0.269, 0.333 whose most stable framework we have to derive ratio
deviation from 0.1, 0.2, 0.3, 0.4 on the AHP ratio scales from homogeneous judgements. When the
scale metric is 1.093, which is near the upper limit elements are close, more accurate results are
of the guideline for acceptable compatibility. If, on obtained by comparing them with other elements
the other hand, we use their scale values for the than by invoking decimals, and in this case one
lengths of the segments and normalize them, we may need to expand the scale in the interval from 1
get 0.18, 0.22, 0.26, 0.33. If we now take their to 2 to allow judgements 1.1, 1.2, . . . , 1.9. One
appropriate ratios and approximate them by the should not force an element to be at least twice
nearest integer from the scale 1±9, we get 0.17, another when in fact it is not.
0.17, 0.33, 0.33 whose deviation from that scale on
the ratio scale metric is 1.030, considerably under
the 1.1 limit for acceptable compatibility. Again, DISCRETIZATION AND BOUNDS ON
here is what the authors' proposed scale gives for SCALE VALUES
areas of figures whose measurements are given in
Table IV compared with the scale 1±9. The author's comments about upper and lower
values leave out the consideration of homogeneity.
It is impossible to construct a normalized example
Table IV. Areas of figures
that results in all the values 0.1, 0.2, . . . , 0.9
Circle Square Diamond Rectangle Triangle
included, because the sum would exceed one.
Normalization, or dividing each value by the same
Actual area 0.470 0.230 0.130 0.097 0.050 constant, simply yields relative values and does not
Authors' scale 0.388 0.218 0.173 0.136 0.087 distort anything if one remembers that in any case,
1±9 scale 0.484 0.236 0.123 0.108 0.049 relative values in the eigenvector are not absolute
but ratio scale numbers. The line segments
example shows that the scale 1±9 does give back
Table V. Relative brightness of chairs equally spaced answers for equally spaced stimuli.
On reading the authors' account of the work by
Chair 1 Chair 2 Chair 3 Chair 4
Wedley and Schoner, I called Wedley on the
Actual scale value 0.608 0.219 0.111 0.065
telephone to learn more about the experiment. He
Authors' scale 0.477 0.241 0.157 0.125 told me that he thinks the scale 1±9 works well and
1±9 scale 0.620 0.220 0.100 0.060 does not think it needs changing because of
anything he has done.

& 1997 John Wiley & Sons, Ltd. J. Multi-Crit. Decis. Anal. 6: 320±339 (1997)
332 DISCUSSION

The authors are troubled by the fact that putting matrix allows for both rank preservation and
a bound on the scale decreases the largest value reversal just as the AHP itself does.
any component of the eigenvector can have, as a There are two kinds of rank reversal discussed in
function of the number of components. This is a the literature. The first has to do with lottery
bit of sophistry because they forget to take the comparisons in risky situations where people's
effect of normalization into account. The more preferences for lotteries change owing to their
elements there are, the smaller is the value of the assessment of the risk involved, leading to a
maximum in relative terms because they all have to change in the outcome giving rise to rank reversal.
add to one. That does not mean that its ratio to the This is what is known as the violation of the
weights of the other elements may not be at its invariance principle, a major concern in utility
largest possible value which is an order of theory discussed briefly later.
magnitude but no more because the elements are The other kind that is directly related to the
homogeneous even with this maximum. The other AHP is rank reversal that may be due to the
elements would take on correspondingly small addition or deletion of alternatives. There are now
decimal values. Since none of these values can be myriads of real-life examples which show that
too small relative to the maximum, the more there are cases where rank does and should be
elements there are, the smaller is the value of the allowed to reverse and others which show that
largest element. In addition, the larger the bound rank should not be allowed to reverse. Cases where
M, the smaller the relative values of the other rank reverses may be due to copies and near
elements need to be to still satisfy homogeneity. copies, phantom alternatives, decoy alternatives
Nothing is surprising about this outcome. It is not and others identified by researchers, most of
a problem with the upper and lower values of the whom are practitioners of utility theory. Because
bounds but needs to be considered together with rank reversals occur in practice, a theory cannot
homogeneity. In comparing the area of a large adopt one position with regard to all these
circle that is homogeneous with smaller circles, the situations by laying down a principle to always
more there are of the latter, the smaller is the preserve rank or always to allow rank to reverse.
proportion of the largest circle to the total, as it Let us note that there are decision problems that
would be. But the larger the circle, the greater is its have identical mathematical structure and judge-
normalized share of the total, yet it is still possible ments; in one of these problems rank needs to be
to form ratios with areas of the other circles. preserved whereas in the other it must be allowed
Similarly, the minimum value is correspondingly to reverse.
smaller. This part of the paper, along with the A decision theory needs two methods of
comments about criteria, makes no sense in synthesis to deal with these two possibilities. In
relative measurement and ratios. Along with the AHP, in both its relative and absolute
several mathematical colleagues who read this measurement approaches, there are two such
paper, I remain open to explanations by the modes of synthesis. In relative measurement the
authors as to what they really have in mind, and first mode preserves rank from irrelevant alter-
what relevance it has to saying that one object is natives by comparing all alternatives with an ideal;
five times heavier than another, or a figure has the second mode, called the distributive mode,
three times the area of another figure, and on to allows one to choose the best alternative in the set
criteria and other abstractions in decision making. by comparing them with each other without
reference to an ideal. In absolute measurement,
the normative approach, that is comparable with
RANK PRESERVATION AND REVERSAL utility theory, where alternatives are rated one at a
[K1±19]; PEOPLE WITH BIG PROBLEMS SEE time, the ideal mode preserves rank not only with
MOTE IN BROTHER'S EYE, NOT BEAM IN respect to irrelevant alternatives but also with
OWN EYE respect to any alternatives, whether relevant or
not, and the distributive mode allows it to reverse.
We discuss three ideas in this part: (1) that rank Those who insist on always preserving rank
reversal is legitimate and should be allowed for in have committed themselves to a theory that can
a mathematically precise way; (2) Salo and give wrong answers to problems raised by their
HaÈmaÈlaÈinen's example of the supermatrix is the own colleagues that cannot be resolved by
wrong one for rank preservation; (3) the super- philosophical arguments but need a revised

J. Multi-Crit. Decis. Anal. 6: 320±339 (1997) & 1997 John Wiley & Sons, Ltd.
 DISCUSSION 333

mathematical approach. Some have argued that by watching the process of decision making with an
revising the set-up of the problem they can always alert eye! The matter would have to be dealt with
deal with a decision problem in the context of rank by either a clear rule or an algorithm, not
preservation. It is easy to see that that is according to one's capricious tastes. There is an
impossible because even in the simplest case absence of awareness that mathematics is needed
where copies can cause rank reversal it may not as in the AHP to determine the point at which
be the first copy but the 1555th copy and how is reversals can and should occur because it may not
one to know that in advance without a careful be the first or second added alternative but the
measurement procedure that tells one when millionth one that can legitimately cause rank to
according to one's preference saturation has been reverse.
reached. The dogma about preserving rank when the
There never was a mathematical theorem that alternatives depend on each other has always been
persuasively showed that rank had to be preserved that anything can happen. Thus, when the criteria
under all conditions. It was simply an assumption depend on the alternatives, which implies that
that some people thought was a reasonable one alternatives depend on the alternatives, rank may
and was supported by ambiguous speculations. be allowed to reverse. However, one can have a
The reader who desires quick exposure to the problem with dependence among the criteria, but
problem should read the tortuous versions and with no dependence of criteria on alternatives and
counter-examples provided by Luce and Raiffa in rank may need to be preserved. It is clear that Salo
their book [K9]. I became a sceptic when I used and HaÈmaÈlaÈinen are not familiar with the AHP
utility theory in the cause of disarmament and literature that says that if the alternatives are
found these contradictory statements. Rank rever- independent of each other and the criteria do not
sal is as essential to creative decision making as depend on the alternatives, then the alternatives
allowing inconsistency is to change our assessment are evaluated with the ideal mode outside the
of the relative importance of alternatives with supermatrix and their weights appropriately
respect to a single criterion. When we are inclined synthesized by using the limiting weights of the
to value the presence of a high level of a property criteria from the supermatrix. This procedure is
in an alternative owing to its scarcity, we change described on p. 249 of my 1994 book on
our minds when we find that that property is in fundamentals of the AHP [R3]. Their supermatrix
fact abundant in many alternatives. Such rank example does not prove anything because in it the
reversal does not only occur when assessing the criteria and the alternatives depend on each other
addition of real alternatives, but also of phantom and this implies that the alternatives depend on
or decoy alternatives because we assume that they each other. The supermatrix approach has been
can be there to affect our thinking in choosing the applied extensively to make predictions about the
best alternative. All these are manifestations of turnaround date of the U.S. economy and
change in the relative measurement of alternatives outcomes of games and sports events with very
in which normalization plays an intrinsic role. satisfactory results and hence there seems to be
Normalization takes account of the number of more to it than fanciful number crunching
alternatives (their scarcity, abundance and unique- designed to impress people with its complex
ness) and the strength of measurement of each manipulations. In fact there is an entire book
alternative (its proximity and similarity to and its [R4] written on the supermatrix theory with
contrast with other alternatives). Another impor- abundant applications and there seems to be
tant factor is whether one ranks alternatives wide demand for knowledge about dependence in
relative to ideal standards or whether one wants decision making as I myself have experienced both
to choose the best one in a set of given alternatives. inside and outside academe.
Usually one preserves rank in the former and Now about the violation of the invariance
allows it to reverse in the latter. principle we mentioned earlier, Tversky et al.
Those who have not dealt with relative measure- [K17] write `. . . a growing body of empirical
ment have often hastened to suggest eliminating evidence questions the assumption of invariance,
normalization to make things look like ratio scale which is essential to the theory of rational
numbers used in actual measurement such as choice . . . alternative framings of the same
yards, pounds and dollars. Rank reversals must options . . . give rise to reversal of preferences,
be accounted for by the theory itself and not by and alternative elicitation procedures . . . give rise

& 1997 John Wiley & Sons, Ltd. J. Multi-Crit. Decis. Anal. 6: 320±339 (1997)
334 DISCUSSION

to reversal of preferences. . . . Because invar- sciences', in Encyclopedia of Mathematics and Its

ianceÐunlike independence or even transitivityÐ Applications, 1979.
is normatively unassailable and descriptively R3. Saaty, T. L., Fundamentals of Decision Making with
incorrect, it does not seem possible to construct a the Analytic Hierarchy Process, Pittsburgh, PA:
RWSA Publications, 1994.
theory of choice that is both normatively accep-
R4. Saaty, T. L., The Analytic Network ProcessÐ
table and descriptively adequate'. This says it all to Decision Making with Dependence and Feedback,
those who doubt that the phenomenon is real and Pittsburgh, PA: RWS Publications, 1996.
cannot be brushed away with philosophical argu- R5. Saaty, T. L., Vargas, L. G. and Wendell, R. E.,
ments. `Assessing attribute weights by ratios', OmegaÐInt. J.
The AHP does not deal with lottery compar- Manag. Sci., 11 (1983).
isons in risky situations but has a different R6. Salo, A. A. and HaÈmaÈlaÈinen, R. P., `On the
approach based on direct paired comparisons measurement of preferences in the analytic hierarchy
and feedback to deal with risky situations. A process', J. Multi-crit. Decis. Anal., 6, 309±319 (1997).
major work that has been completed and refereed R7. Srisoepardani, K. P., `The possibility theorem for
group decision making: the analytic hierarchy pro-
will appear in a well-known journal which
cess', Ph.D. Dissertation, University of Pittsburgh,
demonstrates that the judgement process of 1996.
paired comparisons is more general than estimat- R8. Vargas, L. G., `Utility theory and reciprocal pairwise
ing probabilities as one does in lotteries and may comparisons: the eigenvector method', Socio-Econ.
not lead to the difficulties of violating invariance Plan. Sci., 20, 387±391 (1986).
with appropriate posing of a decision problem
under risk.
In their conclusion the authors say `This
interpretation is general and applies to all methods
(including the AHP and SMART) which make use PREDICTION REFERENCES
of ratio statements in the elicitation of hierarchical
models. In fact, when the questions in the AHP are P1. Blair, A., Nachtmann, R., Olson, J. and Saaty, T.,
rephrased according to the value difference inter- `Forecasting foreign exchange rates: an expert judg-
ment approach', Socio-Econ. Plan. Sci., 21, 363±369
pretation, the AHP can be regarded as a variant of
(1987).
multiattribute value measurement. While it is still P2. Blair, A., Nachtmann, R. and Saaty, T. L.,
unclear to what extent the DM's intuitive `Incorporating expert judgment in economic forecasts:
responses to the standard AHP questions conform the case of the U.S. economy in 1992', in Saaty, T. L.
to the value difference interpretation, we feel that (ed.), Fundamentals of Decision Making and Priority
AHP practitioners could improve their analyses by Theory with the Analytic Hierarchy Process, Pitts-
stating the pairwise comparison questions accord- burgh, PA: RWS Publications, 1994.
ingly'. P3. Gholam-Nezhad, H., `1995: the turning point in oil
I believe that I have shown the lack of relevance prices', in Didsbury Jr., H. F. (ed.), The Global
of the author's proposal to the AHP. I have grave Economy: Today, Tomorrow, and the Transition,
Washington, DC: World Future Society, 1985.
doubts about statements made without rigorous
P4. Gholam-Nezhad, H., `Oil price scenarios: 1989
arguments and mathematical proofs. My under- and 1995', Strat. Plan. Energy Manag., 7, 19±31
standing is that scientific research is based on (1987).
evidence and hypothesis testing and not on P5. Saaty, T. L. and Bennett, J. P., `A theory of
speculative statements and observations about analytical hierarchies applied to political candidacy',
examples. Behav. Sci., 22, 237±245 (1977).
P6. Saaty, T. L. and Gholamnezhad, H., `Oil prices:
1985 and 1990', Energy Syst. Policy, 5, 303±318
(1981).
GENERAL REFERENCES P7. Saaty, T. L. and Mariano, R. S., `Rationing energy
to industries: priorities and input±output depen-
R1. MacKay, A. F., Arrow's Theorem: The Paradox of dence', Energy Syst. Policy, 8, 85±111 (1979).
Social Choice: A Case Study in the Philosophy of P8. Saaty, T. L., Rogers, P. C. and Pell, R., `Portfolio
Economics, New Haven, CT: Yale University Press, selection through hierarchies', J. Portfolio Manag.,
1980. No. 3, pp. 16±21 (1980).
R2. Roberts, F. S., `Measurement theory with applica- P9. Saaty, T. L. and Turner, D. S., `Prediction of the
tions to decision-making, utility, and the social 1996 Super Bowl: an application of the AHP with

J. Multi-Crit. Decis. Anal. 6: 320±339 (1997) & 1997 John Wiley & Sons, Ltd.
 DISCUSSION 335

feedback (the supermatrix approach)', Proc. 4th Int. K7. Kahneman, D. and Tversky, A., `Prospect theory:
Symp. on the Analytic Hierarchy Process, Vancou- an analysis of decision under risk', Econometrica, 47,
ver, 1996, Pittsburgh, PA: Expert Choice Inc., 1996. 263±291 (1979).
P10. Saaty, T. L. and Vargas, L. G., `Hierarchical K8. Keeney, R. L. and Raiffa, H., Decisions with
analysis of behavior in competition: prediction in Multiple Objectives: Preference and Value Tradeoffs,
chess', Behav. Sci., 25, 180±191 (1980). New York: Wiley, 1976.
P11. Saaty, T. L. and Vargas, L. G., `Modeling K9. Luce, R. D. and Raiffa, H., Games and Decisions,
behavior in competition: the analytic hierarchy New York: Wiley, 1976.
process', Appl. Math. Comput., 16, 49±92 (1987). K10. McCardle, K. F. and Winkler, R. L., `Repeated
P12. Saaty, T. L. and Vargas, L. G., Prediction, gambles, learning, and risk aversion', Manag. Sci., 38,
Projection and Forecasting, Dordrecht: Kluwer, 1991. 807 (1992).
P13. Saaty, T. L. and Wong, M., `The average family size K11. McCord, M. and de Neufville, R., `Empirical
in rural India', J. Math. Sociol., 9, 181±209 (1981). demonstration that expected utility decision analysis
is not operational', in Wenstop, S. (ed.), Foundation of
Utility and Risk Theory with Applications, Boston,
MA: Reidel, 1983, pp. 181±200.
RANK REVERSAL REFERENCES K12. Pommerehne, W. W., Schneider, F. and Zweifel,
P., `Economic theory of choice and the preference
K1. Buede, D. and Maxwell, D. T., `Rank disagree- reversal phenomenon: a reexamination', Am. Econ.
ment: a comparison of multi-criteria methodologies', Rev., 72, 569±573 (1982).
J. Multi-Crit. Decis. Anal., 4, 1±21 (1995). K13. Saaty, T. L., Fundamentals of Decision Making
K2. Bunge, M., Treatise on Basic Philosophy, Vol. 7 of
with the Analytic Hierarchy Process, Pittsburgh, PA:
Epistemology and Methodology III: Philosophy of
RWS Publications, 1994.
Science and Technology Part II: Life Science, Social
K14. Saaty, T. L. and Vargas, L. G., `Experiments on
Science and Technology, Boston, MA: Reidel, 1985.
K3. Corbin, R. and Marley, A. A. J., `Random utility rank preservation and reversal in relative measure-
models with equality: an apparent, but not actual, ment', Math. Comput. Model., 17(3±4), 13±18 (1993).
generalization of random utility models', J. Math. K15. Tversky, A. and Kahneman, D., `Judgment under
Psychol., 11, 274±293 (1974). uncertainty: heuristics and biases', Science, 185, 1124±
K4. Farquhar, P. H. and Pratkanis, A. R., `Decision 1131 (1974).
structuring with phantom alternatives', Manag. Sci., K16. Tversky, A. and Simonson, I., `Context-dependent
39, 1214±1226 (1993). preferences', Manag. Sci., 39, 1179±1189 (1993).
K5. Freeman, K. M., Pratkanis, A. R. and Farquhar, K17. Tversky, A., Slovic, P. and Kahneman, D., `The
P. H., `Phantoms as psychological motivation: causes of preference reversal', Am. Econ. Rev., 80,
evidence for compliance and reactance processes', 204±215 (1990).
University of California, Santa Cruz and Carnegie K18. Tyszka, T., `Contextual multiattribute decision
Mellon University, 1990. rules', in Sjoberg, L., Tyszka, T. and Wise, J. A. (eds),
K6. Grether, D. M. and Plott, C. R., `Economic theory Human Decision Making, Bodafors: Doxa, 1983.
of choice and the preference reversal phenomenon', K19. Zeleny, M. Multiple Criteria Decision Making,
Am. Econ. Rev., 69, 623±638 (1979). New York: McGraw-Hill, 1982.

Discussion
On the Meaning of Relative Importance
VALERIE BELTON
Department of Management Science, University of Strathclyde, Glasgow, U.K.

TONY GEAR
Aston Business School, Aston University, Birmingham, U.K.

As anyone who is familiar with our earlier process would expect, we are very much in
publications commenting on the analytic hierarchy agreement with Salo and HaÈmaÈlaÈinen who present

& 1997 John Wiley & Sons, Ltd. J. Multi-Crit. Decis. Anal. 6: 320±339 (1997)