Behson (2011)
Behson (2011)
12-1-2011
Part of the Organizational Behavior and Theory Commons, and the Organizational Communication
Commons
Recommended Citation
Behson, Scott J. (2011) "The relative importance of organizational justice dimensions on employee
outcomes: a critical reanalysis using relative weights analysis," Organization Management Journal: Vol. 8:
Iss. 4, Article 3.
Available at: https://scholarship.shu.edu/omj/vol8/iss4/3
Organization Management Journal (2011) 8, 205–217
& 2011 Eastern Academy of Management All rights reserved 1541-6518
A review of the development of the four-factor but interrelated in meaningful ways (Brockner and
model of organizational justice Wiesenfeld, 2005). All four dimensions have been
There has been over 40 years of accumulated extensively studied; the validity and importance of
research into organizational justice. Prior to the these justice dimensions have stood the test of time
research of Adams (1965), most justice research was and the peer-review scientific process.
concerned solely with the perceived favorability of These facets have been validated, have been
decision outcomes. His research, as well as that distinguished from each other using factor anal-
of Thibaut and Walker (1975) expanded this view ysis and other techniques (e.g., Alexander and
to incorporate social exchange (e.g., the input/ Ruderman, 1987; Folger and Konovsky, 1989), and
outcome exchange which is part of the equity their relationships with outcome variables (such as
theory of employee motivation) and to examine satisfaction, commitment, retaliation, and decision
the fairness and favorability of outcomes in legal acceptance) have been distinguished through
decision processes – which became known as distri- regression analyses and structural equations mod-
butive justice. Leventhal (1980) further expanded eling (e.g., Colquitt, 2001). This literature has been
this research to the organizational setting, and, summarized by several meta-analyses (Cohen-
more importantly, introduced the concept of Charash and Spector, 2001; Skitka et al., 2003),
procedural justice – the fairness of the process by including one which performed regression analyses
which decisions are made. Specifically, Leventhal based on the derived meta-analytic data (Colquitt
included the following in his conception of et al., 2001).
procedural justice: (a) consistency of decision When looking at the accumulated literature,
making, (b) freedom from bias, (c) basing decisions there is considerable conceptual and empirical
on accurate information, (d) the ability to correct evidence to support the notions that certain
flawed decisions, (e) conformity with prevailing dimensions of justice should have larger effects on
morals, and (f) consideration of the opinions of certain outcomes than other dimensions of justice
those affected by decisions. (Cropanzano et al., 2002; Conlan et al., 2005;
In 1986, Bies and Moag introduced the concept of Moorman and Byrne, 2005). For instance, it is
interactional justice, expanding the notion of generally posited that distributive justice is closely
justice to incorporate the way people are treated associated with employee reactions tied closely to
during a decision-making process. Greenberg the decision itself, such as decision acceptance and
(1993) forwarded the notion that this construct outcome satisfaction. Similarly, procedural justice
should be split into two separate constructs – is generally seen to be closely associated with
interpersonal justice, which reflects how respect- employee reactions aimed at the larger decision-
fully people are treated during decisions, and making system (Bies, 2005), and often the organi-
informational justice, which reflects how and how zation itself (e.g., satisfaction with a reward system,
well the information regarding a decision process/ organizational commitment). Interpersonal and
outcome is explained to affected parties (see informational justice are most often associated
Colquitt et al., 2005, for an excellent historical with employee reactions focused on the individual
overview) or agent who makes and explains decisions, often a
There has been extensive research on organiza- supervisor (Cropanzano et al., 2002; Bies, 2005).
tional justice in the years since these dimensions The preponderance of research on organizational
were introduced, and a general consensus has justice suggests the following expectations:
emerged of a four-dimensional distributive (based
on outcomes), procedural (based on process), For outcome measures directed closely at the
interpersonal (based on personal treatment), and decision level, such as outcome satisfaction,
informational (based on data-based explanation of distributive justice will be the most important
decisions) understanding of organizational justice predictor.
(Greenberg and Colquitt, 2005). For example, For outcome measures directed at the supervisory
Colquitt (2001) developed measures of all four (agent) or job-related level, such as evaluation of
justice dimensions and found that a four-factor supervisor, withdrawal and job satisfaction, inter-
confirmatory model provided the best fit to the personal and informational justice will be the
data. There is also considerable evidence that, most important predictors.
despite some conceptual and measurement overlap, For outcome measures directed at the system
these dimensions of justice are best seen as distinct or organizational level, such as organizational
commitment and organizational citizenship had been underestimated (Skitka et al., 2003). What
behaviors, procedural justice will be the most remains most open to question is the marginal
important predictor. utility of different justice dimensions – whether the
addition of another justice dimension accounts for
incremental variance in important employee out-
Questions regarding the validity of the four-factor come variables (see Cohen-Charash and Spector,
model 2001). In fact, some have concluded that we may be
One could question the accumulated research on better off studying “overall justice perceptions” as
the four-factor model of organizational justice for the four sub-dimensions may be seen as substi-
both methodological and theoretical reasons. From tutable for each other in affecting employee
a methodological standpoint, justice research attitudes and behaviors (see Ambrose and Arnaud,
has been based exclusively on regression-based 2005). The goal of the current study is to provide
methods to test hypotheses (e.g., Cohen-Charash a test of the four-factor model of organizational
and Spector, 2001, used a comparison of correlation justice. In this way, evidence for the validity of
coefficients; Colquitt et al., 2001, used hierarchical the four-factor model can be assessed against
regression; and Cropanzano et al., 2002, used evidence that questions the four-factor model and
canonical correlation). These techniques, as will the marginal utility of each justice dimension.
be explained in more detail in the methods section,
are prone to bias and, therefore, not the most Methods
appropriate analytic strategies for testing hypoth-
eses aimed at determining the relative strength of Data set and measures
prediction among independent variables (Nunnaly The Colquitt et al. (2001) meta-analysis of organiza-
and Bernstein, 1994). However, to date, no research tional justice research was chosen as the data set for
on organizational justice (and little research in this study because it is the most comprehensive
organizational behavior generally) has used analy- recent quantitative review of this literature (including
tic strategies specifically constructed to validly test 183 published studies over 25 years) and contains
such hypotheses (see Budescu, 1993, Behson, 2005, full correlation information among all four
LeBreton et al., 2007, and Johnson, 2004, for details dimensions of justice as well as a wide array of
and examples). important consequences of justice. By comparison,
In this paper, I use Johnson’s (2000) RWA the Cohen-Charash and Spector (2001) and the
procedure to provide such a statistical test; this is Skitka et al. (2003) meta-analyses do not include
the most appropriate technique for explaining the enough information to calculate a correlation
relative contribution to R2 among multiple inde- matrix nor do they include all four dimensions
pendent variables, taking into account both the of justice. Further, Colquitt et al. (2001) use their
direct effect and the effect of each variable when meta-analytic results as input into a series of
combined with the other variables in a regression hierarchical regression models in order to explore
equation. Because this technique has never before a number of research questions. Their results
been applied to justice literature, the relative provide an excellent opportunity to compare the
importance of the four dimensions of organiza- results of the RWA with those derived from less
tional justice on the explained variance in a appropriate regression analyses. Most importantly,
number of important dependent variables remains this meta-analysis is highly influential; a Google
unclear. Scholar (2011) citation search reveals that this
This is not simply a methodological question, study has been cited in 1381 subsequent scholarly
however. While the accumulated literature is gen- works in the decade since its publication.
erally supportive of the four dimensions of organi- Finally, the Colquitt et al. (2001) meta-analysis
zational justice, this support is not universal. In reports its results two ways: (a) without using
particular, questions remain regarding the indepen- meta-analytic techniques to correct correlations
dence of justice dimensions (e.g., Sweeney and for attenuation due to unreliability, and (b) when
McFarlin, 1997; Bies, 2005), whether interpersonal correcting correlations for attenuation due to
and informational justice should be considered unreliability. As Johnson (2004) states, no study
separately, or as a single “interactional justice” has yet been conducted that compares the results of
construct (Bies and Moag, 1986; Cropanzano et al., an RWA using such a data set, and that it may be
2007), and whether the role of distributive justice important to explore how consistent the results are
or single-item measures. This lack of information (b) regress dependent variable onto the new
is a potential problem with both the original meta- uncorrelated variables, (c) regress the original
analysis and, by extension, the present study. predictors onto the new uncorrelated variables,
and (d) combine the indices from Step 2 with the
Analytic strategy indices from Step 3. Put more simply, this techni-
Traditional multiple regression maximizes prediction que is analogous to the use of an orthogonal
of a dependent variable by assigning weights to rotation during a factor analysis. The development
predictors in such a way that the sum of squares and use of RWA is more fully described by Johnson
attributable to error is minimized (Nunnaly and (2000, 2001) and LeBreton et al. (2007).
Bernstein, 1994). However, multiple regression does In the present study, RWA is applied to the
a poor job in sorting out the relative importance correlation matrices I derived from the Colquitt
of different predictors, especially in the presence et al. (2001) meta-analysis (see Appendix A for these
of multicollinearity (Johnson, 2000). Hierarchical correlation matrices). Analyses were calculated
regression is the most common regression-based using an SPSS syntax program composed by Dr. Jeff
method by which tests of incremental explanation Johnson and run on the PASW18 (formerly SPSS)
of variance or marginal utility of predictors is statistical software program. By using RWA, this
conducted. However, regression, including stepwise study represents the first time this technique,
and hierarchical approaches, as well as structural the most appropriate for explaining the relative
equations models which rely on both factor contribution to R2 among multiple independent
analytic and regression techniques, are susceptible variables, has been applied to organizational justice
to suppressor effects, overestimate the importance research.
of the strongest predictors, underestimate the
importance of the less important predictors, and Results
allow slight differences in inter-predictor correla-
tions to change the pattern of derived regression Comparing the results for the two different
weights (Budescu, 1993; Johnson, 2000). operationalizations of procedural justice
In response to the limitations of multiple Table 1 reports the results of the RWA. Specifically,
regression to reliably and accurately determine the this table lists the total variance in each dependent
relative importance of predictors, a number of variable explained by the four dimensions of
measures of relative importance have been intro- organizational justice (the total R2), and then lists
duced. Instead of focusing simply on a variable’s how much of this total R2 can be attributed to each
incremental contribution to R2, as is commonly of the four dimensions. Both the relative R2 for each
assessed in hierarchical regression, measures of dimension and the percentage of the total R2 are
relative importance focus on a variable’s relative reported. There are two numbers reported in each
contribution to R2, taking into account both its cell of the table. The first number represents
unique contribution and its contribution in the the results using the BDPJ operationalization, and
presence of other predictors. Of these, Budescu’s the second represents the results using PJFP. Thus,
Dominance Analysis (Budescu, 1993; Azen and by comparing the two sets of numbers, one can
Budescu, 2003) and Johnson’s RWA (Johnson, compare results across the two different operationali-
2000, 2001) are seen as the most valid, as both: zations of justice.
(a) contain no logical flaws in their development, Thus, looking at the second set of columns in
(b) are expressed as a proportion of R2 attributable Table 1, we can see that, when using BDPJ, all four
to each independent variable, and (c) consider dimensions, taken together, account for 43.7% of
both direct effects and effects considering the other the total variance in job satisfaction. Further, 18.6%
independent variables in the model (Johnson, of the variance in job satisfaction is explained by
2004; LeBreton et al., 2007). In this way, these procedural justice, representing 42.5% of the total
techniques correct for the effects of multicollinear- R2 explained by all four dimensions.
ity among predictors and more accurately deter- Similarly, using the PJFP operationalization, we
mines each predictor’s relative contributions to the can see that all four dimensions, taken together,
explained variance of the dependent variable. account for 35.2% of the total variance in job
The four steps to conducting a RWA are: satisfaction. Further, 4.9% of the variance in job
(a) transform predictors to uncorrelated variables satisfaction is explained by procedural justice,
that are maximally related to original predictors, representing 13.9% of total R2 explained by all four
Table 1 Results of the relative weights analysis, showing the relative contribution to R2 of the four justice dimensions on employee
outcome variables, data from the meta-analytic results of Colquitt et al. (2001)
Raw relative Relative weights Raw relative Relative weights Raw relative Relative weights
weights as percentage weights as percentage weights as percentage
of R-square of R-square of R-square
BDPJ PJFP BDPJ (%) PJFP (%) BDPJ PJFP BDPJ (%) PJFP (%) BDPJ PJFP BDPJ (%) PJFP (%)
dimensions. In this case, the choice of operationa- strikingly different pattern of results emerge. Dis-
lization leads to a meaningful difference in the tributive justice is seen as the most important
pattern of results. When using BDPJ, an admittedly dimension of justice in explaining variance in
expansive operationalization of procedural justice outcome satisfaction (52.3% EVAF), job satisfaction
that overlaps with other dimensions of justice, it is (55.4% EVAF), organizational commitment (63.0%
not surprising that procedural justice explains the EVAF), and withdrawal (54.4% EVAF). Interpersonal
largest percentage of variance in job satisfaction. (40.8% EVAF) and informational (28.4% EVAF)
However, when a more limited and appropriate justice are still the most important predictors of
operationalization is used, distributive justice organizational citizenship behaviors. As in the prior
explains the most variance in job satisfaction. analysis, no single justice dimension emerged as
When looking at the results in Table 1 that use most important in explaining supervisor evaluation
the BDPJ operationalization of justice, it can be (all dimensions explained from 18.9% to 28.8%).
seen that procedural justice is the most impor- In short, the major difference between the RWA
tant dimension of justice in explaining variance results when the two different procedural justice
for job satisfaction (42.5% of explained variance operationalizations are used is that distributive
accounted for (EVAF)) and organizational commit- justice is more important in explaining variance
ment (50.3% EVAF), and is as predictive as dis- in job satisfaction, organizational commitment,
tributive justice in explaining variance in and withdrawal when the PJFP operationalization
withdrawal (37.4% compared with distributive is used instead of BDPJ. The results for all other
justice’s 36.1%). Distributive justice is seen as the dependent variables are very similar.
strongest predictor of outcome satisfaction (59.6% The muddled results for supervisor evaluation is
EVAF). Interpersonal (46.2% EVAF) and informa- most likely due to the very general nature of the
tional (31.7% EVAF) justice are primary predictors dependent variable, which combined evaluations
of organizational citizenship behaviors. The results of one’s real-life work supervisor (with whom
are less clear for supervisor evaluation, as all the respondent probably has a long-term complex
four dimensions explain similar percentages of relationship) in organizational field studies with
explained variance (ranging from 21.2% to 29.2%). evaluations of an experimenter (with whom the
However, when looking at the results in Table 1 respondent probably had one short-term interac-
that use the PJFP operationalization of justice, a tion) in laboratory studies. Unfortunately, as
Colquitt et al. (2001) combined both types of fourth rows list the dimension of justice that this
studies and did not split out their results, the study found to be most important in explaining
present study cannot differentiate between these variance in each dependent variable, as well as its
studies either. relative weight (this information was also conveyed
Although I did not proffer formal hypotheses in in Table 1).
this study, these results defy expectations. As stated The reason I did not list the full quantitative
earlier, there is general consensus that distributive results from Colquitt et al. (2001) alongside those of
justice should be most important in explaining the present study is that the results were derived
variables such as outcome satisfaction, which are from different analyses. Tests of unique variance
related closely to the decision. However, it is also determine the effect of each justice dimension after
expected that procedural justice would be most completely accounting for (i.e., fully removing) the
predictive of outcome variables directed at the variance explained by all other dimensions, while
larger organization or system, such as organiza- RWA determines the proportionate contribution
tional commitment or organizational citizenship each predictor makes to R2, considering both its
behaviors, and that interpersonal and informa- direct effect and its effect when combined with
tional justice would be most predictive of outcome the other variables in the regression equation.
variables focused on supervisors or decision makers. (It should be noted that Ambrose and Arnaud
While the differences are more substantive when (2005) stated that the over-reliance on tests of
PJFP is used, many of these predictions were not unique variance has led to an emphasis on unique
supported even when BDPJ is used. contributions to R2, rather than acknowledging
the shared variance among justice dimensions. One
Results of direct comparison with Colquitt et al.’s advantage of RWA is that it does account for both
(2001) tests of unique variance unique and shared variance.) As a result, Colquitt’s
This paper’s results can also be compared with results ranged from 0.00 to 0.09, while those of the
those of Colquitt et al.’s (2001) regression-based present study are as large as 0.25. Thus, no true
analysis of the unique effects of the four justice quantitative “apples to apples” comparison or test
dimensions on employee reaction variables (see of statistical significance is feasible. As a result,
Table 2). This analysis tested the meta-analytic readers should interpret these results conservatively
results against three different models of organiza- and be sure to note the cautions included at the
tional justice. These models are: Leventhal’s (1980) bottom of the table.
distributive dominance model, which posited The most illuminating finding is the almost
that distributive justice is the primary driver of all perfect correspondence between the meta-analytic
reactions; Sweeney and McFarlin’s (1997) two- results and the results of the present study that also
factor model, which predicted distributive justice utilized the BDPJ operationalization for procedural
person-referenced reactions (e.g., outcome satisfac- justice. In fact, they differ only very slightly in
tion) and procedural justice would best predict predicting supervisor evaluation, which, as men-
system-referenced reactions (e.g., organizational tioned earlier, is a problematic dependent variable.
commitment); and Bies and Moag’s (1986) agent- Overall, it could be said that the results of Colquitt
system model, which predicts informational and et al. (2001) and the present study that also utilized
interpersonal justice would best predict agent- the BDPJ operationalization for procedural justice
referenced variables (e.g. supervisor evaluation) are relatively invariant regardless of analytic meth-
and procedural justice would predict system-refer- od. Thus, fears that the four-factor model of justice
enced variables. was flawed due to methodological limitations
The dimensions of justice predicted to be most appear largely unfounded.
important in explaining variance in the various In contrast, the results reported in Table 2,
dependent variables according to each of the especially the lack of agreement between the data
models are listed on the top rows of Table 2. The in Row 2 vs Rows 3 and 4, again demonstrate
dimension of justice that Colquitt et al.’s (2001) that results vary widely based on the choice of
found to be most important in explaining variance operationalization of procedural justice.
for each dependent variable, along with their As can be seen in Table 2, the shaded cells
reported percentage of unique explained variance contain results that are consistent with the respec-
(see their Table 6) for that primary justice dimen- tive model of organizational justice. In terms
sion, is listed on the second row. The third and of the three models, the results using the PJFP
Table 2 Results of the comparison between the results of Colquitt et al. (2001) and the present study as they relate to three established
models of organizational justice
operationalization are most consistent with the operationalizations are used. Specifically, distribu-
distributive dominance model, whereas the results tive justice is more important in explaining
using BDPJ are most consistent with the two-factor variance in job satisfaction, organizational com-
model. The results for organizational citizenship mitment, and withdrawal when the PJFP opera-
behaviors and supervisor evaluation are exceptions tionalization is used instead of BDPJ. As PJFP seems
to this. As predicted by the agent-system model, to be the more appropriate choice, this finding does
interactional justice is seen as most important in have conceptual and practical implications, which
predicting organizational citizenship behaviors. are discussed more fully in the discussion.
Thus, there is some support for all three classic Finally, Appendix B is presented in response to
organizational justice models, although none were Johnson’s (2004) observation that no study has yet
fully supported. been conducted that compares the results of an
In summary, there are two major results of this RWA using meta-analytic data that was both
study. First, there are very few substantive differ- corrected and left uncorrected for attenuation due
ences between the findings of Colquitt et al. (2001) to unreliability. The data set used in this study
study and the current study when BDPJ is used as provides this opportunity, but is extraneous to the
the procedural justice operationalization. In this main focus of the paper, and thus provided in an
case, methodological issues seem not to be a major appendix for interested readers. One can compare
issue in creating divergent results and/or mislead- Table 1 and Appendix B and observe that, in
ing interpretations. The results are robust regardless general, there are only small differences in the
of whether RWA or regression-based analyses pattern of results. These small differences are likely
are conducted. Second, substantive differences due to some measures being more reliable than
are found when different procedural justice others, although this is hard to assess more fully
because Colquitt et al. (2001) do not report average even if, in this case, results using this method
reliability coefficients. served to largely confirm past results using
regression-based methods.
Discussion The more provocative finding of this study is that
This study reexamined organizational justice the choice of procedural justice operationalization
research in two important ways. First, it reevaluated can alter results and the implications drawn from
research that has tended to use regression-based justice research studies. When using the broadest
methods with a more appropriate analytic method, possible operationalization of procedural justice,
RWA. Second, it examined the implications of using BDPJ, it is not surprising that it is seen as the
different operationalizations of procedural justice. primary explanatory factor of many dependent
Given the noted limitations of regression-based variables. However, when a more limited, and I
methods (Nunnaly and Bernstein, 1994), the use of would argue, more appropriate operationalization
RWA (Johnson, 2000), a more recently developed is used, procedural justice is seen as less influential,
technique more appropriate for questions of the and distributive justice becomes a more important
relative weight and marginal utility of various explanatory factor.
dimensions of justice, was warranted. As it turns As stated earlier in the paper, BDPJ is seen as
out, there were few, if any, substantive differences a suboptimal operationalization, as it combines
in the patterns of results based on the methods elements of procedural, interpersonal and informa-
being used. This non-finding is important, as it is tional justice, introduces unnecessary multicollin-
another indicator of the robustness of the dimen- earity, and is less suitable for use in either
sionality of organizational justice and the validity regression-based or RWA. As researchers have cau-
of past research in this area. tioned against unnecessary overlap in measuring
As stated earlier, multiple regression maximizes the various dimensions of justice (Colquitt and
the prediction of a dependent variable using a set Shaw, 2005) and have repeatedly confirmed justice
of data, but is not as useful in determining the as having four unique, yet interrelated dimensions
differential effects of each of the included indepen- (e.g., Colquitt, 2001; Bies, 2005), the use of cleaner,
dent variables. RWA accounts for both unique and more precise, and non-overlapping operationaliza-
shared variance among predictors and seems well tions of justice dimensions is seen as the more valid
suited to behavioral sciences research in general, choice. Thus, one should utilize operationalizations
and justice research in particular. LeBreton et al. closer in scope to PJFP, for both methodological
(2007) provide two instructive examples of how (i.e., avoiding unnecessary multicollinearity) and
the interpretation of a study’s results could be quite conceptual (i.e., discriminant validity) reasons.
different when applying RWA as compared with Thus, one conclusion to be drawn from the
regression. Further, Johnson and LeBreton (2004) present study is that the importance of distributive
provide easy-to-follow guidance on how to conduct justice may be overlooked, in part, because a very
RWA. influential meta-analysis may have overstated the
One hopes that the more researchers are exposed importance of procedural justice and understated
to this type of analysis, the more they will use it for the importance of distributive justice.
hypotheses that focus on the relative importance of However, the present study is not the first to
factors in explaining variance in dependent vari- question whether the relative importance of distrib-
ables. Clearly, there are many areas of organiza- utive justice has been undervalued. In fact, some
tional research in which the relative importance earlier work on organizational justice (e.g., Sweeney
of predictors would be extremely interesting and McFarlin, 1997; Cropanzano and Ambrose,
(Johnson, 2001). For example, a measure of relative 2001), found high levels of intercorrelation
importance would be appropriate if one is compar- between distributive and procedural justice, calling
ing the predictive validities of various employment into question the marginal utility of procedural
selection tests and criteria, making decisions for justice in predicting employee reactions. Further, in
reducing the number of items in a scale, or their review of the literature, Skitka et al. (2003)
comparing the contributions of various proposed discovered that “current theorists argue that dis-
antecedents with phenomena such as employee tributive justice has a comparatively limited sphere
turnover (see also LeBreton et al., 2007). Applying of importance relative to procedural fairness”
RWA to the most comprehensive data set in (p. 310) before finding in their empirical meta-
organizational justice is a contribution in itself, analysis that the role of distributive justice had
in this study; however, full information was not Further, RWA is necessarily restricted to analysis
always included in the meta-analysis. This is of main effects, and may ignore the importance of
important, as Colquitt and Shaw (2005) demon- interactive effects that have been identified in the
strated that precision in measuring organizational organizational justice literature. Finally, the data
justice is important but sometimes overlooked. For taken from the Colquitt et al. (2001) meta-analysis
instance, they maintain it is important to distin- is nearly a decade old. While the implications of the
guish between measures that assess the fairness of present study are clear, it is possible that the
an organizational decision vs a decision made by an inclusion of more recent studies could lead to
identifiable single human actor; failures to do so different results. The field seems ripe for an updated
may impact results. meta-analysis.
Another limitation in this study is that there Despite these limitations, this study represents a
is no easily calculable method for determining meaningful contribution to our understanding of
statistically significant differences among relative organizational justice. These results provide evi-
weights (Johnson, 2001; LeBreton et al., 2007). dence that results vary depending on how broadly
Instead, Johnson (2004) and Tonidandel et al. procedural justice is operationalized and measured,
(2009) describe a boot-strapping procedure in and that more precise operationalizations of justice
which one could create a large population of should be used. Further, the importance of dis-
data sets based on the data set in use, and then tributive justice should not be overlooked, as we
calculate confidence intervals around relative increasingly study procedural, interpersonal, and
weight results. The use of this procedure is beyond informational justice. Finally, this study also pro-
the scope of this paper, especially considering that vides a methodological contribution as it is one of
this procedure has never been applied to meta- only a handful of studies in the organizational
analytic data, and it has not been established that it literature (see Behson, 2002, 2005, Johnson and
is valid to do so. I have attempted to be conserva- LeBreton, 2004, and LeBreton et al., 2007) to revisit
tive in interpreting this study’s findings and prior research using RWA or dominance analysis.
encourage the reader to be similarly conservative Such critical reanalyses of established knowledge
so as to avoid overstating small differences in are an important but all too infrequently performed
relative weight. part of the scientific process.
References
Adams, J.S. (1965). Inequity in social exchange. In L. Berkowitz Bies, J.R. & Moag, J.F. (1986). Interactional justice:
(Ed), Advances in experimental social psychology, 267–299. Communication criteria of fairness. In R.J. Lewicki, B.H.
New York: Academic Press. Sheppard and M.H. Bazerman (Eds), Research on negotiations
Alexander, S. & Ruderman, M. (1987). The role of procedural in organizations, 43–55. Greenwich, CT: JAI Press.
and distributive justice in organizational behavior. Social Justice Bies, R.J. (2005). Are procedural and interactional justice concep-
Research, 1(2): 177–198. tually distinct? In J. Greenberg and J.A. Colquitt (Eds), The hand-
Ambrose, M. & Arnaud, A. (2005). Are procedural and book of organizational justice, 88–112. Mahwah, NJ: Erlbaum.
distributive justice conceptually distinct? In J. Greenberg Brockner, J. (2010). A contemporary look at organizational justice;
and J.A. Colquitt (Eds), The handbook of organizational justice, Multiplying insult times injury. Mahwah, NJ: Erlbaum.
59–84. Mahwah, NJ: Erlbaum. Brockner, J. & Wiesenfeld, B. (2005). How, when and why
Ambrose, M., Hess, R.L. & Ganesan, S. (2007). The does outcome favorability interact with procedural justice.
relationship between justice and attitudes: An examination In J. Greenberg and J.A. Colquitt (Eds), The handbook of
of justice effects on event and system-related attitudes. organizational justice, 525–553. Mahwah, NJ: Erlbaum.
Organizational Behavior and Human Decision Processes, Budescu, D.V. (1993). Dominance analysis: A new approach to
103(1): 21–36. the problem of relative importance of predictors in multiple
Ambrose, M. & Schminke, M. (2009). The role of overall justice regression. Psychological Bulletin, 114: 542–551.
judgments in organizational justice research: A test of Cohen-Charash, Y. & Spector, P.E. (2001). The role of justice in
mediation. Journal of Applied Psychology, 94: 491–500. organizations: A meta-analysis. Organizational Behavior and
Azen, R. & Budescu, D.V. (2003). The dominance analysis Human Decision Processes, 86: 278–321.
approach for comparing predictors in multiple regression. Colquitt, J.A. (2001). On the dimensionality of organizational
Psychological Methods, 8: 129–148. justice: A construct validation of a measure. Journal of Applied
Behson, S.J. (2002). Which dominates? The relative importance Psychology, 86: 386–400.
of work-family organizational support and general organiza- Colquitt, J.A., Greenberg, J. & Zapata-Phelan, C.P. (2005). What
tional context on employee outcomes. Journal of Vocational is organizational justice? A historical overview. In J. Greenberg
Behavior, 61: 53–71. and J.A. Colquitt (Eds), The handbook of organizational justice,
Behson, S.J. (2005). The relative contribution of formal and 3–56. Mahwah, NJ: Erlbaum.
informal organizational work-family support. Journal of Voca- Colquitt, J.A. & Shaw, J.C. (2005). How should organizational
tional Behavior, 66: 487–500. justice be measured? In J. Greenberg and J.A. Colquitt (Eds),
The handbook of organizational justice, 113–152. Mahwah, NJ: Research, Vol. V, 231–251. Huntington, NY: Nova Science
Erlbaum. Publishers.
Conlan, D.E., Meyer, C.J. & Nowakowski, J.M. (2005). How does Johnson, J.W. (2004). Factors affecting relative weights: The
organizational justice affect performance, withdrawal and influence of sampling and measurement error. Organizational
counterproductive behavior? In J. Greenberg and J.A. Colquitt Research Methods, 7: 283–299.
(Eds), The handbook of organizational justice, 301–328. Johnson, J.W. & LeBreton, J.M. (2004). History and use
Mahwah, NJ: Erlbaum. of relative importance indices in organizational research.
Cropanzano, R. & Ambrose, M.L. (2001). Procedural and Organizational Research Methods, 7: 238–257.
distributive justice are more similar than you think: A monistic Jonas, E., Schultz-Hardt, S., Frey, D. & Thelen, N. (2001).
perspective and a research agenda. In J. Greenberg and Confirmation bias in sequential information search after
R. Cropanzano (Eds), Advances in organizational justice, preliminary decisions. Journal of Personality and Social Psychol-
119–151. Stanford, CA: Stanford University Press. ogy, 87: 557–571.
Cropanzano, R., Bowen, D.E. & Gilliland, S.W. (2007). The LeBreton, J.M., Hargis, M.B., Griepentrog, B., Oswald, F.L. &
management of organizational justice. Academy of Manage- Ployhart, R.E. (2007). A multidimensional approach for
ment Perspectives, 21: 34–48. evaluating variables in organizational research and practice.
Cropanzano, R., Prehar, C.A. & Chen, P.Y. (2002). Using social Personnel Psychology, 60: 475–498.
exchange theory to distinguish procedural from interactional Leventhal, G.S. (1980). What should be done with equity
justice. Group and Organization Management, 27: 324–351. theory? New approaches to the study of fairness in social
Folger, R. & Konovsky, M.A. (1989). Effects of procedural and relationships. In K. Gergen, M. Greenberg and R. Willis (Eds),
distributive justice on reactions to pay raise decisions. Academy Social exchange: Advances in theory and research, 27–55. New
of Management Journal, 32(1): 115–130. York: Plenum.
Google Scholar (2011). Search for Colquitt, J.A., Conlon, D.E., Lind, E.A. (2001). Fairness heuristic theory: Justice judgments as
Wesson, M.J., Porter, C.O. & Ng, K.Y. (2001). Justice at the pivotal cognitions in organizational relations. In J. Greenberg
millennium: A meta-analytic review of 25 years of organi- and R. Cropanzano (Eds), Advances in organizational justice,
zational justice research. Journal of Applied Psychology, 86: 56–88. Stanford, CA: Stanford University Press.
425–445. Retrieved July 13, 2011 from http://scholar.google Moorman, R.H. & Byrne, Z.S. (2005). How does organizational
.com. justice affect organizational citizenship behavior? In J. Greenberg
Greenberg, J. (1993). The social side of fairness: Interpersonal and and J.A. Colquitt (Eds), The handbook of organizational justice,
informational classes of organizational justice. In R. Cropanzano 355–382. Mahwah, NJ: Erlbaum.
(Ed), Justice in the work place: Approaching fairness in human Nunnaly, J.C. & Bernstein, I.H. (1994). Psychometric theory. New
resource management, 79–103. Hillsdale, NJ: Erlbaum. York: McGraw-Hill.
Greenberg, J. & Colquitt, J.A. (2005). The handbook of Skitka, L.J., Winquist, J. & Hutchinson, S. (2003). Are outcome
organizational justice. Mahwah, NJ: Erlbaum. fairness and outcome favorability distinguishable psycho-
Hollensbe, E., Khazanchi, S. & Masterson, S. (2008). How do I logical constructs? A meta-analytic review. Social Justice
assess if my supervisor and organization are fair? Identifying Research, 16: 309–341.
the rules underlying entity-based justice perceptions. Academy Sweeney, P.D. & McFarlin, D.B. (1997). Process and outcome:
of Management Journal, 51(6): 1099. Gender differences in the assessment of justice. Journal of
Johnson, J.W. (2000). A heuristic method for estimating the Organizational Behavior, 18: 83–98.
relative weight of predictor variables in multiple regression. Thibaut, J. & Walker, L. (1975). Procedural justice: A psychological
Multivariate Behavioral Research, 35: 1–19. analysis. Hillsdale, NJ: Erlbaum.
Johnson, J.W. (2001). Determining the relative importance of Tonidandel, S., LeBreton, J.M. & Johnson, J.W. (2009).
predictors in multiple regression: Practical applications of Determining the statistical significance of relative weights.
relative weights. In F. Columbus (Ed.), Advances in Psychology Psychological Methods, 14: 387–399.
Appendix A
Table A1 Correlation matrices used in relative weight analyses – based on Colquitt et al. (2001) meta-analytic data, data corrected for
attenuation due to unreliability
1 2 3 4 5 1 2 3 4 5
1. Outcome satisfaction 0.48 0.19 0.3 0.61 1. Organizational 0.22 0.29 0.26 0.15
citizenship
2. Procedural justice 0.53 0.69 0.64 0.67 2. Procedural justice 0.25 0.69 0.64 0.67
3. Interpersonal justice 0.19 0.63 0.66 0.42 3. Interpersonal justice 0.29 0.63 0.66 0.42
4. Information justice 0.3 0.58 0.66 0.46 4. Information justice 0.26 0.58 0.66 0.46
5. Distributive justice 0.61 0.57 0.42 0.46 5. Distributive justice 0.15 0.57 0.42 0.46
1. Job satisfaction 0.62 0.35 0.43 0.56 1. Withdrawal 0.46 0.02 0.24 0.5
2. Procedural justice 0.4 0.69 0.64 0.67 2. Procedural justice 0.34 0.69 0.64 0.67
3. Interpersonal justice 0.35 0.63 0.66 0.42 3. Interpersonal justice 0.02 0.63 0.66 0.42
4. Information justice 0.43 0.58 0.66 0.46 4. Information justice 0.24 0.58 0.66 0.46
5. Distributive justice 0.56 0.57 0.42 0.46 5. Distributive justice 0.5 0.57 0.42 0.46
Table A1 Continued
1 2 3 4 5 1 2 3 4 5
1. Organizational 0.57 0.19 0.29 0.51 1. Supervisor evaluation 0.64 0.62 0.65 0.59
commitment
2. Procedural justice 0.37 0.69 0.64 0.67 2. Procedural justice 0.6 0.69 0.64 0.67
3. Interpersonal justice 0.19 0.63 0.66 0.42 3. Interpersonal justice 0.62 0.63 0.66 0.42
4. Information justice 0.29 0.58 0.66 0.46 4. Information justice 0.65 0.58 0.66 0.46
5. Distributive justice 0.51 0.57 0.42 0.46 5. Distributive justice 0.59 0.57 0.42 0.46
Note: Data above the diagonal is based on the Broadly Defined Procedural Justice (BDPJ) operationalization of procedural justice. Data below the
diagonal is based on the Procedural Justice Fairness perceptions (PJFP) operationalization.
Appendix B
Table B1 Results of the relative weights analysis, showing the relative contribution to R2 of the four justice dimensions on employee
outcome variables, data from the meta-analytic results of Colquitt et al. (2001). Data left uncorrected for attenuation due to unreliability.
Raw relative Relative weights as Raw relative Relative weights as Raw relative Relative weights as
weights percentage of weights percentage of weights percentage of
R-square R-square R-square
BDPJ PJFP BDPJ (%) PJFP (%) BDPJ PJFP BDPJ (%) PJFP (%) BDPJ PJFP BDPJ (%) PJFP (%)