FABIAN BEIGANG* Reconciling Algorithmic

Fairness Criteria

I. INTRODUCTION

When the discussion about algorithmic fairness first started to receive aca-
demic attention, much of the debate was about criteria that use aggregate
statistics of observed data to determine whether a predictive model is fair.
At the center of the debate were the two criteria equalized odds and predic-
tive parity. Equalized odds requires that false positive and false negative
error rates be equal for different protected groups (e.g., men and women)
in order for a predictive model to be considered fair. Predictive parity, on
the other hand, stipulates that a model must have equal predictive values
for different protected groups. At first glance, both criteria seem like rea-
sonable and intuitive conditions for algorithmic fairness.
As it turned out, however, the two criteria are (under realistic circum-
stances) mutually incompatible.1 This means, it is in most cases impossible
to satisfy both—when one is satisfied, the other must be violated. This was
a frustrating result since both conditions have some intuitive appeal. At
the same time, these impossibility results inadvertently provided a justifi-
cation for companies, governments, and other organizations to use predic-
tive models which violate one of the fairness criteria: they could simply
argue that the model cannot but violate the criterion since it satisfies the
other.

I am grateful to Christian List, Richard Bradley, and Liam Kofi Bright for their comments
and feedback. Special thanks go to Vito Szeremeta for significant help with the mathematical
proofs in this article. Moreover, I want to thank the two anonymous Associate Editors at Phi-
losophy & Public Affairs for their comments that helped me significantly improve the article.
1. Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan, “Inherent Trade-offs in the
Fair Determination of Risk Scores,” arXiv:1609.05807 (2016); Alexandra Chouldechova, “Fair
Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments,” Big
Data 5, no. 2 (2017): 153–63.

© 2023 The Author. Philosophy & Public Affairs published by Wiley Periodicals LLC.
Philosophy & Public Affairs 9999, no. 9999
This is an open access article under the terms of the Creative Commons Attribution-
NonCommercial-NoDerivs License, which permits use and distribution in any medium,
provided the original work is properly cited, the use is non-commercial and no modifications
or adaptations are made.
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
2 Philosophy & Public Affairs

To illustrate the impossibility of satisfying both equalized odds and pre-

dictive parity in a practical setting, consider a predictive model that esti-
mates a person’s risk of defaulting on a loan. Let us say this model is used
by a bank to renegotiate the loan terms. To retrospectively compare the
model’s performance for two different demographic groups, we could
examine how many people in each group who defaulted on their loan
were predicted to do so. Equalized odds requires that this proportion be
the same for both groups. Alternatively, we could check how many people
in each group who were predicted to default actually did default. Predic-
tive parity requires that this proportion be the same for both groups. How-
ever, if one group has a lower default rate than the other, it is impossible
to satisfy both equalized odds and predictive parity simultaneously. If
equalized odds is satisfied, the proportion of those who do in fact default
among those predicted to default will differ between the two groups, vio-
lating predictive parity. Conversely, if predictive parity is satisfied, equal-
ized odds will be violated. The bank could exploit this fact to justify the
use of a discriminatory predictive model.
Various approaches have been proposed to address the issue of the
impossibility theorem. Some authors have suggested abandoning one of
the fairness criteria,2 while others have proposed that the choice of crite-
rion should depend on the context.3 Still, others have argued for the aban-
donment of both equalized odds and predictive parity in favor of an
alternative criterion.4 However, given the intuitive appeal of these two
criteria, it is hard to accept any of these options. As a result, there remains
a lack of consensus on how best to deal with the impossibility theorem.
In this article, I will argue that both criteria can be modified in a way
that retains their intuitive appeal and renders them universally compati-
ble. Instead of requiring that error rates or predictive values be equal

2. See, e.g., Deborah Hellman, “Measuring Algorithmic Fairness,” Virginia Law Review
106, no. 4 (2020): 811–66; Brian Hedden, “On Statistical Criteria of Algorithmic Fairness,” Phi-
losophy & Public Affairs 49, no. 2 (2021): 209–31.
3. See, e.g., Michele Loi, Anders Herlitz, and Hoda Heidari, “Fair Equality of Chances for
Prediction-based Decisions,” Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics,
and Society (2021): 756.
4. See, e.g., Matt J. Kusner, Joshua R. Loftus, Chris Russell, and Ricardo Silva, “Counter-
factual fairness,” Proceedings of the International Conference on Advances in Neural Informa-
tion Processing Systems 30 (2017): 4066–76; Cynthia Dwor, Moritz Hardt, Toniann Pitassi,
Omer Reingold, and Richard Zemel, “Fairness through awareness,” Proceedings of the 3rd
Innovations in Theoretical Computer Science Conference (2012): 214–26.
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
3 Reconciling Algorithmic Fairness
Criteria

across protected groups, the modified equalized odds and predictive par-
ity criteria require that the protected characteristic does not cause discrep-
ancies in these metrics across groups. To formalize these modified
versions of equalized odds and predictive parity, I will use a method called
matching, which is typically used for causal inference in observational
studies.
The remainder of this article is organized as follows. In Section II, I
introduce the two statistical fairness criteria, equalized odds and predictive
parity, and present the Kleinberg-Chouldechova impossibility theorem
that involves them. I then present criticisms of the two criteria. In Section-
III, I turn away from fairness for a moment to introduce the method of
matching. In Section IV, I utilize this method to define versions of equal-
ized odds and predictive parity that better capture their intended interpre-
tation. As will be shown, the two modified criteria are universally
compatible. Section V concludes the article.

II. AN INTERPRETATION OF THE KLEINBERG-CHOULDECHOVA IMPOSSIBILITY

Let us begin by providing more precise definitions of the two statistical

fairness criteria equalized odds and predictive parity.5 We will here focus
on binary predictions (or classifications) and take the predictive model to
be a function from a set of input variables to the prediction. We will
denote the variable representing the protected characteristic with
A (which will also be assumed to be binary), the prediction with Yb , and
the target variable that is to be predicted with Y.
Let P be a joint probability distribution function of A, Yb , and Y, such
that (i) P(E) ≥ 0 for all events E in the event space defined on the sample
space; (ii) P(Ω) = 1, where Ω is the sample space, which is the set of all
possible outcomes; and (ii) P(E1[E2) = P(E1) + P(E2) for all events E1 and
E2 that are mutually exclusive in the event space, meaning they cannot
both occur simultaneously. The event space is a collection of subsets of
the sample space that represent different possible events. We will also
denote conditional probability (i.e., the probability of an event occurring
given another event) by PðE 1 jE 2 Þ, which is defined as PðE 1 jE 2 Þ ¼ PðEP1ðE\2 EÞ 2 Þ.

5. Moritz Hardt, Eric Price, and Nati Srebro, “Equality of Opportunity in Supervised
Learning,” Advances in Neural Information Processing Systems 29 (2016); Chouldechova, “Fair
Prediction with Disparate Impact.”
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
4 Philosophy & Public Affairs

The probability of a variable V taking on the value v is denoted as P

(V = v), abbreviated as P(v) when there is no ambiguity. The probability of
two events occurring simultaneously, V = v and U = u, will be abbreviated
as P(v, u). We can now formally define the two criteria as follows:

Definition 1: (Equalized odds). A predictive model satisfies equalized

odds (relative to protected characteristics a1, a2DA) if and only if for all
y  Db and yDY, Pðb
b y ja1 , yÞ ¼ Pðb
y ja2 ,y Þ.
Y

Definition 2: (Predictive parity). A predictive model satisfies predictive

parity (relative to protected characteristics a1, a2DA) if and only if for all
b
y  Db and yDY, Pðyja1 ,b yÞ ¼ Pðyja2 ,b
y Þ.
Y
In most contexts, equalized odds and predictive parity cannot be satis-
fied simultaneously. This follows from a theorem of which two versions
were simultaneously and independently proved by Chouldechova6 and
Kleinberg.7 More precisely, the theorem states that whenever the preva-
lence, that is, the relative frequency of an occurrence of the event repre-
sented by the target variable, is different for different protected groups, a
predictive model which satisfies equalized odds must violate predictive
parity, and vice versa.8 For example, a predictive model intended to pre-
dict whether a defendant will reoffend (i.e., commit a future crime) cannot
at the same time produce equal error rates and have equal predictive
values for different ethnic groups, if the prevalence of reoffence (i.e., the
relative frequency of defendants committing another crime in the future)
differs across these groups. To state this more concisely, the theorem can
be formulated as follows:

Theorem 1: (The Kleinberg-Chouldechova impossibility). If the prevalence

differs across protected groups, no (imperfect) predictive model can satisfy
both equalized odds and predictive parity.

6. Chouldechova, “Fair Prediction with Disparate Impact.”

7. Kleinberg, Mullainathan, and Raghavan, “Inherent Trade-offs in the Fair Determination
of Risk Scores.”
8. Strictly speaking, this is only true for imperfect predictive models, that is, models that
are not guaranteed to always predict correctly. Since in real world situations no predictive
model could ever be guaranteed to only make correct predictions, this restriction does not
limit the significance of the impossibility result.
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
5 Reconciling Algorithmic Fairness
Criteria

If both equalized odds and predictive parity were deemed universally

necessary conditions for predictive fairness, this impossibility theorem
would suggest that truly fair predictive models are an unattainable goal.
There is, however, another possible interpretation of this impossibility:
namely that it highlights a flaw in our formalization of what constitutes
fair predictive models. The impossibility result can then be seen as an
indicator that we have to rethink the definitions of the two fairness criteria
and reevaluate whether they actually formalize the more intuitive ideas
they are intended to formalize. The argument pursued here is along those
lines. I will argue that while the intuitive appeal of both fairness criteria is
undeniable, they impose stronger requirements than what is necessary in
order to avoid certain types of unfairness in predictive models.
Before we examine both criteria, let us first consider how we can cash
out the broader idea of fairness for predictive models. Algorithmic fairness
is, in the context of such models, generally taken to mean the absence of
discrimination. Discrimination, in turn, can be understood as the wrongful
disadvantageous treatment of an individual on the basis of a sensitive
characteristic like ethnicity, gender, or religion.9 There is disagreement on
what constitutes wrongful disadvantageous treatment in this context.
However, the most widely accepted definitions are based on the ideas that
the sensitive characteristic is irrelevant in most situations10 and that a per-
son should be treated as an individual.11 Wrongfulness then arises from
basing disadvantageous treatment on an irrelevant sensitive characteristic,
or from treating someone disadvantageously based on presumed statistical
patterns that associate their sensitive characteristic with some other trait
in a way that disregards their individuality.
Predictive models, by definition, are mathematical models used to pre-
dict the value of a specific variable. The output of such a model can be
seen as representing a belief-like propositional attitude. If the possession
of a certain sensitive characteristic leads to disadvantageous predictions,

9. See, e.g., Kasper Lippert-Rasmussen, Born Free and Equal?: A Philosophical Inquiry into
the Nature of Discrimination (New York: Oxford University Press, 2014); Sophia Moreau,
“What is Discrimination?,” Philosophy & Public Affairs 38, no. 2 (2010): 143–79.
10. Lena Halldenius, “Discrimination and Irrelevance,” in The Routledge Handbook of the
Ethics of Discrimination, ed. Kasper Lippert-Rasmussen (New York: Routledge, 2017), 108–18.
11. Benjamin Eidelson, Discrimination and Disrespect, (Oxford: Oxford University
Press, 2015).
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
6 Philosophy & Public Affairs

such as in terms of accuracy or their impact on the decision for which the
model is used, this process can be considered discriminatory.
We can consequently understand equalized odds and predictive parity as
criteria that prevent discriminatory outcomes in predictive models. Equalized
odds can be interpreted as a criterion that prevents discriminatory outcomes
by preventing systematic cognitive bias with regard to a sensitive characteristic.
By systematic cognitive bias, we here mean misjudging how informative a cer-
tain trait is in predicting another trait.12 Assume, for example, that in making
predictions about whether someone will get lung cancer, we overestimate how
informative it is that the person smokes. More precisely, if someone is a
smoker, we predict that they will get lung cancer, and if not, we predict that
they will not get lung cancer. We are clearly biased with regard to smoking in
predicting lung cancer: not everyone who smokes gets lung cancer, and some
people get it without ever having touched a cigarette. It is easy to see that this
will result in different error rates for the group of smokers and the group of
non-smokers. The smokers will have a false negative rate of 0 (simply for the
fact that no smoker was predicted to not get lung cancer) but a false positive
error rate above 0 (some smokers do not get lung cancer). Vice versa, the non-
smokers will have a false negative rate above 0 (there are some who get lung
cancer, but we never predict a non-smoker to get lung cancer) but a false posi-
tive rate of 0 (because no non-smoker is predicted to get lung cancer). One
could say that this model is systematically biased with regard to smoking in
predicting lung cancer. If, however, instead of using the predictive model just
described we used a predictive model which guarantees that the error rates
across smokers and non-smokers are equal, then we could be sure that the
predictor contains no such bias. While being a smoker is typically not consid-
ered a sensitive characteristic, overestimating the informativeness of a sensitive
characteristic like gender could lead to disadvantageous predictions on the
basis of an irrelevant (or less relevant than warranted) sensitive characteristic.13

12. Cognitive here refers to the fact that we only consider bias in terms of belief-like prop-
ositional attitudes, as opposed to bias in terms of other propositional attitudes like
preferences.
13. While one might argue that disadvantageous treatment is problematic irrespective of
whether the trait whose informativeness is being overestimated is a sensitive characteristic or
not, we here adopt the widely accepted assumption that from a moral perspective, disadvan-
tageous treatment is particularly worrisome if it occurs on the basis of a sensitive characteris-
tic. The results presented in this article, however, do not depend on this assumption. For a
discussion, see, e.g., Eidelson, Discrimination and Disrespect, 30ff.
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
7 Reconciling Algorithmic Fairness
Criteria

FIGURE 1 Two Different Causal Models Potentially Producing the Same Discrepancy
in Error Rates across Religious Groups

Yet, it is important to note that a violation of equalized odds across

protected groups is an indicator and not a definition of systematic cogni-
tive bias. To see this, note that the statement relating error rates to bias is
a conditional: if there is bias with regard to trait A, there will be disparities
in error rates between those with trait A and those without. By simple
logic, this implies that whenever there are no disparities in error rates,
there is no bias. Yet, it does not imply that whenever we observe dispar-
ities in error rates between those with trait A and those without, we can
conclude that the predictor is biased with regard to A. In other words,
equalized odds relative to A is a sufficient condition for the absence of
bias with regard to A, but not a necessary one. Hence, trying to deduce
that a predictor is biased from the observation that error rates among
groups differ amounts to committing the well-known fallacy of affirming
the consequent. At best, observing disparities in error rates allows one to
make an inference to the best explanation: when disparities in error rates
between two groups are observed, and there is no other plausible explana-
tion, then one is justified in suspecting that this is due to bias with regard
to the trait that distinguishes the groups. While this inference may seem
plausible in many cases, it is important to note that it is a fallible infer-
ence, and that is what matters for our purposes (Figure 1).
To illustrate this with an example, imagine a health insurance company
that tries to predict the healthcare costs an individual incurs in a given
year in order to decide how to set their customers’ premiums. To simplify
things, imagine the company is trying to predict only whether an individ-
ual’s annual costs are above a certain threshold. This allows us to repre-
sent the target variable and the prediction as binary variables. Now
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
8 Philosophy & Public Affairs

imagine that in country C, citizens of religion R1 are, on average, younger

than citizens of religion R2 (we can imagine that this is due to the fact that
many people of religion R1 in C have recently immigrated, and that people
generally tend to immigrate when they are somewhat younger). Suppose
that, upon examination, the predictions turn out to have a higher false
positive rate for people of religion R1 than for people of religion R2. Can
we conclude that the predictive model the insurance company used has a
discriminatory bias against people of religion R1? The observation of dif-
ferent error rates does not conclusively establish this. Different explana-
tions for this discrepancy are conceivable.
Imagine first a scenario in which the insurance company uses a predic-
tive model which solely takes the individual’s age into account. Now ima-
gine further that the predictor is biased with regard to age, in that it
overestimates how informative young age is of risky behavior, and hence
of increased health costs. This, as we have shown above, will obviously
lead to higher false positive rates for predictions of high health costs
among young people. Because, on average, people of religion R1 are youn-
ger, and the predictive model is biased with regard to age, it will produce
predictions with a higher false positive rate for people of religion R1.
However, it can be argued that the outcomes of this predictive model
do not discriminate against people of religion R1. To see this, consider the
following. Imagine that instead of C, the health insurance company oper-
ated in a different country D, where citizens of religion R2 are, on average,
younger than citizens of religion R1. Again, we can assume that this is
because in this context people of religion R2 are mostly recent immigrants
to D. It is easy to see that here, the predictive model would produce pre-
dictions with a higher false positive rate for people of religion R2. Remem-
ber that this model is exactly the same as the one above and that the
higher false positive rate for R2 would not occur in C, where the reverse
was the case. If we define bias as disparities in observed error rates, we
would come to the somewhat contradictory conclusion that the predictive
model is biased against people of religion R1, but that, had the insurance
company applied the exact same predictive model in a different country,
the model would be biased against people of religion R2. We can see that
which religion a person has does not, in any sense, influence the predic-
tions (or, for that matter, the error rates). It only happens to be the case
that, in the given context, the predictive model works on average less well
for one religious group than for another. While this might be worrisome
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
9 Reconciling Algorithmic Fairness
Criteria

in its own right, it can hardly be considered discrimination on the basis of

religion, as there is no explanatory relation between a customer’s religion
and the predicted health costs.
Compare this with a second scenario, depicted in 1(b), in which the
insurance’s predictor takes a person’s religion into account in order to
make a healthcare cost prediction. From an observational point of view,
the two predictors’ performances might be indistinguishable, as they
could both produce the same discrepancies in error rates between
different religious groups. Yet, on a narrow understanding of systematic
cognitive bias, only the latter can be said to be biased against people of
religion R1.
This is of course not to say that disparities in error rates among differ-
ent protected groups are of no moral concern by themselves. Disparities
in error rates can, if tied to decision-making, lead to unjust distributions
of resources and burdens. Yet, we are here following Eidelson14 in
claiming that (direct and structural) discrimination is conceptually distinct
from matters of distributive justice, and trying to subsume one under the
other will get in the way of a clear analysis of each. Especially in the con-
text of algorithmic decision-making, it seems that distinguishing between
discriminatory predictions and unjust decisions is necessary in order to
determine an appropriate method of mitigating the unfairness without
introducing other unanticipated biases into the decision-making process.15
We are here solely concerned with fairness criteria for predictions. A dis-
cussion of distributive justice arising from algorithmic decision-making,
although important, is beyond the scope of this article.
Let us now turn to predictive parity. Predictive parity is often inter-
preted as a criterion that prohibits the meaning of a prediction from
depending on a person’s sensitive characteristic, as doing so could incen-
tivize discriminatory behavior.16 Here, a similar observation can be made.
Imagine a medical device that tests for a specific disease. Given a person
has the disease, there is a 95 percent probability that the test turns out
positive. When applied to a person who is healthy, there is a 5 percent
probability that the test nonetheless turns out positive. This, we can

14. Eidelson, Discrimination and Disrespect.

15. Fabian Beigang, “On the Advantages of Distinguishing Between Predictive and Allocat-
ive Fairness in Algorithmic Decision-Making,” Minds and Machines 32 (2022): 655–82.
16. Kleinberg, Mullainathan, and Raghavan, “Inherent Trade-offs in the Fair Determina-
tion of Risk Scores.”
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
10 Philosophy & Public Affairs

imagine, can be shown to robustly hold across genders. There is no differ-

ence whatsoever in the likelihood of receiving an erroneous result, no
matter whether a patient is male or female. Intuitively, it seems, there is
no difference in meaning of the prediction for men and for women.
But now imagine that the disease happens to occur more frequently in
men. More specifically, we can imagine that one in every 10 men has the
disease, but only one in every 100 women does. Then the positive predic-
tive value, that is, the probability of actually having the disease given that
one receives a positive test result, is different for men and women. For
men it is roughly 68 percent, whereas for women it is only about 16 per-
cent.17 This means, in this intuitively fair case, predictive parity is not sat-
isfied. But it seems that this is not due to bias in the testing device, but
just to the prevalence of the disease, which differs across genders. In other
words, it is not gender which causes the difference in predictive value
(since the testing device works, by assumption, equally well for a ran-
domly chosen man as for a randomly chosen woman). So it seems that
here, too, we want to distinguish between discrepancies in predictive
value which are (causally) explained by gender, and discrepancies in pre-
dictive value which are due to external factors, such as differences in the
prevalence of a disease.
In light of these criticisms, it seems that the definitions of both equal-
ized odds and predictive parity do not adequately explicate the underly-
ing moral intuitions they were designed to capture.18 This, in turn,
could mean that the Kleinberg-Chouldechova impossibility result is not
so disastrous after all. If neither equalized odds nor predictive parity
are, as they are currently defined, necessary conditions for fairness, the
impossibility loses its bite. There is a chance that the impossibility theo-
rem is just an artifact of the way the criteria are defined. The remainder
of this article will examine this possibility by trying to provide modified
definitions of equalized odds and predictive parity that retain all the
intuitively plausible aspects of the current definitions but avoid the
impossibility.

17. See calculation in Appendix A and Appendix B.

18. Explication is a method which seeks to turn an informal concept into an exact, for-
mally defined concept. For an overview of the method of explication, see, e.g., Catarina
Dutilh Novaes, “Carnapian explication and ameliorative analysis: a systematic comparison,”
Synthese 197, no. 3 (2020): 1011–34.
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
11 Reconciling Algorithmic Fairness
Criteria

III. THE MATCHING METHOD

We can use matching—a method for causal inference on the basis of

observational data19—to define modified versions of equalized odds and
predictive parity. In this section, I will explain the method.
The motivation behind matching stems from the following problem. In
many scenarios, it would be useful to be able to infer whether and to
which degree a given variable has a causal effect on some other variable.
The “gold standard” for estimating causal effects is the so-called random-
ized controlled trial—a specific type of experimental study. Often, however,
it is practically impossible or unethical to run experiments, or the only
available data is observational data. Think, for instance, about studying
the health effects of passive smoking on children. It would be unethical to
actively expose a group of children to secondhand smoke. Yet, there might
be observational data on the health of children that live in a home where
at least one of the parents smokes. Matching aims to replicate the proper-
ties of a randomized controlled trial for observational data as best as pos-
sible, to allow for the estimation of causal effects in cases like the above,
where experimental data is unavailable.
Randomized controlled trials estimate causal effects by comparing the
results of an intervention on one group of randomly selected participants
(the treatment group) to a control group who do not receive the interven-
tion. This ensures that any observed differences between the groups are
due to the intervention and not other factors. For example, in a drug trial
to test the effectiveness of an anti-rheumatic drug, participants could be
randomly assigned to receive either the drug or a placebo, and the effec-
tiveness of the drug could be determined by comparing the frequency of
rheumatic symptoms in the two groups. A key advantage of randomized
controlled trials over observational studies is that they can control for con-
founding factors that may affect both who receives the treatment and the
outcome being measured.
Contrast this with a trial without randomization. If the anti-rheumatic
drug were administered to anyone willing to take it, and the results were
compared to those who did not receive the drug, the results could be con-
founded by various factors. One such factor is the age of the participants.
Older people are more likely to suffer from rheumatic arthritis and might

19. Elizabeth A. Stuart, “Matching Methods for Causal Inference: A Review and a Look
Forward,” Statistical Science 25, no. 1 (2010): 1–21.
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
12 Philosophy & Public Affairs

hence be more willing to take anti-rheumatic drugs. At the same time, age
obviously also affects whether and with which frequency someone experi-
ences rheumatic symptoms. This means that the treatment group may
have a higher average baseline frequency of rheumatic symptoms than
the control group, which could lead to the incorrect conclusion that the
drug is not effective. In a randomized controlled trial, however, the age
distributions of the treatment and control groups would be roughly equal,
allowing for a more accurate assessment of the drug’s effectiveness.
When we only have observational data, we can try to mimic randomiza-
tion via matching. Instead of randomly assigning individuals to the treat-
ment or control group, observational data can be used to create a
synthetic control group that does not systematically differ from the treat-
ment group on any observed or unobserved variables other than the treat-
ment (i.e., causal) variable, effectively achieving randomization. This is
done as follows. Assume that the data consists of information on the
causal variable, the effect variable, and a number of other variables,
which, in this context, will be called the covariates. For each individual in
the treatment group,20 we pick the individual from the initial control
group whose covariate values are as similar as possible to the covariate
values of the individual from the treatment group. These individuals con-
stitute the synthetic control group. We end up with a treatment and a syn-
thetic control group that have identical or very similar distributions over
the covariates, and which—given certain assumptions that we will get to
in a moment—allow for the conclusion that any significant difference
between the groups with regard to the effect variable is caused by the dif-
ference in the causal variable.
To illustrate this, we can apply matching to the example of evaluating
the effectiveness of the supposedly anti-rheumatic drugs on the presence
of rheumatic symptoms. To create a synthetic control group, we would try
to find individuals in the initial control group with ages that are as close
as possible to the ages of the individuals in the treatment group, as well as
similar levels of pretreatment rheumatic symptoms. By creating a synthetic

20. Note that the terms treatment and control group are, in the context of observational
data, to be understood figuratively. No individual is actually assigned to a specific group.
Rather, it is the case that for some individuals, the supposed causal property is present
(T = 1)—these we call the treatment group—and for others, it is not present (T = 0)—these
we call the initial control group. To highlight the analogy to randomized controlled trials, we
will stick to the terms.
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
13 Reconciling Algorithmic Fairness
Criteria

control group with these individuals, we would ensure that the treatment
and control groups have similar distributions of age and symptomaticity,
allowing us to conclude that any significant differences in the presence of
rheumatic symptoms between the groups is likely due to the treatment
rather than age or pretreatment health. Assuming that the drug is effec-
tive, we would expect the frequency of rheumatic symptoms to be lower
in the matched control group than in the treatment group. Of course,
there may still be other unobserved confounding factors that could affect
the results, but matching allows us to control for observed confounding
factors and provide a more accurate assessment of the treatment’s
effectiveness.
Let us now address a central methodological question, namely how to
choose covariates. In order for matching to be as good a method for esti-
mating causal effects as randomization, it is crucial that the set of
covariates contains all the variables that influence both the causal and the
effect variable (i.e., all confounding variables). This is important because it
entails that there are no unobserved differences between the control and
treatment group conditional on the observed covariates.21 The assumption
that this is the case is typically called ignorability. In our example above,
ignorability is most likely not satisfied: there could be other variables that
influence both treatment and effect variables, like for instance weight, pre-
vious injuries, and so on. Even though we have removed some con-
founding by matching on age and pretreatment rheumatic symptoms, we
could still get a better estimate of the drug’s effect if we had moreover
matched the data on these other confounding factors. Another important
norm for choosing covariates is not to include any variables that are caus-
ally influenced by the causal variable. This might lead to an underestima-
tion of the causal effect and thereby distort the analysis.22
Lastly, let us mention that there is an entire family of matching
methods. The make-up of the obtained synthetic control group depends
on the distance metric used, whether one uses 1:1 or 1:n matching (where
the matched data point is the average of the n most similar data points),
and whether matching is done with replacement or without (i.e., whether
a data point can be matched more than once). The specific choice of

21. Stuart, “Matching Methods for Causal Inference,” 3f.

22. Judea Pearl, “The Foundations of Causal Inference,” Sociological Methodology 40, no.
1 (2010): 75–149.
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
14 Philosophy & Public Affairs

matching method, however, is of no importance to the overall argument

presented here.

IV. MODIFYING EQUALIZED ODDS AND PREDICTIVE PARITY

Let us now return to the attempt to modify equalized odds and predictive
parity to avoid the Kleinberg-Chouldechova impossibility. We will con-
sider both fairness criteria in turn.

IV.A. Matched Equalized Odds

Let us begin by laying out the intended interpretation of the fairness crite-
rion which is to replace equalized odds. We will call this criterion matched
equalized odds. Matched equalized odds requires that the protected char-
acteristic has no direct effect on the predictor in a way that affects its error
rates. For instance, if a recidivism predictor satisfies matched equalized
odds, it means that the fact that a defendant is African- American does
not increase the probability of receiving a false positive recidivism
prediction.23
How can we determine if a predictor satisfies matched equalized odds?
We can approach this as a causal inference problem by identifying the
causal effect of the protected characteristic on a predictor’s error rates.
This problem, as explained in the previous section, can be addressed
using matching. We treat the protected characteristic as the treatment var-
iable and select an appropriate set of covariates. We then create a mat-
ched control group, such that the treatment and control group (i.e., the
two protected groups) exhibit no systematic differences other than in their
protected characteristics and the predictions they receive. Finally, we

23. Although not within the scope of this article, it is worth noting that there exists an
active discourse on the interpretability of sensitive characteristics such as race and gender as
causes. In agreement with Liam Kofi Bright and Daniel Malinsky, I adopt the stance that they
can indeed be interpreted as causes. For a comprehensive examination of this topic, see,
e.g., Daniel Malinsky and Liam Kofi Bright, “On the Causal Effects of Race and Mechanisms
of Racism” (unpublished manuscript, 2021); Liam Kofi Bright, Daniel Malinsky, and Morgan
Thompson, “Causally Interpreting Intersectionality Theory,” Philosophy of Science 83, no.
1 (2016): 60–81; Paul W Holland, “Causation and Race,” in White logic, White methods: Rac-
ism and Methodology, eds. Tukufu Zuberi and Eduardo Bonilla-Silva (Lanham, MD:
Rowman & Littlefield Publishers, Inc. 2008): 93–109; Lily Hu, “What’s ‘Race’ in Algorithmic
Discrimination on the Basis of Race,” Journal of Moral Philosophy (forthcoming).
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
15 Reconciling Algorithmic Fairness
Criteria

compare the error rates of the two groups. If (and only if) the error rates
are equal, matched equalized odds is satisfied.
Some more words are to be said about the choice of covariates. The
first point is that we can, of course, only determine whether a predictor
satisfies matched equalized odds if the covariates satisfy the ignorability
assumption. Any reliable evaluation of a predictor’s fairness hence
requires data on an appropriate amount of variables so that we can ensure
that there are no unobserved differences between the two matched
protected groups. The second point is that, in contrast to standard appli-
cations of matching, in which we aim to determine the total causal effect
of some treatment variable on the effect variable, we are here specifically
interested in the natural direct effect24 of the protected characteristic on
the prediction. More precisely, we are interested in the causal effect along
those causal paths that are not mediated by the target variable. In many
cases, namely when the protected characteristic does not influence the
target variable, this just is the total causal effect. But there are some cases
where the protected characteristic does in fact influence the target vari-
able. In those cases, we want to exclude the causal effect along this path
from the analysis. Technically, this means that we have to include the tar-
get variable in the set of covariates.
This is normatively justified because, from the point of view of moral
permissibility, causal effects along paths mediated by the target variable
seem unproblematic. When we are trying to predict the value of a vari-
able, the variable that is to be predicted should influence the prediction.
Think, for instance, of a medical classifier to detect whether a person is
color blind. Obviously, the fact that a person is color blind should influ-
ence the classifier’s prediction of whether they are color blind. In the ideal
case, the prediction perfectly aligns with whether the person is actually
color blind. Color blindness, however, seems to be influenced by sex-
specific genetic differences.25 This, in turn, implies that in our example,
there is a causal link from a person’s sex to the prediction of whether they

24. Chen Avin, Ilya Shpitser, and Judea Pearl, “Identifiability of Path-Specific Effects,” Pro-
ceedings of International Joint Conference on Artificial Intelligence (2005): 357–63; Judea Pearl,
“Interpretation and Identification of Causal Mediation,” Psychological Methods 19, no.
4 (2014): 459–81; Naftali Weinberger, “Path-Specific Effects,” The British Journal for the Phi-
losophy of Science 70, no. 1 (2019): 53–76.
25. Israel Abramov, James Gordon, Olga Feldman, and Alla Chavarga, “Sex and vision II:
color appearance of monochromatic lights,” Biology of Sex Differences 3, no. 21 (2012): 1–15.
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
16 Philosophy & Public Affairs

are color blind via the target variable which represents whether they actu-
ally are color blind. Figure 2 illustrates this. Since we are interested in
whether discrepancies in error rates arise from an unjustified direct influ-
ence of the protected attribute on the prediction (which cannot be attrib-
uted to group-specific differences in the target variable distribution), we
exclude the mediated causal path from our analysis. This is achieved by
adding the target variable to the covariates.
We can make the above precise by giving a formal definition of mat-
ched equalized odds. To this end, let P* be the joint probability distribu-
tion function of A, Yb , and Y in the data set that is obtained by applying
matching to the original data set such that A = a1 indicates the treatment
and A = a2 the control group, and where the covariates include
Y (as opposed to P, which refers to the distribution of these variables on
the original data set). Matched equalized odds can then be defined as
follows:

Definition 3: (Matched equalized odds). A predictive model satisfies mat-

ched equalized odds (relative to protected characteristics a1, a2  DA) if
and only if for all b
y  Db and y  DY,
Y

P* ðb
yja1 , y Þ ¼ P* ðb
y ja2 , yÞ,

Let us now consider whether this definition escapes the criticism

against equalized odds offered in Section II. Recall that we have argued
that one reasonable way of interpreting equalized odds is to think of it as
a criterion against systematic cognitive bias, where this is understood as a
misjudgment of how informative a given trait is in predicting another trait.
We considered an example in which a predictor of health costs misjudges
the informativeness of its only input variable age, but, by assumption, not
of religion (which is not among the input variables). Its predictions none-
theless exhibit a higher false positive rate for people of a specific religion
due to the fact that members of this religious group happen to be younger
on average than members of other religious groups. Despite a discrepancy
in observed error rates, we would not want to say that this predictor is
unfairly biased against this religious group.
Matched equalized odds gets this right. In order to evaluate the predic-
tor, we would take the data on all the individuals from the relevant
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
17 Reconciling Algorithmic Fairness
Criteria

FIGURE 2 Direct and Indirect (Dashed) Causal Effect from Sex to the Prediction of
Color Blindness. Matched Equalized Odds is Concerned Only with the Direct Causal
Effect

religious group as the treatment group and create a synthetic control

group from the data on individuals from other religious groups. We would
do this by matching them on a number of covariates, which (minimally)
have to include age (since this is a confounder) and actual health costs
(the target variable). This would, consequently, result in a treatment and a
synthetic control group which have equal (or very similar) distributions
for the variables age and actual health costs. Since by assumption the pre-
dictor only takes age as an input variable, the two groups would not
exhibit significant discrepancies in error rates.
We can conclude that matched equalized odds preserves the intuitive
appeal of equalized odds, while avoiding certain types of counterexam-
ples, as demonstrated in the health cost prediction example. A predictor
that satisfies matched equalized odds ensures that the informativeness of
a protected characteristic is not misjudged. However, it does allow for sit-
uations in which one protected group has a different observed error rate,
as long as this difference is not attributable to the protected characteristic
itself.

IV.B. Matched Predictive Parity

We can define a modified version of predictive parity, which we will call
matched predictive parity, using a similar approach as we did for mat-
ched equalized odds. The aim of matched predictive parity is to ensure
that the protected characteristic does not affect the meaning of the pre-
diction, which we define as the confidence conveyed in a given predic-
tion. In other words, matched predictive parity aims to prevent a
protected characteristic from influencing the predictor in a way that
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
18 Philosophy & Public Affairs

results in differences in positive or negative predictive value. While this

criterion allows for the possibility that a given protected group may have
different average predictive values than another group, it ensures that
any such differences are not due to the protected characteristic (or any
properties affected by it).
As above, we can frame the evaluation of whether a predictor satisfies
matched predictive parity as a causal inference problem we address using
matching. Again, the target variable as well as all confounder variables
need to be defined as covariates. This means, the data set used for deter-
mining matched predictive parity is the same one used for determining
matched equalized odds. We can formally define matched predictive par-
ity as follows:

Definition 4: (Matched predictive parity). A predictive model satisfies

matched predictive parity (relative to protected characteristics a1, a2  DA)
if and only if for all b
y  Db and y  DY,
Y

P* ðyja1 ,b
y Þ ¼ P* ðyja2 ,b
yÞ,

can matched predictive parity handle the counterexample described in

Section II? We imagined a medical testing device that, independent of
gender, has a 95 percent probability of correctly predicting that an individ-
ual has the disease (i.e., its true positive rate, TPR), and a 5 percent proba-
bility of incorrectly predicting that an individual who is actually healthy
has the disease (i.e., its false positive rate, FPR). However, assuming the
disease is much more common among men than among women, the
device would produce predictions that have (on average) a lower positive
predictive value (PPV) for women than for men. We concluded that the
discrepancy in predictive value should not be considered discriminatory
bias despite violating predictive parity because the discrepancy is due only
to the different prevalence levels of the disease in women and men. It is
easy to see that this device would satisfy matched predictive parity. As a
direct consequence of Bayes’ theorem, we know that generally:

TPR*p
PPV ¼
TPR*p þ FPR*ð1  pÞ
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
19 Reconciling Algorithmic Fairness
Criteria

where p is the prevalence of the target variable (i.e., P(Y = 1)). First, by
assumption, TPR, and FPR do not differ across gender groups. Second, we
examine the relative frequencies after matching on various variables,
including the target variable. Therefore, in the matched dataset, the preva-
lence of the target variable is equal or nearly equal between men and
women (i.e., pmale = pfemale). Consequently, the positive predictive value
(PPV) is also equal for men and women. This demonstrates that matched
predictive parity can address some types of counterexamples to predictive
parity while preserving the intuitively plausible aspects of the original
criterion.

IV.C. The Kleinberg-Chouldechova Impossibility Revisited

Let us now return to our initial question, namely whether we can under-
stand the Kleinberg-Chouldechova impossibility theorem as an indication
that the plausible intuitions motivating equalized odds and predictive par-
ity have been formalized in inadequate ways. If this is the right interpreta-
tion, then adequately modified versions of equalized odds and predictive
parity should be universally compatible and hence escape the impossibil-
ity result. We will now analyze whether this is the case for the two modi-
fied criteria we proposed above.
First, note that, assuming ideal matching conditions, we know that for
the relative frequency function P* on the matched data set it is always the
case that P* ðyja1 Þ ¼ P * ðyja2 Þ (for any two groups a1, a2  DA and all y 
DY). This simply follows from the fact that we are required to include the
target variable Y in the set of covariates. The matching thus must result in
a data set where the distribution of Y is identical for both groups a1 and
a2. We can now show that matched equalized odds and matched predic-
tive parity are not only universally compatible, but moreover, turn out to
be mathematically equivalent.

Proof: To see this, we will first show that if a predictive model satisfies
matched equalized odds, it also satisfies matched predictive parity. To this
end, assume we want to evaluate the predictive model relative to a1, a2 
DA. By the assumption of matched equalized odds, the predictive model
satisfies P* ðb
yja1 , yÞ ¼ P * ðb
y ja2 , yÞ, for all y  DY and b
y  Db. From this
Y
assumption, together with the equality stated above, it follows that:
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
20 Philosophy & Public Affairs

P* ðby, a1 jyÞ*P * ðyja1 Þ

P* ðb
y, a1 jyÞ*P* ðyja1 Þ þ P* ðb y, a1 j¬yÞ*P* ð ¬yja1 Þ
ð1Þ
P* ðb y, a2 jy Þ*P* ðyja2 Þ
¼ *
P ðb y, a2 jy Þ*P ðyja2 Þ þ P* ðb
*
y , a2 j¬y Þ*P* ð ¬yja2 Þ

where y and ¬y abbreviate Y = 1 and Y = 0, respectively. By Bayes’ theo-

rem, it follows that for all y  DY.
and b y  Db:
Y

P* ðyja1 ,b
yÞ ¼ P * ðyja2 ,b
yÞ ð2Þ

This is the definition of matched predictive parity. We have hence

shown that matched equalized odds implies matched predictive parity.
Likewise, if we assume matched predictive parity, i.e., that (given fixed
a1 and a2  DA), for all y  DY and b
y  Db the predictive model satisfies
Y
P * ðyja1 ,b
y Þ ¼ P* ðyja2 ,b
yÞ, we can, together with the assumption that
* *
P ðyja1 Þ ¼ P ðyja2 Þ, use Bayes’ theorem to show that this implies matched
equalized odds. As both directions of the proof have an identical structure,
the proof that matched predictive parity implies matched equalized odds
is omitted.
The upshot is then that the infamous impossibility theorem loses its
force when we consider adequately modified versions of equalized odds
and predictive parity. These modifications are normatively justified by
counterexamples that demonstrate the intuitive implausibility of the origi-
nal criteria. Not only do these modifications address the counterexamples,
but they also resolve the Kleinberg-Chouldechova impossibility. Surpris-
ingly, under perfect matching conditions the two criteria even turn out to
be equivalent. This can be interpreted as meaning that we have found the
unique and robust baseline notion of algorithmic fairness which can be
strengthened in at least two (mutually inconsistent) ways.
This conclusion, however, has to be qualified in one respect. We
assumed for the proof that the matching conditions are ideal and that
hence P* ðyja1 Þ ¼ P* ðyja2 Þ for any two groups a1, a2  DA and all y  DY.
In many realistic situations, however, the conditions for matching are not
ideal. Especially if there is a big number of covariates, it is unlikely that for
every individual in the treatment group, one can find an exact match in
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
21 Reconciling Algorithmic Fairness
Criteria

the control group. But only exact matching can guarantee the above
equality. Typically, the methods of choice are distance-based or propen-
sity score matching. Both can give very good results with regard to the
matched data set obtained, in that the distributions over individual
covariates in the treatment and control groups are very similar. Yet, these
two methods cannot guarantee identical distributions over the covariates.
The following two theorems, however, show that even though under
imperfect matching26 the impossibility persists, the smaller the difference
in prevalence, the smaller the trade-off between equalized odds and pre-
dictive parity. More precisely, if equalized odds is satisfied, then the
smaller the difference in prevalence between the groups, the smaller the
difference in positive and negative predictive value. Conversely, if predic-
tive parity is satisfied, then the smaller the difference in prevalence
between the group, the smaller the difference in false positive and false
negative error rates (the proof for the following two theorems can be
found in Appendix B).

Theorem 2: Suppose a predictive model satisfies equalized odds (relative to

a1 and a2  DA). Let Δp denote the difference in prevalence between two
groups a1 and a2, and let ΔPPV(Δp) and ΔNPV(Δp) denote the functions
which return the difference in positive predictive value and the difference in
negative predictive value between the groups for a given value of Δp, respec-
tively. Then ΔPPV(Δp) and ΔNPV(Δp) are both monotonically increasing
in the interval [0, 1].

Theorem 3: Suppose a predictive model satisfies predictive parity (relative to

a1 and a2  DA). Let Δp denote the difference in prevalence between two
groups a1 and a2, and let ΔFPR(Δp) and ΔFNR(Δp) denote the functions
which return the difference in false positive and the difference in false negative
error rates between the groups for a given value of Δp, respectively. Then ΔFPR
(Δp) and ΔFNR(Δp) are both monotonically increasing in the interval [0, 1].
How then do we have to understand the results obtained in this sec-
tion? One important insight is that at least conceptually, there is no
incompatibility between adequately modified versions of equalized odds
and predictive parity. Even if in realistic situations, we can often only

26. Note that this is only a relaxation of the assumption that the distribution of the target
variable is identical for both groups. We still assume that the ignorability assumption holds.
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
22 Philosophy & Public Affairs

approximate whether predictive models satisfy the criteria, this should

give us confidence that we are applying plausible criteria of fairness. The-
orem 2 and 3, moreover, provide us with evidence that the better the data
is matched, the closer we come to approximating the underlying criteria
of fairness.

V. CONCLUSION

In this article, I have argued that in light of the Kleinberg-Chouldechova

impossibility, the two fairness criteria equalized odds and predictive parity
seem too demanding. In particular, I have argued that by using the causal
inference method matching, we can modify both criteria in a way that
retains their intuitive appeal but makes them compatible. The modified
versions of equalized odds and predictive parity were, moreover, shown to
be not only compatible but also equivalent. Additionally, I demonstrated
that an approximate version of this result holds when perfect matching is
not feasible.

APPENDIX A: ILLUSTRATIVE CALCULATION OF PPV FOR MEN AND WOMEN

Let the variable T denote the test result of the device (1 indicates a posi-
tive and 0 a negative test result), D whether a person has the disease
(value 1 indicates the presence of the disease and 0 the absence), and
A the gender of the person (male or female, indicated by m and f ). To cal-
culate the positive predictive values for men and women, we assume that
the device has the same true and false positive rates (0.95 and 0.05,
respectively) for men and for women, that is:

TPR ¼ PðT ¼ 1jD ¼ 1Þ ¼ PðT ¼ 1 j D ¼ 1, A ¼ mÞ ¼ PðT ¼ 1 j D ¼ 1, A ¼ f Þ

¼ 0:95
ðA1Þ

FPR ¼ PðT ¼ 1jD ¼ 0Þ ¼ P ðT ¼ 1 j D ¼ 0, A ¼ mÞ ¼ P ðT ¼ 1 j D ¼ 0, A ¼ f Þ

¼ 0:05
ðA2Þ

We also know that the disease is more common in men than in

women:
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
23 Reconciling Algorithmic Fairness
Criteria

P ðD ¼ 1jA ¼ mÞ ¼ 0:1 ðA3Þ

P ðD ¼ 1jA ¼ f Þ ¼ 0:01 ðA4Þ

Now we can use Bayes’ theorem to calculate the positive predictive

value for men and women:

P ðT ¼ 1jD ¼ 1, A ¼ mÞ  P ðD ¼ 1jA ¼ mÞ
PPV male ¼
P ðT ¼ 1jD ¼ 1, A ¼ mÞ*P ðD ¼ 1jA ¼ mÞ þ PðT ¼ 1jD ¼ 0, A ¼ mÞ*P ðD ¼ 0jA ¼ mÞ
0:95  0:1
¼ ≈ 0:68
0:95  0:1 þ 0:05  0:9
ðA5Þ
P ðT ¼ 1jD ¼ 1, A ¼ mÞ  PðD ¼ 1jA ¼ mÞ
PPV female ¼
PðT ¼ 1jD ¼ 1, A ¼ mÞ  P ðD ¼ 1jA ¼ mÞ þ P ðT ¼ 1jD ¼ 0, A ¼ mÞ  P ðD ¼ 0jA ¼ mÞ
0:95  0:01
¼ ≈ 0:16
0:95  0:01 þ 0:05  0:99
ðA6Þ

APPENDIX B: PROOF OF THEOREMS 2 AND 3

To prove Theorem 2, suppose that the predictive models we consider sat-

isfy equalized odds relative to a1 and a2  DA, i.e., Pðb
yja1 , y Þ ¼ P ðb
yja2 , yÞ
for all b
y  Db and y  DY. We here ignore whether the data is matched or
Y
unmatched, as we are only interested in how the difference in positive
and negative predictive value between the groups changes when the dif-
ference in prevalence changes.
For the sake of simplicity, we will assume that probabilities never take
extreme values, i.e., they never take the value 0 or 1. Without loss of gen-
erality, we moreover assume that group a1 has a higher prevalence than
a2. This allows us to define the difference in prevalence as

PðY ¼ 1ja1 Þ  PðY ¼ 1ja2 Þ ¼ Δp ðB1Þ

To make the calculations more readable, we will abbreviate expressions

as follows:
   
u ≔ P Yb ¼ 1 j Y ¼ 1, A ¼ a1 v ≔ P Yb ¼ 1 j Y ¼ 0, A ¼ a1 w ≔ P ðY ¼ 1jA ¼ a2 Þ
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
24 Philosophy & Public Affairs

By Bayes’ theorem, equalized odds, and the definition of prevalence,

we can then express the positive predictive value of groups a1 and a2 as
follows:

uðw þ ΔpÞ
PPV a1 ¼ ðB2Þ
uðw þ ΔpÞ þ vð1  w  ΔpÞ
uw
PPV a2 ¼ ðB3Þ
uw þ vð1  wÞ

We can now define the difference between the two groups’ positive pre-
dictive value as a function of their difference in prevalence:

uðw þ ΔpÞ uw
ΔPPV ðΔpÞ ¼ PPV a1  PPV a2 ¼
uðw þ ΔpÞ þ vð1  w  ΔpÞ uw þ vð1  wÞ
ðB4Þ

We want to show that the smaller the difference in prevalence Δp, the
smaller the difference in positive predictive value, i.e., the smaller ΔPPV
(Δp). To do this, we need to show that ΔPPV(Δp) is strictly increasing on
the interval (0, 1). To do this, we calculate the derivative of ΔPPV(Δp):

uΔp þ wðu þ vÞ u2
Δ:j PPV ðΔpÞ ¼ þ
:u:Δp þ w þ v:  Δp  w þ 1 u:Δp þ w þ v:  Δp  w þ 1
ðB5Þ

This derivative is always positive for u, w, v, Δp  (0, 1). To show this,

let us abbreviate PPAa1 as follows:

uðw þ ΔpÞ A
PPV a1 ¼ ¼ ðB6Þ
uðw þ ΔpÞ þ vð1  w  ΔpÞ B

Consequently, ΔPPVJ(Δp) can be abbreviated and transformed as

follows:

Aðu þ vÞ u Bu  Au þ Av
ΔPPV j ðΔpÞ ¼ þ ¼ ðB7Þ
B2 B B2
 10884963, 0, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/papa.12233 by CAPES, Wiley Online Library on [12/04/2023]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
25 Reconciling Algorithmic Fairness
Criteria

Now, because PPAa1 is a probability, 0 ≤ PPAa1 ≤ 1. Since u, w, v, Δp 

(0, 1), A > 0. It follows that A < B. This, in turn, entails that Bu  Au >0.
Since trivially, B2, Av > 0, it follows that ΔPPV J(Δp) is always positive.
ΔPPV(Δp) is hence strictly increasing on the interval (0, 1).
Analogously, we can show that the difference in negative predictive
value ΔNPV(Δp) strictly increases on the interval (0, 1).
Theorem 3 can be proved in a similar fashion, for which reason the
proof is omitted.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflict of interest.

DATA AVAILABILITY STATEMENT

Data sharing not applicable to this article as no datasets were generated

or analysed during the current study.