Measuring the Biases that Matter: The Ethical and Causal Foundations for Measures of Fairness in Algorithms


Measures of algorithmic bias can be roughly classified into four categories, distinguished by the conditional probabilistic dependencies to which they are sensitive. First, measures of "procedural bias" diagnose bias when the score returned by an algorithm is probabilistically dependent on a sensitive class variable (e.g. race or sex). Second, measures of "outcome bias" capture probabilistic dependence between class variables and the outcome for each subject (e.g. parole granted or loan denied). Third, measures of "behavior-relative error bias" capture probabilistic dependence between class variables and the algorithmic score, conditional on target behaviors (e.g. recidivism or loan default). Fourth, measures of "score-relative error bias" capture probabilistic dependence between class variables and behavior, conditional on score. Several recent discussions have demonstrated a tradeoff between these different measures of algorithmic bias, and at least one recent paper has suggested conditions under which tradeoffs may be minimized. In this paper we use the machinery of causal graphical models to show that, under standard assumptions, the underlying causal relations among variables forces some tradeoffs. We delineate a number of normative considerations that are encoded in different measures of bias, with reference to the philosophical literature on the wrongfulness of disparate treatment and disparate impact. While both kinds of error bias are nominally motivated by concern to avoid disparate impact, we argue that consideration of causal structures shows that these measures are better understood as complicated and unreliable measures of procedural biases (i.e. disparate treatment). Moreover, while procedural bias is indicative of disparate treatment, we show that the measure of procedural bias one ought to adopt is dependent on the account of the wrongfulness of disparate treatment one endorses. Finally, given that neither score-relative nor behavior-relative measures of error bias capture the relevant normative considerations, we suggest that error bias proper is best measured by score-based measures of accuracy, such as the Brier score.

Download options


    Upload a copy of this work     Papers currently archived: 72,722

External links

Setup an account with your affiliations in order to access resources via your University's proxy server

Through your library


Added to PP

13 (#775,675)

6 months
4 (#163,373)

Historical graph of downloads
How can I increase my downloads?

Author Profiles

Jonathan Herington
University of Rochester
Bruce Glymour
Kansas State University

References found in this work

No references found.

Add more references

Citations of this work

Add more citations

Similar books and articles

Measuring Fairness in an Unfair World.Jonathan Herington - 2020 - Proceedings of AAAI/ACM Conference on AI, Ethics, and Society 2020:286-292.
Democratizing Algorithmic Fairness.Pak-Hang Wong - 2020 - Philosophy and Technology 33 (2):225-244.
A Moral Framework for Understanding of Fair ML Through Economic Models of Equality of Opportunity.Hoda Heidari - 2019 - Proceedings of the Conference on Fairness, Accountability, and Transparency 1.
What's Fair About Individual Fairness?Will Fleisher - 2021 - Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society.
On Statistical Criteria of Algorithmic Fairness.Brian Hedden - 2021 - Philosophy and Public Affairs 49 (2):209-231.