SCANCOR at the Weatherhead Seminar
Date and Time
Location
In the Land of the Blind, the One-Eyed Man is a Burden: A Case of Algorithmic Bias in University Admissions
Presenter
- Emrah Karakaya, Associate Professor, Department of Industrial Economics and Management, KTH Royal Institute of Technology.
Moderator
- Jocelyn Viterna, Codirector, SCANCOR at the Weatherhead Center; Faculty Associate; Harvard Academy Senior Scholar. Professor of Sociology, Department of Sociology; Chair, Committee on Degrees in Studies of Women, Gender, and Sexuality, Harvard University.
Contact
Diana Ocampo Belloso
dianaocampobelloso@wcfia.harvard.edu
Abstract
Co-author: Charlotta Kronblad, Associate Senior Lecturer, Department of Business Administration, University of Gothenburg.
University admissions are human labor-intensive, and fairness is often contested. Algorithmic decision-making (ADM) is consequently promoted as a remedy, promising efficiency and neutrality. But not all ADM systems fit with this ideal. This paper examines a biased ADM system used by a European university to rank applicants for international master programs based on group-level data such as previous university, country and region. Drawing on a first-hand experience of a master program director – pseudonymized as Leila – who provided us with primary data, we explore how the ADM system reinforced institutional bias and unjust decisions. With the gradual unveiling of biases and open critique by program directors, such as Leila who has both algorithmic and domain expertise, the biased ADM system was partially stopped after two years of experimentation. The contribution of our paper is twofold. Empirically, we provide a compelling case of how algorithmic injustice in practice has been discovered and protested by a de-facto knowledge broker while overlooked and downplayed by others, including managers and developers. Theoretically, we contribute to the ongoing discussion on ADM, algorithmic (in)justice and multilevel blackboxing by demonstrating how opacity and human oversight can function as an organizational asset, shielding managers and developers from contestation. Contrasting Waardenburg et al. (2022 in Organization Science), who depict knowledge brokers as “one-eyed kings,” our case reveals brokers as burdens, whose insights are resisted by the institutions that use the ADM, highlighting the limits of using the “human in the loop” as a safeguard.