AI and Holistic Review: Informing Human Reading in College Admissions
This addresses bias concerns in college admissions by showing how data auditing could inform holistic review, though it is incremental as it applies existing methods to new data.
The study analyzed 283,676 college application essays to assess if demographic characteristics like gender and household income could be inferred, finding that a logistic regression classifier predicted these with high accuracy.
College admissions in the United States is carried out by a human-centered method of evaluation known as holistic review, which typically involves reading original narrative essays submitted by each applicant. The legitimacy and fairness of holistic review, which gives human readers significant discretion over determining each applicant's fitness for admission, has been repeatedly challenged in courtrooms and the public sphere. Using a unique corpus of 283,676 application essays submitted to a large, selective, state university system between 2015 and 2016, we assess the extent to which applicant demographic characteristics can be inferred from application essays. We find a relatively interpretable classifier (logistic regression) was able to predict gender and household income with high levels of accuracy. Findings suggest that data auditing might be useful in informing holistic review, and perhaps other evaluative systems, by checking potential bias in human or computational readings.