GPT is Not an Annotator: The Necessity of Human Annotation in Fairness Benchmark Construction
This addresses the need for reliable fairness benchmarks in AI, showing that human annotation remains essential for sensitive tasks, and is incremental by extending previous work to a new community.
The paper tackled the problem of measuring social biases in LLMs by exploring if GPT-3.5-Turbo could assist in developing a bias benchmark dataset from community surveys, specifically for the Jewish community and antisemitism, and found that it performed poorly with unacceptable quality issues.
Social biases in LLMs are usually measured via bias benchmark datasets. Current benchmarks have limitations in scope, grounding, quality, and human effort required. Previous work has shown success with a community-sourced, rather than crowd-sourced, approach to benchmark development. However, this work still required considerable effort from annotators with relevant lived experience. This paper explores whether an LLM (specifically, GPT-3.5-Turbo) can assist with the task of developing a bias benchmark dataset from responses to an open-ended community survey. We also extend the previous work to a new community and set of biases: the Jewish community and antisemitism. Our analysis shows that GPT-3.5-Turbo has poor performance on this annotation task and produces unacceptable quality issues in its output. Thus, we conclude that GPT-3.5-Turbo is not an appropriate substitute for human annotation in sensitive tasks related to social biases, and that its use actually negates many of the benefits of community-sourcing bias benchmarks.