On the Challenges of Using Black-Box APIs for Toxicity Evaluation in Research
This work highlights a reproducibility issue for researchers and practitioners relying on automated toxicity scores to evaluate AI models, cautioning against direct comparisons across studies due to evolving API metrics.
The study examined how changes in black-box toxicity detection APIs, like Perspective API, affect the reproducibility of research comparing models and methods for reducing toxicity, finding that rescoring models from the HELM benchmark with an updated API version altered the ranking of widely used foundation models.
Perception of toxicity evolves over time and often differs between geographies and cultural backgrounds. Similarly, black-box commercially available APIs for detecting toxicity, such as the Perspective API, are not static, but frequently retrained to address any unattended weaknesses and biases. We evaluate the implications of these changes on the reproducibility of findings that compare the relative merits of models and methods that aim to curb toxicity. Our findings suggest that research that relied on inherited automatic toxicity scores to compare models and techniques may have resulted in inaccurate findings. Rescoring all models from HELM, a widely respected living benchmark, for toxicity with the recent version of the API led to a different ranking of widely used foundation models. We suggest caution in applying apples-to-apples comparisons between studies and lay recommendations for a more structured approach to evaluating toxicity over time. Code and data are available at https://github.com/for-ai/black-box-api-challenges.