The Slodderwetenschap (Sloppy Science) of Stochastic Parrots -- A Plea for Science to NOT take the Route Advocated by Gebru and Bender
This paper addresses concerns about scientific ethics and methodology within the computer science, machine learning, and AI communities, particularly regarding the evaluation of large language models.
This paper critiques "On the Dangers of Stochastic Parrots" (the "Parrot Paper"), arguing it lacks proper scientific rigor. The author contends the Parrot Paper fails to acknowledge its nature as an advocacy piece, articulate presuppositions, and consider cost/benefit trade-offs.
This article is a position paper written in reaction to the now-infamous paper titled "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" by Timnit Gebru, Emily Bender, and others who were, as of the date of this writing, still unnamed. I find the ethics of the Parrot Paper lacking, and in that lack, I worry about the direction in which computer science, machine learning, and artificial intelligence are heading. At best, I would describe the argumentation and evidentiary practices embodied in the Parrot Paper as Slodderwetenschap (Dutch for Sloppy Science) -- a word which the academic world last widely used in conjunction with the Diederik Stapel affair in psychology [2]. What is missing in the Parrot Paper are three critical elements: 1) acknowledgment that it is a position paper/advocacy piece rather than research, 2) explicit articulation of the critical presuppositions, and 3) explicit consideration of cost/benefit trade-offs rather than a mere recitation of potential "harms" as if benefits did not matter. To leave out these three elements is not good practice for either science or research.