"Alexa, can you forget me?" Machine Unlearning Benchmark in Spoken Language Understanding
It addresses the need for responsible AI by providing a benchmark for the 'right to be forgotten' in speech-related tasks, though it is incremental as it builds on existing unlearning concepts.
This paper tackles the problem of evaluating machine unlearning methods for spoken language understanding by introducing UnSLU-BENCH, the first benchmark in this area, which tests eight techniques across four datasets and languages, revealing significant differences in effectiveness and computational feasibility.
Machine unlearning, the process of efficiently removing specific information from machine learning models, is a growing area of interest for responsible AI. However, few studies have explored the effectiveness of unlearning methods on complex tasks, particularly speech-related ones. This paper introduces UnSLU-BENCH, the first benchmark for machine unlearning in spoken language understanding (SLU), focusing on four datasets spanning four languages. We address the unlearning of data from specific speakers as a way to evaluate the quality of potential "right to be forgotten" requests. We assess eight unlearning techniques and propose a novel metric to simultaneously better capture their efficacy, utility, and efficiency. UnSLU-BENCH sets a foundation for unlearning in SLU and reveals significant differences in the effectiveness and computational feasibility of various techniques.