Fast-MIA: Efficient and Scalable Membership Inference for LLMs
This addresses the need for efficient and reproducible MIA evaluation in LLMs, which is crucial for copyright, security, and data privacy concerns, though it is incremental as it builds on existing methods.
The authors tackled the high computational cost and lack of standardized implementations for membership inference attacks (MIA) on Large Language Models (LLMs) by developing Fast-MIA, a Python library that provides fast batch inference and unified evaluation, released as open-source to support scalable research.
We propose Fast-MIA (https://github.com/Nikkei/fast-mia), a Python library for efficiently evaluating membership inference attacks (MIA) against Large Language Models (LLMs). MIA against LLMs has emerged as a crucial challenge due to growing concerns over copyright, security, and data privacy, and has attracted increasing research attention. However, the progress of this research is significantly hindered by two main obstacles: (1) the high computational cost of inference in LLMs, and (2) the lack of standardized and maintained implementations of MIA methods, which makes large-scale empirical comparison difficult. To address these challenges, our library provides fast batch inference and includes implementations of representative MIA methods under a unified evaluation framework. This library supports easy implementation of reproducible benchmarks with simple configuration and extensibility. We release Fast-MIA as an open-source (Apache License 2.0) tool to support scalable and transparent research on LLMs.