Parameter-Efficient Token Embedding Editing for Clinical Class-Level Unlearning
This addresses privacy and policy compliance in clinical AI by enabling targeted unlearning, though it is incremental as it builds on existing parameter-efficient and unlearning methods.
The paper tackled the problem of efficiently removing sensitive information from clinical language models without full retraining, introducing STEU which achieved near-complete forgetting (forget F1 = 0.0004) while maintaining competitive utility (retain avg F1 = 0.4766) by modifying only 0.19% of parameters.
Machine unlearning is increasingly important for clinical language models, where privacy regulations and institutional policies may require removing sensitive information from deployed systems without retraining from scratch. In practice, deletion requests must balance effective forgetting of targeted information with preservation of model utility and minimal parameter modification. We introduce Sparse Token Embedding Unlearning (STEU), a parameter-efficient method for behavioral class-level unlearning that updates only PMI-selected token embeddings together with a small classifier head while keeping all encoder layers frozen. Across experiments on MIMIC-IV, MIMIC-III, and eICU using BioClinicalBERT, BERT-base, and DistilBERT, STEU consistently suppresses the target class while largely preserving retained task performance. In the primary MIMIC-IV setting, STEU achieves near-complete forgetting (forget F1 = 0.0004) while maintaining competitive retained utility (retain avg F1 = 0.4766) after modifying only 0.19\% of model parameters. These results suggest that targeted behavioral unlearning can be achieved through sparse embedding edits without modifying deeper encoder representations.