HM3: Heterogeneous Multi-Class Model Merging
This addresses the issue of increased inference costs and complexity for AI deployments using auxiliary models, though it appears incremental as it builds on existing training-free merging techniques.
The paper tackles the problem of high complexity and cost in deploying multiple guard-rail models alongside foundation language models by proposing a training-free merging technique called HM3 to consolidate multi-class classifiers with heterogeneous label spaces into a single model. It reports results such as an average F1-score higher than source models and up to 44% reduction in inference time for BERT-based guard models.
Foundation language model deployments often include auxiliary guard-rail models to filter or classify text, detecting jailbreak attempts, biased or toxic output, or ensuring topic adherence. These additional models increase the complexity and cost of model inference, especially since many are also large language models. To address this issue, we explore training-free model merging techniques to consolidate these models into a single, multi-functional model. We propose Heterogeneous Multi-Class Model Merging (HM3) as a simple technique for merging multi-class classifiers with heterogeneous label spaces. Unlike parameter-efficient fine-tuning techniques like LoRA, which require extensive training and add complexity during inference, recent advancements allow models to be merged in a training-free manner. We report promising results for merging BERT-based guard models, some of which attain an average F1-score higher than the source models while reducing the inference time by up to 44%. We introduce self-merging to assess the impact of reduced task-vector density, finding that the more poorly performing hate speech classifier benefits from self-merging while higher-performing classifiers do not, which raises questions about using task vector reduction for model tuning.