Ensemble Bayesian Inference: Leveraging Small Language Models to Achieve LLM-level Accuracy in Profile Matching Tasks
This enables high-performance AI systems with limited computational resources, though it is incremental as it builds on existing ensemble methods and open-source LLM research.
The study tackled the problem of achieving large language model (LLM) accuracy with small language models (SLMs) by proposing Ensemble Bayesian Inference (EBI), which combines multiple SLMs using Bayesian estimation to exceed individual performance limits, as demonstrated in tasks like aptitude assessments and consumer profile analysis across Japanese and English.
This study explores the potential of small language model(SLM) ensembles to achieve accuracy comparable to proprietary large language models (LLMs). We propose Ensemble Bayesian Inference (EBI), a novel approach that applies Bayesian estimation to combine judgments from multiple SLMs, allowing them to exceed the performance limitations of individual models. Our experiments on diverse tasks(aptitude assessments and consumer profile analysis in both Japanese and English) demonstrate EBI's effectiveness. Notably, we analyze cases where incorporating models with negative Lift values into ensembles improves overall performance, and we examine the method's efficacy across different languages. These findings suggest new possibilities for constructing high-performance AI systems with limited computational resources and for effectively utilizing models with individually lower performance. Building on existing research on LLM performance evaluation, ensemble methods, and open-source LLM utilization, we discuss the novelty and significance of our approach.