Affinity and Diversity: A Unified Metric for Demonstration Selection via Internal Representations
This addresses the inconsistency in demonstration selection for In-Context Learning, which is incremental as it unifies existing approaches.
The paper tackled the problem of inconsistent demonstration selection in In-Context Learning by proposing a unified metric based on affinity and diversity using internal representations, and experiments showed strong correlations with test accuracies.
The performance of In-Context Learning (ICL) is highly sensitive to the selected demonstrations. Existing approaches to demonstration selection optimize different objectives, yielding inconsistent results. To address this, we propose a unified metric--affinity and diversity--that leverages ICL model's internal representations. Our experiments show that both affinity and diversity strongly correlate with test accuracies, indicating their effectiveness for demonstration selection. Moreover, we show that our proposed metrics align well with various previous works to unify the inconsistency.