FLARE: Task-agnostic embedding model evaluation through a normalization process
For practitioners needing to select embedding models without labels, FLARE provides a reliable evaluation metric that works in high dimensions, solving a known bottleneck in labelless evaluation.
FLARE introduces a flow-based labelless method to evaluate embedding models without task-specific labels, achieving a Spearman's ρ of 0.90 against supervised benchmarks and remaining stable in high-dimensional spaces (d ≥ 3,584) where existing methods fail.
When task-specific labels are not available, it becomes difficult to select an embedding model for a specific target corpus. Existing labelless measures based on kernel estimators or Gaussian mixes fail in high-dimensional space, resulting in unstable rankings. We propose a flow-based labelless representation embedding evaluation (FLARE), which utilizes normalized streams to estimate information sufficiency directly from log-likelihood and avoid distance-based density estimation. We give a finite sample boundary, indicating that the estimation error depends on the intrinsic dimension of the data manifold rather than the original embedding dimension. On 11 datasets and 8 embedders, FLARE reached Spearman's $ρ$ of 0.90 under the supervised benchmark and remained stable in high-dimensional embeddings ($d \geq 3{,}584$) as the existing labelless baseline collapsed.