Local Neighborhood Instability in Parametric Projections: Quantitative and Visual Analysis
For analysts using parametric projections, this framework reveals hidden instability that standard metrics miss, enabling more reliable interpretation.
Parametric projections can be locally unstable under input perturbations, and existing metrics fail to detect this. The paper introduces a framework combining quantitative measures and visualizations to identify unstable regions, demonstrated on MNIST and Fashion-MNIST with UMAP/t-SNE projectors.
Parametric projections let analysts embed new points in real time, but input variations from measurement noise or data drift can produce unpredictable shifts in the 2D layout. Whether and where a projection is locally stable remains largely unexamined. In this paper, we present a stability evaluation framework that probes parametric projections with Gaussian perturbations around selected anchor points and assesses how neighborhoods deform in the 2D embedding. Our approach combines quantitative measures of mean displacement, bias, and nearest-anchor assignment error with per-anchor visualizations of displacement vectors, local PCA ellipsoids, and Voronoi misassignment for detailed inspection. We demonstrate the framework's effectiveness on UMAP- and t-SNE-based neural projectors of varying network sizes and study the effect of Jacobian regularization as a gradient-based robustness strategy. We apply our framework to the MNIST and Fashion-MNIST datasets. The results show that our framework identifies unstable projection regions invisible to reconstruction error or neighborhood-preservation metrics.