Supervised Distributional Reduction via Optimal Transport and Dependence Maximization
For practitioners needing compact, supervised representations that preserve task-relevant structure, SDR offers a principled method, though its novelty is incremental as it augments an existing Fused Gromov-Wasserstein objective.
The paper proposes Supervised Distributional Reduction (SDR), an algorithm that combines optimal transport with dependence maximization to learn target-aware representations that balance compression with predictive fidelity. SDR induces adaptive kernels for Gaussian Process modeling, improving performance on tasks requiring non-stationary geometry.
Learning representations that capture both intrinsic data geometry and target-relevant structure remains a fundamental challenge, particularly in settings where data reduction must balance compression with predictive fidelity. While distributional reduction-encompassing joint clustering and dimensionality reduction-offers a principled way to summarize data, its supervised variants remain relatively under-explored, despite the importance of retaining task-relevant signal for downstream prediction and decision-making. We propose Supervised Distributional Reduction (SDR), an algorithm for learning target-aware representations by combining optimal transport with explicit dependence maximization. SDR builds on the Fused Gromov-Wasserstein (FGW) objective to align the relational structure of the input distribution with a set of representative points, while augmenting it with a direct dependence term that encourages the learned embeddings to capture predictive signal more explicitly. This results in compact representations that reflect both geometric structure and supervision. Beyond representation learning, SDR naturally induces a data-dependent, non-stationary geometry that can be leveraged for settings such as Gaussian Process (GP) modelling. By redefining distances through target-aware distributional alignment, SDR enables the construction of adaptive kernels that respond to local variations in both data geometry and supervision, offering an optimal transport-based perspective on non-stationary kernel design.