Identifying Metric Structures of Deep Latent Variable Models
This provides a principled method for extracting trustworthy conclusions from deep latent variable models, addressing a foundational issue in machine learning interpretation.
The paper tackles the problem of non-identifiability in deep latent variable models, which hinders reliable interpretation by domain experts, by proposing to identify relationships like distances and angles between latent variables instead of the variables themselves, proving feasibility under mild conditions and showing empirical improvements in latent distances.
Deep latent variable models learn condensed representations of data that, hopefully, reflect the inner workings of the studied phenomena. Unfortunately, these latent representations are not statistically identifiable, meaning they cannot be uniquely determined. Domain experts, therefore, need to tread carefully when interpreting these. Current solutions limit the lack of identifiability through additional constraints on the latent variable model, e.g. by requiring labeled training data, or by restricting the expressivity of the model. We change the goal: instead of identifying the latent variables, we identify relationships between them such as meaningful distances, angles, and volumes. We prove this is feasible under very mild model conditions and without additional labeled data. We empirically demonstrate that our theory results in more reliable latent distances, offering a principled path forward in extracting trustworthy conclusions from deep latent variable models.