Orthogonal Dendritic Intrinsic Networks: An Architecture for Significance-Ordered, Orthogonal Latent Spaces
For practitioners needing interpretable latent representations from deep autoencoders, ODIN provides a principled method to recover PCA-like structure without sacrificing nonlinear expressivity.
ODIN introduces an autoencoder architecture that enforces orthogonal, variance-ordered latent dimensions, achieving PCA-like interpretability in deep nonlinear models. Empirical results on synthetic and real datasets demonstrate structured feature learning.
Principal Component Analysis or PCA-like properties (orthogonality, variance ranking) are seldom realized in deep autoencoder architectures. In this work, we present ODIN (Orthogonal Dendritic Intrinsic Network), a novel autoencoder architecture that recovers PCA-like latent structure in a fully non-linear regime. By incorporating a set of geometric constraints directly into the training objective, ODIN encourages latent dimensions to be mutually orthogonal and ordered by explained variance, mirroring the interpretable decomposition of PCA while retaining the expressive power of deep networks. We provide theoretical grounding for these constraints and demonstrate their compatibility with standard encoder-decoder frameworks. We also establish empirical results for both synthetic and real world datasets, establishing a principled path toward interpretable, structured feature learning and dimensionality reduction.