Maximum Entropy Linear Manifold for Learning Discriminative Low-dimensional Representation
This addresses the need for efficient and interpretable low-dimensional representations in machine learning, particularly for tasks like classification and data visualization, though it appears incremental as a generalization of an existing model.
The paper tackles the problem of learning discriminative low-dimensional representations by proposing Maximum Entropy Linear Manifold (MELM), a method that finds linear projections to maximize class discriminativeness, resulting in embeddings useful for classification, dimensionality reduction, and visualization.
Representation learning is currently a very hot topic in modern machine learning, mostly due to the great success of the deep learning methods. In particular low-dimensional representation which discriminates classes can not only enhance the classification procedure, but also make it faster, while contrary to the high-dimensional embeddings can be efficiently used for visual based exploratory data analysis. In this paper we propose Maximum Entropy Linear Manifold (MELM), a multidimensional generalization of Multithreshold Entropy Linear Classifier model which is able to find a low-dimensional linear data projection maximizing discriminativeness of projected classes. As a result we obtain a linear embedding which can be used for classification, class aware dimensionality reduction and data visualization. MELM provides highly discriminative 2D projections of the data which can be used as a method for constructing robust classifiers. We provide both empirical evaluation as well as some interesting theoretical properties of our objective function such us scale and affine transformation invariance, connections with PCA and bounding of the expected balanced accuracy error.