A Theory of Feature Learning
This foundational work tackles the theoretical gap in feature learning, which underpins deep learning, for researchers and practitioners in machine learning.
The paper addresses the lack of theoretical understanding in feature learning by developing a theoretical framework that characterizes when features can be learned unsupervised and provides methods to judge feature quality using rate-distortion theory.
Feature Learning aims to extract relevant information contained in data sets in an automated fashion. It is driving force behind the current deep learning trend, a set of methods that have had widespread empirical success. What is lacking is a theoretical understanding of different feature learning schemes. This work provides a theoretical framework for feature learning and then characterizes when features can be learnt in an unsupervised fashion. We also provide means to judge the quality of features via rate-distortion theory and its generalizations.