Empirical Evaluation and Theoretical Analysis for Representation Learning: A Survey
It provides a comprehensive overview for researchers and practitioners to understand and assess representation learning approaches, though it is incremental as an extended version of prior work.
This survey reviews evaluation methods and theoretical analyses for representation learning algorithms, highlighting their flexibility and state-of-the-art performance in various tasks, and discusses future directions based on the findings.
Representation learning enables us to automatically extract generic feature representations from a dataset to solve another machine learning task. Recently, extracted feature representations by a representation learning algorithm and a simple predictor have exhibited state-of-the-art performance on several machine learning tasks. Despite its remarkable progress, there exist various ways to evaluate representation learning algorithms depending on the application because of the flexibility of representation learning. To understand the current representation learning, we review evaluation methods of representation learning algorithms and theoretical analyses. On the basis of our evaluation survey, we also discuss the future direction of representation learning. Note that this survey is the extended version of Nozawa and Sato (2022).