Components of Machine Learning: Binding Bits and FLOPS
It provides a conceptual framework for understanding ML methods, but is incremental as it reviews existing components without new empirical results.
The paper analyzes machine learning methods as combinations of data, hypothesis space, and loss function, and discusses trade-offs between statistical and computational properties.
Many machine learning problems and methods are combinations of three components: data, hypothesis space and loss function. Different machine learning methods are obtained as combinations of different choices for the representation of data, hypothesis space and loss function. After reviewing the mathematical structure of these three components, we discuss intrinsic trade-offs between statistical and computational properties of machine learning methods.