Bilinear Models for Machine Learning
This work addresses the inefficiency of linear operations in ML for structured data like images, offering a domain-specific improvement.
The paper tackles the problem of conventional linear operations in machine learning by introducing bilinear models that adapt to the structure of data, such as monochromatic images, resulting in significantly fewer parameters needed to achieve the same performance, as demonstrated with classification on the MNIST dataset.
In this work we define and analyze the bilinear models which replace the conventional linear operation used in many building blocks of machine learning (ML). The main idea is to devise the ML algorithms which are adapted to the objects they treat. In the case of monochromatic images, we show that the bilinear operation exploits better the structure of the image than the conventional linear operation which ignores the spatial relationship between the pixels. This translates into significantly smaller number of parameters required to yield the same performance. We show numerical examples of classification in the MNIST data set.