How Could Polyhedral Theory Harness Deep Learning?
This addresses the challenge of architecture design for deep learning practitioners, but it is incremental as it outlines potential directions rather than presenting a proven solution.
The paper tackles the problem of automatically designing optimal deep learning architectures based on computational resources, input size, and training data, proposing research directions using polyhedral theory and mixed-integer representability as an analytical alternative to empirical methods.
The holy grail of deep learning is to come up with an automatic method to design optimal architectures for different applications. In other words, how can we effectively dimension and organize neurons along the network layers based on the computational resources, input size, and amount of training data? We outline promising research directions based on polyhedral theory and mixed-integer representability that may offer an analytical approach to this question, in contrast to the empirical techniques often employed.