Mathematical Challenges in Deep Learning
It addresses foundational problems for researchers and practitioners in AI, but it is incremental as it lists existing challenges without proposing new solutions.
The paper identifies key mathematical challenges in deep learning, such as training, inference, generalization bounds, and optimization, to facilitate communication with mathematicians and other theorists, aiming to benefit the tech industry in the long run.
Deep models are dominating the artificial intelligence (AI) industry since the ImageNet challenge in 2012. The size of deep models is increasing ever since, which brings new challenges to this field with applications in cell phones, personal computers, autonomous cars, and wireless base stations. Here we list a set of problems, ranging from training, inference, generalization bound, and optimization with some formalism to communicate these challenges with mathematicians, statisticians, and theoretical computer scientists. This is a subjective view of the research questions in deep learning that benefits the tech industry in long run.