Multi-objective Deep Learning: Taxonomy and Survey of the State of the Art
It organizes existing methods for researchers and practitioners working with multi-objective deep learning, but it is incremental as it surveys rather than introduces new techniques.
This survey addresses the challenges of applying multi-objective optimization to deep learning due to high computational costs and model complexities, providing a taxonomy and overview of recent advancements across supervised, unsupervised, reinforcement learning, and generative modeling.
Simultaneously considering multiple objectives in machine learning has been a popular approach for several decades, with various benefits for multi-task learning, the consideration of secondary goals such as sparsity, or multicriteria hyperparameter tuning. However - as multi-objective optimization is significantly more costly than single-objective optimization - the recent focus on deep learning architectures poses considerable additional challenges due to the very large number of parameters, strong nonlinearities and stochasticity. This survey covers recent advancements in the area of multi-objective deep learning. We introduce a taxonomy of existing methods - based on the type of training algorithm as well as the decision maker's needs - before listing recent advancements, and also successful applications. All three main learning paradigms supervised learning, unsupervised learning and reinforcement learning are covered, and we also address the recently very popular area of generative modeling.