Genetic algorithms with DNN-based trainable crossover as an example of partial specialization of general search
This work addresses the challenge of improving search efficiency in machine learning for researchers, though it appears incremental as it builds on existing genetic algorithm and neural network methods.
The paper tackles the problem of balancing generality and efficiency in universal induction by proposing genetic algorithms with a trainable crossover operator, implemented as a deep neural network, and shows that this approach can be more efficient than both general genetic algorithms and discriminative models.
Universal induction relies on some general search procedure that is doomed to be inefficient. One possibility to achieve both generality and efficiency is to specialize this procedure w.r.t. any given narrow task. However, complete specialization that implies direct mapping from the task parameters to solutions (discriminative models) without search is not always possible. In this paper, partial specialization of general search is considered in the form of genetic algorithms (GAs) with a specialized crossover operator. We perform a feasibility study of this idea implementing such an operator in the form of a deep feedforward neural network. GAs with trainable crossover operators are compared with the result of complete specialization, which is also represented as a deep neural network. Experimental results show that specialized GAs can be more efficient than both general GAs and discriminative models.