Ensemble of heterogeneous flexible neural trees using multiobjective genetic programming
This work addresses the need for more effective and interpretable ensemble models in machine learning, though it appears incremental as it builds on existing neural tree and genetic programming techniques.
The authors tackled the problem of balancing approximation error and model complexity in machine learning by proposing a multiobjective genetic programming approach to create heterogeneous flexible neural trees, which were then used to form an ensemble system, achieving improved efficiency over existing methods on classification, regression, and time-series datasets.
Machine learning algorithms are inherently multiobjective in nature, where approximation error minimization and model's complexity simplification are two conflicting objectives. We proposed a multiobjective genetic programming (MOGP) for creating a heterogeneous flexible neural tree (HFNT), tree-like flexible feedforward neural network model. The functional heterogeneity in neural tree nodes was introduced to capture a better insight of data during learning because each input in a dataset possess different features. MOGP guided an initial HFNT population towards Pareto-optimal solutions, where the final population was used for making an ensemble system. A diversity index measure along with approximation error and complexity was introduced to maintain diversity among the candidates in the population. Hence, the ensemble was created by using accurate, structurally simple, and diverse candidates from MOGP final population. Differential evolution algorithm was applied to fine-tune the underlying parameters of the selected candidates. A comprehensive test over classification, regression, and time-series datasets proved the efficiency of the proposed algorithm over other available prediction methods. Moreover, the heterogeneous creation of HFNT proved to be efficient in making ensemble system from the final population.