Mapping Learning Algorithms on Data, a useful step for optimizing performances and their comparison
This work addresses the need for better optimization and comparison of machine learning algorithms, though it appears incremental as it builds on existing concepts without introducing a new paradigm.
The paper tackles the problem of understanding and comparing learning algorithm performances across parameter spaces by proposing a methodology to map algorithms on data, resulting in enhanced insights for selecting optimal configurations and comparing learners across different contexts.
In the paper, we propose a novel methodology to map learning algorithms on data (performance map) in order to gain more insights in the distribution of their performances across their parameter space. This methodology provides useful information when selecting a learner's best configuration for the data at hand, and it also enhances the comparison of learners across learning contexts. In order to explain the proposed methodology, the study introduces the notions of learning context, performance map, and high performance function. It then applies these concepts to a variety of learning contexts to show how their use can provide more insights in a learner's behavior, and can enhance the comparison of learners across learning contexts. The study is completed by an extensive experimental study describing how the proposed methodology can be applied.