AutoML @ NeurIPS 2018 challenge: Design and Results
This work addresses the problem of lifelong learning with data drift for machine learning practitioners, but it is incremental as it describes a competition design and results without introducing new methods.
The authors organized the NeurIPS 2018 AutoML challenge, focusing on autonomous lifelong machine learning with drift, where participants developed programs for supervised learning without the i.i.d. assumption, attracting over 300 participants.
We organized a competition on Autonomous Lifelong Machine Learning with Drift that was part of the competition program of NeurIPS 2018. This data driven competition asked participants to develop computer programs capable of solving supervised learning problems where the i.i.d. assumption did not hold. Large data sets were arranged in a lifelong learning and evaluation scenario and CodaLab was used as the challenge platform. The challenge attracted more than 300 participants in its two month duration. This chapter describes the design of the challenge and summarizes its main results.