Predictability of Machine Learning Algorithms and Related Feature Extraction Techniques
This work addresses the need for efficient model selection and performance estimation in machine learning, but it appears incremental as it builds on existing prediction methods without introducing a major breakthrough.
The thesis tackled the problem of predicting classification accuracy for machine learning models on various datasets using matrix factorization, achieving results on predictability of fine-tuned models, MLP with feature extraction, and model performance using implicit feedback across over fifty datasets.
This thesis designs a prediction system based on matrix factorization to predict the classification accuracy of a specific model on a particular dataset. In this thesis, we conduct comprehensive empirical research on more than fifty datasets that we collected from the openml website. We study the performance prediction of three fundamental machine learning algorithms, namely, random forest, XGBoost, and MultiLayer Perceptron(MLP). In particular, we obtain the following results: 1. Predictability of fine-tuned models using coarse-tuned variants. 2. Predictability of MLP using feature extraction techniques. 3. Predict model performance using implicit feedback.