Regularising Non-linear Models Using Feature Side-information
This addresses the issue of underutilizing feature properties in machine learning, offering a domain-specific improvement for tasks with available side-information.
The paper tackles the problem of improving model prediction by incorporating feature side-information, which provides detailed properties of features, into the learning process, resulting in significant predictive performance gains on benchmark datasets over baselines.
Very often features come with their own vectorial descriptions which provide detailed information about their properties. We refer to these vectorial descriptions as feature side-information. In the standard learning scenario, input is represented as a vector of features and the feature side-information is most often ignored or used only for feature selection prior to model fitting. We believe that feature side-information which carries information about features intrinsic property will help improve model prediction if used in a proper way during learning process. In this paper, we propose a framework that allows for the incorporation of the feature side-information during the learning of very general model families to improve the prediction performance. We control the structures of the learned models so that they reflect features similarities as these are defined on the basis of the side-information. We perform experiments on a number of benchmark datasets which show significant predictive performance gains, over a number of baselines, as a result of the exploitation of the side-information.