Sparse learning of maximum likelihood model for optimization of complex loss function
This addresses the gap between model training and evaluation metrics for practitioners in domains like aviation safety and intrusion detection, though it is incremental as it builds on existing optimization frameworks.
The paper tackles the problem of directly optimizing complex performance measures like F-score and ROC curves, rather than minimizing simple loss functions, by proposing a sparse maximum likelihood model and an iterative optimization algorithm. Experiments across three real-world applications show advantages over state-of-the-art methods.
Traditional machine learning methods usually minimize a simple loss function to learn a predictive model, and then use a complex performance measure to measure the prediction performance. However, minimizing a simple loss function cannot guarantee that an optimal performance. In this paper, we study the problem of optimizing the complex performance measure directly to obtain a predictive model. We proposed to construct a maximum likelihood model for this problem, and to learn the model parameter, we minimize a com- plex loss function corresponding to the desired complex performance measure. To optimize the loss function, we approximate the upper bound of the complex loss. We also propose impose the sparsity to the model parameter to obtain a sparse model. An objective is constructed by combining the upper bound of the loss function and the sparsity of the model parameter, and we develop an iterative algorithm to minimize it by using the fast iterative shrinkage- thresholding algorithm framework. The experiments on optimization on three different complex performance measures, including F-score, receiver operating characteristic curve, and recall precision curve break even point, over three real-world applications, aircraft event recognition of civil aviation safety, in- trusion detection in wireless mesh networks, and image classification, show the advantages of the proposed method over state-of-the-art methods.