A Random Block-Coordinate Douglas-Rachford Splitting Method with Low Computational Complexity for Binary Logistic Regression
This is an incremental improvement for optimization in machine learning, specifically for sparse logistic regression.
The paper tackles binary logistic regression by proposing a stochastic block-coordinate Douglas-Rachford splitting method that uses mini-batches and proximity operators with the generalized Lambert W function, demonstrating efficiency over stochastic gradient methods on standard datasets.
In this paper, we propose a new optimization algorithm for sparse logistic regression based on a stochastic version of the Douglas-Rachford splitting method. Our algorithm sweeps the training set by randomly selecting a mini-batch of data at each iteration, and it allows us to update the variables in a block coordinate manner. Our approach leverages the proximity operator of the logistic loss, which is expressed with the generalized Lambert W function. Experiments carried out on standard datasets demonstrate the efficiency of our approach w.r.t. stochastic gradient-like methods.