Online Dual Coordinate Ascent Learning
This work addresses the need for scalable optimization in online learning scenarios where data streams continuously, offering an incremental improvement over existing S-DCA methods.
The paper tackled the limitation of stochastic dual coordinate-ascent (S-DCA) for online learning by developing an online dual coordinate-ascent (O-DCA) algorithm that handles streaming data without revisiting past data, enabling continuous adaptation and tracking.
The stochastic dual coordinate-ascent (S-DCA) technique is a useful alternative to the traditional stochastic gradient-descent algorithm for solving large-scale optimization problems due to its scalability to large data sets and strong theoretical guarantees. However, the available S-DCA formulation is limited to finite sample sizes and relies on performing multiple passes over the same data. This formulation is not well-suited for online implementations where data keep streaming in. In this work, we develop an {\em online} dual coordinate-ascent (O-DCA) algorithm that is able to respond to streaming data and does not need to revisit the past data. This feature embeds the resulting construction with continuous adaptation, learning, and tracking abilities, which are particularly attractive for online learning scenarios.