Structured Learning via Logistic Regression
This work provides a method for structured learning that generalizes beyond linear factors, potentially benefiting researchers in machine learning and optimization.
The paper tackles structured learning by showing that smoothing the inference problem with entropy terms reduces the learning objective to a traditional logistic regression problem for fixed messages, enabling the extension of structured energy functions to any function class with a logistic loss minimization oracle.
A successful approach to structured learning is to write the learning objective as a joint function of linear parameters and inference messages, and iterate between updates to each. This paper observes that if the inference problem is "smoothed" through the addition of entropy terms, for fixed messages, the learning objective reduces to a traditional (non-structured) logistic regression problem with respect to parameters. In these logistic regression problems, each training example has a bias term determined by the current set of messages. Based on this insight, the structured energy function can be extended from linear factors to any function class where an "oracle" exists to minimize a logistic loss.