Sparse Topical Coding
This work addresses the need for more flexible and efficient topic modeling in machine learning, though it appears incremental as it builds on existing probabilistic topic models with specific relaxations.
The authors tackled the problem of discovering latent representations in large data collections by introducing sparse topical coding (STC), a non-probabilistic topic model that relaxes normalization constraints to enable sparsity control, integration with supervised learning, and efficient optimization, resulting in improved classification accuracy and time efficiency.
We present sparse topical coding (STC), a non-probabilistic formulation of topic models for discovering latent representations of large collections of data. Unlike probabilistic topic models, STC relaxes the normalization constraint of admixture proportions and the constraint of defining a normalized likelihood function. Such relaxations make STC amenable to: 1) directly control the sparsity of inferred representations by using sparsity-inducing regularizers; 2) be seamlessly integrated with a convex error function (e.g., SVM hinge loss) for supervised learning; and 3) be efficiently learned with a simply structured coordinate descent algorithm. Our results demonstrate the advantages of STC and supervised MedSTC on identifying topical meanings of words and improving classification accuracy and time efficiency.