Computing Word Classes Using Spectral Clustering
This work addresses the need for effective word clustering in NLP to handle sparse data and improve tasks like semantic role labeling and dependency parsing, but it is incremental as it adapts an existing method to a new domain.
The authors tackled the problem of clustering a lexicon of words by applying spectral clustering, a method previously unexplored in this context, and evaluated it on semantic role labeling and dependency parsing tasks, showing it performs similarly to Brown clustering and outperforms other methods.
Clustering a lexicon of words is a well-studied problem in natural language processing (NLP). Word clusters are used to deal with sparse data in statistical language processing, as well as features for solving various NLP tasks (text categorization, question answering, named entity recognition and others). Spectral clustering is a widely used technique in the field of image processing and speech recognition. However, it has scarcely been explored in the context of NLP; specifically, the method used in this (Meila and Shi, 2001) has never been used to cluster a general word lexicon. We apply spectral clustering to a lexicon of words, evaluating the resulting clusters by using them as features for solving two classical NLP tasks: semantic role labeling and dependency parsing. We compare performance with Brown clustering, a widely-used technique for word clustering, as well as with other clustering methods. We show that spectral clusters produce similar results to Brown clusters, and outperform other clustering methods. In addition, we quantify the overlap between spectral and Brown clusters, showing that each model captures some information which is uncaptured by the other.