Online Inference for Relation Extraction with a Reduced Feature Set
This work addresses scalability issues for researchers and practitioners in natural language processing aiming to build knowledge bases from web-scale data, but it is incremental as it focuses on empirical assessment of an existing inference scheme.
The paper tackled the scalability problem of probabilistic models for relation extraction from large unannotated corpora by empirically assessing a sublinear time sparse stochastic variational inference (SSVI) scheme applied to RelLDA, finding that online inference yields relatively strong qualitative results but also identifying pathologies that need addressing for large-scale use.
Access to web-scale corpora is gradually bringing robust automatic knowledge base creation and extension within reach. To exploit these large unannotated---and extremely difficult to annotate---corpora, unsupervised machine learning methods are required. Probabilistic models of text have recently found some success as such a tool, but scalability remains an obstacle in their application, with standard approaches relying on sampling schemes that are known to be difficult to scale. In this report, we therefore present an empirical assessment of the sublinear time sparse stochastic variational inference (SSVI) scheme applied to RelLDA. We demonstrate that online inference leads to relatively strong qualitative results but also identify some of its pathologies---and those of the model---which will need to be overcome if SSVI is to be used for large-scale relation extraction.