Knowledge Base Index Compression via Dimensionality and Precision Reduction
This work addresses memory and compute resource challenges in scaling up knowledge-intensive NLP tasks, but it is incremental as it applies existing compression techniques to a specific domain.
The paper tackles the problem of compressing knowledge base indices for neural retrieval in NLP tasks by exploring dimensionality and precision reduction methods, achieving up to 100x compression with 75% performance retention and 24x compression with 92% performance retention on HotpotQA.
Recently neural network based approaches to knowledge-intensive NLP tasks, such as question answering, started to rely heavily on the combination of neural retrievers and readers. Retrieval is typically performed over a large textual knowledge base (KB) which requires significant memory and compute resources, especially when scaled up. On HotpotQA we systematically investigate reducing the size of the KB index by means of dimensionality (sparse random projections, PCA, autoencoders) and numerical precision reduction. Our results show that PCA is an easy solution that requires very little data and is only slightly worse than autoencoders, which are less stable. All methods are sensitive to pre- and post-processing and data should always be centered and normalized both before and after dimension reduction. Finally, we show that it is possible to combine PCA with using 1bit per dimension. Overall we achieve (1) 100$\times$ compression with 75%, and (2) 24$\times$ compression with 92% original retrieval performance.