Learning Directly from Grammar Compressed Text
This addresses a bottleneck in processing large text datasets for machine learning, though it is incremental as it builds on existing compression and neural methods.
The paper tackled the problem of applying neural sequence models directly to grammar-compressed text without decompression, achieving memory and computational efficiency while maintaining moderate performance in experiments on real datasets.
Neural networks using numerous text data have been successfully applied to a variety of tasks. While massive text data is usually compressed using techniques such as grammar compression, almost all of the previous machine learning methods assume already decompressed sequence data as their input. In this paper, we propose a method to directly apply neural sequence models to text data compressed with grammar compression algorithms without decompression. To encode the unique symbols that appear in compression rules, we introduce composer modules to incrementally encode the symbols into vector representations. Through experiments on real datasets, we empirically showed that the proposal model can achieve both memory and computational efficiency while maintaining moderate performance.