Forward asymmetric numeral systems coding for natural language text compression
For researchers and practitioners in data compression, this solves the adaptive ANS problem, enabling efficient compression with forward modeling.
The paper proposes a method combining forward modeling and adaptive coding for asymmetric numeral systems (ANS), achieving compression ratios close to Shannon entropy while enabling adaptive ANS—a long-standing problem. The approach maintains high encoding/decoding speeds.
Compression based on asymmetric numeral systems (ANS) combines high encoding and decoding speeds with a compression ratio close to Shannon entropy, while forward modeling of the information source makes it possible to obtain an estimated compressed message size that is less than the entropy. This paper proposes combining these modeling and adaptive coding methods. In addition to ensuring high data processing speeds and compression ratios, this approach enables one to implement the adaptive ANS, which has long remained an important scientific and practical problem.