A two-step sequential approach for hyperparameter selection in finite context models
This provides a more efficient alternative to exhaustive hyperparameter tuning for researchers and practitioners using FCMs in domains like DNA compression, though it appears incremental as it builds on existing statistical measures.
The paper tackles the computationally expensive problem of hyperparameter selection in finite-context models (FCMs) for compressing symbolic sequences like DNA, proposing a two-step sequential approach that decomposes joint optimization into independent stages for context length and smoothing parameter estimation. Results show the method achieves compression performance comparable to exhaustive grid search in terms of average bitrate while substantially reducing computational cost.
Finite-context models (FCMs) are widely used for compressing symbolic sequences such as DNA, where predictive performance depends critically on the context length k and smoothing parameter α. In practice, these hyperparameters are typically selected through exhaustive search, which is computationally expensive and scales poorly with model complexity. This paper proposes a statistically grounded two-step sequential approach for efficient hyperparameter selection in FCMs. The key idea is to decompose the joint optimization problem into two independent stages. First, the context length k is estimated using categorical serial dependence measures, including Cramér's ν, Cohen's \k{appa} and partial mutual information (pami). Second, the smoothing parameter α is estimated via maximum likelihood conditional on the selected context length k. Simulation experiments were conducted on synthetic symbolic sequences generated by FCMs across multiple (k, α) configurations, considering a four-letter alphabet and different sample sizes. Results show that the dependence measures are substantially more sensitive to variations in k than in α, supporting the sequential estimation strategy. As expected, the accuracy of the hyperparameter estimation improves with increasing sample size. Furthermore, the proposed method achieves compression performance comparable to exhaustive grid search in terms of average bitrate (bits per symbol), while substantially reducing computational cost. Overall, the results on simulated data show that the proposed sequential approach is a practical and computationally efficient alternative to exhaustive hyperparameter tuning in FCMs.