Chunlin Huang

Hongxi Li, Chunlin Huang

We present a theory of feature learning in wide L2-regularized networks showing that supervised learning is inherently compressive. We derive a kernel ODE that predicts a "water-filling" spectral evolution and prove that for any stable steady state, the kernel rank is bounded by the number of classes ($C$). We further demonstrate that SGD noise is similarly low-rank ($O(C)$), confining dynamics to the task-relevant subspace. This framework unifies the deterministic and stochastic views of alignment and contrasts the low-rank nature of supervised learning with the high-rank, expansive representations of self-supervision.

Zihui Song, Shihao Ji, Hongxi Li et al.

We investigate the hypothesis that real-data scaling laws are governed by progressive coverage of a latent predictive contribution spectrum rather than by token-frequency tails alone. We work with a suffix-automaton representation of text corpora and define a data-intrinsic global-KL predictive contribution spectrum, in which each state contributes according to its empirical mass times its KL deviation from a global next-token baseline. Across 12 real corpora, the tail slope of this spectrum is already strongly correlated with the empirical data-scaling exponent of a fixed small GPT learner. We then go beyond slope correlation and define, for each training size N, an effective truncation rank K(N) by matching the observed excess loss to the residual tail mass of the prepared 1000k global-KL spectrum. Empirically, log K is close to linear in log N, with pooled R^2 about 0.96 for the raw spectrum and R^2 about 0.90 for the smoothed spectrum. These findings provide strong empirical support for a simple mechanism picture: training scale advances an effective frontier through a predictive state spectrum, and the residual tail mass of that spectrum tracks the remaining excess loss.

Fusen Wang, Jun Sang, Qi Liu et al.

In this paper, we propose a known-plaintext attack (KPA) method based on deep learning for traditional chaotic encryption scheme. We employ the convolutional neural network to learn the operation mechanism of chaotic cryptosystem, and accept the trained network as the final decryption system. To evaluate the attack performance of different networks on different chaotic cryptosystem, we adopt two neural networks to perform known-plaintext attacks on two distinct chaotic encryption schemes. The experimental results demonstrate the potential of deep learning-based method for known-plaintext attack against chaotic cryptosystem. Different from the previous known-plaintext attack methods, which were usually limited to a specific chaotic cryptosystem, a neural network can be applied to the cryptanalysis of various chaotic cryptosystems with deep learning-based approach, while several different networks can be designed for the cryptanalysis of chaotic cryptosystems. This paper provides a new idea for the cryptanalysis of chaotic image encryption algorithm.