The Symmetric Perceptron: a Teacher-Student Scenario
This work addresses a foundational problem in statistical physics and machine learning by providing insights into learning dynamics in high-dimensional models, though it appears incremental as it adapts existing formulations.
The authors tackled the problem of transforming the symmetric binary Perceptron from a storage model into a teacher-student inference problem with guaranteed solutions at any sample density, and they mapped the phase diagram using free-entropy calculations to identify transitions in learning scenarios.
We introduce and solve a teacher-student formulation of the symmetric binary Perceptron, turning a traditionally storage-oriented model into a planted inference problem with a guaranteed solution at any sample density. We adapt the formulation of the symmetric Perceptron which traditionally considers either the u-shaped potential or the rectangular one, by including labels in both regions. With this formulation, we analyze both the Bayes-optimal regime at for noise-less examples and the effect of thermal noise under two different potential/classification rules. Using annealed and quenched free-entropy calculations in the high-dimensional limit, we map the phase diagram in the three control parameters, namely the sample density $α$, the distance between the origin and one of the symmetric hyperplanes $κ$ and temperature $T$, and identify a robust scenario where learning is organized by a second-order instability that creates teacher-correlated suboptimal states, followed by a first-order transition to full alignment. We show how this structure depends on the choice of potential, the interplay between metastability of the suboptimal solution and its melting towards the planted configuration, which is relevant for Monte Carlo-based optimization algorithms.