Layer-Parallel Inference Reduces Encrypted Nonlinear Depth in Transformers
For practitioners of encrypted neural network inference, SNLP offers a method to reduce the computational bottleneck of sequential nonlinear layers in FHE-based Transformers.
Structured Newton Layer Parallelism (SNLP) reduces the sequential nonlinear depth in encrypted Transformer inference, cutting symbolic bootstraps from 53 to 20 (2.65x) with only +1.2% perplexity degradation on a 0.5B model, and lowers error amplification across all tested models.
Fully homomorphic encryption (FHE) enables computation on encrypted data, but practical encrypted Transformer inference is bottlenecked by the sequential composition of many nonlinear blocks. We study whether Structured Newton Layer Parallelism (SNLP) can make this inter-layer composition more FHE-friendly: each Transformer block still requires polynomial approximations for operations such as softmax and RMSNorm, but SNLP reduces the layerwise sequential nonlinear depth from L stages to a small number of solver iterations plus linear structured corrections. Using a simulation framework based on Chebyshev polynomial approximations, we measure error accumulation under sequential versus SNLP inference across 8 models and 4 architecture families. On a 0.5B IDN-trained model, SNLP reduces symbolic bootstraps from 53 to 20 (2.65x) with only +1.2% perplexity degradation, while lowering error amplification (1.36x vs. 1.42x). Across all tested models, SNLP has lower amplification than sequential inference. Ablations show that softmax approximation dominates the error budget and CKKS arithmetic noise is negligible in our setting, suggesting that SNLP is complementary to block-level FHE-friendly operator design rather than a replacement for it.