AERO: Entropy-Guided Framework for Private LLM Inference
This addresses privacy concerns in LLM deployment for users needing secure inference, though it is incremental as it builds on existing transformer architectures.
The paper tackled the problem of high latency and communication overheads in privacy-preserving LLM inference by introducing AERO, a framework that strategically removes nonlinear operations, resulting in 3.4x communication savings and 1.4x latency reduction without performance loss.
Privacy-preserving computation enables language model inference directly on encrypted data yet suffers from prohibitive latency and communication overheads, primarily due to nonlinear functions. Removing nonlinearities, however, can trigger one of two failure modes restricting the potential for nonlinearity removal: entropy collapse in deeper layers, which destabilizes training, and entropic overload in early layers, causing under-utilization of attention heads. To address these challenges, we introduce AERO, an entropy-guided framework to strategically eliminates costly nonlinear operations from transformer architectures, which employs an adaptive recalibration through a head-wise entropy regularizer with learnable per-head strengths, enabling each head to adjust its entropy level while penalizing extreme entropies and fostering functional diversity through a tolerance margin. Experiments show AERO can save 3.4$\times$ communication and 1.4$\times$ latency, without any performance penalty.