Zhijing Li

Hao-Jun Michael Shi, Tsung-Hsien Lee, Shintaro Iwasaki et al. · mila

Shampoo is an online and stochastic optimization algorithm belonging to the AdaGrad family of methods for training neural networks. It constructs a block-diagonal preconditioner where each block consists of a coarse Kronecker product approximation to full-matrix AdaGrad for each parameter of the neural network. In this work, we provide a complete description of the algorithm as well as the performance optimizations that our implementation leverages to train deep networks at-scale in PyTorch. Our implementation enables fast multi-GPU distributed data-parallel training by distributing the memory and computation associated with blocks of each parameter via PyTorch's DTensor data structure and performing an AllGather primitive on the computed search directions at each iteration. This major performance enhancement enables us to achieve at most a 10% performance reduction in per-step wall-clock time compared against standard diagonal-scaling-based adaptive gradient methods. We validate our implementation by performing an ablation study on training ImageNet ResNet50, demonstrating Shampoo's superiority over standard training recipes with minimal hyperparameter tuning.

Zhijing Li, Yuwei Ye, Stephen Neuendorffer et al.

As customized accelerator design has become increasingly popular to keep up with the demand for high performance computing, it poses challenges for modern simulator design to adapt to such a large variety of accelerators. Existing simulators tend to two extremes: low-level and general approaches, such as RTL simulation, that can model any hardware but require substantial effort and long execution times; and higher-level application-specific models that can be much faster and easier to use but require one-off engineering effort. This work proposes a compiler-driven simulation workflow that can model configurable hardware accelerator. The key idea is to separate structure representation from simulation by developing an intermediate language that can flexibly represent a wide variety of hardware constructs. We design the Event Queue (EQueue) dialect of MLIR, a dialect that can model arbitrary hardware accelerators with explicit data movement and distributed event-based control; we also implement a generic simulation engine to model EQueue programs with hybrid MLIR dialects representing different abstraction levels. We demonstrate two case studies of EQueue-implemented accelerators: the systolic array of convolution and SIMD processors in a modern FPGA. In the former we show EQueue simulation is as accurate as a state-of-the-art simulator, while offering higher extensibility and lower iteration cost via compiler passes. In the latter we demonstrate our simulation flow can guide designer efficiently improve their design using visualizable simulation outputs.

Yuxin Chen, Huiying Li, Steven Nagels et al.

Recent works have explained the principle of using ultrasonic transmissions to jam nearby microphones. These signals are inaudible to nearby users, but leverage "hardware nonlinearity" to induce a jamming signal inside microphones that disrupts voice recordings. This has great implications on audio privacy protection. In this work, we gain a deeper understanding on the effectiveness of ultrasonic jammer under practical scenarios, with the goal of disabling both visible and hidden microphones in the surrounding area. We first experiment with existing jammer designs (both commercial products and that proposed by recent papers), and find that they all offer limited angular coverage, and can only target microphones in a particular direction. We overcome this limitation by building a circular transducer array as a wearable bracelet. It emits ultrasonic signals simultaneously from many directions, targeting surrounding microphones without needing to point at any. More importantly, as the bracelet moves with the wearer, its motion increases jamming coverage and diminishes blind spots (the fundamental problem facing any transducer array). We evaluate the jammer bracelet under practical scenarios, confirming that it can effectively disrupt visible and hidden microphones in the surrounding areas, preventing recognition of recorded speech. We also identify limitations and areas for improvement.