Yang Wang

Peichen Xie, Yang Wang, Fan Yang et al.

The rapidly growing computation demands of deep neural networks (DNNs) have driven hardware vendors to integrate matrix multiplication accelerators (MMAs), such as NVIDIA Tensor Cores and AMD Matrix Cores, into modern GPUs. However, due to distinct and undocumented arithmetic specifications for floating-point matrix multiplication, some MMAs can lead to numerical imprecision and inconsistency that can compromise the stability and reproducibility of DNN training and inference. This paper presents MMA-Sim, the first bit-accurate reference model that reveals the detailed arithmetic behaviors of the MMAs from ten GPU architectures (eight from NVIDIA and two from AMD). By dissecting the MMAs using a combination of targeted and randomized tests, our methodology derives nine arithmetic algorithms to simulate the floating-point matrix multiplication of the MMAs. Large-scale validation confirms bitwise equivalence between MMA-Sim and the real hardware. Using MMA-Sim, we investigate arithmetic behaviors that affect DNN training stability, and identify undocumented behaviors that could lead to significant errors.

Yixuan Zhang, Zhidong Li, Yang Wang et al.

Machine learning algorithms often struggle to eliminate inherent data biases, particularly those arising from unreliable labels, which poses a significant challenge in ensuring fairness. Existing fairness techniques that address label bias typically involve modifying models and intervening in the training process, but these lack flexibility for large-scale datasets. To address this limitation, we introduce a data selection method designed to efficiently and flexibly mitigate label bias, tailored to more practical needs. Our approach utilizes a zero-shot predictor as a proxy model that simulates training on a clean holdout set. This strategy, supported by peer predictions, ensures the fairness of the proxy model and eliminates the need for an additional holdout set, which is a common requirement in previous methods. Without altering the classifier's architecture, our modality-agnostic method effectively selects appropriate training data and has proven efficient and effective in handling label bias and improving fairness across diverse datasets in experimental evaluations.

Rui Li, Zhibin Pan, Yang Wang

Tensor decomposition (TD) is widely used in hyperspectral image (HSI) compression. The initialization of factor matrix in tensor decomposition can determine the HSI compression performance. It is worth noting that HSI is highly correlated in bands. However, this phenomenon is ignored by the previous TD method. Aiming at improving the HSI compression performance, we propose a method called correlation-based TD initialization algorithm. As HSI is well approximated by means of a reference band. In accordance with the SVD result of the reference band, the initialized factor matrices of TD are produced. We compare our methods with random and SVD-based initialization methods. The experimental results reveal that our correlation-based TD initialization method is capable of significantly reducing the computational cost of TD while keeping the initialization quality and compression performance.

Mark Iwen, Aditya Viswanathan, Yang Wang

In this short note we propose a simple two-stage sparse phase retrieval strategy that uses a near-optimal number of measurements, and is both computationally efficient and robust to measurement noise. In addition, the proposed strategy is fairly general, allowing for a large number of new measurement constructions and recovery algorithms to be designed with minimal effort.