Fan Zhang
The design of deep neural networks remains somewhat of an art rather than precise science. By tentatively adopting ergodic theory considerations on top of viewing the network as the time evolution of a dynamical system, with each layer corresponding to a temporal instance, we show that some rules of thumb, which might otherwise appear mysterious, can be attributed heuristics.
Yudao Sun, Juan Yin, Juan Zhao et al.
Neural network language models (LMs) are confronted with significant challenges in generalization and robustness. Currently, many studies focus on improving either generalization or robustness in isolation, without methods addressing both aspects simultaneously, which presents a significant challenge in developing LMs that are both robust and generalized. In this paper, we propose a bi-stage optimization framework to uniformly enhance both the generalization and robustness of LMs, termed UEGR. Specifically, during the forward propagation stage, we enrich the output probability distributions of adversarial samples by adaptive dropout to generate diverse sub models, and incorporate JS divergence and adversarial losses of these output distributions to reinforce output stability. During backward propagation stage, we compute parameter saliency scores and selectively update only the most critical parameters to minimize unnecessary deviations and consolidate the model's resilience. Theoretical analysis shows that our framework includes gradient regularization to limit the model's sensitivity to input perturbations and selective parameter updates to flatten the loss landscape, thus improving both generalization and robustness. The experimental results show that our method significantly improves the generalization and robustness of LMs compared to other existing methods across 13 publicly available language datasets, achieving state-of-the-art (SOTA) performance.
Xiaoxuan Lou, Fan Zhang, Zheng Leong Chua et al.
Rowhammer is a hardware-based bug that allows the attacker to modify the data in the memory without accessing it, just repeatedly and frequently accessing (or hammering) physically adjacent memory rows. So that it can break the memory isolation between processes, which is seen as the cornerstone of modern system security, exposing the sensitive data to unauthorized and imperceptible corruption. A number of previous works have leveraged the rowhammer bug to achieve various critical attacks. In this work, we propose a unified reference framework for analyzing the rowhammer attacks, indicating three necessary factors in a practical rowhammer attack: the attack origin, the intended implication and the methodology. Each factor includes multiple primitives, the attacker can select primitives from three factors to constitute an effective attack. In particular, the methodology further summarizes all existing attack techniques, that are used to achieve its three primitives: Location Preparation (LP), Rapid Hammering (RH), and Exploit Verification (EV). Based on the reference framework, we analyze all previous rowhammer attacks and corresponding countermeasures. Our analysis shows that how primitives in different factors are combined and used in previous attacks, and thus points out new possibility of rowhammer attacks, enabling proactive prevention before it causes harm. Under the framework, we propose a novel expressive rowhammer attack that is capable of accumulating injected memory changes and achieving rich attack semantics. We conclude by outlining future research directions.