Rui Yang

Jiawei Xu, Rui Yang, Shuang Qiu et al. · tencent-ai, tsinghua

Learning policy from offline datasets through offline reinforcement learning (RL) holds promise for scaling data-driven decision-making while avoiding unsafe and costly online interactions. However, real-world data collected from sensors or humans often contains noise and errors, posing a significant challenge for existing offline RL methods, particularly when the real-world data is limited. Our study reveals that prior research focusing on adapting predominant offline RL methods based on temporal difference learning still falls short under data corruption when the dataset is limited. In contrast, we discover that vanilla sequence modeling methods, such as Decision Transformer, exhibit robustness against data corruption, even without specialized modifications. To unlock the full potential of sequence modeling, we propose Robust Decision Rransformer (RDT) by incorporating three simple yet effective robust techniques: embedding dropout to improve the model's robustness against erroneous inputs, Gaussian weighted learning to mitigate the effects of corrupted labels, and iterative data correction to eliminate corrupted data from the source. Extensive experiments on MuJoCo, Kitchen, and Adroit tasks demonstrate RDT's superior performance under various data corruption scenarios compared to prior methods. Furthermore, RDT exhibits remarkable robustness in a more challenging setting that combines training-time data corruption with test-time observation perturbations. These results highlight the potential of sequence modeling for learning from noisy or corrupted offline datasets, thereby promoting the reliable application of offline RL in real-world scenarios. Our code is available at https://github.com/jiawei415/RobustDecisionTransformer.

Xiao Fu, Rui Yang, Xiaojiang Du et al.

Recently, the IaaS (Infrastructure as a Service) Cloud (e.g., Amazon EC2) has been widely used by many organizations. However, some IaaS security issues create serious threats to its users. A typical issue is the timing channel. This kind of channel can be a cross-VM information channel, as proven by many researchers. Because it is covert and traceless, the traditional identification methods cannot build an accurate analysis model and obtain a compromised result. We investigated the underlying behavior of the timing channel from the perspective of the memory activity records and summarized the signature of the timing channel in the underlying memory activities. An identification method based on long-term behavior signatures was proposed. We proposed a complete set of forensics steps including evidence extraction, identification, record reserve, and evidence reports. We studied four typical timing channels, and the experiments showed that these channels can be detected and investigated, even with disturbances from normal processes.