Shuai Feng

Shuai Feng, Pietro Tesi

In this paper, we study networked control systems in the presence of Denial-of-Service (DoS) attacks, namely attacks that prevent transmissions over the communication network. The control objective is to maximize frequency and duration of the DoS attacks under which closed-loop stability is not destroyed. Analog and digital predictor-based controllers with state resetting are proposed, which achieve the considered control objective for a general class of DoS signals. An example is given to illustrate the proposed solution approach.

Shuai Feng, Pietro Tesi, Claudio De Persis

In this paper, we consider networked distributed systems in the presence of Denial-of-Service (DoS) attacks, namely attacks that prevent transmissions over the communication network. First, we consider a simple and typical scenario where communication sequence is purely Round-robin and we explicitly calculate a bound of attack frequency and duration, under which the interconnected large-scale system is asymptotically stable. Second, trading-off system resilience and communication load, we design a hybrid transmission strategy consisting of Zeno-free distributed event-triggered control and Round-robin. We show that with lower communication loads, the hybrid communication strategy enables the systems to have the same resilience as in pure Round-robin.

Wenyu Jiang, Yuxin Ge, Hao Cheng et al.

Detecting out-of-distribution (OOD) samples is crucial to the safe deployment of a classifier in the real world. However, deep neural networks are known to be overconfident for abnormal data. Existing works directly design score function by mining the inconsistency from classifier for in-distribution (ID) and OOD. In this paper, we further complement this inconsistency with reconstruction error, based on the assumption that an autoencoder trained on ID data can not reconstruct OOD as well as ID. We propose a novel method, READ (Reconstruction Error Aggregated Detector), to unify inconsistencies from classifier and autoencoder. Specifically, the reconstruction error of raw pixels is transformed to latent space of classifier. We show that the transformed reconstruction error bridges the semantic gap and inherits detection performance from the original. Moreover, we propose an adjustment strategy to alleviate the overconfidence problem of autoencoder according to a fine-grained characterization of OOD data. Under two scenarios of pre-training and retraining, we respectively present two variants of our method, namely READ-MD (Mahalanobis Distance) only based on pre-trained classifier and READ-ED (Euclidean Distance) which retrains the classifier. Our methods do not require access to test time OOD data for fine-tuning hyperparameters. Finally, we demonstrate the effectiveness of the proposed methods through extensive comparisons with state-of-the-art OOD detection algorithms. On a CIFAR-10 pre-trained WideResNet, our method reduces the average FPR@95TPR by up to 9.8% compared with previous state-of-the-art.

Shuai Feng, Pietro Tesi

In this paper, we study networked systems in the presence of Denial-of-Service (DoS) attacks, namely attacks that prevent transmissions over the communication network. Previous studies have shown that co-located architectures (control unit co-located with the actuators and networked sensor channel) can ensure a high level of robustness against DoS. However, co-location requires a wired or dedicated actuator channel, which could not meet flexibility and cost requirements. In this paper we consider a control architecture that approximates co-location while enabling remote implementation (networked sensor and actuator channels). We analyze closed-loop stability and quantify the robustness "gap" between this architecture and the co-located one.

Yucen Wang, Shenghua Wan, Le Gan et al.

Model-based methods have significantly contributed to distinguishing task-irrelevant distractors for visual control. However, prior research has primarily focused on heterogeneous distractors like noisy background videos, leaving homogeneous distractors that closely resemble controllable agents largely unexplored, which poses significant challenges to existing methods. To tackle this problem, we propose Implicit Action Generator (IAG) to learn the implicit actions of visual distractors, and present a new algorithm named implicit Action-informed Diverse visual Distractors Distinguisher (AD3), that leverages the action inferred by IAG to train separated world models. Implicit actions effectively capture the behavior of background distractors, aiding in distinguishing the task-irrelevant components, and the agent can optimize the policy within the task-relevant state space. Our method achieves superior performance on various visual control tasks featuring both heterogeneous and homogeneous distractors. The indispensable role of implicit actions learned by IAG is also empirically validated.

Kaichen Huang, Hai-Hang Sun, Shenghua Wan et al.

Imitating skills from low-quality datasets, such as sub-optimal demonstrations and observations with distractors, is common in real-world applications. In this work, we focus on the problem of Learning from Noisy Demonstrations (LND), where the imitator is required to learn from data with noise that often occurs during the processes of data collection or transmission. Previous IL methods improve the robustness of learned policies by injecting an adversarially learned Gaussian noise into pure expert data or utilizing additional ranking information, but they may fail in the LND setting. To alleviate the above problems, we propose Denoised Imitation learning based on Domain Adaptation (DIDA), which designs two discriminators to distinguish the noise level and expertise level of data, facilitating a feature encoder to learn task-related but domain-agnostic representations. Experiment results on MuJoCo demonstrate that DIDA can successfully handle challenging imitation tasks from demonstrations with various types of noise, outperforming most baseline methods.

Kaichen Huang, Minghao Shao, Shenghua Wan et al.

In many real-world visual Imitation Learning (IL) scenarios, there is a misalignment between the agent's and the expert's perspectives, which might lead to the failure of imitation. Previous methods have generally solved this problem by domain alignment, which incurs extra computation and storage costs, and these methods fail to handle the \textit{hard cases} where the viewpoint gap is too large. To alleviate the above problems, we introduce active sensoring in the visual IL setting and propose a model-based SENSory imitatOR (SENSOR) to automatically change the agent's perspective to match the expert's. SENSOR jointly learns a world model to capture the dynamics of latent states, a sensor policy to control the camera, and a motor policy to control the agent. Experiments on visual locomotion tasks show that SENSOR can efficiently simulate the expert's perspective and strategy, and outperforms most baseline methods.

Shuai Feng, Yuxin Ge, Yuntao Du et al.

Out-of-distribution (OOD) detection is crucial for deploying robust machine learning models. However, when training data follows a long-tailed distribution, the model's ability to accurately detect OOD samples is significantly compromised, due to the confusion between OOD samples and head/tail classes. To distinguish OOD samples from both head and tail classes, the separate class learning (SCL) approach has emerged as a promising solution, which separately conduct head-specific and tail-specific class learning. To this end, we examine the limitations of existing works of SCL and reveal that the OOD detection performance is notably influenced by the use of static scaling temperature value and the presence of uninformative outliers. To mitigate these limitations, we propose a novel approach termed Refined Separate Class Learning (RSCL), which leverages dynamic class-wise temperature adjustment to modulate the temperature parameter for each in-distribution class and informative outlier mining to identify diverse types of outliers based on their affinity with head and tail classes. Extensive experiments demonstrate that RSCL achieves superior OOD detection performance while improving the classification accuracy on in-distribution data.

Shuai Feng, Wei-Chuang Chan, Srishti Chouhan et al.

The integration of large language models (LLMs) into global applications necessitates effective cultural alignment for meaningful and culturally-sensitive interactions. Current LLMs often lack the nuanced understanding required for diverse cultural contexts, and adapting them typically involves costly full fine-tuning. To address this, we introduce a novel soft prompt fine-tuning framework that enables efficient and modular cultural alignment. Our method utilizes vectorized prompt tuning to dynamically route queries to a committee of culturally specialized 'expert' LLM configurations, created by optimizing soft prompt embeddings without altering the base model's parameters. Extensive experiments demonstrate that our framework significantly enhances cultural sensitivity and adaptability, improving alignment scores from 0.208 to 0.820, offering a robust solution for culturally-aware LLM deployment. This research paves the way for subsequent investigations into enhanced cultural coverage and dynamic expert adaptation, crucial for realizing autonomous AI with deeply nuanced understanding in a globally interconnected world.

Shenghua Wan, Ziyuan Chen, Le Gan et al.

Model-based offline reinforcement Learning (RL) is a promising approach that leverages existing data effectively in many real-world applications, especially those involving high-dimensional inputs like images and videos. To alleviate the distribution shift issue in offline RL, existing model-based methods heavily rely on the uncertainty of learned dynamics. However, the model uncertainty estimation becomes significantly biased when observations contain complex distractors with non-trivial dynamics. To address this challenge, we propose a new approach - \emph{Separated Model-based Offline Policy Optimization} (SeMOPO) - decomposing latent states into endogenous and exogenous parts via conservative sampling and estimating model uncertainty on the endogenous states only. We provide a theoretical guarantee of model uncertainty and performance bound of SeMOPO. To assess the efficacy, we construct the Low-Quality Vision Deep Data-Driven Datasets for RL (LQV-D4RL), where the data are collected by non-expert policy and the observations include moving distractors. Experimental results show that our method substantially outperforms all baseline methods, and further analytical experiments validate the critical designs in our method. The project website is \href{https://sites.google.com/view/semopo}{https://sites.google.com/view/semopo}.

Shuai Feng, Ahmet Cetinkaya, Hideaki Ishii et al.

We study communication-constrained networked control problems for linear time-invariant systems in the presence of Denial-of-Service (DoS) attacks, namely attacks that prevent transmissions over the communication network. Our work aims at exploring the relationship between system resilience and network bandwidth capacity. Given a class of DoS attacks, we first characterize time-invariant bit-rate bounds that are dependent on the unstable eigenvalues of the dynamic matrix of the plant and the parameters of DoS attacks, beyond which exponential stability of the closed-loop system can be guaranteed. Second, we design the time-varying bit-rate protocol and show that it can enable the system to maintain the comparable robustness as the one under the time-invariant bit-rate protocol and meanwhile promote the possibility of transmitting fewer bits especially when the attack levels are low. Our characterization clearly shows the trade-off between the communication bandwidth and resilience against DoS. An example is given to illustrate the proposed solution approach.