11.6AIApr 12
A Progressive Training Strategy for Vision-Language Models to Counteract Spatio-Temporal Hallucinations in Embodied ReasoningXiaoda Yang, Shuai Yang, Can Wang et al.
Vision-Language Models (VLMs) have made significant strides in static image understanding but continue to face critical hurdles in spatiotemporal reasoning. A major bottleneck is "multi-image reasoning hallucination", where a massive performance drop between forward and reverse temporal queries reveals a dependence on superficial shortcuts instead of genuine causal understanding. To mitigate this, we first develop a new Chain-of-Thought (CoT) dataset that decomposes intricate reasoning into detailed spatiotemporal steps and definitive judgments. Building on this, we present a progressive training framework: it initiates with supervised pre-training on our CoT dataset to instill logical structures, followed by fine-tuning with scalable weakly-labeled data for broader generalization. Our experiments demonstrate that this approach not only improves backbone accuracy but also slashes the forward-backward performance gap from over 70\% to only 6.53\%. This confirms the method's ability to develop authentic dynamic reasoning and reduce the inherent temporal biases of current VLMs.
2.3CRAug 31, 2023Code
Towards Low-Barrier Cybersecurity Research and Education for Industrial Control SystemsColman McGuan, Chansu Yu, Qin Lin
The protection of Industrial Control Systems (ICS) that are employed in public critical infrastructures is of utmost importance due to catastrophic physical damages cyberattacks may cause. The research community requires testbeds for validation and comparing various intrusion detection algorithms to protect ICS. However, there exist high barriers to entry for research and education in the ICS cybersecurity domain due to expensive hardware, software, and inherent dangers of manipulating real-world systems. To close the gap, built upon recently developed 3D high-fidelity simulators, we further showcase our integrated framework to automatically launch cyberattacks, collect data, train machine learning models, and evaluate for practical chemical and manufacturing processes. On our testbed, we validate our proposed intrusion detection model called Minimal Threshold and Window SVM (MinTWin SVM) that utilizes unsupervised machine learning via a one-class SVM in combination with a sliding window and classification threshold. Results show that MinTWin SVM minimizes false positives and is responsive to physical process anomalies. Furthermore, we incorporate our framework with ICS cybersecurity education by using our dataset in an undergraduate machine learning course where students gain hands-on experience in practicing machine learning theory with a practical ICS dataset. All of our implementations have been open-sourced.
7.0CRMay 13
MQTT Across a Raspberry Pi 5 IoT Network Utilizing Quantum-resistant Signature AlgorithmsRay Feingold, Chansu Yu
The rapid expansion of the Internet of Things (IoT) has introduced millions of resource-constrained devices into critical infrastructures, consumer environments, and industrial systems. These devices rely on lightweight communication protocols such as MQTT to support low-power, intermittent, and bandwidth-limited operation. However, common TLS algorithms used to secure MQTT communications are vulnerable to quantum attacks made feasible by Shor's algorithm. As a result, IoT infrastructures must evaluate and adopt post-quantum cryptographic (PQC) methods capable of providing long-term resilience. This report investigates the implementation of PQC algorithms within an MQTT-based IoT networks using three Raspberry Pis. Specifically, it integrates the FALCON digital signature scheme, one of NIST's selected post-quantum signature algorithms, to maintain message authenticity and integrity across resource-constrained MQTT clients and brokers. By measuring system performance, the research characterizes the practical trade-offs of deploying lattice-based PQC on lightweight hardware.
8.2ROFeb 9
$χ_{0}$: Resource-Aware Robust Manipulation via Taming Distributional InconsistenciesChecheng Yu, Chonghao Sima, Gangcheng Jiang et al.
High-reliability long-horizon robotic manipulation has traditionally relied on large-scale data and compute to understand complex real-world dynamics. However, we identify that the primary bottleneck to real-world robustness is not resource scale alone, but the distributional shift among the human demonstration distribution, the inductive bias learned by the policy, and the test-time execution distribution -- a systematic inconsistency that causes compounding errors in multi-stage tasks. To mitigate these inconsistencies, we propose $χ_{0}$, a resource-efficient framework with effective modules designated to achieve production-level robustness in robotic manipulation. Our approach builds off three technical pillars: (i) Model Arithmetic, a weight-space merging strategy that efficiently soaks up diverse distributions of different demonstrations, varying from object appearance to state variations; (ii) Stage Advantage, a stage-aware advantage estimator that provides stable, dense progress signals, overcoming the numerical instability of prior non-stage approaches; and (iii) Train-Deploy Alignment, which bridges the distribution gap via spatio-temporal augmentation, heuristic DAgger corrections, and temporal chunk-wise smoothing. $χ_{0}$ enables two sets of dual-arm robots to collaboratively orchestrate long-horizon garment manipulation, spanning tasks from flattening, folding, to hanging different clothes. Our method exhibits high-reliability autonomy; we are able to run the system from arbitrary initial state for consecutive 24 hours non-stop. Experiments validate that $χ_{0}$ surpasses the state-of-the-art $π_{0.5}$ in success rate by nearly 250%, with only 20-hour data and 8 A100 GPUs. Code, data and models will be released to facilitate the community.
16.9ROOct 30, 2024
EMOS: Embodiment-aware Heterogeneous Multi-robot Operating System with LLM AgentsJunting Chen, Checheng Yu, Xunzhe Zhou et al.
Heterogeneous multi-robot systems (HMRS) have emerged as a powerful approach for tackling complex tasks that single robots cannot manage alone. Current large-language-model-based multi-agent systems (LLM-based MAS) have shown success in areas like software development and operating systems, but applying these systems to robot control presents unique challenges. In particular, the capabilities of each agent in a multi-robot system are inherently tied to the physical composition of the robots, rather than predefined roles. To address this issue, we introduce a novel multi-agent framework designed to enable effective collaboration among heterogeneous robots with varying embodiments and capabilities, along with a new benchmark named Habitat-MAS. One of our key designs is $\textit{Robot Resume}$: Instead of adopting human-designed role play, we propose a self-prompted approach, where agents comprehend robot URDF files and call robot kinematics tools to generate descriptions of their physics capabilities to guide their behavior in task planning and action execution. The Habitat-MAS benchmark is designed to assess how a multi-agent framework handles tasks that require embodiment-aware reasoning, which includes 1) manipulation, 2) perception, 3) navigation, and 4) comprehensive multi-floor object rearrangement. The experimental results indicate that the robot's resume and the hierarchical design of our multi-agent system are essential for the effective operation of the heterogeneous multi-robot system within this intricate problem context.
17.4CVApr 3, 2025
OmniCam: Unified Multimodal Video Generation via Camera ControlXiaoda Yang, Jiayang Xu, Kaixuan Luan et al.
Camera control, which achieves diverse visual effects by changing camera position and pose, has attracted widespread attention. However, existing methods face challenges such as complex interaction and limited control capabilities. To address these issues, we present OmniCam, a unified multimodal camera control framework. Leveraging large language models and video diffusion models, OmniCam generates spatio-temporally consistent videos. It supports various combinations of input modalities: the user can provide text or video with expected trajectory as camera path guidance, and image or video as content reference, enabling precise control over camera motion. To facilitate the training of OmniCam, we introduce the OmniTr dataset, which contains a large collection of high-quality long-sequence trajectories, videos, and corresponding descriptions. Experimental results demonstrate that our model achieves state-of-the-art performance in high-quality camera-controlled video generation across various metrics.
1.2NINov 7, 2017
Pre-shared Key Agreement for Secure Public Wi-FiSeokseong Jeon, Chansu Yu, Young-Joo Suh
This paper presents a novel pre-shared key (PSK) agreement scheme to establish a secure connection between a Wi-Fi client and access point (AP) without prior knowledge of a password. The standard IEEE 802.11 security method, Robust Security Network Association, widely known as Wi-Fi Protected Access (WPA) and WPA2, derives a shared cryptographic key if and only if a user provides an identical password which an AP possesses, causing ofinconvenience of obtaining and entering the password. In this paper, a proposed scheme, Secure Open AP (SOAP), adopts two public key algorithms, the elliptic curve Diffie-Hellman key exchange algorithm (ECDH) and digital signature algorithm (ECDSA) to establish a secure connection between a client and an AP without having prior knowledge of a password. Implementation and experiment results demonstrate the viability of the proposed scheme.