Juan Ren

Juan Ren, Mark Dras, Usman Naseem

Large Vision-Language Models (LVLMs) unlock powerful multimodal reasoning but also expand the attack surface, particularly through adversarial inputs that conceal harmful goals in benign prompts. We propose SHIELD, a lightweight, model-agnostic preprocessing framework that couples fine-grained safety classification with category-specific guidance and explicit actions (Block, Reframe, Forward). Unlike binary moderators, SHIELD composes tailored safety prompts that enforce nuanced refusals or safe redirection without retraining. Across five benchmarks and five representative LVLMs, SHIELD consistently lowers jailbreak and non-following rates while preserving utility. Our method is plug-and-play, incurs negligible overhead, and is easily extendable to new attack types -- serving as a practical safety patch for both weakly and strongly aligned LVLMs.

Juan Ren, Mark Dras, Usman Naseem

Large Vision-Language Models (LVLMs) have shown remarkable capabilities across a wide range of multimodal tasks. However, their integration of visual inputs introduces expanded attack surfaces, thereby exposing them to novel security vulnerabilities. In this work, we conduct a systematic representational analysis to uncover why conventional adversarial attacks can circumvent the safety mechanisms embedded in LVLMs. We further propose a novel two stage evaluation framework for adversarial attacks on LVLMs. The first stage differentiates among instruction non compliance, outright refusal, and successful adversarial exploitation. The second stage quantifies the degree to which the model's output fulfills the harmful intent of the adversarial prompt, while categorizing refusal behavior into direct refusals, soft refusals, and partial refusals that remain inadvertently helpful. Finally, we introduce a normative schema that defines idealized model behavior when confronted with harmful prompts, offering a principled target for safety alignment in multimodal systems.

Shengwen Xie, Juan Ren

In this paper, we present a hybrid position/force controller for operating joint robots. The hybrid controller has two goals -- motion tracking and force regulating. As long as these two goals are not mutually exclusive, they can be decoupled in some way. In this work, we make use of the smooth and invertible mapping from joint space to task space to decouple the two control goals and design controllers separately. The traditional motion controller in task space is used for motion control, while the force controller is designed through manipulating the desired trajectory to regulate the force indirectly. Two case studies -- contour tracking/polishing surfaces and grabbing boxes with two robotic arms -- are presented to show the efficacy of the hybrid controller, and simulations with physics engines are carried out to validate the efficacy of the proposed method.