Junan Lin

Junan Lin, Zhichao Sun, Enjie Cao et al.

Skeletal Action recognition from an egocentric view is important for applications such as interfaces in AR/VR glasses and human-robot interaction, where the device has limited resources. Most of the existing skeletal action recognition approaches use 3D coordinates of hand joints and 8-corner rectangular bounding boxes of objects as inputs, but they do not capture how the hands and objects interact with each other within the spatial context. In this paper, we present a new framework called Contact-aware Skeletal Action Recognition (CaSAR). It uses novel representations of hand-object interaction that encompass spatial information: 1) contact points where the hand joints meet the objects, 2) distant points where the hand joints are far away from the object and nearly not involved in the current action. Our framework is able to learn how the hands touch or stay away from the objects for each frame of the action sequence, and use this information to predict the action class. We demonstrate that our approach achieves the state-of-the-art accuracy of 91.3% and 98.4% on two public datasets, H2O and FPHA, respectively.

Junan Lin, Daizong Liu, Xianke Chen et al.

Video Moment Retrieval (VMR) aims to retrieve a specific moment semantically related to the given query. To tackle this task, most existing VMR methods solely focus on the visual and textual modalities while neglecting the complementary but important audio modality. Although a few recent works try to tackle the joint audio-vision-text reasoning, they treat all modalities equally and simply embed them without fine-grained interaction for moment retrieval. These designs are counter-practical as: Not all audios are helpful for video moment retrieval, and the audio of some videos may be complete noise or background sound that is meaningless to the moment determination. To this end, we propose a novel Importance-aware Multi-Granularity fusion model (IMG), which learns to dynamically and selectively aggregate the audio-vision-text contexts for VMR. Specifically, after integrating the textual guidance with vision and audio separately, we first design a pseudo-label-supervised audio importance predictor that predicts the importance score of the audio, and accordingly assigns weights to mitigate the interference caused by noisy audio. Then, we design a multi-granularity audio fusion module that adaptively fuses audio and visual modalities at local-, event-, and global-level, fully capturing their complementary contexts. We further propose a cross-modal knowledge distillation strategy to address the challenge of missing audio modality during inference. To evaluate our method, we further construct a new VMR dataset, i.e., Charades-AudioMatter, where audio-related samples are manually selected and re-organized from the original Charades-STA to validate the model's capability in utilizing audio modality. Extensive experiments validate the effectiveness of our method, achieving state-of-the-art with audio-video fusion in VMR methods. Our code is available at https://github.com/HuiGuanLab/IMG.

Junan Lin, Paul J. Goulart, Luca Furieri

The Alternating Direction Method of Multipliers (ADMM) is a widely used method for structured convex optimization, and its practical performance depends strongly on the choice of penalty and relaxation parameters. Motivated by settings such as Model Predictive Control (MPC), where one repeatedly solves related optimization problems with fixed structure and changing parameter values, we propose learning online updates of the relaxation parameter to improve performance on problem classes of interest. This choice is computationally attractive in OSQP-like architectures, since adapting relaxation does not trigger the matrix refactorizations associated with penalty updates. We establish convergence guarantees for ADMM with time-varying penalty and relaxation parameters under mild assumptions, and show on benchmark quadratic programs that the resulting learned policies improve both iteration count and wall-clock time over baseline OSQP.

Junan Lin, Tal Mor, Roman Shapira

The field of quantum information is becoming more known to the general public. However, effectively demonstrating the concepts underneath quantum science and technology to the general public can be a challenging job. We investigate, extend, and greatly expand here "quantum candies" (invented by Jacobs), a pedagogical model for intuitively describing some basic concepts in quantum information, including quantum bits, complementarity, the no-cloning principle, and entanglement. Following Jacob's quantum candies description of the well-known quantum key distribution protocol BB84, we explicitly demonstrate additional quantum cryptography protocols and quantum communication protocols, using generalized quantum candies (including correlated pairs of qandies). These demonstrations are done in an approachable manner, that can be explained to high-school students, without using the hard-to-grasp concept of superpositions and its mathematics. The intuitive model we investigate has a fascinating overlap with some of the most basic features of quantum theory. Hence, it can be a valuable tool for science and engineering educators who would like to help the general public to gain more insights into quantum science and technology. For the experts, the model we present, due to not employing quantum superpositions, enables - in some sense - extending far beyond quantum theory. Most remarkably, "quantum" candies of some unique type can be defined, such that non-local boxes (of the Popescu-Rohrlich type) as well as regular (correlated) quantum candies can be generated by a single `"quantum" candies machine.

Junan Lin, Tal Mor

The field of quantum information is becoming more known to the general public. However, effectively demonstrating the concepts underneath quantum science and technology to the general public can be a challenging job. We investigate, extend, and much expand here "quantum candies" (invented by Jacobs), a pedagogical model for intuitively describing some basic concepts in quantum information, including quantum bits, complementarity, the no-cloning principle, and entanglement. Following Jacob's quantum candies description of the well known quantum key distribution protocol BB84, we explicitly demonstrate various additional quantum cryptography protocols using quantum candies in an approachable manner. The model we investigate can be a valuable tool for science and engineering educators who would like to help the general public to gain more insights about quantum science and technology: most parts of this paper, including many protocols for quantum cryptography, are expected to be easily understandable by a layperson without any previous knowledge of mathematics, physics, or cryptography.