Junjie Zhou

Junjie Zhou, Haijun Xiong, Junhao Lu et al.

Skeleton-based gait emotion recognition has received significant attention due to its wide-ranging applications. However, existing methods primarily focus on extracting spatial and local temporal motion information, failing to capture long-range temporal representations. In this paper, we propose \textbf{CGTGait}, a novel framework that collaboratively integrates graph convolution and transformers to extract discriminative spatiotemporal features for gait emotion recognition. Specifically, CGTGait consists of multiple CGT blocks, where each block employs graph convolution to capture frame-level spatial topology and the transformer to model global temporal dependencies. Additionally, we introduce a Bidirectional Cross-Stream Fusion (BCSF) module to effectively aggregate posture and motion spatiotemporal features, facilitating the exchange of complementary information between the two streams. We evaluate our method on two widely used datasets, Emotion-Gait and ELMD, demonstrating that our CGTGait achieves state-of-the-art or at least competitive performance while reducing computational complexity by approximately \textbf{82.2\%} (only requiring 0.34G FLOPs) during testing. Code is available at \small{https://github.com/githubzjj1/CGTGait.}

Junjie Zhou, Zheng Liu, Shitao Xiao et al.

Multi-modal retrieval becomes increasingly popular in practice. However, the existing retrievers are mostly text-oriented, which lack the capability to process visual information. Despite the presence of vision-language models like CLIP, the current methods are severely limited in representing the text-only and image-only data. In this work, we present a new embedding model VISTA for universal multi-modal retrieval. Our work brings forth threefold technical contributions. Firstly, we introduce a flexible architecture which extends a powerful text encoder with the image understanding capability by introducing visual token embeddings. Secondly, we develop two data generation strategies, which bring high-quality composed image-text to facilitate the training of the embedding model. Thirdly, we introduce a multi-stage training algorithm, which first aligns the visual token embedding with the text encoder using massive weakly labeled data, and then develops multi-modal representation capability using the generated composed image-text data. In our experiments, VISTA achieves superior performances across a variety of multi-modal retrieval tasks in both zero-shot and supervised settings. Our model, data, and source code are available at https://github.com/FlagOpen/FlagEmbedding.

Junjie Zhou, Yan Shu, Bo Zhao et al.

The evaluation of Long Video Understanding (LVU) performance poses an important but challenging research problem. Despite previous efforts, the existing video understanding benchmarks are severely constrained by several issues, especially the insufficient lengths of videos, a lack of diversity in video types and evaluation tasks, and the inappropriateness for evaluating LVU performances. To address the above problems, we propose a new benchmark called MLVU (Multi-task Long Video Understanding Benchmark) for the comprehensive and in-depth evaluation of LVU. MLVU presents the following critical values: \textit{1)} The substantial and flexible extension of video lengths, which enables the benchmark to evaluate LVU performance across a wide range of durations. \textit{2)} The inclusion of various video genres, e.g., movies, surveillance footage, egocentric videos, cartoons, game videos, etc., which reflects the models' LVU performances in different scenarios. \textit{3)} The development of diversified evaluation tasks, which enables a comprehensive examination of MLLMs' key abilities in long-video understanding. The empirical study with 23 latest MLLMs reveals significant room for improvement in today's technique, as all existing methods struggle with most of the evaluation tasks and exhibit severe performance degradation when handling longer videos. Additionally, it suggests that factors such as context length, image-understanding ability, and the choice of LLM backbone can play critical roles in future advancements. We anticipate that MLVU will advance the research of long video understanding by providing a comprehensive and in-depth analysis of MLLMs.

Huichen Ma, Junjie Zhou, Jian Zhang et al.

In this work, we address the inverse kinetics problem of motion planning of soft biomimetic actuators driven by three chambers. Soft biomimetic actuators have been applied in many applications owing to their intrinsic softness. Although a mathematical model can be derived to describe the inverse dynamics of this actuator, it is still not accurate to capture the nonlinearity and uncertainty of the material and the system. Besides, such a complex model is time-consuming, so it is not easy to apply in the real-time control unit. Therefore, developing a model-free approach in this area could be a new idea. To overcome these intrinsic problems, we propose a back-propagation (BP) neural network learning the inverse kinetics of the soft biomimetic actuator moving in three-dimensional space. After training with sample data, the BP neural network model can represent the relation between the manipulator tip position and the pressure applied to the chambers. The proposed algorithm is more precise than the analytical model. The results show that a desired terminal position can be achieved with a degree of accuracy of 2.46% relative average error with respect to the total actuator length.

Tianyu Shi, Zhichao Wang, Chong hao Zou et al.

In this paper,we design a formation control systrm for multi-unmanned ground vehicles(UGV) from the prospective of path planning and path tracking.The master-slave control is adopted by electing out a main vehicle to address the problem of possible accumulation,tranmission and amplification of errors.In the process of formation transformation,we first generate an expected path by combing the methods of dynamic window and potential energy field.Then a path tracking algorithm based on Hermite curve is adopted to make the formation transformation process more stable and accurate.Finally,the evaluation system of the formation control system is constructed,which combines the expected position,the actual position,the expected speed, the actual speed and the actual acceleration,giving an evalutaion on the performance of the formation transformation,response of the formation driving process and the performance of the formation stability.