7.1LGDec 17, 2025
On-device Large Multi-modal Agent for Human Activity RecognitionMd Shakhrul Iman Siam, Ishtiaque Ahmed Showmik, Guanqun Song et al.
Human Activity Recognition (HAR) has been an active area of research, with applications ranging from healthcare to smart environments. The recent advancements in Large Language Models (LLMs) have opened new possibilities to leverage their capabilities in HAR, enabling not just activity classification but also interpretability and human-like interaction. In this paper, we present a Large Multi-Modal Agent designed for HAR, which integrates the power of LLMs to enhance both performance and user engagement. The proposed framework not only delivers activity classification but also bridges the gap between technical outputs and user-friendly insights through its reasoning and question-answering capabilities. We conduct extensive evaluations using widely adopted HAR datasets, including HHAR, Shoaib, Motionsense to assess the performance of our framework. The results demonstrate that our model achieves high classification accuracy comparable to state-of-the-art methods while significantly improving interpretability through its reasoning and Q&A capabilities.
2.0CVDec 23, 2023
Beyond the Frame: Single and mutilple video summarization method with user-defined lengthVahid Ahmadi Kalkhorani, Qingquan Zhang, Guanqun Song et al.
Video smmarization is a crucial method to reduce the time of videos which reduces the spent time to watch/review a long video. This apporach has became more important as the amount of publisehed video is increasing everyday. A single or multiple videos can be summarized into a relatively short video using various of techniques from multimodal audio-visual techniques, to natural language processing approaches. Audiovisual techniques may be used to recognize significant visual events and pick the most important parts, while NLP techniques can be used to evaluate the audio transcript and extract the main sentences (timestamps) and corresponding video frames from the original video. Another approach is to use the best of both domain. Meaning that we can use audio-visual cues as well as video transcript to extract and summarize the video. In this paper, we combine a variety of NLP techniques (extractive and contect-based summarizers) with video processing techniques to convert a long video into a single relatively short video. We design this toll in a way that user can specify the relative length of the summarized video. We have also explored ways of summarizing and concatenating multiple videos into a single short video which will help having most important concepts from the same subject in a single short video. Out approach shows that video summarizing is a difficult but significant work, with substantial potential for further research and development, and it is possible thanks to the development of NLP models.
6.5LGJan 8, 2021
Benchmarking Machine Learning: How Fast Can Your Algorithms Go?Zeyu Ning, Hugues Nelson Iradukunda, Qingquan Zhang et al.
This paper is focused on evaluating the effect of some different techniques in machine learning speed-up, including vector caches, parallel execution, and so on. The following content will include some review of the previous approaches and our own experimental results.
4.2LGDec 22, 2020
MailLeak: Obfuscation-Robust Character Extraction Using Transfer LearningWei Wang, Emily Sallenback, Zeyu Ning et al.
The following work presents a new algorithm for character recognition from obfuscated images. The presented method is an example of a potential threat to current postal services. This paper both analyzes the efficiency of the given algorithm and suggests countermeasures to prevent such threats from occurring.