3.6ROJun 26, 2023
A Self-supervised Contrastive Learning Method for Grasp Outcomes PredictionChengliang Liu, Binhua Huang, Yiwen Liu et al.
In this paper, we investigate the effectiveness of contrastive learning methods for predicting grasp outcomes in an unsupervised manner. By utilizing a publicly available dataset, we demonstrate that contrastive learning methods perform well on the task of grasp outcomes prediction. Specifically, the dynamic-dictionary-based method with the momentum updating technique achieves a satisfactory accuracy of 81.83% using data from one single tactile sensor, outperforming other unsupervised methods. Our results reveal the potential of contrastive learning methods for applications in the field of robot grasping and highlight the importance of accurate grasp prediction for achieving stable grasps.
MoCrop: Training Free Motion Guided Cropping for Efficient Video Action RecognitionBinhua Huang, Wendong Yao, Shaowu Chen et al.
We introduce MoCrop, a motion-aware adaptive cropping module for efficient video action recognition in the compressed domain. MoCrop uses motion vectors that are available in H.264 video to locate motion-dense regions and produces a single clip-level crop that is applied to all I-frames at inference. The module is training free, adds no parameters, and can be plugged into diverse backbones. A lightweight pipeline that includes denoising & merge (DM), Monte Carlo sampling (MCS), and adaptive cropping (AC) via a motion-density submatrix search yields robust crops with negligible overhead. On UCF101, MoCrop improves accuracy or reduces compute. With ResNet-50, it delivers +3.5% Top-1 accuracy at equal FLOPs (attention setting), or +2.4% Top-1 accuracy with 26.5% fewer FLOPs (efficiency setting). Applied to CoViAR, it reaches 89.2% Top-1 accuracy at the original cost and 88.5% Top-1 accuracy while reducing compute from 11.6 to 8.5 GFLOPs. Consistent gains on MobileNet-V3, EfficientNet-B1, and Swin-B indicate strong generality and make MoCrop practical for real-time deployment in the compressed domain. Our code and models are available at https://github.com/microa/MoCrop.
MoCLIP-Lite: Efficient Video Recognition by Fusing CLIP with Motion VectorsBinhua Huang, Ni Wang, Arjun Pakrashi et al.
Video action recognition is a fundamental task in computer vision, but state-of-the-art models are often computationally expensive and rely on extensive video pre-training. In parallel, large-scale vision-language models like Contrastive Language-Image Pre-training (CLIP) offer powerful zero-shot capabilities on static images, while motion vectors (MV) provide highly efficient temporal information directly from compressed video streams. To synergize the strengths of these paradigms, we propose MoCLIP-Lite, a simple yet powerful two-stream late fusion framework for efficient video recognition. Our approach combines features from a frozen CLIP image encoder with features from a lightweight, supervised network trained on raw MV. During fusion, both backbones are frozen, and only a tiny Multi-Layer Perceptron (MLP) head is trained, ensuring extreme efficiency. Through comprehensive experiments on the UCF101 dataset, our method achieves a remarkable 89.2% Top-1 accuracy, significantly outperforming strong zero-shot (65.0%) and MV-only (66.5%) baselines. Our work provides a new, highly efficient baseline for video understanding that effectively bridges the gap between large static models and dynamic, low-cost motion cues. Our code and models are available at https://github.com/microa/MoCLIP-Lite.
DCentNet: Decentralized Multistage Biomedical Signal Classification using Early ExitsXiaolin Li, Binhua Huang, Barry Cardiff et al.
DCentNet is a novel decentralized multistage signal classification approach designed for biomedical data from IoT wearable sensors, integrating early exit points (EEP) to enhance energy efficiency and processing speed. Unlike traditional centralized processing methods, which result in high energy consumption and latency, DCentNet partitions a single CNN model into multiple sub-networks using EEPs. By introducing encoder-decoder pairs at EEPs, the system compresses large feature maps before transmission, significantly reducing wireless data transfer and power usage. If an input is confidently classified at an EEP, processing stops early, optimizing efficiency. Initial sub-networks can be deployed on fog or edge devices to further minimize energy consumption. A genetic algorithm is used to optimize EEP placement, balancing performance and complexity. Experimental results on ECG classification show that with one EEP, DCentNet reduces wireless data transmission by 94.54% and complexity by 21%, while maintaining original accuracy and sensitivity. With two EEPs, sensitivity reaches 98.36%, accuracy 97.74%, wireless data transmission decreases by 91.86%, and complexity is reduced by 22%. Implemented on an ARM Cortex-M4 MCU, DCentNet achieves an average power saving of 73.6% compared to continuous wireless ECG transmission.