Junfei Huang

Panquan Yang, Junfei Huang, Zongzhangbao Yin et al.

3D visual grounding aims to localize the object in 3D point cloud scenes that semantically corresponds to given natural language sentences. It is very critical for roadside infrastructure system to interpret natural languages and localize relevant target objects in complex traffic environments. However, most existing datasets and approaches for 3D visual grounding focus on the indoor and outdoor driving scenes, outdoor monitoring scenarios remain unexplored due to scarcity of paired point cloud-text data captured by roadside infrastructure sensors. In this paper, we introduce a novel task of 3D Visual Grounding for Outdoor Monitoring Scenarios, which enables infrastructure-level understanding of traffic scenes beyond the ego-vehicle perspective. To support this task, we construct MoniRefer, the first real-world large-scale multi-modal dataset for roadside-level 3D visual grounding. The dataset consists of about 136,018 objects with 411,128 natural language expressions collected from multiple complex traffic intersections in the real-world environments. To ensure the quality and accuracy of the dataset, we manually verified all linguistic descriptions and 3D labels for objects. Additionally, we also propose a new end-to-end method, named Moni3DVG, which utilizes the rich appearance information provided by images and geometry and optical information from point cloud for multi-modal feature learning and 3D object localization. Extensive experiments and ablation studies on the proposed benchmarks demonstrate the superiority and effectiveness of our method. Our dataset and code will be released.

Junfei Huang, Guochu Shou

Service Oriented Architecture (SOA) is commonly employed in the design and implementation of web service systems. The key technology to enable media communications in the context of SOA is the Service Oriented Communication. To exploit the advantage of SOA, we design and implement a web-based multimedia conferencing system that provides users with a hybrid orchestration of web and communication services. As the current SOA lacks effective QoS guarantee solutions for multimedia services, the user satisfaction is greatly challenged with QoS violations, e.g., low video PSNR (Peak Signal-to-Noise Ratio) and long playback delay. Motivated by addressing the critical problem, we firstly employ the Business Process Execution Language (BPEL) service engine for the hybrid services orchestration and execution. Secondly, we propose a novel context-aware approach to quantify and leverage the causal relationships between QoS metrics and available contexts based on Bayesian networks (CABIN). This approach includes three phases: (1) information discretization, (2) causal relationship profiling, and (3) optimal context tuning. We implement CABIN in a real-life multimedia conferencing system and compare its performance with existing delay and throughput oriented schemes. Experimental results show that CABIN outperforms the competing approaches in improving the video quality in terms of PSNR. It also provides a one-stop shop controls both the web and communication services.