Ke Hu

Shutong Ding, Yimiao Zhou, Ke Hu et al.

Recent advances in diffusion-based reinforcement learning (RL) methods have demonstrated promising results in a wide range of continuous control tasks. However, existing works in this field focus on the application of diffusion policies while leaving the diffusion critics unexplored. In fact, since policy optimization fundamentally relies on the critic, accurate value estimation is far more important than policy expressiveness. Furthermore, given the stochasticity of most reinforcement learning tasks, it has been confirmed that the critic is more appropriately depicted with a distributional model. Motivated by these points, we propose a novel distributional RL method with Diffusion Bridge Critics (DBC). DBC directly models the inverse cumulative distribution function (CDF) of the Q value. This allows us to accurately capture the value distribution and prevents it from collapsing into a trivial Gaussian distribution owing to the strong distribution-matching capability of the diffusion bridge. Moreover, we further derive an analytic integral formula to address discretization errors in DBC, which is essential in value estimation. To our knowledge, DBC is the first work to employ the diffusion bridge model as the critic. Notably, DBC is also a plug-and-play component and can be integrated into most existing RL frameworks. Experimental results on MuJoCo robot control benchmarks demonstrate the superiority of DBC compared with previous distributional critic models.

Xun Yuan, Ke Hu, Song Chen

The local feature detector and descriptor are essential in many computer vision tasks, such as SLAM and 3D reconstruction. In this paper, we introduce two separate CNNs, lightweight SobelNet and DesNet, to detect key points and to compute dense local descriptors. The detector and the descriptor work in parallel. Sobel filter provides the edge structure of the input images as the input of CNN. The locations of key points will be obtained after exerting the non-maximum suppression (NMS) process on the output map of the CNN. We design Gaussian loss for the training process of SobelNet to detect corner points as keypoints. At the same time, the input of DesNet is the original grayscale image, and circle loss is used to train DesNet. Besides, output maps of SobelNet are needed while training DesNet. We have evaluated our method on several benchmarks including HPatches benchmark, ETH benchmark, and FM-Bench. SobelNet achieves better or comparable performance with less computation compared with SOTA methods in recent years. The inference time of an image of 640x480 is 7.59ms and 1.09ms for SobelNet and DesNet respectively on RTX 2070 SUPER.

Ke Hu, Ashfaqur Rahman, Hassan Habibi Gharakheili et al.

Nowadays air pollution becomes one of the biggest world issues in both developing and developed countries. Helping individuals understand their air pollution exposure and health risks, the traditional way is to utilize data from static monitoring stations and estimate air pollution qualities in a large area by government agencies. Data from such sensing system is very sparse and cannot reflect real personal exposure. In recent years, several research groups have developed participatory air pollution sensing systems which use wearable or portable units coupled with smartphones to crowd-source urban air pollution data. These systems have shown remarkable improvement in spatial granularity over government-operated fixed monitoring systems. In this paper, we extend the paradigm to HazeDose system, which can personalize the individuals' air pollution exposure. Specifically, we combine the pollution concentrations obtained from an air pollution estimation system with the activity data from the individual's on-body activity monitors to estimate the personal inhalation dosage of air pollution. Users can visualize their personalized air pollution exposure information via a mobile application. We show that different activities, such as walking, cycling, or driving, impact their dosage, and commuting patterns contribute to a significant proportion of an individual's daily air pollution dosage. Moreover, we propose a dosage minimization algorithm, with the trial results showing that up to 14.1% of a biker's daily exposure can be reduced while using alternative routes the driver can inhale 25.9% less than usual. One heuristic algorithm is also introduced to balance the execution time and dosage reduction for alternative routes scenarios. The results show that up to 20.3% dosage reduction can be achieved when the execution time is almost one seventieth of the original one.