Yu Lei

Bishan Wang, Jingwei He, Lei Yu et al.

With extremely high temporal resolution, event cameras have a large potential for robotics and computer vision. However, their asynchronous imaging mechanism often aggravates the measurement sensitivity to noises and brings a physical burden to increase the image spatial resolution. To recover high-quality intensity images, one should address both denoising and super-resolution problems for event cameras. Since events depict brightness changes, with the enhanced degeneration model by the events, the clear and sharp high-resolution latent images can be recovered from the noisy, blurry and low-resolution intensity observations. Exploiting the framework of sparse learning, the events and the low-resolution intensity observations can be jointly considered. Based on this, we propose an explainable network, an event-enhanced sparse learning network (eSL-Net), to recover the high-quality images from event cameras. After training with a synthetic dataset, the proposed eSL-Net can largely improve the performance of the state-of-the-art by 7-12 dB. Furthermore, without additional training process, the proposed eSL-Net can be easily extended to generate continuous frames with frame-rate as high as the events.

Fang Xu, Shijie Lin, Wen Yang et al.

The event camera has appealing properties: high dynamic range, low latency, low power consumption and low memory usage, and thus provides complementariness to conventional frame-based cameras. It only captures the dynamics of a scene and is able to capture almost "continuous" motion. However, different from frame-based camera that reflects the whole appearance as scenes are, the event camera casts away the detailed characteristics of objects, such as texture and color. To take advantages of both modalities, the event camera and frame-based camera are combined together for various machine vision tasks. Then the cross-modal matching between neuromorphic events and color images plays a vital and essential role. In this paper, we propose the Event-Based Image Retrieval (EBIR) problem to exploit the cross-modal matching task. Given an event stream depicting a particular object as query, the aim is to retrieve color images containing the same object. This problem is challenging because there exists a large modality gap between neuromorphic events and color images. We address the EBIR problem by proposing neuromorphic Events-Color image Feature Learning (ECFL). Particularly, the adversarial learning is employed to jointly model neuromorphic events and color images into a common embedding space. We also contribute to the community N-UKbench and EC180 dataset to promote the development of EBIR problem. Extensive experiments on our datasets show that the proposed method is superior in learning effective modality-invariant representation to link two different modalities.

Shaofu Xu, Rui Wang, Jianping Chen et al.

In regular microwave photonic (MWP) receiving systems, broadband signals are processed in the analog domain before they are transformed to the digital domain for further processing and storage. However, the quality of the signals may be degraded by defective photonic analog links, especially in a complicated MWP system. Here, we show a unified deep learning scheme that recovers the distorted broadband signals as they are transformed to the digital domain. The neural network could automatically learn the end-to-end inverse responses of the distortion effects of actual photonic analog links from data without expert knowledge and system priors. Hence, by shifting or augmenting the datasets, the neural network is potential to be generalized to various MWP receiving systems. We conduct experiments by nontrivial MWP systems with complicated waveforms. Results validate the effectiveness, general applicability and the noise-robustness of the proposed scheme, showing its superior performance in practical MWP systems. Therefore, the proposed deep learning scheme facilitates the low-cost performance improvement of MWP receiving systems, as well as the next-generation broadband transceivers, including radars, communications, and microwave imaging.