Can Chen

Can Chen, Luca Zanotti Fragonara, Antonios Tsourdos

Autonomous systems need to localize and track surrounding objects in 3D space for safe motion planning. As a result, 3D multi-object tracking (MOT) plays a vital role in autonomous navigation. Most MOT methods use a tracking-by-detection pipeline, which includes object detection and data association processing. However, many approaches detect objects in 2D RGB sequences for tracking, which is lack of reliability when localizing objects in 3D space. Furthermore, it is still challenging to learn discriminative features for temporally-consistent detection in different frames, and the affinity matrix is normally learned from independent object features without considering the feature interaction between detected objects in the different frames. To settle these problems, We firstly employ a joint feature extractor to fuse the 2D and 3D appearance features captured from both 2D RGB images and 3D point clouds respectively, and then propose a novel convolutional operation, named RelationConv, to better exploit the correlation between each pair of objects in the adjacent frames, and learn a deep affinity matrix for further data association. We finally provide extensive evaluation to reveal that our proposed model achieves state-of-the-art performance on KITTI tracking benchmark.

Can Chen, Luca Zanotti Fragonara, Antonios Tsourdos

When localizing and detecting 3D objects for autonomous driving scenes, obtaining information from multiple sensor (e.g. camera, LIDAR) typically increases the robustness of 3D detectors. However, the efficient and effective fusion of different features captured from LIDAR and camera is still challenging, especially due to the sparsity and irregularity of point cloud distributions. This notwithstanding, point clouds offer useful complementary information. In this paper, we would like to leverage the advantages of LIDAR and camera sensors by proposing a deep neural network architecture for the fusion and the efficient detection of 3D objects by identifying their corresponding 3D bounding boxes with orientation. In order to achieve this task, instead of densely combining the point-wise feature of the point cloud and the related pixel features, we propose a novel fusion algorithm by projecting a set of 3D Region of Interests (RoIs) from the point clouds to the 2D RoIs of the corresponding the images. Finally, we demonstrate that our deep fusion approach achieves state-of-the-art performance on the KITTI 3D object detection challenging benchmark.

Can Chen, Luca Zanotti Fragonara, Antonios Tsourdos

In order to achieve better performance for point cloud analysis, many researchers apply deeper neural networks using stacked Multi-Layer-Perceptron (MLP) convolutions over irregular point cloud. However, applying dense MLP convolutions over large amount of points (e.g. autonomous driving application) leads to inefficiency in memory and computation. To achieve high performance but less complexity, we propose a deep-wide neural network, called ShufflePointNet, to exploit fine-grained local features and reduce redundancy in parallel using group convolution and channel shuffle operation. Unlike conventional operation that directly applies MLPs on high-dimensional features of point cloud, our model goes wider by splitting features into groups in advance, and each group with certain smaller depth is only responsible for respective MLP operation, which can reduce complexity and allows to encode more useful information. Meanwhile, we connect communication between groups by shuffling groups in feature channel to capture fine-grained features. We claim that, multi-branch method for wider neural networks is also beneficial to feature extraction for point cloud. We present extensive experiments for shape classification task on ModelNet40 dataset and semantic segmentation task on large scale datasets ShapeNet part, S3DIS and KITTI. We further perform ablation study and compare our model to other state-of-the-art algorithms in terms of complexity and accuracy.

Can Chen, Luca Zanotti Fragonara, Antonios Tsourdos

The analyses relying on 3D point clouds are an utterly complex task, often involving million of points, but also requiring computationally efficient algorithms because of many real-time applications; e.g. autonomous vehicle. However, point clouds are intrinsically irregular and the points are sparsely distributed in a non-Euclidean space, which normally requires point-wise processing to achieve high performances. Although shared filter matrices and pooling layers in convolutional neural networks (CNNs) are capable of reducing the dimensionality of the problem and extracting high-level information simultaneously, grids and highly regular data format are required as input. In order to balance model performance and complexity, we introduce a novel neural network architecture exploiting local features from a manually subsampled point set. In our network, a recursive farthest point sampling method is firstly applied to efficiently cover the entire point set. Successively, we employ the k-nearest neighbours (knn) algorithm to gather local neighbourhood for each group of the subsampled points. Finally, a multiple layer perceptron (MLP) is applied on the subsampled points and edges that connect corresponding point and neighbours to extract local features. The architecture has been tested for both shape classification and segmentation using the ModelNet40 and ShapeNet part datasets, in order to show that the network achieves the best trade-off in terms of competitive performance when compared to other state-of-the-art algorithms.