Ryosuke Watanabe

Ryosuke Watanabe, Shashank N. Sridhara, Haoran Hong et al.

Point clouds are a general format for representing realistic 3D objects in diverse 3D applications. Since point clouds have large data sizes, developing efficient point cloud compression methods is crucial. However, excessive compression leads to various distortions, which deteriorates the point cloud quality perceived by end users. Thus, establishing reliable point cloud quality assessment (PCQA) methods is essential as a benchmark to develop efficient compression methods. This paper presents an accurate full-reference point cloud quality assessment (FR-PCQA) method called full-reference quality assessment using support vector regression (FRSVR) for various types of degradations such as compression distortion, Gaussian noise, and down-sampling. The proposed method demonstrates accurate PCQA by integrating five FR-based metrics covering various types of errors (e.g., considering geometric distortion, color distortion, and point count) using support vector regression (SVR). Moreover, the proposed method achieves a superior trade-off between accuracy and calculation speed because it includes only the calculation of these five simple metrics and SVR, which can perform fast prediction. Experimental results with three types of open datasets show that the proposed method is more accurate than conventional FR-PCQA methods. In addition, the proposed method is faster than state-of-the-art methods that utilize complicated features such as curvature and multi-scale features. Thus, the proposed method provides excellent performance in terms of the accuracy of PCQA and processing speed. Our method is available from https://github.com/STAC-USC/FRSVR-PCQA.

Ryosuke Watanabe, Keisuke Nonaka, Eduardo Pavez et al.

Point clouds in 3D applications frequently experience quality degradation during processing, e.g., scanning and compression. Reliable point cloud quality assessment (PCQA) is important for developing compression algorithms with good bitrate-quality trade-offs and techniques for quality improvement (e.g., denoising). This paper introduces a full-reference (FR) PCQA method utilizing spectral graph wavelets (SGWs). First, we propose novel SGW-based PCQA metrics that compare SGW coefficients of coordinate and color signals between reference and distorted point clouds. Second, we achieve accurate PCQA by integrating several conventional FR metrics and our SGW-based metrics using support vector regression. To our knowledge, this is the first study to introduce SGWs for PCQA. Experimental results demonstrate the proposed PCQA metric is more accurately correlated with subjective quality scores compared to conventional PCQA metrics.

Jun Chen, Ryosuke Watanabe, Keisuke Nonaka et al.

We present a billboard-based free-viewpoint video synthesizing algorithm for sports scenes that can robustly reconstruct and render a high-fidelity billboard model for each object, including an occluded one, in each camera. Its contributions are (1) applicable to a challenging shooting condition where a high precision 3D model cannot be built because a small number of cameras featuring wide-baseline are equipped; (2) capable of reproducing appearances of occlusions, that is one of the most significant issues for billboard-based approaches due to the ineffective detection of overlaps. To achieve contributions above, the proposed method does not attempt to find a high-quality 3D model but utilizes a raw 3D model that is obtained directly from space carving. Although the model is insufficiently accurate for producing an impressive visual effect, precise objects segmentation and occlusions detection can be performed by back-projecting it onto each camera plane. The billboard model of each object in each camera is rendered according to whether it is occluded or not, and its location in the virtual stadium is determined considering the location of its 3D model. We synthesized free-viewpoint videos of two soccer sequences recorded by five cameras with the proposed and state-of-art methods to demonstrate its performance.

Jun Chen, Ryosuke Watanabe, Keisuke Nonaka et al.

In this paper, we report on a parallel freeviewpoint video synthesis algorithm that can efficiently reconstruct a high-quality 3D scene representation of sports scenes. The proposed method focuses on a scene that is captured by multiple synchronized cameras featuring wide-baselines. The following strategies are introduced to accelerate the production of a free-viewpoint video taking the improvement of visual quality into account: (1) a sparse point cloud is reconstructed using a volumetric visual hull approach, and an exact 3D ROI is found for each object using an efficient connected components labeling algorithm. Next, the reconstruction of a dense point cloud is accelerated by implementing visual hull only in the ROIs; (2) an accurate polyhedral surface mesh is built by estimating the exact intersections between grid cells and the visual hull; (3) the appearance of the reconstructed presentation is reproduced in a view-dependent manner that respectively renders the non-occluded and occluded region with the nearest camera and its neighboring cameras. The production for volleyball and judo sequences demonstrates the effectiveness of our method in terms of both execution time and visual quality.

Jun Chen, Keisuke Nonaka, Ryosuke Watanabe et al.

This paper proposes a new parallel approach to solve connected components on a 2D binary image implemented with CUDA. We employ the following strategies to accelerate neighborhood exploration after dividing an input image into independent blocks. In the local labeling stage, a coarse-labeling algorithm, including row-column connection and label-equivalence list unification, is applied first to sort out the mess of an initialized local label map; a refinement algorithm is then introduced to merge separated sub-regions from a single component. In the block merge stage, we scan the pixels located on the boundary of each block instead of solving the connectivity of all the pixels. With the proposed method, the length of label-equivalence lists is compressed, and the number of memory accesses is reduced. Thus, the efficiency of connected components labeling is improved. Experimental results show that our method outperforms the other approaches between $29\%$ and $80\%$ on average.