Cédric Thébault

Bharath Bhushan Damodaran, Emmanuel Jolly, Gilles Puy et al.

We present FacialFilmroll, a solution for spatially and temporally consistent editing of faces in one or multiple shots. We build upon unwrap mosaic [Rav-Acha et al. 2008] by specializing it to faces. We leverage recent techniques to fit a 3D face model on monocular videos to (i) improve the quality of the mosaic for edition and (ii) permit the automatic transfer of edits from one shot to other shots of the same actor. We explain how FacialFilmroll is integrated in post-production facility. Finally, we present video editing results using FacialFilmroll on high resolution videos.

Abdallah Dib, Gaurav Bharaj, Junghyun Ahn et al.

We present a differentiable ray-tracing based novel face reconstruction approach where scene attributes - 3D geometry, reflectance (diffuse, specular and roughness), pose, camera parameters, and scene illumination - are estimated from unconstrained monocular images. The proposed method models scene illumination via a novel, parameterized virtual light stage, which in-conjunction with differentiable ray-tracing, introduces a coarse-to-fine optimization formulation for face reconstruction. Our method can not only handle unconstrained illumination and self-shadows conditions, but also estimates diffuse and specular albedos. To estimate the face attributes consistently and with practical semantics, a two-stage optimization strategy systematically uses a subset of parametric attributes, where subsequent attribute estimations factor those previously estimated. For example, self-shadows estimated during the first stage, later prevent its baking into the personalized diffuse and specular albedos in the second stage. We show the efficacy of our approach in several real-world scenarios, where face attributes can be estimated even under extreme illumination conditions. Ablation studies, analyses and comparisons against several recent state-of-the-art methods show improved accuracy and versatility of our approach. With consistent face attributes reconstruction, our method leads to several style -- illumination, albedo, self-shadow -- edit and transfer applications, as discussed in the paper.

Lucas Mourot, François Le Clerc, Cédric Thébault et al.

Human Pose Estimation is a low-level task useful forsurveillance, human action recognition, and scene understandingat large. It also offers promising perspectives for the animationof synthetic characters. For all these applications, and especiallythe latter, estimating the positions of many joints is desirablefor improved performance and realism. To this purpose, wepropose a novel method called JUMPS for increasing the numberof joints in 2D pose estimates and recovering occluded ormissing joints. We believe this is the first attempt to addressthe issue. We build on a deep generative model that combines aGenerative Adversarial Network (GAN) and an encoder. TheGAN learns the distribution of high-resolution human posesequences, the encoder maps the input low-resolution sequencesto its latent space. Inpainting is obtained by computing the latentrepresentation whose decoding by the GAN generator optimallymatches the joints locations at the input. Post-processing a 2Dpose sequence using our method provides a richer representationof the character motion. We show experimentally that thelocalization accuracy of the additional joints is on average onpar with the original pose estimates.