Recovering Geometric Information with Learned Texture Perturbations
This addresses the issue of detail loss in neural network-inferred cloth for computer graphics applications, though it appears incremental as it builds on existing regularization and texture techniques.
The paper tackles the problem of neural networks losing high-frequency detail due to regularization, which leads to unnatural or time-incoherent wrinkles in inferred cloth. It proposes a method to embed high-frequency information into low-frequency data, resulting in recovered geometric detail that corrects over-smoothed appearances across multiple camera views.
Regularization is used to avoid overfitting when training a neural network; unfortunately, this reduces the attainable level of detail hindering the ability to capture high-frequency information present in the training data. Even though various approaches may be used to re-introduce high-frequency detail, it typically does not match the training data and is often not time coherent. In the case of network inferred cloth, these sentiments manifest themselves via either a lack of detailed wrinkles or unnaturally appearing and/or time incoherent surrogate wrinkles. Thus, we propose a general strategy whereby high-frequency information is procedurally embedded into low-frequency data so that when the latter is smeared out by the network the former still retains its high-frequency detail. We illustrate this approach by learning texture coordinates which when smeared do not in turn smear out the high-frequency detail in the texture itself but merely smoothly distort it. Notably, we prescribe perturbed texture coordinates that are subsequently used to correct the over-smoothed appearance of inferred cloth, and correcting the appearance from multiple camera views naturally recovers lost geometric information.