RenderBender: A Survey on Adversarial Attacks Using Differentiable Rendering
This survey addresses the need for a structured approach to adversarial machine learning research involving differentiable rendering, which is incremental as it organizes existing work rather than introducing new methods.
The paper tackles the problem of unifying adversarial attacks using differentiable rendering by proposing the first framework that categorizes diverse goals and tasks, facilitating comparison and identifying research gaps in the field.
Differentiable rendering techniques like Gaussian Splatting and Neural Radiance Fields have become powerful tools for generating high-fidelity models of 3D objects and scenes. Their ability to produce both physically plausible and differentiable models of scenes are key ingredient needed to produce physically plausible adversarial attacks on DNNs. However, the adversarial machine learning community has yet to fully explore these capabilities, partly due to differing attack goals (e.g., misclassification, misdetection) and a wide range of possible scene manipulations used to achieve them (e.g., alter texture, mesh). This survey contributes the first framework that unifies diverse goals and tasks, facilitating easy comparison of existing work, identifying research gaps, and highlighting future directions - ranging from expanding attack goals and tasks to account for new modalities, state-of-the-art models, tools, and pipelines, to underscoring the importance of studying real-world threats in complex scenes.