TactStyle: Generating Tactile Textures with Generative AI for Digital Fabrication
This addresses the lack of tactile information in digital fabrication for creators, enabling haptic design, though it is incremental as it builds on existing generative AI methods.
The paper tackles the problem of generating 3D models with desired tactile properties from image prompts, presenting TactStyle, which uses a fine-tuned generative AI model to create heightfields for textures, and results show it successfully generates a wide range of tactile features from single images.
Recent work in Generative AI enables the stylization of 3D models based on image prompts. However, these methods do not incorporate tactile information, leading to designs that lack the expected tactile properties. We present TactStyle, a system that allows creators to stylize 3D models with images while incorporating the expected tactile properties. TactStyle accomplishes this using a modified image-generation model fine-tuned to generate heightfields for given surface textures. By optimizing 3D model surfaces to embody a generated texture, TactStyle creates models that match the desired style and replicate the tactile experience. We utilize a large-scale dataset of textures to train our texture generation model. In a psychophysical experiment, we evaluate the tactile qualities of a set of 3D-printed original textures and TactStyle's generated textures. Our results show that TactStyle successfully generates a wide range of tactile features from a single image input, enabling a novel approach to haptic design.