Detecting and Restoring Non-Standard Hands in Stable Diffusion Generated Images
This addresses a specific problem for users of Stable Diffusion by improving image quality, but it is incremental as it builds on existing models like ControlNet and InstructPix2Pix.
The paper tackles anatomical inaccuracies in hand images generated by Stable Diffusion by introducing a pipeline that detects and restores anomalies, resulting in enhanced realism as demonstrated experimentally.
We introduce a pipeline to address anatomical inaccuracies in Stable Diffusion generated hand images. The initial step involves constructing a specialized dataset, focusing on hand anomalies, to train our models effectively. A finetuned detection model is pivotal for precise identification of these anomalies, ensuring targeted correction. Body pose estimation aids in understanding hand orientation and positioning, crucial for accurate anomaly correction. The integration of ControlNet and InstructPix2Pix facilitates sophisticated inpainting and pixel-level transformation, respectively. This dual approach allows for high-fidelity image adjustments. This comprehensive approach ensures the generation of images with anatomically accurate hands, closely resembling real-world appearances. Our experimental results demonstrate the pipeline's efficacy in enhancing hand image realism in Stable Diffusion outputs. We provide an online demo at https://fixhand.yiqun.io