Style Transfer to Calvin and Hobbes comics using Stable Diffusion
This is an incremental application of existing methods for style transfer in a specific domain (comics).
The researchers tackled the problem of converting input images into the Calvin and Hobbes comic style using stable diffusion fine-tuning, achieving visually appealing results with limited training time and data quality.
This project report summarizes our journey to perform stable diffusion fine-tuning on a dataset containing Calvin and Hobbes comics. The purpose is to convert any given input image into the comic style of Calvin and Hobbes, essentially performing style transfer. We train stable-diffusion-v1.5 using Low Rank Adaptation (LoRA) to efficiently speed up the fine-tuning process. The diffusion itself is handled by a Variational Autoencoder (VAE), which is a U-net. Our results were visually appealing for the amount of training time and the quality of input data that went into training.