Generating coherent comic with rich story using ChatGPT and Stable Diffusion
This addresses the challenge of creating AI-generated comics with consistent narratives and artistic styles, though it appears incremental by combining existing models.
The paper tackles the problem of generating coherent comics with rich stories by using ChatGPT for storyline/dialogue generation and Stable Diffusion for image generation, achieving state-of-the-art performance on character fidelity and art style through fine-tuning techniques like LoRA and ControlNet.
Past work demonstrated that using neural networks, we can extend unfinished music pieces while maintaining the music style of the musician. With recent advancements in large language models and diffusion models, we are now capable of generating comics with an interesting storyline while maintaining the art style of the artist. In this paper, we used ChatGPT to generate storylines and dialogue and then generated the comic using stable diffusion. We introduced a novel way to evaluate AI-generated stories, and we achieved SOTA performance on character fidelity and art style by fine-tuning stable diffusion using LoRA, ControlNet, etc.