Calliffusion: Chinese Calligraphy Generation and Style Transfer with Diffusion Modeling
This addresses the problem of automated calligraphy generation and style transfer for artists and designers, but it is incremental as it applies existing diffusion methods to a new domain.
The paper tackles generating high-quality Chinese calligraphy and transferring styles using diffusion models, achieving results that are difficult to distinguish from real artworks and enabling effective control over characters, scripts, and styles.
In this paper, we propose Calliffusion, a system for generating high-quality Chinese calligraphy using diffusion models. Our model architecture is based on DDPM (Denoising Diffusion Probabilistic Models), and it is capable of generating common characters in five different scripts and mimicking the styles of famous calligraphers. Experiments demonstrate that our model can generate calligraphy that is difficult to distinguish from real artworks and that our controls for characters, scripts, and styles are effective. Moreover, we demonstrate one-shot transfer learning, using LoRA (Low-Rank Adaptation) to transfer Chinese calligraphy art styles to unseen characters and even out-of-domain symbols such as English letters and digits.