RichSpace: Enriching Text-to-Video Prompt Space via Text Embedding Interpolation
This addresses a specific bottleneck in text-to-video generation for AI content creation, though it appears incremental as it builds on existing embedding and interpolation techniques.
The paper tackles the problem of text-to-video generation models struggling with complex features due to inaccurate text embeddings by proposing an interpolation method in the embedding space to select optimal embeddings, enabling the generation of desired videos.
Text-to-video generation models have made impressive progress, but they still struggle with generating videos with complex features. This limitation often arises from the inability of the text encoder to produce accurate embeddings, which hinders the video generation model. In this work, we propose a novel approach to overcome this challenge by selecting the optimal text embedding through interpolation in the embedding space. We demonstrate that this method enables the video generation model to produce the desired videos. Additionally, we introduce a simple algorithm using perpendicular foot embeddings and cosine similarity to identify the optimal interpolation embedding. Our findings highlight the importance of accurate text embeddings and offer a pathway for improving text-to-video generation performance.