LLaVA-Interactive: An All-in-One Demo for Image Chat, Segmentation, Generation and Editing
This work addresses the problem of creating interactive multimodal systems for users needing seamless AI-driven image and text interactions, but it is incremental as it combines existing models without novel training.
The authors tackled the challenge of multimodal human-AI interaction by developing LLaVA-Interactive, a cost-efficient prototype that integrates pre-built models for visual chat, segmentation, generation, and editing, enabling multi-turn dialogues with visual prompts without additional training.
LLaVA-Interactive is a research prototype for multimodal human-AI interaction. The system can have multi-turn dialogues with human users by taking multimodal user inputs and generating multimodal responses. Importantly, LLaVA-Interactive goes beyond language prompt, where visual prompt is enabled to align human intents in the interaction. The development of LLaVA-Interactive is extremely cost-efficient as the system combines three multimodal skills of pre-built AI models without additional model training: visual chat of LLaVA, image segmentation from SEEM, as well as image generation and editing from GLIGEN. A diverse set of application scenarios is presented to demonstrate the promises of LLaVA-Interactive and to inspire future research in multimodal interactive systems.