Chat-crowd: A Dialog-based Platform for Visual Layout Composition
This addresses the need for building multimodal goal-oriented dialog tasks requiring spatial reasoning, but it is incremental as it focuses on platform development rather than novel AI methods.
The paper tackles the problem of visual layout composition by introducing Chat-crowd, an interactive dialog-based platform that enables multiple agents to collaboratively place objects on a canvas, resulting in a system designed for data collection with quality controls.
In this paper we introduce Chat-crowd, an interactive environment for visual layout composition via conversational interactions. Chat-crowd supports multiple agents with two conversational roles: agents who play the role of a designer are in charge of placing objects in an editable canvas according to instructions or commands issued by agents with a director role. The system can be integrated with crowdsourcing platforms for both synchronous and asynchronous data collection and is equipped with comprehensive quality controls on the performance of both types of agents. We expect that this system will be useful to build multimodal goal-oriented dialog tasks that require spatial and geometric reasoning.