A Neural, Interactive-predictive System for Multimodal Sequence to Sequence Tasks
This is an incremental improvement for users in tasks requiring text generation, focusing on enhancing human-computer interaction efficiency.
The paper presents a neural interactive-predictive system for multimodal sequence-to-sequence tasks like machine translation and captioning, aiming to reduce human effort by allowing users to correct predictions and the system to react with alternatives.
We present a demonstration of a neural interactive-predictive system for tackling multimodal sequence to sequence tasks. The system generates text predictions to different sequence to sequence tasks: machine translation, image and video captioning. These predictions are revised by a human agent, who introduces corrections in the form of characters. The system reacts to each correction, providing alternative hypotheses, compelling with the feedback provided by the user. The final objective is to reduce the human effort required during this correction process. This system is implemented following a client-server architecture. For accessing the system, we developed a website, which communicates with the neural model, hosted in a local server. From this website, the different tasks can be tackled following the interactive-predictive framework. We open-source all the code developed for building this system. The demonstration in hosted in http://casmacat.prhlt.upv.es/interactive-seq2seq.