Attract me to Buy: Advertisement Copywriting Generation with Multimodal Multi-structured Information
This work addresses the lack of datasets and evaluation frameworks for multimodal advertisement copywriting generation in e-commerce, though it appears incremental as it builds on existing multimodal text generation approaches.
The authors tackled the problem of generating advertisement copywriting by creating a new dataset, E-MMAD, and a baseline method with a faithfulness evaluation metric, which significantly outperformed previous methods on all metrics.
Recently, online shopping has gradually become a common way of shopping for people all over the world. Wonderful merchandise advertisements often attract more people to buy. These advertisements properly integrate multimodal multi-structured information of commodities, such as visual spatial information and fine-grained structure information. However, traditional multimodal text generation focuses on the conventional description of what existed and happened, which does not match the requirement of advertisement copywriting in the real world. Because advertisement copywriting has a vivid language style and higher requirements of faithfulness. Unfortunately, there is a lack of reusable evaluation frameworks and a scarcity of datasets. Therefore, we present a dataset, E-MMAD (e-commercial multimodal multi-structured advertisement copywriting), which requires, and supports much more detailed information in text generation. Noticeably, it is one of the largest video captioning datasets in this field. Accordingly, we propose a baseline method and faithfulness evaluation metric on the strength of structured information reasoning to solve the demand in reality on this dataset. It surpasses the previous methods by a large margin on all metrics. The dataset and method are coming soon on \url{https://e-mmad.github.io/e-mmad.net/index.html}.