Knowledge-Centric Templatic Views of Documents
This work addresses the issue of redundant and disjointed document generation for authors and content creators, offering a more efficient and adaptable solution, though it is incremental in building on existing LLM capabilities.
The paper tackles the problem of fragmented document generation across formats like slide decks and reports by proposing a unified approach that treats them as templatic views of underlying knowledge, showing that LLMs can generate these formats with minimal supervision and that a structured intermediate representation improves performance, with human evaluation preferring 82% of documents generated by their method.
Authors seeking to communicate with broader audiences often share their ideas in various document formats, such as slide decks, newsletters, reports, and posters. Prior work on document generation has generally tackled the creation of each separate format to be a different task, leading to fragmented learning processes, redundancy in models and methods, and disjointed evaluation. We consider each of these documents as templatic views of the same underlying knowledge/content, and we aim to unify the generation and evaluation of these templatic views. We begin by showing that current LLMs are capable of generating various document formats with little to no supervision. Further, a simple augmentation involving a structured intermediate representation can improve performance, especially for smaller models. We then introduce a novel unified evaluation framework that can be adapted to measuring the quality of document generators for heterogeneous downstream applications. This evaluation is adaptable to a range of user defined criteria and application scenarios, obviating the need for task specific evaluation metrics. Finally, we conduct a human evaluation, which shows that people prefer 82% of the documents generated with our method, while correlating more highly with our unified evaluation framework than prior metrics in the literature.