Creative Procedural-Knowledge Extraction From Web Design Tutorials
This addresses the lack of procedural knowledge extraction in creative fields like design, though it is incremental as it builds on existing text models and highlights their limitations.
The paper tackled the problem of extracting Creative Procedural-Knowledge (CPK) from web design tutorials to enable semi-autonomous design tasks, resulting in a dataset with 819 commands, 47,491 actions, and 2,022 workflows and goals from 19.6K webpages.
Complex design tasks often require performing diverse actions in a specific order. To (semi-)autonomously accomplish these tasks, applications need to understand and learn a wide range of design procedures, i.e., Creative Procedural-Knowledge (CPK). Prior knowledge base construction and mining have not typically addressed the creative fields, such as design and arts. In this paper, we formalize an ontology of CPK using five components: goal, workflow, action, command and usage; and extract components' values from online design tutorials. We scraped 19.6K tutorial-related webpages and built a web application for professional designers to identify and summarize CPK components. The annotated dataset consists of 819 unique commands, 47,491 actions, and 2,022 workflows and goals. Based on this dataset, we propose a general CPK extraction pipeline and demonstrate that existing text classification and sequence-to-sequence models are limited in identifying, predicting and summarizing complex operations described in heterogeneous styles. Through quantitative and qualitative error analysis, we discuss CPK extraction challenges that need to be addressed by future research.