DOM-LM: Learning Generalizable Representations for HTML Documents
This addresses the challenge of effective HTML document representation for applications like Question Answering and Web Search, offering a novel method that improves over existing approaches by combining text and structure, though it is incremental in building on transformer-based techniques.
The paper tackles the problem of representing HTML documents for machine understanding by introducing DOM-LM, which encodes both text and DOM tree structure using a transformer-based encoder and self-supervised pre-training. The result shows that DOM-LM consistently outperforms baselines on tasks like Attribute Extraction, Open Information Extraction, and Question Answering, with better generalization in few-shot and zero-shot settings.
HTML documents are an important medium for disseminating information on the Web for human consumption. An HTML document presents information in multiple text formats including unstructured text, structured key-value pairs, and tables. Effective representation of these documents is essential for machine understanding to enable a wide range of applications, such as Question Answering, Web Search, and Personalization. Existing work has either represented these documents using visual features extracted by rendering them in a browser, which is typically computationally expensive, or has simply treated them as plain text documents, thereby failing to capture useful information presented in their HTML structure. We argue that the text and HTML structure together convey important semantics of the content and therefore warrant a special treatment for their representation learning. In this paper, we introduce a novel representation learning approach for web pages, dubbed DOM-LM, which addresses the limitations of existing approaches by encoding both text and DOM tree structure with a transformer-based encoder and learning generalizable representations for HTML documents via self-supervised pre-training. We evaluate DOM-LM on a variety of webpage understanding tasks, including Attribute Extraction, Open Information Extraction, and Question Answering. Our extensive experiments show that DOM-LM consistently outperforms all baselines designed for these tasks. In particular, DOM-LM demonstrates better generalization performance both in few-shot and zero-shot settings, making it attractive for making it suitable for real-world application settings with limited labeled data.