Beyond Text: Characterizing Domain Expert Needs in Document Research
This work addresses the gap between NLP systems and expert needs in document research, highlighting the importance of accessibility and social awareness for building useful tools.
The study interviewed sixteen domain experts to analyze their document research processes, revealing that these processes are idiosyncratic, iterative, and heavily reliant on social context, and found that existing NLP approaches focusing on documents as objects better reflect expert priorities but are often less accessible.
Working with documents is a key part of almost any knowledge work, from contextualizing research in a literature review to reviewing legal precedent. Recently, as their capabilities have expanded, primarily text-based NLP systems have often been billed as able to assist or even automate this kind of work. But to what extent are these systems able to model these tasks as experts conceptualize and perform them now? In this study, we interview sixteen domain experts across two domains to understand their processes of document research, and compare it to the current state of NLP systems. We find that our participants processes are idiosyncratic, iterative, and rely extensively on the social context of a document in addition its content; existing approaches in NLP and adjacent fields that explicitly center the document as an object, rather than as merely a container for text, tend to better reflect our participants' priorities, though they are often less accessible outside their research communities. We call on the NLP community to more carefully consider the role of the document in building useful tools that are accessible, personalizable, iterative, and socially aware.