Text-Transport: Toward Learning Causal Effects of Natural Language
This addresses the need for robust causal inference in natural language processing for real-world applications like social media analysis, though it is incremental as it builds on existing distribution shift methods.
The paper tackles the problem of estimating causal effects of linguistic attributes on reader responses across different text distributions, introducing Text-Transport to transport causal effects between domains without strong assumptions, with empirical results supporting its validity and application to hate speech on social media showing significant shifts in effects.
As language technologies gain prominence in real-world settings, it is important to understand how changes to language affect reader perceptions. This can be formalized as the causal effect of varying a linguistic attribute (e.g., sentiment) on a reader's response to the text. In this paper, we introduce Text-Transport, a method for estimation of causal effects from natural language under any text distribution. Current approaches for valid causal effect estimation require strong assumptions about the data, meaning the data from which one can estimate valid causal effects often is not representative of the actual target domain of interest. To address this issue, we leverage the notion of distribution shift to describe an estimator that transports causal effects between domains, bypassing the need for strong assumptions in the target domain. We derive statistical guarantees on the uncertainty of this estimator, and we report empirical results and analyses that support the validity of Text-Transport across data settings. Finally, we use Text-Transport to study a realistic setting--hate speech on social media--in which causal effects do shift significantly between text domains, demonstrating the necessity of transport when conducting causal inference on natural language.