Yanfang Hou

Yanfang Hou, Peter van der Putten, Suzan Verberne

During the COVID-19 pandemic, large amounts of COVID-19 misinformation are spreading on social media. We are interested in the stance of Twitter users towards COVID-19 misinformation. However, due to the relative recent nature of the pandemic, only a few stance detection datasets fit our task. We have constructed a new stance dataset consisting of 2631 tweets annotated with the stance towards COVID-19 misinformation. In contexts with limited labeled data, we fine-tune our models by leveraging the MNLI dataset and two existing stance detection datasets (RumourEval and COVIDLies), and evaluate the model performance on our dataset. Our experimental results show that the model performs the best when fine-tuned sequentially on the MNLI dataset and the combination of the undersampled RumourEval and COVIDLies datasets. Our code and dataset are publicly available at https://github.com/yanfangh/covid-rumor-stance

Yanfang Hou, Carlos Fernández-Loría

Causal forests estimate how treatment effects vary across individuals, guiding personalized interventions in areas like marketing, operations, and public policy. A standard modeling practice with this method is honest estimation: dividing the data so that the subgroups used to model treatment effect variation are formed separately from the data used to estimate those effects. This is intended to reduce overfitting and is the default in many software packages. But is it always the right choice? In this paper, we show that honest estimation can reduce the accuracy of individual-level treatment effect estimates, especially when there are substantial differences in how individuals respond to treatment, and the data is rich enough to uncover those differences. The core issue is a classic bias-variance trade-off: honesty lowers the risk of overfitting but increases the risk of underfitting, because it limits the data available to detect patterns. Across 7,500 benchmark datasets, we find that the cost of using honesty by default can be as high as requiring 75% more data to match the performance of models trained without it. We argue that honesty is best understood as a form of regularization, and like any regularization choice, its use should be guided by out-of-sample performance, not adopted reflexively.