Yang Lü

Subin An, Yugyeong Ji, Junyoung Kim et al.

Open-ended survey responses provide valuable insights in marketing research, but low-quality responses not only burden researchers with manual filtering but also risk leading to misleading conclusions, underscoring the need for effective evaluation. Existing automatic evaluation methods target LLM-generated text and inadequately assess human-written responses with their distinct characteristics. To address such characteristics, we propose a two-stage evaluation framework specifically designed for human survey responses. First, gibberish filtering removes nonsensical responses. Then, three dimensions-effort, relevance, and completeness-are evaluated using LLM capabilities, grounded in empirical analysis of real-world survey data. Validation on English and Korean datasets shows that our framework not only outperforms existing metrics but also demonstrates high practical applicability for real-world applications such as response quality prediction and response rejection, showing strong correlations with expert assessment.

Yang Lu, Shujun Li, Athina Ioannou et al.

Although there are privacy-enhancing tools designed to protect users' online privacy, it is surprising to see a lack of user-centric solutions allowing privacy control based on the joint assessment of privacy risks and benefits, due to data disclosure to multiple platforms. In this paper, we propose a conceptual framework to fill the gap: aiming at the user-centric privacy protection, we show the framework can not only assess privacy risks in using online services but also the added values earned from data disclosure. Through following a human-in-the-loop approach, it is expected the framework provides a personalized solution via preference learning, continuous privacy assessment, behavior monitoring and nudging. Finally, we describe a case study towards "leisure travelers" and several future areas to be studied in the ongoing project.