Persuasion Meets AI: Ethical Considerations for the Design of Social Engineering Countermeasures
This work highlights ethical concerns for designers of AI-based countermeasures targeting social engineering in social networks, focusing on user agency and autonomy.
The paper addresses the ethical challenges of using AI and persuasive nudging to counteract social engineering attacks on social network sites, endorsing personalized risk awareness solutions as an ethical and effective approach to promote reflective privacy decisions.
Privacy in Social Network Sites (SNSs) like Facebook or Instagram is closely related to people's self-disclosure decisions and their ability to foresee the consequences of sharing personal information with large and diverse audiences. Nonetheless, online privacy decisions are often based on spurious risk judgements that make people liable to reveal sensitive data to untrusted recipients and become victims of social engineering attacks. Artificial Intelligence (AI) in combination with persuasive mechanisms like nudging is a promising approach for promoting preventative privacy behaviour among the users of SNSs. Nevertheless, combining behavioural interventions with high levels of personalization can be a potential threat to people's agency and autonomy even when applied to the design of social engineering countermeasures. This paper elaborates on the ethical challenges that nudging mechanisms can introduce to the development of AI-based countermeasures, particularly to those addressing unsafe self-disclosure practices in SNSs. Overall, it endorses the elaboration of personalized risk awareness solutions as i) an ethical approach to counteract social engineering, and ii) as an effective means for promoting reflective privacy decisions.