PRISMe: A Novel LLM-Powered Tool for Interactive Privacy Policy Assessment
This addresses the challenge of online privacy policy comprehension for general users, but it is incremental as it builds on existing LLM technology for a specific application.
The researchers tackled the problem of users struggling to comprehend lengthy and complex website privacy policies by developing PRISMe, an LLM-powered browser extension tool that provides quick overviews and interactive chats for assessment. In a user study with 22 participants, the tool improved privacy awareness but faced issues with consistency and trust.
Protecting online privacy requires users to engage with and comprehend website privacy policies, but many policies are difficult and tedious to read. We present PRISMe (Privacy Risk Information Scanner for Me), a novel Large Language Model (LLM)-driven privacy policy assessment tool, which helps users to understand the essence of a lengthy, complex privacy policy while browsing. The tool, a browser extension, integrates a dashboard and an LLM chat. One major contribution is the first rigorous evaluation of such a tool. In a mixed-methods user study (N=22), we evaluate PRISMe's efficiency, usability, understandability of the provided information, and impacts on awareness. While our tool improves privacy awareness by providing a comprehensible quick overview and a quality chat for in-depth discussion, users note issues with consistency and building trust in the tool. From our insights, we derive important design implications to guide future policy analysis tools.