Simeon Tverdal

Arda Goknil, Femke B. Gelderblom, Simeon Tverdal et al.

Privacy policies are often obfuscated by their complexity, which impedes transparency and informed consent. Conventional machine learning approaches for automatically analyzing these policies demand significant resources and substantial domain-specific training, causing adaptability issues. Moreover, they depend on extensive datasets that may require regular maintenance due to changing privacy concerns. In this paper, we propose, apply, and assess PAPEL (Privacy Policy Analysis through Prompt Engineering for LLMs), a framework harnessing the power of Large Language Models (LLMs) through prompt engineering to automate the analysis of privacy policies. PAPEL aims to streamline the extraction, annotation, and summarization of information from these policies, enhancing their accessibility and comprehensibility without requiring additional model training. By integrating zero-shot, one-shot, and few-shot learning approaches and the chain-of-thought prompting in creating predefined prompts and prompt templates, PAPEL guides LLMs to efficiently dissect, interpret, and synthesize the critical aspects of privacy policies into user-friendly summaries. We demonstrate the effectiveness of PAPEL with two applications: (i) annotation and (ii) contradiction analysis. We assess the ability of several LLaMa and GPT models to identify and articulate data handling practices, offering insights comparable to existing automated analysis approaches while reducing training efforts and increasing the adaptability to new analytical needs. The experiments demonstrate that the LLMs PAPEL utilizes (LLaMA and Chat GPT models) achieve robust performance in privacy policy annotation, with F1 scores reaching 0.8 and above (using the OPP-115 gold standard), underscoring the effectiveness of simpler prompts across various advanced language models.

Simeon Tverdal, Phu Nguyen, Arda Goknil et al.

Software-intensive organizations need effective ways to identify where maintenance and refactoring efforts will yield the greatest practical benefit. Although software analytics such as code health, hotspots, and coupling provide valuable signals, they do not always capture the experienced burden of change that slows software evolution in practice. This paper presents a multi-case industrial study of technical debt friction as a prioritization-oriented concept for identifying where technical debt most strongly affects maintenance and evolution. We investigate how practitioners interpret the concept, whether friction-related analysis aligns with perceived maintenance pain points and refactoring needs, and what broader maintenance and evolution insights friction can provide beyond individual refactoring candidates. To this end, we conducted structured walkthrough sessions with practitioners across multiple industrial cases using analysis artifacts including code health, hotspots, coupling, refactoring targets, and socio-technical views. Our findings show that practitioners generally considered technical debt friction useful for reasoning about maintenance burden, especially when interpreted together with complementary technical and socio-technical views. At the file level, friction often aligned with known problematic areas and, in several cases, with files that later received maintenance attention, although its practical relevance depended strongly on context. In addition, our exploratory project-level analysis suggests that friction distributions may reveal broader maintenance and evolution patterns. These results indicate that technical debt friction is promising as a decision-support concept, but most effective when used with contextual knowledge and supporting evidence.