5.7SEApr 9
Vulnerability Detection with Interprocedural Context in Multiple Languages: Assessing Effectiveness and Cost of Modern LLMsKevin Lira, Baldoino Fonseca, Davy Baía et al.
Large Language Models (LLMs) have been a promising way for automated vulnerability detection. However, most prior studies have explored the use of LLMs to detect vulnerabilities only within single functions, disregarding those related to interprocedural dependencies. These studies overlook vulnerabilities that arise from data and control flows that span multiple functions. Thus, leveraging the context provided by callers and callees may help identify vulnerabilities. This study empirically investigates the effectiveness of detection, the inference cost, and the quality of explanations of four modern LLMs (Claude Haiku 4.5, GPT-4.1 Mini, GPT-5 Mini, and Gemini 3 Flash) in detecting vulnerabilities related to interprocedural dependencies. To do that, we conducted an empirical study on 509 vulnerabilities from the ReposVul dataset, systematically varying the level of interprocedural context (target function code-only, target function + callers, and target function + callees) and evaluating the four modern LLMs across C, C++, and Python. The results show that Gemini 3 Flash offers the best cost-effectiveness trade-off for C vulnerabilities, achieving F1 >= 0.978 at an estimated cost of $0.50-$0.58 per configuration, and Claude Haiku 4.5 correctly identified and explained the vulnerability in 93.6% of the evaluated cases. Overall, the findings have direct implications for the design of AI-assisted security analysis tools that can generalize across codebases in multiple programming languages.
6.3SEMar 19
Where are the Hidden Gems? Applying Transformer Models for Design Discussion DetectionLawrence Arkoh, Daniel Feitosa, Wesley K. G. Assunção
Design decisions are at the core of software engineering and appear in Q\&A forums, mailing lists, pull requests, issue trackers, and commit messages. Design discussions spanning a project's history provide valuable information for informed decision-making, such as refactoring and software modernization. Machine learning techniques have been used to detect design decisions in natural language discussions; however, their effectiveness is limited by the scarcity of labeled data and the high cost of annotation. Prior work adopted cross-domain strategies with traditional classifiers, training on one domain and testing on another. Despite their success, transformer-based models, which often outperform traditional methods, remain largely unexplored in this setting. The goal of this work is to investigate the performance of transformer-based models (i.e., BERT, RoBERTa, XLNet, LaMini-Flan-T5-77M, and ChatGPT-4o-mini) for detecting design-related discussions. To this end, we conduct a conceptual replication of prior cross-domain studies while extending them with modern transformer architectures and addressing methodological issues in earlier work. The models were fine-tuned on Stack Overflow and evaluated on GitHub artifacts (i.e., pull requests, issues, and commits). BERT and RoBERTa show strong recall across domains, while XLNet achieves higher precision but lower recall. ChatGPT-4o-mini yields the highest recall and competitive overall performance, whereas LaMini-Flan-T5-77M provides a lightweight alternative with stronger precision but less balanced performance. We also evaluated similar-word injection for data augmentation, but unlike prior findings, it did not yield meaningful improvements. Overall, these results highlight both the opportunities and trade-offs of using modern language models for detecting design discussion.
9.6SEJun 28
On the JI-RADAR: Uncovering Sustainability Tool Support for Requirements EngineeringMarco Stadler, Pascal Taurer, Johannes Sametinger et al.
Context: Software-intensive systems are integral to nearly all facets of modern society [1]. Consequently, both their sustainability and their role in facilitating sustainable processes must be established by design [2], [3]. Software sustainability is defined as "the preservation of the long-term and beneficial use of software, and its appropriate evolution, in a context that continuously changes" [2]. RE Problem & Motivation: Regulatory initiatives increasingly require (software) organizations to integrate sustainability into their day-to-day business and operational processes. The United Nations 2030 Agenda formulated 17 Sustainable Development Goals (SDGs) [6], while the EU passed the Corporate Sustainability Reporting Directive (CSRD), which requires companies to publish and audit sustainability-related information [7]. Regulations and laws require organizations in the software development sector to disclose both qualitative and quantitative sustainability metrics, among other obligations [1]. Consequently, integrating sustainability reporting processes into the software development life cycle becomes increasingly important. RE processes often lack systematic methods to elicit, analyze, and prioritize sustainability requirements alongside functional and non-functional requirements, and studies indicate that tool support for this integration remains limited [4]. To address this gap, we introduce JI-RADAR, which supports stakeholders involved in system design (e.g., developers, requirements engineers, project managers, and usability engineers) [5] by providing practical tools to integrate sustainability into the RE process. We extend the widely used Atlassian Jira platform [8] by implementing a ready-to-use plugin that can be directly adopted in industrial practice.