14.9SEJul 12, 2025Code
SPICE: An Automated SWE-Bench Labeling Pipeline for Issue Clarity, Test Coverage, and Effort EstimationGustavo A. Oliva, Gopi Krishnan Rajbahadur, Aaditya Bhatia et al.
High-quality labeled datasets are crucial for training and evaluating foundation models in software engineering, but creating them is often prohibitively expensive and labor-intensive. We introduce SPICE, a scalable, automated pipeline for labeling SWE-bench-style datasets with annotations for issue clarity, test coverage, and effort estimation. SPICE combines context-aware code navigation, rationale-driven prompting, and multi-pass consensus to produce labels that closely approximate expert annotations. SPICE's design was informed by our own experience and frustration in labeling more than 800 instances from SWE-Gym. SPICE achieves strong agreement with human-labeled SWE-bench Verified data while reducing the cost of labeling 1,000 instances from around \$100,000 (manual annotation) to just \$5.10. These results demonstrate SPICE's potential to enable cost-effective, large-scale dataset creation for SE-focused FMs. To support the community, we release both SPICE tool and SPICE Bench, a new dataset of 6,802 SPICE-labeled instances curated from 291 open-source projects in SWE-Gym (over 13x larger than SWE-bench Verified).
21.9AIJan 16, 2025
A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation StrategyHuandong Wang, Wenjie Fu, Yingzhou Tang et al.
While large language models (LLMs) present significant potential for supporting numerous real-world applications and delivering positive social impacts, they still face significant challenges in terms of the inherent risk of privacy leakage, hallucinated outputs, and value misalignment, and can be maliciously used for generating toxic content and unethical purposes after been jailbroken. Therefore, in this survey, we present a comprehensive review of recent advancements aimed at mitigating these issues, organized across the four phases of LLM development and usage: data collecting and pre-training, fine-tuning and alignment, prompting and reasoning, and post-processing and auditing. We elaborate on the recent advances for enhancing the performance of LLMs in terms of privacy protection, hallucination reduction, value alignment, toxicity elimination, and jailbreak defenses. In contrast to previous surveys that focus on a single dimension of responsible LLMs, this survey presents a unified framework that encompasses these diverse dimensions, providing a comprehensive view of enhancing LLMs to better serve real-world applications.
16.0LGFeb 19, 2021
Analytics and Machine Learning in Vehicle Routing ResearchRuibin Bai, Xinan Chen, Zhi-Long Chen et al.
The Vehicle Routing Problem (VRP) is one of the most intensively studied combinatorial optimisation problems for which numerous models and algorithms have been proposed. To tackle the complexities, uncertainties and dynamics involved in real-world VRP applications, Machine Learning (ML) methods have been used in combination with analytical approaches to enhance problem formulations and algorithmic performance across different problem solving scenarios. However, the relevant papers are scattered in several traditional research fields with very different, sometimes confusing, terminologies. This paper presents a first, comprehensive review of hybrid methods that combine analytical techniques with ML tools in addressing VRP problems. Specifically, we review the emerging research streams on ML-assisted VRP modelling and ML-assisted VRP optimisation. We conclude that ML can be beneficial in enhancing VRP modelling, and improving the performance of algorithms for both online and offline VRP optimisations. Finally, challenges and future opportunities of VRP research are discussed.
7.3SIAug 15, 2019
When Your Friends Become Sellers: An Empirical Study of Social Commerce Site BeidianHancheng Cao, Zhilong Chen, Fengli Xu et al.
Past few years have witnessed the emergence and phenomenal success of strong-tie based social commerce. Embedded in social networking sites, these E-Commerce platforms transform ordinary people into sellers, where they advertise and sell products to their friends and family in online social networks. These sites can acquire millions of users within a short time, and are growing fast at an accelerated rate. However, little is known about how these social commerce develop as a blend of social relationship and economic transactions. In this paper we present the first measurement study on the full-scale data of Beidian, one of the fastest growing social commerce sites in China, which involves 11.8 million users. We first analyzed the topological structure of the Beidian platform and highlighted its decentralized nature. We then studied the site's rapid growth and its growth mechanism via invitation cascade. Finally, we investigated purchasing behavior on Beidian, where we focused on user proximity and loyalty, which contributes to the site's high conversion rate. As the consequences of interactions between strong ties and economic logics, emerging social commerce demonstrates significant property deviations from all known social networks and E-Commerce in terms of network structure, dynamics and user behavior. To the best of our knowledge, this work is the first quantitative study on the network characteristics and dynamics of emerging social commerce platforms.