19.4AIAug 1
DeepSurvey-Bench: Evaluating Academic Value of Automatically Generated Scientific SurveysGuo-Biao Zhang, Xian-Ling Mao, Ding-Yuan Liu et al.
The rapid development of automated survey generation technology has made it increasingly important to establish a comprehensive benchmark to evaluate the quality of generated surveys. Most existing benchmarks first construct ground-truth datasets by selecting human-written surveys based on limited selection criteria, such as citation counts and structural coherence, and evaluate generated surveys primarily based on conventional quality dimensions, including structural quality and reference relevance. However, these benchmarks have two key issues: (1) the datasets are insufficiently reliable because the selection criteria only identify highly cited or structurally coherent surveys without verifying their academic value; (2) the evaluation metrics mainly reflect the surface-level quality of generated surveys and are insufficient to assess their academic value. Together, these issues prevent existing benchmarks from effectively assessing the academic value of generated surveys. To address the above problems, we propose DeepSurvey-Bench, a comprehensive benchmark for evaluating the academic value of automatically generated surveys. Specifically, our proposed benchmark introduces a set of academic value evaluation criteria covering three dimensions: informational value, scholarly communication value, and research guidance value. We first construct a reliable dataset with academic value annotations based on these criteria, and then evaluate the academic value of generated surveys according to these criteria through a multi-LLM-as-a-judge approach. Extensive experiments demonstrate that DeepSurvey-Bench not only aligns closely with human assessments in evaluating the academic value of surveys, but also reveals underlying academic value beyond the reach of surface-level quality metrics, providing a foundation for fine-grained diagnosis and iterative improvement of generated surveys.
18.6CLAug 1
A Heuristic Perspective on Debiasing Language ModelsTian Lan, Yemin Wang, Chuancheng Shi et al.
Language models (LMs) often acquire various biases during pre-training and may express them in interactions, potentially causing social harm. Existing methods often rely on counterfactual augmentation or representation projection. These strategies remain limited in practice due to their high computational costs and difficulty in scaling to larger models. Additionally, many of these strategies require manual data annotation, narrowing their scope to specific cultures and bias categories. To overcome these limitations, we propose HEIMAT, a HEurIstic-style autoMATic debiasing framework for LMs. HEIMAT consists of two main steps: bias disclosure and debiasing fine-tuning. In the first step, it uses simple templates to construct heuristic prompts, which are applied to reveal model biases and generate corresponding context prompts. In the second step, it fine-tunes the model by minimizing the Jensen-Shannon divergence of predictions on these context prompts to reduce bias. Extensive experiments show that HEIMAT effectively mitigates bias in different cultures while maintaining the model's natural language understanding (NLU) performance.
2.2NIAug 2
Learning Not to Optimize: Physics-Informed Action-Space Reshaping for Intent-Based Network ControlZuyuan Zhang, Vaneet Aggarwal, Tian Lan
Modern network policy control maps intent to sequential placement-control decisions. Bellman-style policy optimization primarily asks which action to optimize, while constraints are commonly handled through penalty, barrier, or Lagrangian mechanisms. We observe that before a value function can certify the best deployment, intermediate signals may already identify many candidates that should be excluded from further optimization. This motivates a complementary direction: \emph{Learning Not to Optimize}. Before a value function is accurate enough to select the best placement-control decision, intermediate signals may already show that candidates are equivalent under state--intent relabeling (quotienting), lead to a uniformly worse future state (dominance), or violate executable network laws (residual screening). \LNOQRD{} uses these computed or learned signals as a shadow process to reshape the domain on which primal policy optimization is performed, thereby reducing the action space. We prove lossless quotienting and dominance under explicit equivariance and monotonicity conditions, bound frontier size and ranking cost, and quantify losses from approximate certificates and primal estimates. Experiments show that \LNOQRD{} reduces small-instance candidates by $75.9\%$ while retaining $90.8\%$ near-oracle coverage and, on large instances, achieves the highest utility and intent satisfaction, the lowest hard-law violation and post-generation latency, and a $73.0\%$ average reduction among candidate-based baselines.