Alvaro Velasquez

Yongliang Sun, Ismail Alkhouri, Cheng-Han Huang et al.

Recent studies suggest that gradient-based methods applied to relaxed box-constrained Quadratic Unconstrained Binary Optimization (QUBO) formulations can outperform classical heuristics in some large-scale regimes, often relying on heavy parallelization. However, these methods still underperform heuristics in other settings. In this work, we clarify this apparent discrepancy through a detailed analysis of the relaxed non-convex QUBO local maxima for both the Maximum Independent Set (MIS) and Maximum Cut (MaxCut) problems, and by introducing a new quadratic objective for MaxCut. Motivated by this analysis, we propose a mutation-based differentiable global reset algorithm, combined with local search to escape local maxima. We term our approach mQO, standing for mutation-based Quadratic combinatorial Optimization. The proposed strategy dramatically improves the performance of gradient-based solvers without heavy reliance on GPU parallelized initializations, indicating that stalling, rather than model capacity or compute, is the dominant bottleneck. As a result, on large-scale graphs, mQO achieves superior performance against state-of-the-art heuristics, commercial integer programming solvers, and recent GPU methods.

Patrick Cooper, Alireza Nadali, Ashutosh Trivedi et al.

Large language models (LLMs) are known to exhibit brittle behavior under adversarial prompts and jailbreak attacks, even after extensive alignment and fine-tuning. This fragility reflects a broader challenge of modern neural language models: small, carefully structured perturbations in high-dimensional input spaces can induce large and unpredictable changes in internal semantic representations and output. We investigate monotonicity as an architectural inductive bias for improving the robustness of Transformer-based language models. Monotonicity constrains semantic transformations so that strengthening information, evidence, or constraints cannot lead to regressions in the corresponding internal representations. Such order-preserving behavior has long been exploited in control and safety-critical systems to simplify reasoning and improve robustness, but has traditionally been viewed as incompatible with the expressivity required by neural language models. We show that this trade-off is not inherent. By enforcing monotonicity selectively in the feed-forward sublayers of sequence-to-sequence Transformers -- while leaving attention mechanisms unconstrained -- we obtain monotone language models that preserve the performance of their pretrained counterparts. This architectural separation allows negation, contradiction, and contextual interactions to be introduced explicitly through attention, while ensuring that subsequent semantic refinement is order-preserving. Empirically, monotonicity substantially improves robustness: adversarial attack success rates drop from approximately 69% to 19%, while standard summarization performance degrades only marginally.