Alex Hu

Ruitao Liu, Qinghao Hu, Alex Hu et al.

Reinforcement learning with verifiable rewards (RLVR) has become a powerful paradigm for improving language models on reasoning-intensive tasks, but its effectiveness is often limited by exploration. For example, models often fail on hard problems, leaving little useful reward signal. External expert traces offer a natural source of guidance, yet they may also expose reward-relevant content along the critical path to the verifier target, such as final answers, intermediate values, executable implementations, or answer-related entities. This content can create an unintended reward hacking channel, allowing the policy to obtain reward by copying the trace rather than learning the underlying reasoning or agentic behavior. Existing guided-RL methods reduce this risk by using partial trajectories, but they mainly control how much expert information is shown heuristically rather than which parts should be hidden. To this end, we propose Semantic Masked Expert Policy Optimization (SMEPO), a fine-grained semantic masking strategy for expert-guided RLVR. Instead of truncating traces coarsely or revealing them unchanged, SMEPO masks reward-relevant semantic spans along the critical path while preserving the expert's decomposition, plan, and procedural structure. This turns hard problems from reasoning from scratch into a fill-in-the-blank process: the policy can follow the expert's problem-solving route, but must still reconstruct the missing values, code, or entities by itself. SMEPO is simple to apply and requires no changes to the reward function or RL objective. Across diverse domains, including math, code, and agentic search, SMEPO improves accuracy by up to 3.2 points over GRPO and reduces training time by up to 4.2x. The code is available at https://github.com/mit-han-lab/SMEPO.

Han Yue, Eric Zhou, Alex Hu et al.

Biomedical question answering (QA) encompasses diverse question types, including yes/no, factoid, list, and summary questions, each requiring distinct forms of evidence and reasoning. However, most retrieval-augmented QA systems rely on a unified retrieval pipeline, regardless of the information needs of different question categories. This one-size-fits-all approach may limit the effectiveness of evidence acquisition and downstream answer generation. In this work, we propose an adaptive retrieval framework that selects retrieval and evidence aggregation strategies according to question type. The system combines query understanding, biomedical document retrieval, reranking, knowledge graph augmentation, document clustering, and large language model-based answer generation. For yes/no questions, it focuses on precise evidence retrieval; for factoid and list questions, it emphasizes entity-oriented retrieval and clustering; and for summary questions, it performs broader evidence collection and synthesis. We evaluate the proposed framework on the BioASQ benchmark and demonstrate that adaptive retrieval strategies improve evidence relevance and answer quality across multiple question types. Our results suggest that aligning retrieval mechanisms with question-specific information needs provides an effective direction for enhancing retrieval-augmented biomedical QA systems.