C-MORAL: Controllable Multi-Objective Molecular Optimization with Reinforcement Alignment for LLMs
This work addresses the challenge of aligning LLMs with multiple competing drug-design constraints, offering a practical post-training solution for molecular optimization.
C-MORAL introduces a reinforcement learning post-training framework for controllable multi-objective molecular optimization, achieving state-of-the-art Success Optimized Rates of 48.9% on in-domain and 39.5% on out-of-domain tasks while preserving scaffold similarity.
Large language models (LLMs) show promise for molecular optimization, but aligning them with selective and competing drug-design constraints remains challenging. We propose C-Moral, a reinforcement learning post-training framework for controllable multi-objective molecular optimization. C-Moral combines group-based relative optimization, property score alignment for heterogeneous objectives, and continuous non-linear reward aggregation to improve stability across competing properties. Experiments on the C-MuMOInstruct benchmark show that C-Moral consistently outperforms state-of-the-art models across both in-domain and out-of-domain settings, achieving the best Success Optimized Rate (SOR) of 48.9% on IND tasks and 39.5% on OOD tasks, while largely preserving scaffold similarity. These results suggest that RL post-training is an effective way to align molecular language models with continuous molecular design objectives. Our code and models are publicly available at https://github.com/Rwigie/C-MORAL.