Mitigating the Length-Scaling Tax with Online Distillation

Xu Wan, Wenyue Xu, Shengjie Zhao, Mingyang Sun

• Length Self-Distillation (LSD) reduces unnecessary response-length growth on easy queries during language model RL post-training.

Buffer Matters: Unleashing the Power of Off-Policy Reinforcement Learning in Large Language Model Reasoning [Code]

Xu Wan, Yansheng Wang, Wenqi Huang, Mingyang Sun

• BAPO is an off-policy RLVR framework to improve the data efficiency in large language models post-training.

The Shadow Price of Reasoning: Economic Perspective on Optimal Budget Allocation for LLMs [Code]

Xu Wan, SpeedZhu, Jiawei Cai, Guang Chen, Ximing Huang, Wiggin Zhou, Mingyang Sun

• CLEAR implements a Lambert W policy to execute strategic abandonment, sacrificing insolvent tasks to redistribute critical computational resources to solvable complex queries.

AdapThink: Adaptive Thinking Preferences for Reasoning Language Model [Code]

Xu Wan*(co-first author), Wenyue Xu*, Wei Wang, Wotao Yin, Wenqi Huang, Shengjie Zhao, Mingyang Sun

• AdapThink is an adaptive length penalty method for efficient thinking of reasoning language models.

Think Twice, Act Once: A Co-Evolution Framework of LLM and RL for Large-Scale Decision Making

Xu Wan, Wenyue Xu, Chao Yang, Mingyang Sun

• Agents Co-Evolution (ACE) is a synergistic framework between LLMs and RL agents for large-scale decision-making scenarios.

ProOPF: Benchmarking and Improving LLMs for Professional-Grade Power Systems Optimization Modeling [Code]

Chao Shen, Zihan Guo, Xu Wan*(co-first author), Zhenghao Yang, Yifan Zhang, Wengi Huang, Jie Song, Zongyan Zhang, Mingyang Sun

• ProOPF introduces a 12K-instance dataset and a 121-case expert benchmark for evaluating and improving LLMs on professional-grade optimal power flow modeling from natural language.

Fuz-RL: A Fuzzy-Guided Robust Framework for Safe Reinforcement Learning under Uncertainty [Code]

Xu Wan, Chao Yang, Cheng Yang, Jie Song, Mingyang Sun

• Fuz-RL is a novel fuzzy-guided robust framework for safe RL.

SrSv: Integrating Sequential Rollouts with Sequential Value Estimation for Multi-agent Reinforcement Learning [Code]

Xu Wan, Chao Yang, Cheng Yang, Jie Song, Mingyang Sun

• SrSv aims to capture agent interdependence and provide a scalable solution for cooperative MARL.