Shenao Zhang

I am a first-year Ph.D. student at Northwestern University, advised by Prof. Zhaoran Wang. I was a Student Researcher at Microsoft GenAI working with Donghan Yu. I received my M.S. degree from Georgia Tech, where I was fortunate to work with Prof. Tuo Zhao and Prof. Bo Dai. I obtained my Bachelor's degree from South China University of Technology. In the spring of 2019, I visited UC Berkeley EECS. Previously, I was a research intern at ByteDance AML, Microsoft Research, and Tencent AI Lab.

CV  /  Email  /  Google Scholar  /  LinkedIn  /  Twitter

profile photo

Research

My research centers around Large Language Models (LLMs) and Reinforcement Learning (RL). I’m currently interested in aligning LLMs with RLHF, studying active synthetic data generation and reward overoptimization. Previously, I built autonomous LLM agents and developed data-efficient decision-making algorithms with applications to robotic and multi-agent systems.


(* indicates equal contribution, indicates equal advising)

project image

Self-Exploring Language Models: Active Preference Elicitation for Online Alignment


Shenao Zhang, Donghan Yu, Hiteshi Sharma, Ziyi Yang, Shuohang Wang, Hany Hassan, Zhaoran Wang
Preprint, 2024
paper / code / models / tweet /

project image

Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizer


Zhihan Liu*, Miao Lu*, Shenao Zhang, Boyi Liu, Hongyi Guo, Yingxiang Yang, Jose Blanchet, Zhaoran Wang
Preprint, 2024
paper /

project image

Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency


Zhihan Liu*, Hao Hu*, Shenao Zhang*, Hongyi Guo, Shuqi Ke, Boyi Liu, Zhaoran Wang
International Conference on Machine Learning (ICML), 2024
paper / code / website / tweet /

project image

Adaptive-Gradient Policy Optimization: Enhancing Policy Learning in Non-Smooth Differentiable Simulations


Feng Gao*, Liangzhi Shi*, Shenao Zhang, Zhaoran Wang, Yi Wu
International Conference on Machine Learning (ICML), 2024
paper /

project image

Model-Based Reparameterization Policy Gradient Methods: Theory and Practical Algorithms


Shenao Zhang, Boyi Liu, Zhaoran Wang, Tuo Zhao
Neural Information Processing Systems (NeurIPS), 2023
paper / code /

project image

Maximize to Explore: One Objective Function Fusing Estimation, Planning, and Exploration


Zhihan Liu*, Miao Lu*, Wei Xiong*, Han Zhong, Hao Hu, Shenao Zhang, Sirui Zheng, Zhuoran Yang, Zhaoran Wang
Neural Information Processing Systems (NeurIPS), 2023 (Spotlight)
paper / code /

project image

Adaptive Barrier Smoothing for First-Order Policy Gradient with Contact Dynamics


Shenao Zhang, Wanxin Jin, Zhaoran Wang
International Conference on Machine Learning (ICML), 2023
paper / website /

project image

Conservative Dual Policy Optimization for Efficient Model-Based Reinforcement Learning


Shenao Zhang
Neural Information Processing Systems (NeurIPS), 2022
paper / video / website / poster / video /

project image

Learning Meta Representation for Agents in Multi-Agent Reinforcement Learning


Shenao Zhang, Li Shen, Lei Han, Li Shen
Conference on Lifelong Learning Agents (CoLLAs), 2023 (Oral)
paper / poster /

project image

How Can LLM Guide RL? A Value-Based Approach


Shenao Zhang*, Sirui Zheng*, Shuqi Ke, Zhihan Liu, Wanxin Jin, Jianbo Yuan, Yingxiang Yang, Hongxia Yang, Zhaoran Wang
Preprint, 2023
paper / code /

project image

Asking Before Action: Gather Information in Embodied Decision Making with Language Models


Xiaoyu Chen, Shenao Zhang, Pushi Zhang, Li Zhao, Jianyu Chen
Preprint, 2023
paper /

project image

Structure-Regularized Attention for Deformable Object Representation


Shenao Zhang, Li Shen, Zhifeng Li, Wei Liu
NeurIPS Workshop on Object Representations for Learning and Reasoning, 2020
paper / code / website / poster /

Professional Service

Conference Review: NeurIPS 2020-24, ICLR 2022-24, ICML 2022-24, AISTATS 2022-24, RSS 2021.

Journal Review: Neurocomputing, IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI).

Teaching

Graduate Teaching Assistant: Head TA of CS 7648: Interactive Robot Learning (Fall 2021) at Georgia Tech.




Source code from Jon Barron's website