论文arXiv 2611.07679今天

PhySA解耦人形运动美学与物理奖励，MuJoCo关节加速度降32%但难迁移

研究揭示VLM美学评分与人形机器人物理平滑度呈正相关（r=0.66），提出PhySA框架解耦双通道奖励。在MuJoCo Humanoid-v5中关节加速度降低26%-32%，但在IsaacLab及Unitree G1真机验证中未能复现该收益。

AI 整理

仿真与训练
具身模型
评测与安全

宇树 G1

本文由 AI 根据公开内容整理，可能有误，请以原文为准：PhySA: Decoupling Aesthetic and Physical Rewards for Humanoid Locomotion（arXiv 2611.07679）

核心内容

针对人形机器人步态训练缺乏显式平滑目标导致动作抖动问题，arXiv论文提出PhySA方法。研究发现四款主流VLM对高关节加速度片段给出更高美学评分（Pearson r=+0.66），证实视觉偏好与物理约束冲突。PhySA通过分离异步VLM美学奖励与基于仿真状态的相位触发物理修正通道解决此矛盾。在MuJoCo Humanoid-v5基准测试2.6M步数下，相比PPO、Text2Reward及Eureka，PhySA使关节加速度降低26%至32%。

技术实现上，PhySA将稀疏VLM评分与物理惩罚权重调整解耦，消融实验确认平滑增益源于物理修正通道。尽管单次长时运行较Prompt2Policy降低59.5%加速度，但三种子复现仅获7.2%非显著降幅，显示稳定性不足。更关键的是，该算法增益无法迁移至IsaacLab仿真环境及Unitree G1实体机器人，即使重新缩放平滑项参数仍失效，表明纯视觉美学反馈难以直接作为通用具身控制信号。

关键看点

VLM审美偏差：四款生产级VLM对高关节加速度视频打分更高，平均Pearson相关系数达+0.66，证明美学偏好与物理平滑性背离
双通道解耦架构：PhySA独立设置异步VLM美学评分与相位触发物理修正通道，消融实验证实平滑度提升完全来自物理修正模块
MuJoCo性能边界：在Humanoid-v5任务2.6M步训练中，相较PPO/Eureka等基线降低26%-32%关节加速度，但多籽复现结果波动大
跨域迁移失效：算法收益未能泛化至IsaacLab仿真器或Unitree G1真机平台，重调平滑权重参数后依然无效，限制工程落地价值

工程与产业观察

该研究量化揭示了VLM作为人形运动奖励函数的内在缺陷：视觉美学与动力学平滑存在系统性错位。虽然PhySA在特定仿真器验证了解耦设计的有效性，但其向IsaacLab及Unitree G1迁移失败警示业界，当前VLM反馈尚不足以支撑Sim-to-Real闭环。工程实践中应谨慎使用纯视觉美学指标优化底层控制，需结合本体感知数据构建对齐物理规律的混合奖励体系。

英文摘要 (Abstract)

Humanoid locomotion policies trained without an explicit smoothness objective tend to produce jerky, high-energy gaits that stress actuators. Vision-language models (VLMs) offer an appealing reward source: given a rendered rollout, a VLM returns an aesthetic quality score on [0,10] without task-specific feature engineering. We find, however, that aesthetic preference and physical smoothness point in opposite directions: across four production VLMs, clips with higher joint acceleration receive higher aesthetic scores (mean Pearson r=+0.66). We study PhySA (Physical Smoothness and Aesthetics), which keeps the two on separate, independently removable reward channels: a sparse, asynchronous VLM score and a phase-triggered correction channel that adjusts physical penalty weights from simulator state. Ablations attribute the smoothness gain to the correction channel. On MuJoCo Humanoid-v5 at 2.6M steps, PhySA lowers joint acceleration by 26 to 32% relative to vanilla PPO, Text2Reward, and Eureka (three seeds), at the cost of shorter episodes. In a single extended run it lowers joint acceleration by 59.5% relative to Prompt2Policy, while a three-seed replication shows a smaller, non-significant reduction (7.2%). The gain does not transfer to IsaacLab or the Unitree G1, even after rescaling the smoothness term.

作者：Yunwan Fu, Sicheng He, Jiageng Mao, Yue Wang