外观
论文arXiv 2610.2115213 天前
KODA：用于人形机器人从人类第一视角演示中学习的运动学-观测数据对齐
KODA将第一视角人类RGB演示转换为全身VLA训练数据，并在Unitree G1四个移动操作任务和两种控制器上验证。
宇树 G1原文链接：KODA: Kinematics--Observation Data Alignment for Humanoid Learning from Human Egocentric Demonstration（arXiv 2610.21152 · 发布日期：2026-10-15）
KODA: Kinematics--Observation Data Alignment for Humanoid Learning from Human Egocentric Demonstration
KODA将第一视角人类RGB演示转换为全身VLA训练数据，并在Unitree G1四个移动操作任务和两种控制器上验证。
核心要点
- 全身VLA策略训练依赖昂贵且对齐的机器人演示，人类第一视角演示存在视角错位和运动线索稀疏。
- KODA用可复用规范轨迹引导机器人视角合成，并生成特定控制器的全身状态-动作监督。
- 其先生成经仿真器检查的全身轨迹，再由该轨迹确定机载相机位姿并形成规范记录。
- 在固定机器人数据时，未被机器人数据覆盖条件的宏平均成功率分别提升45.0和46.25个百分点。
验证范围与局限：验证范围为四个Unitree G1移动操作任务，以及SONIC和HEFT两种全身控制器。
英文摘要 (Abstract)
Training humanoid whole-body vision-language-action (VLA) policies requires costly robot demonstrations with aligned robot-view observations and controller-compatible whole-body state-action labels. Egocentric human demonstrations offer scalable supervision but have misaligned views and sparse motion cues. Existing approaches either pretrain policies that still require robot data for adaptation or generate controller-specific commands that limit reuse. Conversion also requires robot-view observations aligned with the generated whole-body motion. We present KODA (Kinematics-Observation Data Alignment), which converts egocentric RGB demonstrations into whole-body VLA training data using a reusable canonical trajectory to guide robot-view synthesis and generate controller-specific whole-body state-action supervision. KODA first generates a simulator-checked whole-body trajectory using a sparse tracker with sampling-based command optimization. This generated trajectory directly determines the onboard camera poses used to synthesize corresponding robot-view observations. The motion, observations, and task instruction form a canonical record. For each deployment controller, KODA optimizes commands to track the stored motion and records the corresponding action labels and proprioceptive states. When switching controllers, KODA retains this canonical record and regenerates whole-body state-action supervision for co-training with matching robot demonstrations. We evaluate KODA on four Unitree G1 loco-manipulation tasks using two whole-body controllers, SONIC and HEFT. With robot data held fixed, macro-average success on conditions covered by human demonstrations but absent from robot data improves by 45.0 and 46.25 percentage points, respectively. Under both controllers, this metric shows an upward trend across the evaluated human-data budgets, suggesting potential for further scaling.
作者：Zibo Zhou, Yueru Chen, Junsong Wu, Weiji Xie, Qingyao Xu, Sheng Yin, Shiqing Wang, Jiyuan Shi, Siheng Chen, Chenjia Bai
来源
本页事实对应的公开页面，以原文为准。