# AutodidactWAM：从生成视频到机器人动作的跨模态自蒸馏

> 原文链接：https://robotics.hellobike.com/papers/2026-10-06-2610-08119
> 频道：papers | 更新日期：2026-10-06 | 标签：arxiv, cs.RO
> 核心摘要：针对WAM迁移到未见机器人时动作偏靶，AutodidactWAM从生成视频恢复动作并自蒸馏，在Unitree G1真机任务上验证。

# AutodidactWAM：从生成视频到机器人动作的跨模态自蒸馏

::: v-pre
> 原文链接：[AutodidactWAM: Cross-Modal Self-Distillation from Generated Video to Robot Actions](https://arxiv.org/abs/2610.08119)（arXiv 2610.08119 · 发布日期：2026-10-06）

**AutodidactWAM: Cross-Modal Self-Distillation from Generated Video to Robot Actions**

针对WAM迁移到未见机器人时动作偏靶，AutodidactWAM从生成视频恢复动作并自蒸馏，在Unitree G1真机任务上验证。

## 核心要点

- LoRA适配Cosmos 3到未见Unitree G1后，视频合理但动作系统性偏靶。
- 方法用免遥操作训练的手姿态估计器处理生成视频，再经逆运动学恢复动作。
- 恢复动作与原生预测配对，微调动作相关层，并教师强制生成视频。
- 真机三阶段恢复动作门控约75%、47%、42%，DPO+SFT+DTW表现最佳。

**验证范围与局限**：验证限于Unitree G1配BrainCo五指手的闭环真机试验，任务阶段为预抓取、抓取、拾取放置，并含训练物体Oreo和一个未见物体。

## 英文摘要 (Abstract)

World-action models (WAMs) such as Cosmos 3 jointly generate future video and robot actions from an observation and instruction. Adapting one such model with a lightweight LoRA fine-tune to a previously unseen robot, a Unitree G1 humanoid with five-fingered BrainCo hands, exposes a video-action asymmetry: the video renders plausible task executions, while the co-generated action is systematically mis-targeted. We evaluate closed-loop real-robot trials at three cumulative stages: pre-grasp, grasp, and pick-and-place. The native action succeeds only approximately 17%, 10%, and 7% of the time, respectively, and performs worse on held-out objects. We propose AutodidactWAM, a hand-pose estimator trained without teleoperation, followed by inverse kinematics, that runs on the model's generated video to recover action estimates. Paired with the native prediction, these recovered actions provide preferred targets for fine-tuning only the action-related layers, while the generated video is teacher-forced. We compare supervised relabeling with a rectified-flow adaptation of Diffusion-DPO. After one-time embodiment adaptation, self-distillation requires no additional task-specific teleoperation. The recovered-action gate reaches approximately 75%, 47%, and 42% pre-grasp, grasp, and pick-and-place success, compared with 17%, 10%, and 7% for the native action. A hybrid objective combining preference supervision, supervised target fitting, and Cartesian trajectory anchoring (DPO+SFT+DTW) performs best: on Oreo, the training object, it reaches 90% pre-grasp and 20% full-task success; on a held-out object, it reaches 80% and 30%. Plain Flow-DPO reaches 0% success despite 1.000 validation preference accuracy, indicating that the combination of training objectives, rather than the contrastive objective alone, drives the observed gains.

作者：Sergei Kurchev, Iaroslav Kolomiets, Miguel Altamirano Cabrera, Artem Lykov, Dzmitry Tsetserukou
:::

## 原始参考来源 (Sources)

- [arXiv 2610.08119：AutodidactWAM: Cross-Modal Self-Distillation from Generated Video to Robot Actions](https://arxiv.org/abs/2610.08119)