# KINO：用于人形机器人移动操作中 VLM 规划与全身控制的关键帧接口

> 原文链接：https://robotics.hellobike.com/papers/2026-09-16-2609-18869
> 频道：papers | 更新日期：2026-09-16 | 标签：arxiv, cs.RO
> 核心摘要：提出以运动关键帧连接 VLM 规划与 RL 全身控制，并在仿真和 Unitree G1 上验证物体操作任务。

# KINO：用于人形机器人移动操作中 VLM 规划与全身控制的关键帧接口

::: v-pre
> 原文链接：[KINO: A Keyframe Interface for VLM Planning and Whole-Body Control in Humanoid Loco-Manipulation](https://arxiv.org/abs/2609.18869)（arXiv 2609.18869 · 发布日期：2026-09-16）

**KINO: A Keyframe Interface for VLM Planning and Whole-Body Control in Humanoid Loco-Manipulation**

提出以运动关键帧连接 VLM 规划与 RL 全身控制，并在仿真和 Unitree G1 上验证物体操作任务。

## 核心要点

- 面向人形移动操作中理解指令、场景语义并协调全身运动的需求。
- VLM 根据语言、场景和反馈从预定义库选择关键帧，并按场景重定向。
- 关键帧条件全身策略生成关节动作，到达目标机器人姿态和物体姿态。
- 显著性关键帧采样将稀疏 VLM 关键帧下任务成功率从 44% 提升到 92%。

**验证范围与局限**：验证限于仿真及一台 Unitree G1 上的物体拾取、搬运和放置任务。

## 英文摘要 (Abstract)

Humanoid loco-manipulation requires robots to interpret task instructions and scene semantics while executing coordinated whole-body motions. We propose a hierarchical framework that uses motion keyframes as an intermediate representation between Vision-Language Model (VLM) planning and Reinforcement Learning (RL) control. Each keyframe specifies a target whole-body robot pose and, when applicable, an object pose. Given a language instruction, scene observations, and execution feedback, the VLM selects successive task-relevant keyframes from a predefined library. The selected keyframes are retargeted to the current scene to account for object poses and dimensions. A keyframe-conditioned whole-body policy then generates joint-level actions to reach these goals. We introduce a saliency-based keyframe sampling strategy for low-level policy training that improves end-to-end task success rate from 44% to 92% when using sparse VLM keyframes. We evaluate our framework on object pickup, transport, and placement tasks in simulation and on a Unitree G1 humanoid. The system successfully performs both one- and two-handed manipulation and generalises to placement locations beyond the training reference data.

作者：Sitong Chen, Fatemeh Zargarbashi, Jin Cheng, Tianxu An, Stelian Coros
:::

## 原始参考来源 (Sources)

- [arXiv 2609.18869：KINO: A Keyframe Interface for VLM Planning and Whole-Body Control in Humanoid Loco-Manipulation](https://arxiv.org/abs/2609.18869)