# 通过空间语言建模统一策略学习与状态预测

> 原文链接：https://robotics.hellobike.com/papers/2026-10-08-2610-12172
> 频道：papers | 更新日期：2026-10-08 | 标签：arxiv, cs.RO
> 核心摘要：提出空间语言建模，用共享离散坐标与语义词元统一动作生成和状态预测，并在Push-T仿真与真实机器人上验证。

# 通过空间语言建模统一策略学习与状态预测

::: v-pre
> 原文链接：[Unifying Policy Learning and State Prediction through Spatial Language Modeling](https://arxiv.org/abs/2610.12172)（arXiv 2610.12172 · 发布日期：2026-10-08）

**Unifying Policy Learning and State Prediction through Spatial Language Modeling**

提出空间语言建模，用共享离散坐标与语义词元统一动作生成和状态预测，并在Push-T仿真与真实机器人上验证。

## 核心要点

- 动机是学习动作如何改变场景几何，为目标导向操作提供互补监督。
- 方法用任务特定语法，将轮廓、目标、动作目标和未来状态组织为空间序列。
- 单个自回归Transformer通过共同的下一词元目标学习动作生成与状态预测。
- 实验显示仿真性能有竞争力，真实机器人成功率和目标覆盖优于所评基线；消融表明联合训练和random-play预训练有增益。

**验证范围与局限**：验证范围为Push-T仿真和真实机器人实验。

## 英文摘要 (Abstract)

Learning how actions change scene geometry can provide complementary supervision for goal-directed manipulation. We introduce Spatial Language Modeling, which represents scene contours, goals, action targets, and future states with a shared vocabulary of discrete coordinates and semantic tokens. A task-specific grammar organizes these elements into spatial sequences, allowing one autoregressive Transformer to learn action generation and action-conditioned state prediction through a common next-token objective. We train the model from scratch using random-play transition pretraining followed by joint action and state training on expert demonstrations. During pretraining, recorded action coordinates condition subsequent state predictions and are excluded from the prediction loss. During control, the model decodes only executable action targets and updates its history with newly observed states. We evaluate the approach on Push-T in simulation and on a real robot. The model achieves competitive simulation performance and higher task success and target coverage than the evaluated real-robot policy baselines. Training ablations show improved control with joint action and state sequences, with further gains from random-play pretraining. Given supplied action trajectories, the same model also predicts successive scene states, capturing the geometric effects of pushing.

作者：Minye Wu, Zehao Wang, Tinne Tuytelaars
:::

## 原始参考来源 (Sources)

- [arXiv 2610.12172：Unifying Policy Learning and State Prediction through Spatial Language Modeling](https://arxiv.org/abs/2610.12172)