论文/arXiv 2610.11416·20 天前

重连语义、动力学与控制：一种简单而有效的以动作为中心的三流 Transformer

针对 VLA 缺乏物理动力学先验的问题，ACT^3 以三流 Transformer 融合语义与动力学信息，并在仿真和真实机器人操作基准上验证。

AI 整理

• 本体 SDK 与控制
• 具身模型
• 评测与安全

原文链接：Rewiring Semantics, Dynamics, and Control: A Simple yet Effective Action-Centric Tri-Stream Transformer（arXiv 2610.11416 · 发布日期：2026-10-08）

Rewiring Semantics, Dynamics, and Control: A Simple yet Effective Action-Centric Tri-Stream Transformer

针对 VLA 缺乏物理动力学先验的问题，ACT^3 以三流 Transformer 融合语义与动力学信息，并在仿真和真实机器人操作基准上验证。

核心要点

• VLA 借助预训练 VLM 的语义理解，但物理动力学先验不足，限制机器人策略泛化。
• ACT^3 采用以动作为中心的三流 Transformer，将语义与动力学信息融合到控制动作中。
• 方法让动作专家通过逐层注意力访问 VLM 与 WM 表征，同时各骨干仅在自身流内注意。
• 仿真与真实机器人操作基准实验显示，ACT^3 的结果优于其对应方法。

验证范围与局限：验证范围为仿真和真实世界机器人操作基准。

英文摘要 (Abstract)

Vision-Language-Action (VLA) models have emerged as a prominent framework for complex robotic manipulation, building on the strong semantic understanding of pretrained Vision-Language Models (VLMs). However, such VLM backbones offer insufficient physical dynamics priors, which limits the generalization capabilities of robot policies. Recent efforts therefore integrate video-generation World Models (WMs) into robot policies through various strategies, using predictive dynamics to facilitate action generation. Despite these advances, harnessing semantic understanding and dynamics prediction as complementary guidance for action generation remains challenging. In this paper, we introduce $\mathrm{ACT}^3$, a simple yet effective Action-Centric Tri-Stream Transformer that fuses semantic and dynamics information into control actions while preserving the distinct roles of context streams. Specifically, $\mathrm{ACT}^3$ enables the dedicated action expert to access VLM and WM representations through layerwise attention, with each backbone attending only within its own stream. This straightforward interaction design maintains independent forward propagation in the context streams while allowing both backbones to be updated through control supervision. Experiments on both simulated and real-world robotic manipulation benchmarks show that the proposed $\mathrm{ACT}^3$ yields results superior to its counterparts.

作者：Shuang Luo, Yilun Kong, Yunpeng Qing, Yihang Jiao, Zhi Hou, Shunyu Liu, Xiaogang Wang, Dacheng Tao

来源

本页事实对应的公开页面，以原文为准。

1. arXiv 2610.11416：Rewiring Semantics, Dynamics, and Control: A Simple yet Effective Action-Centric Tri-Stream Transformer访问 2026-10-21