外观
论文arXiv 2610.1047921 天前
Agentic RSR框架打通实景重建与策略开发，真机任务成功率达仿真80%
本文由 AI 根据公开内容整理，可能有误，请以原文为准：Agentic RSR: Real-to-Sim-to-Real through Scene Reconstruction and Execution-Grounded Robot Policies（arXiv 2610.10479）
核心内容
针对具身智能中场景重建与策略开发割裂的痛点，研究团队提出Agentic Real-to-Sim-to-Real（Agentic RSR）框架。该方案利用视频输入与已知机器人模型，在MuJoCo中恢复度量尺度并迭代优化场景，通过编码Agent自动生成可执行策略。实验覆盖两款机器人的18个场景，全部通过联合视觉检查，真机聚合任务成功率达到仿真环境的80%，验证了Sim-to-Real迁移的有效性。
技术上，系统采用Claude Opus 5.5结合DeepSeek Harness驱动，支持策略在单次调用中交错多模态观测与动作，并利用执行反馈进行重试或修正。共享任务级接口将累积经验无缝传递至真机，配合实时安全校验引导执行。量化指标显示，重建场景的四视图深度MAE为0.0808m，Lab ΔE76为8.01，灰度SSIM达0.7338，确保了仿真环境与物理世界在几何与语义上的高度对齐。
关键看点
- Sim2Real迁移率：真机聚合任务成功率达到仿真环境的80%，证实框架在保留仿真性能方面具有显著实效
- 场景重建精度：18个场景四视图深度MAE仅0.0808m，灰度SSIM为0.7338，保障策略训练的几何一致性
- Agent闭环机制：编码Agent支持单次调用内交错观测与动作，利用执行反馈自动重试或修订策略逻辑
- 统一任务接口：通过共享接口将仿真积累的策略与经验直接部署至真机，结合实时观测与安全校验执行
工程与产业观察
该工作解决了具身智能中“重建归重建、训练归训练”的工程断层问题。通过LLM Agent将视觉反馈、代码生成与物理执行串联，实现了从视频到真机策略的端到端自动化。80%的真机成功率与开源承诺，为降低Sim-to-Real门槛提供了高价值参考范式，尤其适合需快速适配新场景的操纵任务研发。
英文摘要 (Abstract)
A simulation of a real robot workspace must preserve task-relevant interactions, while policies developed in it must operate on observations available to the real robot. Yet scene reconstruction and policy development are often treated separately. We present Agentic Real-to-Sim-to-Real (Agentic RSR), a framework that links scene reconstruction, policy development, and real-robot execution through the same manipulation task. Given a workspace video, a task description, and a known robot model, an agent recovers metric scale, iteratively refines the scene using visual feedback, and checks task-relevant interactions in MuJoCo. A coding agent then develops an executable policy, progressing from privileged object poses to visual observations and randomized simulation. The policy can interleave multiple observations and actions within one invocation, while the agent uses execution feedback to continue, retry, or revise its approach. A shared task-level interface carries the policy and accumulated experience to the real robot, where fresh observations and safety checks guide execution. Using Claude Opus 5.5 through DeepSeek Harness (DSH), all 18 scenes across two robots pass the joint visual check; mean four-view Depth MAE against reference depth estimates is 0.0808 m, mean Lab $ΔE_{76}$ is 8.01, and mean grayscale SSIM is 0.7338. In real-robot experiments, the aggregate task success rate reaches 80% of the simulation task success rate, indicating substantial retention of simulated performance on hardware. Code and reconstructed scene data will be made publicly available.
作者：Yihan Li, Yating Feng, Shengjiu Sun, Jianing Chen, Hao Ren, Bowen Yang, Weisheng Xu, Qiwei Wu, Hui Cheng, Renjing Xu
来源
本页事实对应的公开页面，以原文为准。