具身开发者社区

外观

论文arXiv 2609.014181 个月前

可证明安全的 Sim-to-Real 迁移 ​

将安全 Sim-to-Real 迁移表述为无奖励安全 RL，提出结合可靠仿真与安全采样的算法，并通过理论分析给出可行近优保证。

AI 整理
  • 仿真与训练
  • 数据与采集
  • 具身模型

原文链接：Provably Safe Sim-to-Real Transfer（arXiv 2609.01418 · 发布日期：2026-09-01）

Provably Safe Sim-to-Real Transfer

将安全 Sim-to-Real 迁移表述为无奖励安全 RL，提出结合可靠仿真与安全采样的算法，并通过理论分析给出可行近优保证。

核心要点 ​

  • 动机是仿真数据廉价但存在 Sim-to-Real 偏差，纠偏所需真实交互本身也必须保持安全。
  • 方法将问题建模为无奖励安全强化学习，一次采集数据并复用于任意奖励函数的规划。
  • 算法识别仿真与真实动力学差异，可靠处使用认证仿真转移，失配处用安全采集数据估计。
  • 结果给出高概率可行性与近优性保证，并在特定情形下改进已知样本复杂度。

英文摘要 (Abstract) ​

We address safe sim-to-real transfer, in which an agent leverages an imperfect simulator and limited real-world interaction while ensuring safety throughout data collection in the real system. This problem arises in applications such as robotics and healthcare: simulators provide cheap data, but sim-to-real mismatch makes direct transfer unreliable, and collecting real-world data to correct this mismatch must itself be safe. Moreover, deployment objectives may vary across tasks, making it costly to collect new data for each reward function. We therefore formulate safe sim-to-real transfer as a reward-free safe reinforcement learning (RL) problem, in which data are collected once and reused to plan for arbitrary reward functions. We develop a computationally efficient algorithm that identifies where the simulator and real dynamics differ, uses certified simulator transitions where they are reliable, and estimates mismatched transitions from safely collected data. With high probability, every policy deployed during learning is feasible, and the collected data support the computation of a feasible and near-optimal policy for any reward function. When the simulator is uninformative, our algorithm recovers online reward-free safe RL while improving the best-known sample complexity by a factor of (\widetildeΘ(H/ξ^2)), where (ξ) is the safety margin of a baseline policy. When the simulator is accurate on most transitions, this improvement grows to (\widetildeΘ(H^2|\mc S||\mc A|/(ξ^2|\mc B|))), where (|\mc B|) denotes the size of the sim-to-real mismatch region.

作者：Tingting Ni, Maryam Kamgarpour

来源

本页事实对应的公开页面，以原文为准。

  1. arXiv 2609.01418：Provably Safe Sim-to-Real Transfer访问 2026-10-21