具身开发者社区

外观

论文arXiv 2610.1214020 天前

在强化学习中利用人在回路示范实现数字孪生驱动的机器人灵活性 ​

提出结合数字孪生、强化学习与人类示范的在线训练框架，并在 Ufactory Xarm5 避障到达任务上验证适应性。

AI 整理
  • 仿真与训练
  • 数据与采集
  • 具身模型

原文链接：Leveraging Human-In-The-Loop Demonstrations in Reinforcement Learning for Digital Twin-Driven Robot Flexibility（arXiv 2610.12140 · 发布日期：2026-10-08）

Leveraging Human-In-The-Loop Demonstrations in Reinforcement Learning for Digital Twin-Driven Robot Flexibility

提出结合数字孪生、强化学习与人类示范的在线训练框架，并在 Ufactory Xarm5 避障到达任务上验证适应性。

核心要点 ​

  • 协作机器人面向更可变环境，提升了对任务执行中自适应能力的需求。
  • 数字孪生通过相机与物理系统实时同步，虚拟机器人用真实反馈更新观测与策略。
  • 双 actor 框架融入模仿学习但不向 RL actor 加直接模仿损失，以示范引导适应。
  • 实验显示工作空间变化后可恢复训练；在非最优和 VR 人类示范下均优于模仿损失方法。

验证范围与局限：验证范围为 Ufactory Xarm5 协作机器人末端避障到达目标位置任务。

英文摘要 (Abstract) ​

Growing automation makes collaborative robots work in more variable environments, increasing the need for adaptation. We propose a human-in-the-loop online training framework combining a digital twin (DT), reinforcement learning (RL), and human demonstrations. Unlike DTs used mainly to generate synthetic data before task execution, our DT is synchronized with the physical system in real time through camera feeds, allowing the virtual robot to update its observations and policy from real-world feedback. A dual actor framework integrates imitation learning (IL) without adding a direct imitation loss to the RL actor, so demonstrations can guide adaptation instead of manual reprogramming. The proposed framework is demonstrated on the Ufactory Xarm5 collaborative robot, where the robot's end-effector aims to reach the target position while avoiding obstacles. The experiments show that the framework can resume training after a change in the physical workspace and that, with a fixed set of non-optimal demonstrations, the dual actor framework achieves a much higher final success rate than two methods that add an imitation loss to the actor. The same pattern holds with real human demonstrations collected in virtual reality (VR): with demonstrations that never reach the goal, the dual actor framework reached 83-100% mean deterministic evaluation success, against 0-17% for the two imitation-loss methods.

作者：Yuzhu Sun, Mien Van, Nguyen Minh Nhat, Stephen McIlvanna, Seán McLoone

来源

本页事实对应的公开页面，以原文为准。

  1. arXiv 2610.12140：Leveraging Human-In-The-Loop Demonstrations in Reinforcement Learning for Digital Twin-Driven Robot Flexibility访问 2026-10-21