外观
论文arXiv 2610.0535924 天前
未建模状态依赖下结合学习型 Critic 的最优控制
原文链接：Optimal Control with Learned Critics under Unmodeled State Dependencies（arXiv 2610.05359 · 发布日期：2026-10-04）
Optimal Control with Learned Critics under Unmodeled State Dependencies
针对难建模状态依赖限制MPC的问题，提出结合残差动力学与动作价值critic的学习型MPC，并在有偏和不完整建模控制任务中验证性能提升。
核心要点
- MPC具备结构化与约束感知优势，但依赖易优化解析动力学，难处理接触等状态依赖。
- 方法用残差动力学网络补偿名义模型缺失效应，并以动作价值critic注入长时域MDP结构。
- 开发GPU加速批量iLQR，在最优控制循环中评估网络并并行求解大量轨迹优化问题。
- 实验显示在有偏和不完整建模控制任务中提升闭环控制性能，同时保留模型式约束优化结构。
验证范围与局限：验证范围为机器人仿真器中的有偏和不完整建模控制任务。
英文摘要 (Abstract)
Model Predictive Control (MPC) provides a structured and constraint-aware mechanism for decision-making, but its reliance on optimization-friendly analytical dynamics models limits its use in tasks with contacts and other hard-to-model state dependencies. Model-free reinforcement learning avoids explicit modeling assumptions but typically requires large amounts of interaction data. We present a learning-based MPC framework that combines the data efficiency and structure of local model-based planning with learned components that compensate for incomplete dynamics and finite-horizon myopia. The method augments a nominal analytical model with a residual dynamics network that learns missing state-dependent effects from data and combines the resulting planner with a learned action-value critic that injects long-horizon MDP structure into the local iLQR optimization. To make this practical at reinforcement-learning scale, we develop a GPU-accelerated batched iLQR solver that evaluates learned dynamics and critic networks inside the optimal-control loop and solves thousands of trajectory-optimization problems in parallel. The complete system is integrated into a robotics simulator, enabling scalable model-based reinforcement learning under incomplete dynamics. Experiments on biased and incompletely modeled control tasks show that the approach improves closed-loop control performance while preserving the model-based structure needed for efficient constrained trajectory optimization.
作者：Philipp Schoch, Markus Ryll
来源
本页事实对应的公开页面，以原文为准。