# 未建模状态依赖下结合学习型 Critic 的最优控制

> 原文链接：https://robotics.hellobike.com/papers/2026-10-04-2610-05359
> 频道：papers | 更新日期：2026-10-04 | 标签：arxiv, cs.RO
> 核心摘要：针对难建模状态依赖限制MPC的问题，提出结合残差动力学与动作价值critic的学习型MPC，并在有偏和不完整建模控制任务中验证性能提升。

# 未建模状态依赖下结合学习型 Critic 的最优控制

::: v-pre
> 原文链接：[Optimal Control with Learned Critics under Unmodeled State Dependencies](https://arxiv.org/abs/2610.05359)（arXiv 2610.05359 · 发布日期：2026-10-04）

**Optimal Control with Learned Critics under Unmodeled State Dependencies**

针对难建模状态依赖限制MPC的问题，提出结合残差动力学与动作价值critic的学习型MPC，并在有偏和不完整建模控制任务中验证性能提升。

## 核心要点

- MPC具备结构化与约束感知优势，但依赖易优化解析动力学，难处理接触等状态依赖。
- 方法用残差动力学网络补偿名义模型缺失效应，并以动作价值critic注入长时域MDP结构。
- 开发GPU加速批量iLQR，在最优控制循环中评估网络并并行求解大量轨迹优化问题。
- 实验显示在有偏和不完整建模控制任务中提升闭环控制性能，同时保留模型式约束优化结构。

**验证范围与局限**：验证范围为机器人仿真器中的有偏和不完整建模控制任务。

## 英文摘要 (Abstract)

Model Predictive Control (MPC) provides a structured and constraint-aware mechanism for decision-making, but its reliance on optimization-friendly analytical dynamics models limits its use in tasks with contacts and other hard-to-model state dependencies. Model-free reinforcement learning avoids explicit modeling assumptions but typically requires large amounts of interaction data. We present a learning-based MPC framework that combines the data efficiency and structure of local model-based planning with learned components that compensate for incomplete dynamics and finite-horizon myopia. The method augments a nominal analytical model with a residual dynamics network that learns missing state-dependent effects from data and combines the resulting planner with a learned action-value critic that injects long-horizon MDP structure into the local iLQR optimization. To make this practical at reinforcement-learning scale, we develop a GPU-accelerated batched iLQR solver that evaluates learned dynamics and critic networks inside the optimal-control loop and solves thousands of trajectory-optimization problems in parallel. The complete system is integrated into a robotics simulator, enabling scalable model-based reinforcement learning under incomplete dynamics. Experiments on biased and incompletely modeled control tasks show that the approach improves closed-loop control performance while preserving the model-based structure needed for efficient constrained trajectory optimization.

作者：Philipp Schoch, Markus Ryll
:::

## 原始参考来源 (Sources)

- [arXiv 2610.05359：Optimal Control with Learned Critics under Unmodeled State Dependencies](https://arxiv.org/abs/2610.05359)