外观
论文arXiv 2609.071111 个月前
从 LLM 生成的规格到学习得到的四足运动
原文链接：From LLM-Generated Specifications to Learned Quadruped Locomotion（arXiv 2609.07111 · 发布日期：2026-09-07）
From LLM-Generated Specifications to Learned Quadruped Locomotion
研究用LLM生成PSTL规格并转为平滑奖励，在MJX中训练并评估四足运动策略。
核心要点
- 四足强化学习常依赖手工奖励，且局部奖励能否诱导期望全局行为并不清晰。
- GPT-5.5与Qwen 3.6根据自然语言目标和受限语法生成PSTL/STL模板。
- 参数由专家轨迹实例化，仅保留符合专家行为的规格，再用PPO训练策略。
- Qwen 3.6步态感知规格在全测试速度达100%存活与指令成功，并在高速匹配目标步态。
验证范围与局限：验证在MuJoCo XLA（MJX）中进行，覆盖步态感知和步态无关设置及0.3--2.1 m/s测试速度。
英文摘要 (Abstract)
Quadruped robot locomotion policies are often trained using reinforcement learning, which in turn relies heavily on hand-crafted reward functions. Designing reward functions requires substantial manual engineering, and it is often unclear which local rewards will induce the desired global behavior. Shaped rewards from formal specifications in languages like Signal Temporal Logic (STL) can make rewards more interpretable, but writing STL specifications itself still requires domain expertise. We study whether large language models (LLMs) can fill this gap by generating Parametric Signal Temporal Logic (PSTL) specifications that are subsequently used for policy learning. Given a natural language locomotion objective and a constrained specification grammar, GPT-5.5 and Qwen 3.6 independently propose STL templates for command tracking, safety, and gait structure. We instantiate the parameters of the generated PSTL templates using expert trajectories and retain only specifications that are consistent with demonstrated expert behavior. The resulting specifications are then transformed into smooth, finite-history reward functions and used to train a quadruped locomotion policy with Proximal Policy Optimization (PPO) in MuJoCo XLA (MJX). We evaluate both \emph{gait-aware} and \emph{gait-agnostic} settings. The former specifies walking-trot, trot, and bound regimes, while the latter allows contact patterns to emerge from the task objective. We compare against hand-engineered rewards, Text2Reward-style LLM-generated reward code, and an expert-switching oracle. Gait-aware Qwen 3.6 specifications achieved 100% survival and command success across all tested speeds (0.3--2.1 m/s) and matched the target gait at high speeds, whereas Text2Reward achieved 0% for both metrics at $\geq 1.9$ m/s. Videos: https://stl-locomotion.github.io/
作者：Merve Atasever, Keyan Azbijari, Cagan Bakirci, Alfredo Reina Corona, Tolga İzdaş, Richard Yang, Erdem Bıyık, Jyotirmoy V. Deshmukh
来源
本页事实对应的公开页面，以原文为准。