# 多种成功之道：面向 VLA 泛化的多样性驱动 RL 微调

> 原文链接：https://robotics.hellobike.com/papers/2026-10-07-2610-09943
> 频道：papers | 更新日期：2026-10-07 | 标签：arxiv, cs.RO
> 核心摘要：针对VLA微调分布外泛化受限，DRIVE将成功行为多样性纳入RL目标，并在多个基准和真机上验证提升。

# 多种成功之道：面向 VLA 泛化的多样性驱动 RL 微调

::: v-pre
> 原文链接：[Many Ways to Succeed: Diversity-Driven RL Fine-Tuning for VLA Generalization](https://arxiv.org/abs/2610.09943)（arXiv 2610.09943 · 发布日期：2026-10-07）

**Many Ways to Succeed: Diversity-Driven RL Fine-Tuning for VLA Generalization**

针对VLA微调分布外泛化受限，DRIVE将成功行为多样性纳入RL目标，并在多个基准和真机上验证提升。

## 核心要点

- RL微调可通过闭环经验改进VLA策略，但超出微调分布的泛化仍然受限。
- 分析发现RL整体收缩行为，却会多样化成功轨迹并覆盖更多潜在有效解空间。
- DRIVE在匹配任务条件下分组rollout，经时间对齐比较轨迹，生成成功条件内在奖励。
- 在三套基准上，OOD平均性能较普通RL微调在π0和π0.5上分别提升5.3和2.0点；真机提升9.2点。

**验证范围与局限**：验证范围包括LIBERO-Plus、ManiSkill3、RoboTwin 2.0，以及双臂AgileX PiPER-X平台上的OOD表现。

## 英文摘要 (Abstract)

Reinforcement learning (RL) fine-tuning improves vision-language-action (VLA) policies through closed-loop experience, yet generalization beyond the fine-tuning distribution remains limited. Our analysis reveals a selective reshaping of exploration: RL contracts behavior globally, yet diversifies successful trajectories, elicits success with fewer rollouts, and covers more of the latent task-valid solution space than supervised fine-tuning. Broader successful-mode coverage may provide alternative strategies under distribution shifts. Inspired by this, we introduce DRIVE (Diversity-driven RL fIne-tuning for VLA gEneralization), which turns successful-behavior diversity into an explicit RL objective. DRIVE groups rollouts under matched task conditions, compares their trajectories with temporal alignment, and derives a success-conditioned intrinsic reward from relative behavioral diversity. This design encourages broader coverage of feasible solutions without rewarding diverse failures or superficial timing differences. Across LIBERO-Plus, ManiSkill3, and RoboTwin 2.0, DRIVE improves the average out-of-domain (OOD) performance over vanilla RL fine-tuning by 5.3 points on $π_0$ and 2.0 points on $π_{0.5}$. On a dual-arm AgileX PiPER-X platform, DRIVE further increases average OOD success from 64.1% to 73.3% (+9.2 points), demonstrating gains that persist under physical deployment.

作者：Haoru Li, Jinmei Liu, Zhiyong Wang, Xiaoming Li, Zhenhong Sun, Daoyi Dong, Chunlin Chen, Zhi Wang
:::

## 原始参考来源 (Sources)

- [arXiv 2610.09943：Many Ways to Succeed: Diversity-Driven RL Fine-Tuning for VLA Generalization](https://arxiv.org/abs/2610.09943)