外观
论文arXiv 2610.1394519 天前
Splat2Act：从采集场景到个性化 VLA 操作
原文链接：Splat2Act: From Captured Scenes to Personalized VLA Manipulation（arXiv 2610.13945 · 发布日期：2026-10-09）
Splat2Act: From Captured Scenes to Personalized VLA Manipulation
Splat2Act用手机扫描和物体图像生成仿真数据来个性化VLA拾取策略，并在101个扫描场景和真实Stretch 3上验证。
核心要点
- 通用操作策略进入家庭或办公室时需适应用户物体与场景，π_{0.5}在采集部署场景中未经适配拾取成功率为0%。
- 该流程用手机扫描和物体图像构建3D Gaussian Splatting数字孪生与碰撞几何，自动发现承载面并生成可行拾取任务。
- 系统用特权仿真oracle生成成功演示，并分两阶段微调π_{0.5}，先适配公共物体资产，再个性化到用户物体。
- 在101个扫描校园环境中生成852个承载面和超过68000个演示；最佳个性化策略在目标场景达62.5%聚合成功率。
验证范围与局限：验证范围包括101个扫描校园环境中的仿真拾取，以及真实Stretch 3上的12任务子集。
英文摘要 (Abstract)
A user who brings a generalist manipulation policy into their home or office wants it to pick up their objects from their shelves, counters, and tables. Yet even a strong generalist like $π_{0.5}$ achieves 0% pickup success in our captured deployment scenes without adaptation, and collecting real demonstrations in every user's environment is prohibitively expensive. We present Splat2Act, an automated real-to-sim-to-real pipeline that personalizes a pretrained vision-language-action (VLA) policy to a specific scene and object set using only a handheld phone scan and images of the user's objects. Unlike prior real-to-sim work that reconstructs individual tabletops, Splat2Act operates on building-scale indoor environments: it builds a photorealistic digital twin with 3D Gaussian Splatting and collision geometry, automatically discovers tabletop and non-tabletop receptacles, constructs feasible pickup tasks, and generates successful demonstrations with a privileged simulation oracle, with only scene and object capture required from a human. Applied to 101 scanned campus environments with a median navigable area of 104.87 m$^2$ and 7 receptacles, the pipeline yields 852 receptacles and over 68,000 pickup demonstrations. We fine-tune $π_{0.5}$ in two stages: Stage 1 adapts to hundreds of public object assets, drawn either from the deployment scene alone or from all 101 scenes; Stage 2 personalizes to the user's own objects on the deployment scene's receptacles. Our strongest personalized policy reaches 62.5% aggregate pickup success in the target scene. Deployed on a real Stretch 3, the same policy achieves 63.9% success on a 12-task subset, closely tracking its 66.7% simulated success on those tasks. These results suggest that automated real-to-sim-to-real training can personalize VLA manipulation policies using only simulation-oracle data.
作者：Xiaomeng Ye, Krishna Jaganathan, Gyubeum Lim, Gunjan Chhablani, Muhammad Zubair Irshad, Zsolt Kira
来源
本页事实对应的公开页面，以原文为准。