外观
论文arXiv 2610.1043821 天前
ECHO：人类搬运物体的具身相机观测
原文链接：ECHO: Embodied Camera Observations of Human Object Carrying（arXiv 2610.10438 · 发布日期：2026-10-07）
ECHO: Embodied Camera Observations of Human Object Carrying
为预测搬运中物体目的地，提出情境物体放置任务与ECHO合成数据集，并在其上用探针和端到端基线评估。
核心要点
- 现有数据集缺少可导航重建场景、人类活动和物体自然目的地标注，限制了相关研究。
- ECHO将室内RGB-D扫描与具身人类搬运日常物体到合适目的地的记录配对。
- 数据集含3,805个标注情节、159个楼层、115个HM3D场景和198种不同物体。
- 实验显示单一输入模态不足，需要联合推理场景结构、人类活动和上下文知识。
验证范围与局限：验证范围限于ECHO合成数据集中的室内场景与人类搬运物体情节。
英文摘要 (Abstract)
Embodied and assistive agents must do more than recognize objects: they must reason about where an object belongs given the layout of an environment and the habits of the people who live in it. Progress on this problem has been limited, in part because no dedicated benchmark or dataset exists to define and evaluate it. Existing RGB-D scan datasets reconstruct static rooms without human activity, while human-object-interaction datasets capture motion without a navigable, fully reconstructed scene or a ground-truth notion of an object's natural destination. We introduce contextual object placement as a benchmark task: predicting an object's destination during an observed object-carrying episode. To support this task, we present Embodied Camera observations of Human Object carrying (ECHO), a large-scale synthetic dataset that pairs dense RGB-D scans of indoor scenes with recordings of an embodied human carrying everyday objects to context-appropriate destinations. ECHO is the first publicly available dataset to combine reconstructed scenes, human activity, natural language, and contextual-placement annotations. It comprises 3,805 human-annotated episodes across 159 floors of 115 HM3D scenes, involving 198 distinct objects. Each floor includes a complete RGB-D scan with human-annotated room labels and a surface list. Each episode provides synchronized RGB-D encounter clips; 6-DoF camera, human, and object trajectories; start and destination surfaces; an action caption; and a human-written context: a single sentence describing the inhabitant's routine that implies the destination without naming it. We evaluate contextual object placement using input-masked probes and an end-to-end baseline. Results show that no single input modality is sufficient, highlighting the need to jointly reason over scene structure, human activity, and contextual knowledge.
作者：Xuefei Sun, Lorin Achey, Kali Hamilton, Alberto Speranzon B, Gregory S. Grebe, Yonatan Bisk, Christoffer Heckman
来源
本页事实对应的公开页面，以原文为准。