# FACT模型融合EgoForce触觉数据实现人机灵巧操作跨具身迁移

> 原文链接：https://robotics.hellobike.com/papers/2026-10-15-2610-19155
> 频道：papers | 更新日期：2026-10-15 | 标签：auto, arxiv
> 核心摘要：提出FACT基础模型与EgoForce大规模人类触觉数据集，通过视觉规划与力觉精炼级联架构解决人机数据异构难题。仅用10条机器人轨迹微调，成功率较T-Rex提升60%，验证了力觉监督在跨具身知识迁移中的关键作用。

# FACT模型融合EgoForce触觉数据实现人机灵巧操作跨具身迁移

::: v-pre
> 本文由 AI 根据公开内容整理，可能有误，请以原文为准：[FACT: A Multimodal Force-Action-Track Foundation Model for Human-to-Robot Dexterity](https://arxiv.org/abs/2610.19155)（arXiv 2610.19155）

## 核心内容

针对人类演示数据缺乏力觉反馈导致灵巧操作迁移困难的问题，研究发布EgoForce大规模人类触觉数据集及FACT机器人基础模型。该工作旨在通过包含丰富力觉信息的人类预训练数据，实现向机器人策略的高效知识迁移。实验表明，在仅需10条机器人轨迹配合90条人类轨迹进行微调的极低样本设定下，FACT在工具使用、乐器演奏及精密实验室操作等10项任务中，成功率较基线模型T-Rex最高提升60个百分点。

FACT采用双组件级联架构：Track-Guided Coarse Planner联合预测动作、未来视频与视觉轨迹，构建跨具身共享任务表征以生成粗粒度动作；Force-Action Joint Refiner则基于粗规划联合预测未来力并精炼动作，使预期力与动作相互塑造。消融实验证实，人类预训练阶段的力觉监督对下游性能至关重要，且视觉轨迹预测与未来力预测显著增强执行精度。不同人机数据比例测试进一步显示，模型从人类演示中学习的效果几乎等同于机器人真机数据。

## 关键看点

- **极低样本微调**：仅需10条机器人轨迹搭配90条人类轨迹微调，在10项精细操作任务中成功率较T-Rex提升达60%。
- **级联推理架构**：Coarse Planner生成视觉轨迹引导的粗动作，Joint Refiner利用未来力预测实时修正，实现力-动耦合。
- **跨具身表征对齐**：利用未来视频与视觉轨迹作为共享任务表示，有效弥合人类与机器人本体差异，支撑知识迁移。
- **力觉数据价值验证**：消融实验确认人类数据中的力觉监督是下游机器人灵巧操作性能的关键，纯视觉预训练效果受限。

## 工程与产业观察

该工作确立了“人类触觉数据+力觉精炼”在具身智能预训练中的技术范式，证明高质量力觉标注能大幅降低真机数据采集成本。对于算法工程师，其级联式力-动联合优化架构为多模态大模型落地精细操作提供了可复现路径；对产业界而言，EgoForce数据集的开源有望加速通用灵巧操作能力的规模化验证与迭代。

## 英文摘要 (Abstract)

Human demonstrations offer a scalable alternative to costly robot data collection for dexterous manipulation, but most human datasets lack force measurements. Dexterous manipulation requires not only visual and proprioceptive understanding but also the ability to anticipate and regulate interaction forces. To scale force-aware human-robot co-training, we introduce EgoForce, a large-scale human tactile dataset capturing force-rich manipulation experience, and combine it with robot demonstrations for pretraining. We present FACT, a robot foundation model that cascades from prospective visual planning to force-aware refinement for dexterous manipulation. FACT comprises two jointly trained components. A Track-Guided Coarse Planner jointly predicts actions, future video, and visual tracks. Future video and visual tracks also provide a shared task representation that facilitates cross-embodiment knowledge transfer and coarse action generation. Building on this coarse plan, a Force-Action Joint Refiner jointly predicts future force and refines the coarse action, allowing anticipated force and action to shape each other during refinement. We evaluate the model on ten tasks requiring fine-grained coordination and force feedback, including tool use, musical instruments, and precision laboratory procedures. Using as few as 10 robot trajectories with 90 human trajectories for fine-tuning, FACT outperforms both baselines, with success-rate gains of up to 60 percentage points over T-Rex. Ablations show that force supervision in human pretraining data is crucial for downstream robot performance after fine-tuning, while visual track prediction and training with future-force prediction improve dexterous execution. Experiments varying the human-robot data ratio during fine-tuning further suggest that FACT learns from human demonstrations nearly as effectively as from robot demonstrations.

作者：Zhi Cao, Yining Hong, Jiaolong Yang, Yejin Choi, Li Fei-Fei, Jiajun Wu, Huang Huang
:::

## 原始参考来源 (Sources)

- [arXiv 2610.19155：FACT: A Multimodal Force-Action-Track Foundation Model for Human-to-Robot Dexterity](https://arxiv.org/abs/2610.19155)