# 用于从多模态人类活动序列进行层级意图识别的神经符号图 Transformer

> 原文链接：https://robotics.hellobike.com/papers/2026-10-13-2610-18176
> 频道：papers | 更新日期：2026-10-13 | 标签：arxiv, cs.RO
> 核心摘要：提出NSGT，将HR-G偏置与FOL损失用于多模态层级意图识别，并在合成NTU-Hierarchy上验证。

# 用于从多模态人类活动序列进行层级意图识别的神经符号图 Transformer

::: v-pre
> 原文链接：[Neuro-Symbolic Graph Transformers for Hierarchical Intention Recognition from Multimodal Human Activity Sequences](https://arxiv.org/abs/2610.18176)（arXiv 2610.18176 · 发布日期：2026-10-13）

**Neuro-Symbolic Graph Transformers for Hierarchical Intention Recognition from Multimodal Human Activity Sequences**

提出NSGT，将HR-G偏置与FOL损失用于多模态层级意图识别，并在合成NTU-Hierarchy上验证。

## 核心要点

- 当前方法多聚焦扁平动作分类，难捕捉运动与高层目标之间的结构依赖。
- NSGT将Human Reasoning Graph以可学习拓扑偏置融入自注意力，并加入可微FOL损失。
- 在合成NTU-Hierarchy上评估，包含四级本体层次和119,383个复杂层级片段。
- λ=0.5时逻辑一致性达94.59%、矛盾减少33%，并保持89.90% top-1动作准确率。

**验证范围与局限**：验证范围为合成NTU-Hierarchy数据集，该数据集由NTU RGB+D 120扩展得到。

## 英文摘要 (Abstract)

Proactive human-robot collaboration requires autonomous systems to infer not only immediate actions but also the underlying hierarchical intentions governing human behavior. Current deep learning approaches predominantly focus on flat action classification, failing to capture the structural dependencies connecting kinematic motion to high-level goals. A Neuro-Symbolic Graph Transformer (NSGT) is proposed within this multimodal framework, a Human Reasoning Graph (HR-G) is integrated into the self-attention mechanism via learnable topological biases. Furthermore, the architecture is augmented with a differentiable First-Order Logic (FOL) loss, acting as a structural regularizer by which ontological consistency is enforced across four levels: atomic actions, composite activities, Low-Level Intentions, and High-Level Intentions. Evaluation is performed on the synthesized NTU-Hierarchy dataset, a four-level ontological extension of NTU RGB+D 120, wherein existing sequences were mapped into 119,383 complex hierarchical episodes (85 actions, 16 activities, 8 LLIs, 4 HLIs). To prevent circular supervision, strict isolation between label generation and FOL constraints was maintained. In the base configuration, 91.65% top-1 action accuracy is achieved by the NSGT. When the FOL loss is incorporated into the baseline, logic consistency is improved by 3.26 percentage points to 96.67%. Through the HR-G biases, a tunable trade-off between accuracy and structural coherence is established. In the primary consistency-maximizing configuration ($λ$ = 0.5), a logic consistency of 94.59% is reached and ontological contradictions are reduced by 33%, while a robust 89.90% top-1 action accuracy is retained. Finally, interpretable reasoning evidence is provided by the learned HR-G weights, wherein transition edges are strongly exploited ($β$ trans $\approx$ +2.6) without explicit supervision.

作者：Farnaz Soleimani, Abdelghani Chibani, Yacine Amirat, Ghazaleh Khodabandelou
:::

## 原始参考来源 (Sources)

- [arXiv 2610.18176：Neuro-Symbolic Graph Transformers for Hierarchical Intention Recognition from Multimodal Human Activity Sequences](https://arxiv.org/abs/2610.18176)