外观
论文HF Daily Papers · arXiv 2610.309326 天前
UserArena发布6.4万真实电商轨迹基准与OPTD蒸馏算法
本文由 AI 根据公开内容整理，可能有误，请以原文为准：UserArena: Benchmarking Interactive User Simulation for Long-Horizon Shopping Agents（HF Daily Papers · arXiv 2610.30932）
核心内容
为解决购物智能体长程决策评估缺乏真实数据与验证手段的问题，研究团队推出UserArena基准。该数据集涵盖11,010名用户的64,729条会话轨迹、566,102个商品及13种动作类型，并配套交互式环境与生产级A/B测试结果，支持从输出有效性到最终业务成果的五级离线评估体系。
为提升长程交互模拟器的保真度，提出On-Policy Trajectory Distillation（OPTD）算法。该方法利用轨迹级意图为学生模型生成的rollouts提供密集教师目标，有效缓解长时序误差累积。实验表明，OPTD在UserArena评估协议下综合保真度最优，且在离线模拟中最接近真实生产环境的策略效果。
关键看点
- 真实轨迹规模：收录64,729条真实电商会话轨迹，覆盖13种动作类型与56万+商品，填补长程交互数据空白
- 五级评估体系：构建涵盖输出有效性、局部决策、序列行为、个性化及业务结果的完整离线验证框架
- OPTD蒸馏算法：通过轨迹级意图提供密集教师信号，解决学生模型生成长程rollouts时的误差累积问题
- 生产对齐验证：支持基于受控生产A/B测试结果的离线校验，确保模拟环境与真实业务策略高度一致
工程与产业观察
UserArena与OPTD为具身智能及决策Agent提供了稀缺的真实长程交互数据与高保真仿真方法。其五级评估与生产对齐机制，解决了Sim-to-Real中用户行为模拟失真的核心痛点，对服务机器人交互策略训练与离线强化学习具有重要参考价值。
英文摘要 (Abstract)
Agentic recommendation pursues shopper goals through sequences of context-dependent decisions. Developing and evaluating such policies requires faithful recommendation world models that capture both the interactive shopping environment and users' behavioral responses over long interactions. Existing resources rarely provide long-horizon, fine-grained real-user trajectories or support trajectory-level evaluation and offline validation against production A/B results. We introduce UserArena, a real-world e-commerce interactive benchmark containing 64,729 session trajectories from 11,010 users, spanning 566,102 items and 13 action types. UserArena pairs these trajectories with an interactive environment and supports offline validation against outcomes from a controlled production A/B test, enabling five-level evaluation from output validity and local decisions to sequential behavior, personalization, and outcomes. To improve simulator fidelity over these long-horizon interactions, we further propose On-Policy Trajectory Distillation (OPTD), which uses trajectory-level intent to provide dense teacher targets on student-generated rollouts. Experiments show that OPTD achieves the strongest overall fidelity among the evaluated simulators under UserArena's evaluation protocol and most closely reproduces the production policy effect in offline simulation.
Hugging Face Daily Papers 热度：41 upvotes 作者：Jujia Zhao, TangJiakai, Jing Yu, Wen Chen, Jian Wu, Han Zhu, Fei Sun, Suzan Verberne, Zhaochun Ren
来源
本页事实对应的公开页面，以原文为准。