结合时序触觉与顺应控制的双臂人机意图感知递物系统

My Perspective on Human-Robot Handover

I'm tackling the challenge of reliable robot-to-human handover. It's clear that the robot needs to accurately infer when a human is ready to receive an object and then release it in a safe and comfortable manner at the right time. The tricky part is that simply relying on visual observations can be misleading. It's difficult to distinguish a clear intention to take an object from accidental contact, a weak grasp, unintended forces, or transient interactions.

To address this, I'm treating human-robot handover as inherently multimodal. My approach combines a VLA model with a compliance controller. The compliance controller is crucial as it reduces interaction forces during the handover process. I'm training and fine-tuning the VLA model using human demonstrations, leveraging RGB visual data, temporally encoded tactile feedback, and proprioception.

I'm evaluating the complete system through a human-subject study, comparing it against two baselines: one without any tactile feedback and another that incorporates tactile feedback but lacks compliance control. I hypothesize that combining compliance and the temporal encoding of tactile information will result in the most reliable and comfortable handovers. The compliance facilitates physical interaction, while the temporal tactile information captures sustained taking intent. Performance is measured using objective metrics and a tailored questionnaire. My results show that these two components offer complementary benefits and significantly outperform the baselines. The code and data will be released once the paper is accepted.

可靠的机器人向人传递物品要求机器人能够推断出人类何时准备好接收物体，并在适当时机安全、舒适地将其释放。这极具挑战性，因为仅凭视觉观测可能无法将明确的接物意图与意外触碰、微弱抓握、错误方向的施力或短暂的接触区分开来。

在这项工作中，我们将人机物品传递视为一个本质上的多模态问题。我们的方法将 VLA 模型与柔顺控制器相结合，以减少物体传递过程中的交互力。我们利用人类示范数据，结合 RGB 观测、时序编码的触觉反馈和本体感觉，对 VLA 模型进行了微调。

我们通过一项真人受试者实验对整个系统进行了评估，并与两个基线进行了对比：一个没有触觉反馈，另一个使用触觉反馈但没有柔顺控制。我们假设，将柔顺控制与时序触觉编码相结合能够实现最可靠、最舒适的传递，因为柔顺性有助于物理交互，而触觉历史信息则能捕捉到持续的接物意图。系统表现通过客观指标和专门设计的调查问卷进行衡量。结果表明，这两个组件具有互补优势，性能显著优于基线方法。代码和数据将在论文录用后公开发布。