OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction
点击次数：
发表刊物：IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
摘要：Learning how humans manipulate objects requires machines to acquire knowledge from two perspectives: one for understanding object affordances and the other for learning human's interactions based on the affordances. Even though these two knowledge bases are crucial, we find that current databases lack a comprehensive awareness of them. In this work, we propose a multi-modal and rich-annotated knowledge repository, OakInk, for visual and cognitive understanding of hand-object interactions. We start to collect 1,800 common household objects and annotate their affordances to construct the first knowledge base: Oak. Given the affordance, we record rich human interactions with 100 selected objects in Oak. Finally, we transfer the interactions on the 100 recorded objects to their virtual counterparts through a novel method: Tink. The recorded and transferred hand-object interactions constitute the second knowledge base: Ink. As a result, OakInk contains 50,000 distinct affordance-aware and intent-oriented hand-object interactions. We benchmark OakInk on pose estimation and grasp generation tasks. Moreover, we propose two practical applications of OakInk: intent-based interaction generation and handover generation. Our dataset and source code are publicly available at www. oakink. net.
合写作者：Liu Liu,Xinyu Zhan,Kailin Li
第一作者：Lixin Yang
论文类型：其它
通讯作者：Cewu Lu
页面范围：20953-20962
是否译文：否
发表时间：2022-07-13