eduzhai > Applied Sciences > Engineering >

Two-stream Fusion Model for Dynamic Hand Gesture Recognition using 3D-CNN and 2D-CNN Optical Flow guided Motion Template

  • king
  • (0) Download
  • 20210506
  • Save

... pages left unread,continue reading

Document pages: 7 pages

Abstract: The use of hand gestures can be a useful tool for many applications in thehuman-computer interaction community. In a broad range of areas hand gesturetechniques can be applied specifically in sign language recognition, roboticsurgery, etc. In the process of hand gesture recognition, proper detection, andtracking of the moving hand become challenging due to the varied shape and sizeof the hand. Here the objective is to track the movement of the handirrespective of the shape, size, and color of the hand. And, for this, a motiontemplate guided by optical flow (OFMT) is proposed. OFMT is a compactrepresentation of the motion information of a gesture encoded into a singleimage. In the experimentation, different datasets using bare hand with an openpalm, and folded palm wearing green-glove are used, and in both cases, we couldgenerate the OFMT images with equal precision. Recently, deep network-basedtechniques have shown impressive improvements as compared to conventionalhand-crafted feature-based techniques. Moreover, in the literature, it is seenthat the use of different streams with informative input data helps to increasethe performance in the recognition accuracy. This work basically proposes atwo-stream fusion model for hand gesture recognition and a compact yetefficient motion template based on optical flow. Specifically, the two-streamnetwork consists of two layers: a 3D convolutional neural network (C3D) thattakes gesture videos as input and a 2D-CNN that takes OFMT images as input. C3Dhas shown its efficiency in capturing spatio-temporal information of a video.Whereas OFMT helps to eliminate irrelevant gestures providing additional motioninformation. Though each stream can work independently, they are combined witha fusion scheme to boost the recognition results. We have shown the efficiencyof the proposed two-stream network on two databases.

Please select stars to rate!


0 comments Sign in to leave a comment.

    Data loading, please wait...