eduzhai > Applied Sciences > Engineering >

EBBINNOT A Hardware Efficient Hybrid Event-Frame Tracker for Stationary Neuromorphic Vision Sensors

  • Save

... pages left unread,continue reading

Document pages: 15 pages

Abstract: In this paper, we present a hybrid event-frame approach for detecting andtracking objects recorded by a stationary neuromorphic vision sensor (NVS) usedin the application of traffic monitoring with a hardware efficient processingpipeline that optimizes memory and computational needs. The usage of NVS givesthe advantage of rejecting background while it has a unique disadvantage offragmented objects due to lack of events generated by smooth areas such asglass windows. To exploit the background removal, we propose an event basedbinary image (EBBI) creation that signals presence or absence of events in aframe duration. This reduces memory requirement and enables usage of simplealgorithms like median filtering and connected component labeling (CCL) fordenoise and region proposal (RP) respectively. To overcome the fragmentationissue, a YOLO inspired neural network based detector and classifier (NNDC) tomerge fragmented region proposals has been proposed. Finally, a simplifiedversion of Kalman filter, termed overlap based tracker (OT), exploiting overlapbetween detections and tracks is proposed with heuristics to overcomeocclusion.The proposed pipeline is evaluated using more than 5 hours of trafficrecordings. Our proposed hybrid architecture outperformed (AUC = $0.45$) Deeplearning (DL) based tracker SiamMask (AUC = $0.33$) operating on simultaneouslyrecorded RGB frames while requiring $2200 times$ less computations. Compared topure event based mean shift (AUC = $0.31$), our approach requires $68 times$more computations but provides much better performance. Finally, we alsoevaluated our performance on two different NVS: DAVIS and CeleX anddemonstrated similar gains. To the best of our knowledge, this is the firstreport where an NVS based solution is directly compared to other simultaneouslyrecorded frame based method and shows tremendous promise.

Please select stars to rate!


0 comments Sign in to leave a comment.

    Data loading, please wait...