Egocentric activity recognition with multimodal fisher vector

Song, Sibo; Cheung, Ngai-Man; Chandrasekhar, Vijay; Mandal, Bappaditya; Liri, Jie

Abstract

With the increasing availability of wearable devices, research on egocentric activity recognition has received much attention recently. In this paper, we build a Multimodal Egocentric Activity dataset which includes egocentric videos and sensor data of 20 fine-grained and diverse activity categories. We present a novel strategy to extract temporal trajectory-like features from sensor data. We propose to apply the Fisher Kernel framework to fuse video and temporal enhanced sensor features. Experiment results show that with careful design of feature extraction and fusion algorithm, sensor data can enhance information-rich video data. We make publicly available the Multimodal Egocentric Activity dataset to facilitate future research.

Citation

Song, S., Cheung, N., Chandrasekhar, V., Mandal, B., & Liri, J. (2016). Egocentric activity recognition with multimodal fisher vector. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). https://doi.org/10.1109/icassp.2016.7472171

• Conference Name: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• Conference Location: Shanghai, China
• Start Date: Mar 20, 2016
• End Date: Mar 25, 2016
• Online Publication Date: May 19, 2016
• Publication Date: 2016-03
• Deposit Date: Nov 17, 2023
• Publisher: Institute of Electrical and Electronics Engineers (IEEE)
• Book Title: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• ISBN: 978-1-4799-9987-3
• DOI: https://doi.org/10.1109/icassp.2016.7472171