Deep fusion of multi-channel neurophysiological signal for emotion recognition and monitoring

Li, Xiang; Song, Dawei; Zhang, Peng; Hou, Yuexian and Hu, Bin (2017). Deep fusion of multi-channel neurophysiological signal for emotion recognition and monitoring. International Journal of Data Mining and Bioinformatics, 18(1) pp. 1–27.

DOI: https://doi.org/10.1504/IJDMB.2017.086097

Abstract

How to fuse multi-channel neurophysiological signals for emotion recognition is emerging as a hot research topic in community of Computational Psychophysiology. Nevertheless, prior feature engineering based approaches require extracting various domain knowledge related features at a high time cost. Moreover, traditional fusion method cannot fully utilise correlation information between different channels and frequency components. In this paper, we design a hybrid deep learning model, in which the 'Convolutional Neural Network (CNN)' is utilised for extracting task-related features, as well as mining inter-channel and inter-frequency correlation, besides, the 'Recurrent Neural Network (RNN)' is concatenated for integrating contextual information from the frame cube sequence. Experiments are carried out in a trial-level emotion recognition task, on the DEAP benchmarking dataset. Experimental results demonstrate that the proposed framework outperforms the classical methods, with regard to both of the emotional dimensions of Valence and Arousal.

Viewing alternatives

Metrics

Item Actions

Export

You can export this page using these formats Digital Object Identifier - DOI

About

• Item ORO ID
• 52877
• Item Type
• Journal Item
• ISSN
• 1748-5681
• Project Funding Details

• Funded Project Name	Project ID	Funding Body
  Chinese National Program on Key Basic Research Project (973 Program)	2014CB744604	Not Set
  Chinese National Program on Key Basic Research Project (973 Program)	2013CB329303	Not Set
  Chinese National Program on Key Basic Research Project (973 Program)	2013CB329304	Not Set
  Chinese 863 Program	2015AA015403	Not Set
  Not Set	U1636203	Natural Science Foundation of China
  Not Set	61402324	Natural Science Foundation of China
  Tianjin Research Program of Application Foundation and Advanced Technology	15JCQNJC41700	Not Set

• Keywords
• affective computing; CNN; time series data analysis; EEG; emotion recognition; LSTM; multi-channel data fusion; multi-modal data fusion; physiological signal; RNN
• Academic Unit or School
• Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications
  Faculty of Science, Technology, Engineering and Mathematics (STEM)
• Research Group
• Centre for Research in Computing (CRC)
• Copyright Holders
• © 2017 Inderscience Enterprises Ltd.
• Depositing User
• Dawei Song