The Open UniversitySkip to content
 

Deep fusion of multi-channel neurophysiological signal for emotion recognition and monitoring

Li, Xiang; Song, Dawei; Zhang, Peng; Hou, Yuexian and Hu, Bin (2017). Deep fusion of multi-channel neurophysiological signal for emotion recognition and monitoring. International Journal of Data Mining and Bioinformatics, 18(1) pp. 1–27.

Full text available as:
[img]
Preview
PDF (Accepted Manuscript) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (2MB) | Preview
DOI (Digital Object Identifier) Link: https://doi.org/10.1504/IJDMB.2017.086097
Google Scholar: Look up in Google Scholar

Abstract

How to fuse multi-channel neurophysiological signals for emotion recognition is emerging as a hot research topic in community of Computational Psychophysiology. Nevertheless, prior feature engineering based approaches require extracting various domain knowledge related features at a high time cost. Moreover, traditional fusion method cannot fully utilise correlation information between different channels and frequency components. In this paper, we design a hybrid deep learning model, in which the 'Convolutional Neural Network (CNN)' is utilised for extracting task-related features, as well as mining inter-channel and inter-frequency correlation, besides, the 'Recurrent Neural Network (RNN)' is concatenated for integrating contextual information from the frame cube sequence. Experiments are carried out in a trial-level emotion recognition task, on the DEAP benchmarking dataset. Experimental results demonstrate that the proposed framework outperforms the classical methods, with regard to both of the emotional dimensions of Valence and Arousal.

Item Type: Journal Item
Copyright Holders: 2017 Inderscience Enterprises Ltd.
ISSN: 1748-5681
Project Funding Details:
Funded Project NameProject IDFunding Body
Chinese National Program on Key Basic Research Project (973 Program)2014CB744604Not Set
Chinese National Program on Key Basic Research Project (973 Program)2013CB329303Not Set
Chinese National Program on Key Basic Research Project (973 Program)2013CB329304Not Set
Chinese 863 Program2015AA015403Not Set
Not SetU1636203Natural Science Foundation of China
Not Set61402324Natural Science Foundation of China
Tianjin Research Program of Application Foundation and Advanced Technology15JCQNJC41700Not Set
Keywords: affective computing; CNN; time series data analysis; EEG; emotion recognition; LSTM; multi-channel data fusion; multi-modal data fusion; physiological signal; RNN
Academic Unit/School: Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications
Faculty of Science, Technology, Engineering and Mathematics (STEM)
Research Group: Centre for Research in Computing (CRC)
Item ID: 52877
Depositing User: Dawei Song
Date Deposited: 12 Jan 2018 15:44
Last Modified: 02 May 2018 14:37
URI: http://oro.open.ac.uk/id/eprint/52877
Share this page:

Metrics

Altmetrics from Altmetric

Citations from Dimensions

Download history for this item

These details should be considered as only a guide to the number of downloads performed manually. Algorithmic methods have been applied in an attempt to remove automated downloads from the displayed statistics but no guarantee can be made as to the accuracy of the figures.

Actions (login may be required)

Policies | Disclaimer

© The Open University   contact the OU