The Open UniversitySkip to content
 

A Quantum-Inspired Multimodal Sentiment Analysis Framework

Zhang, Yazhou; Song, Dawei; Zhang, Peng; Wang, Panpan; Li, Jingfei; Li, Xiang and Wang, Benyou (2018). A Quantum-Inspired Multimodal Sentiment Analysis Framework. Theoretical Computer Science, 752 pp. 21–40.

Full text available as:
[img]
Preview
PDF (Accepted Manuscript) - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (2MB) | Preview
DOI (Digital Object Identifier) Link: https://doi.org/10.1016/j.tcs.2018.04.029
Google Scholar: Look up in Google Scholar

Abstract

Multimodal sentiment analysis aims to capture diversified sentiment information implied in data that are of different modalities (e.g., an image that is associated with a textual description or a set of textual labels). The key challenge is rooted on the “semantic gap” between different low-level content features and high-level semantic information. Existing approaches generally utilize a combination of multimodal features in a somehow heuristic way. However, how to employ and combine multiple information from different sources effectively is still an important yet largely unsolved problem. To address the problem, in this paper, we propose a Quantum-inspired Multimodal Sentiment Analysis (QMSA) framework. The framework consists of a Quantum-inspired Multimodal Representation (QMR) model (which aims to fill the “semantic gap” and model the correlations between different modalities via density matrix), and a Multimodal decision Fusion strategy inspired by Quantum Interference (QIMF) in the double-slit experiment (in which the sentiment label is analogous to a photon, and the data modalities are analogous to slits). Extensive experiments are conducted on two large scale datasets, which are collected from the Getty Images and Flickr photo sharing platform. The experimental results show that our approach significantly outperforms a wide range of baselines and state-of-the-art methods.

Item Type: Journal Item
ISSN: 0304-3975
Keywords: Multimodal sentiment analysis; Quantum theory; Decision fusion; Information fusion
Academic Unit/School: Faculty of Science, Technology, Engineering and Mathematics (STEM) > Computing and Communications
Faculty of Science, Technology, Engineering and Mathematics (STEM)
Item ID: 54814
Depositing User: ORO Import
Date Deposited: 26 Apr 2018 12:28
Last Modified: 25 May 2019 19:15
URI: http://oro.open.ac.uk/id/eprint/54814
Share this page:

Metrics

Altmetrics from Altmetric

Citations from Dimensions

Download history for this item

These details should be considered as only a guide to the number of downloads performed manually. Algorithmic methods have been applied in an attempt to remove automated downloads from the displayed statistics but no guarantee can be made as to the accuracy of the figures.

Actions (login may be required)

Policies | Disclaimer

© The Open University   contact the OU