Neuro-symbolic learning for dealing with sparsity in cultural heritage image archives: an empirical journey

Chiatti, Agnese and Daga, Enrico (2022). Neuro-symbolic learning for dealing with sparsity in cultural heritage image archives: an empirical journey. In: Proceedings of the Workshop on Deep Learning for Knowledge Graphs (DL4KG), co-located with the 21st International Semantic Web Conference (ISWC), 23-24 Oct 2022, Virtual, CEUR Workshop Proceedings 3342.

URL: https://iswc2022.semanticweb.org/

Abstract

Deep Learning (DL) methods have proved to be very successful for many image classification tasks. In the SPICE project, we are researching on an intelligent system that classifies artworks to support several tasks such as metadata curation and linking across image collections. However, applying DL methods to real-world cultural heritage collections for the task of artwork subject classification is problematic. Objects in this domain are characterised by different levels of heterogeneity: of media and techniques, of categories, of time-periods, just to mention a few. This heterogeneity makes the related training features sparsely distributed. In this paper, we report on an empirical investigation where we apply neuro-symbolic, Deep Learning techniques to a paradigmatic case of cultural heritage archive: the Tate Gallery collection open data. We pose the question of what type of feature engineering could help in reducing the impact of data sparsity in this domain. Crucially, we explore how neuro-symbolic learning, combining image features, textual metadata, and Knowledge Graph embeddings, could help in mitigating the problems derived from data sparsity in cultural heritage image archives.

Viewing alternatives

Item Actions

Export

You can export this page using these formats