A Hybrid Model for Image Aesthetic Assessment Integrating Theme Prediction and ViT-based Composition Analysis

Abstract

This paper proposes a novel hybrid model for Image Aesthetic Assessment (IAA), which integrates theme prediction and Vision Transformer (ViT)-based composition analysis. Unlike traditional deep learning methods that rely solely on aesthetic scores for training, this approach combines semantic and photographic themes with the spatial relationships of visual elements. By leveraging theme labels and compositional relationships, the model captures aesthetic qualities from a more human-centric perspective. The model comprises three modules: a pre-trained Theme Understanding Network (TUN) to extract semantic and photographic theme features, a ViT module to analyze the spatial dependencies of visual elements, and a scoring module to concatenate features from the previous modules and then generate the prediction of images’ aesthetic score. This integration enhances interpretability by providing attention heat maps and detailed theme and semantic labels, facilitating a deeper exploration of human aesthetic intelligence. The Theme and Composition-aware IAA (TC-IAA) model achieves a Mean Squared Error (MSE) of 0.47 on the Aesthetic Visual Analysis (AVA) dataset, demonstrating its effectiveness. Its practical implications extend to iamges aesthetic recommendation, photo-editing, and targeted image advertising, highlighting its potential impact on real-world applications.