Recommendation Algorithm Fusing Multimodal Auxiliary Information for Multi-view Social Recommendation
DOI:
https://doi.org/10.54097/p9dwaw40Keywords:
Social Recommendation, Multimodal Fusion, Attention MechanismAbstract
Aiming at the problems of data sparsity, cold start and the limitation of single-modal auxiliary information in recommendation systems, this paper proposes a multi-view social recommendation algorithm that fuses multimodal auxiliary information. The algorithm consists of a user-consistent social recommendation module and a multimodal information fusion module. The former dynamically measures the preference similarity between the target user and social neighbors with an attention mechanism, selects preference-consistent neighbors through a differentiable gate for neighborhood aggregation, and extracts item features from both the knowledge graph view and the user-item interaction view, which are integrated by a cross-attention fusion unit. The latter encodes user raw features into low-dimensional dense vectors with multi-layer perceptrons, performs multi-layer information propagation with residual connections, and adopts attention-based late fusion for textual and visual modalities so that modality weights adapt to users and scenarios dynamically. Experiments are conducted on two public datasets, Yelp Open and LastFM-2k, compared with six baselines (BPR-MF, NeuMF, LightGCN, SocialMF, MMGCN, VBPR) using Recall, NDCG and Precision as evaluation metrics. Experimental results show that the proposed algorithm achieves the best performance on both datasets, and ablation studies verify the effectiveness of user consistency filtering, social aggregation, multi-view item feature extraction and dynamic multimodal fusion.
Downloads
References
[1] Resnick, P., & Varian, H. R. (1997). Recommender systems. Communications of the ACM, 40(3), 56 58. https://doi.org/10. 1145/ 245108.245126. DOI: https://doi.org/10.1145/245108.245121
[2] Adomavicius, G., & Tuzhilin, A. (2005). Toward the next generation of recommender systems: A survey of the state of the art and possible extensions. IEEE Transactions on Knowledge and Data Engineering, 17(6), 734 749. https://doi. org/ 10. 1109/TKDE.2005.99. DOI: https://doi.org/10.1109/TKDE.2005.99
[3] Su, X. Y., & Khoshgoftaar, T. M. (2009). A survey of collaborative filtering techniques. Advances in Artificial Intelligence, 2009, 421425. https://doi.org/ 10. 1155/ 2009/ 421425. DOI: https://doi.org/10.1155/2009/421425
[4] Wu, L., Sun, P. J., Hong, R. C., & Wang, M. (2018). SocialGCN: An efficient graph convolutional network based model for social recommendation. CoRR, abs/1811.02815.
[5] Yu, J. L., Yin, H. Z., Li, J. D., Wang, Q., & Hung, N. Q. V. (2021). Self supervised multi channel hypergraph convolutional network for social recommendation. In Proceedings of the Web Conference 2021 (pp. 413 424). https: // doi.org/10.1145/3442381.3449835. DOI: https://doi.org/10.1145/3442381.3449844
[6] BaltruĊĦaitis, T., Ahuja, C., & Morency, L. P. (2019). Multimodal machine learning: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(2), 423 443. https://doi.org/10.1109/TPAMI.2018.2798607. DOI: https://doi.org/10.1109/TPAMI.2018.2798607
[7] Ma, H., Yang, H. X., Lyu, M. R., & King, I. (2008). SoRec: Social recommendation using probabilistic matrix factorization. In Proceedings of the 17th ACM Conference on Information and Knowledge Management (pp. 931 940). https:// doi.org/ 10. 1145/ 1458083.1458205. DOI: https://doi.org/10.1145/1458082.1458205
[8] Yang, B., Lei, Y., Liu, J. M., & Li, X. (2017). Social collaborative filtering by trust. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(8), 1633 1647. https:// doi. org/10.1109/TPAMI.2016.2612259. DOI: https://doi.org/10.1109/TPAMI.2016.2605085
[9] Fan, W. Q., Ma, Y., Li, Q., He, Y., Zhao, E., Tang, J. L., & Yin, D. W. (2019). Graph neural networks for social recommendation. In Proceedings of the World Wide Web Conference (pp. 417 426). https://doi.org/ 10.1145/ 3308558. 3313488. DOI: https://doi.org/10.1145/3308558.3313488
[10] Wu, L., Li, J. W., Sun, P. J., Hong, R. C., Ge, Y., & Wang, M. (2022). DiffNet++: A neural influence and interest diffusion network for social recommendation. IEEE Transactions on Knowledge and Data Engineering, 34(10), 4753 4766. https:// doi. org/10.1109/TKDE.2021.3092445. DOI: https://doi.org/10.1109/TKDE.2020.3048414
[11] Zhang, F. Z., Yuan, N. J., Lian, D. F., Xie, X., & Sun, G. Z. (2016). Collaborative knowledge base embedding for recommender systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 353 362). https://doi.org/ 10. 1145/ 293 9672.2939673. DOI: https://doi.org/10.1145/2939672.2939673
[12] Wang, X., He, X. N., Cao, Y. X., Liu, M., & Chua, T. S. (2019). KGAT: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 950 958). https://doi.org/10. 1145/ 3292500. 3330989. DOI: https://doi.org/10.1145/3292500.3330989
[13] He, R. N., & McAuley, J. (2016). VBPR: Visual Bayesian personalized ranking from implicit feedback. In Proceedings of the 30th AAAI Conference on Artificial Intelligence (pp. 144 150). https://doi.org/10.1609/aaai.v30i1.9894. DOI: https://doi.org/10.1609/aaai.v30i1.9973
[14] Wei, Y. W., Wang, X., Nie, L. Q., & Chua, T. S. (2019). MMGCN: Multi modal graph convolution network for personalized recommendation of micro video. In Proceedings of the 27th ACM International Conference on Multimedia (pp. 1437 1445). https://doi.org/10.1145/3343031.3350940. DOI: https://doi.org/10.1145/3343031.3351034
[15] Rendle, S., Freudenthaler, C., Gantner, Z., & Schmidt Thieme, L. (2012). BPR: Bayesian personalized ranking from implicit feedback. CoRR, abs/1205.2618.
[16] He, X. N., Liao, L. Z., Zhang, H. W., Nie, L. Q., Hu, X., & Chua, T. S. (2017). Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web (pp. 173 182). https://doi.org/10. 1145/ 30389 12. 3052569. DOI: https://doi.org/10.1145/3038912.3052569
[17] He, X. N., Deng, K., Wang, X., Li, Y., Zhang, Y. D., & Wang, M. (2020). LightGCN: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 639 648). https: // doi.org/10.1145/3397271.3401063. DOI: https://doi.org/10.1145/3397271.3401063
[18] Jamali, M., & Ester, M. (2010). A matrix factorization technique with trust propagation for recommendation in social networks. In Proceedings of the 4th ACM Conference on Recommender Systems (pp. 135 142). https://doi.org/ 10. 1145/ 1864708.1864736. DOI: https://doi.org/10.1145/1864708.1864736
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Frontiers in Computing and Intelligent Systems
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
