Enhanced protein function prediction by fusion embedding based on protein language models
DOI:
https://doi.org/10.54097/hset.v66i.11697Keywords:
Protein Function; Language Models; Fusion Embeddings.Abstract
Natural language models can accomplish non-natural language tasks such as protein prediction, but the actual prediction effect is low and occupies large computational resources. In this paper, a fusion embedding model is proposed to improve the prediction effect of the model and reduce the computational cost of the model by fusing information of different dimensions. The paper is validated by the downstream task of protein function prediction, which provides a reference for solving practical tasks using fusion embedding methods.
Downloads
References
Jumper, J. M. et al. Highly accurate protein structure prediction with AlphaFold. Nature 596, 583–589 (2021).
Elnaggar, A. et al. ProtTrans: Toward Understanding the Language of Life through Self-Supervised Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 7112–7127 (2022).
Ashburner, M. et al. Gene Ontology: tool for the unification of biology. Nature Genetics 25, 25–29 (2000).
Wang, S., Peng, J., Ma, J. & Xu, J. Protein Secondary Structure Prediction Using Deep Convolutional Neural Fields. Scientific Reports 6, (2016).
Vaswani, A. et al. Attention is all you need. arXiv (Cornell University) vol. 30 5998–6008 (Cornell University, 2017).
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (Devlin et al., NAACL 2019).
Raffel, C. et al. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research 21, 1–67 (2020).
Gligorijević, V., Barot, M. & Bonneau, R. deepNF: deep network fusion for protein function prediction. Bioinformatics 34, 3873–3881 (2018).
Brandes, N., Ofer, D., Peleg, Y., Rappoport, N. & Linial, M. ProteinBERT: a universal deep-learning model of protein sequence and function. Bioinformatics 38, 2102–2110 (2022).
Dallago, C., Schütze, K., Heinzinger, M., Olenyi, T., Littmann, M., Lu, A. X., Yang, K. K., Min, S., Yoon, S., Morton, J. T., & Rost, B. (2021). Learned Embeddings from Deep Learning to Visualize and Predict Protein Sets. Current protocols, 1(5), e113.
Huang, S.-C., Pareek, A., Seyyedi, S., Rubin, D. L. & Lungren, M. P. Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines. Npj Digital Medicine 3, (2020).
Hirasawa, T., Yamagishi, H., Matsumura, Y. & Komachi, M. Multimodal Machine Translation with Embedding Prediction. (2019).
Lange, L., Adel, H., Strötgen, J. & Klakow, D. FAME: Feature-Based Adversarial Meta-Embeddings for Robust Input Representations. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (2021).
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
