A Survey of Few-Shot Learning Research Based on Deep Neural Network

Pengjin Wu

doi:10.54097/fcis.v2i1.3177

Authors

Pengjin Wu

DOI:

https://doi.org/10.54097/fcis.v2i1.3177

Keywords:

Few-shot learning, Low-shot learning, Deep learning, Deep neural network

Abstract

With the successful development of deep learning techniques in recent years, deep neural networks have achieved excellent results in both computer vision and natural language processing by relying on large-scale datasets but still face significant challenges in solving the problem of learning from few-shot. Inspired by the ability of humans to learn to recognize objects as a way to simulate the cognitive process of learning from a small sample size, few-shot learning is a hot topic of research in deep neural networks today. It is also a significant and challenging problem. This paper first introduces the research background and definition of few-shot learning, introduces the relevant models, and summarizes and analyzes the common approaches to the problem of few-shot learning based on deep neural networks at the present stage, which are divided into four types: data augmentation, model fine-tuning, metric learning and meta-learning. Finally, popular datasets for few-shot learning are described, the paper is concluded and future research directions are discussed.

Downloads

Download data is not yet available.

References

Biederman, I. (1987). Recognition-by-components: A theory of human image understanding. Psychological Review, 94(2), 115–147. https://doi.org/10.1037/0033-295X.94.2.115

Fu, Y., Xiang, L., Zahid, Y., Ding, G., Mei, T., Shen, Q., & Han, J. (2022). Long-tailed visual recognition with deep models: A methodological survey and evaluation. Neurocomputing.

Fe-Fei, L. (2003). A Bayesian approach to unsupervised one-shot learning of object categories. In proceedings ninth IEEE international conference on computer vision (pp. 1134-1141). IEEE.

Wang, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2020). Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur), 53(3), 1-34.

Hinton, G. E., & Salakhutdinov, R. R. (2006). Reducing the dimensionality of data with neural networks. science, 313(5786), 504-507.

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).

Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., ... & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1-9).

Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.

LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.

Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences, 79(8), 2554-2558.

Jordan, M. I. (1986). SERIAL ORDER: A PARALLEL DISTRmUTED PROCESSING APPROACH.

Elman, J. L. (1990). Finding structure in time. Cognitive science, 14(2), 179-211.

Palangi, H., Deng, L., Shen, Y., Gao, J., He, X., Chen, J., ... & Ward, R. (2016). Deep sentence embedding using long short-term memory networks: Analysis and application to information retrieval. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 24(4), 694-707.

Hariharan, B., & Girshick, R. (2017). Low-shot visual recognition by shrinking and hallucinating features. In Proceedings of the IEEE international conference on computer vision (pp. 3018-3027).

Wang, Y. X., Girshick, R., Hebert, M., & Hariharan, B. (2018). Low-shot learning from imaginary data. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7278-7286).

Howard, J., & Ruder, S. (2018). Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146.

Nakamura, A., & Harada, T. (2019). Revisiting fine-tuning for few-shot learning. arXiv preprint arXiv:1910.00216.

Bellet, A., Habrard, A., & Sebban, M. (2013). A survey on metric learning for feature vectors and structured data. arXiv preprint arXiv:1306.6709.

Weinberger, K. Q., & Saul, L. K. (2009). Distance metric learning for large margin nearest neighbor classification. Journal of machine learning research, 10(2).

Koch, G., Zemel, R., & Salakhutdinov, R. (2015, July). Siamese neural networks for one-shot image recognition. In ICML deep learning workshop (Vol. 2, p. 0).

Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016, June). Meta-learning with memory-augmented neural networks. In International conference on machine learning (pp. 1842-1850). PMLR.

Vinyals, O., Blundell, C., Lillicrap, T., & Wierstra, D. (2016). Matching networks for one shot learning. Advances in neural information processing systems, 29.

Snell, J., Swersky, K., & Zemel, R. (2017). Prototypical networks for few-shot learning. Advances in neural information processing systems, 30.

Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P. H., & Hospedales, T. M. (2018). Learning to compare: Relation network for few-shot learning. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1199-1208).

Naik, D. K., & Mammone, R. J. (1992, June). Meta-neural networks that learn by learning. In [Proceedings 1992] IJCNN International Joint Conference on Neural Networks (Vol. 1, pp. 437-442). IEEE.

Andrychowicz, M., Denil, M., Gomez, S., Hoffman, M. W., Pfau, D., Schaul, T., ... & De Freitas, N. (2016). Learning to learn by gradient descent by gradient descent. Advances in neural information processing systems, 29.

Vinyals, O., Blundell, C., Lillicrap, T., & Wierstra, D. (2016). Matching networks for one shot learning. Advances in neural information processing systems, 29.

Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015). Human-level concept learning through probabilistic program induction. Science, 350(6266), 1332-1338.

Khosla, A., Jayadevaprakash, N., Yao, B., & Li, F. F. (2011, June). Novel dataset for fine-grained image categorization: Stanford dogs. In Proc. CVPR workshop on fine-grained visual categorization (FGVC) (Vol. 2, No. 1). Citeseer.

Han, X., Zhu, H., Yu, P., Wang, Z., Yao, Y., Liu, Z., & Sun, M. (2018, January). FewRel: A Large-Scale Supervised Few-shot Relation Classification Dataset with State-of-the-Art Evaluation. In EMNLP.

Yu, M., Guo, X., Yi, J., Chang, S., Potdar, S., Cheng, Y., ... & Zhou, B. (2018, January). Diverse Few-Shot Text Classification with Multiple Metrics. In NAACL-HLT.

Geng, R., Li, B., Li, Y., Zhu, X., Jian, P., & Sun, J. (2019, November). Induction Networks for Few-Shot Text Classification. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 3904-3913).