Analysis of Pruning Optimization Technology Based on Deep Learning
DOI:
https://doi.org/10.54097/hset.v4i.921Keywords:
channel pruning, Neuron pruning, Weightpruning, layer pruningAbstract
Neural network models have become a major topic in many areas in recent years due to their excellent problem-solving ability. However, during the training process in the real application, neural network models usually have relatively high requirements on the performance of the platform, which limits their applications, especially in battery-driven devices or embedded platforms with relatively weak performance. Therefore, the importance of optimizing and pruning neural network models is very important. This paper analyzes four categories of pruning techniques, including channel pruning, neuron pruning, weight pruning and layer pruning. This paper investigates its main ideas and particular pruning methods and compares the optimised effects of these pruning techniques in different neural network models through experiments. This paper discusses neuron pruning on the VGG-16 model, weight pruning on ResNet and AlexNet, and layer pruning on the YOLOV3 model, respectively. The parameter amount and inference speed before and after pruning are compared, and the accuracies drop before and after pruning are verified on data sets such as CIFAR-10.
Downloads
References
Y.,He, X.,Zhang, &J. Sun, (2017). Channel pruning for accelerating very deep neural networks. In Proceedings of the IEEE international conference on computer vision (pp. 1389-1397).
J. Luo, H., J., Wu, &W. Lin, (2017). Thanet: A filter level pruning method for deep neural network compression. In Proceedings of the IEEE international conference on computer vision (pp. 5058-5066).
Z., Liu, J., Li, Z., Shen, G., Huang, S., Yan, &C. Zhang, (2017). Learning efficient convolutional networks through network slimming. In Proceedings of the IEEE international conference on computer vision (pp. 2736-2744).
H Hu, R Peng, YW Tai, CK Tang.: Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures. arXiv preprint arXiv:1607.03250, (2016)
Miguel Á. Carreira-Perpiñán, YerlanIdelbayev; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 8532-8541
S., Han, J., Pool, J.,Tran,W. Dally,: Learning both weights and connections for the efficient neural network. In: Advances in Neural Information Processing Systems (NIPS). pp. 1135–1143 (2015)
T.J., Yang, Y.H., Chen, V.Sze,: Designing energy-efficient convolutional neural networks using energy-aware pruning. arXiv preprint arXiv:1611.05128 (2016)
W., Wen, C., Wu, Y., Wang, Y., Chen, H.Li,: Learning structured sparsity in deep neural networks. In: Advances in Neural Information Processing Systems. pp. 2074–2082 (2016)
X., Gong, L.,Ma, &H. Ouyang, (2020, February). An improved method of Tiny YOLOV3. In IOP Conference Series: Earth and Environmental Science (Vol. 440, No. 5, p. 052025). IOP Publishing.
J., Redmon, &A. Farhadi, (2018). Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767.
Q. -C. Mao, H. -M. Sun, Y. -B. Liu and R. -S. Jia, "Mini-YOLOv3: Real-Time Object Detector for Embedded Applications," in IEEE Access, vol. 7, pp. 133529-133538, 2019, doi: 10.1109/ACCESS.2019.2941547.
X. Ma, G. Yuan, S. Lin, Z. Li, H. Sun and Y. Wang, "ResNet Can Be Pruned 60×: Introducing Network Purification and Unused Path Removal (P-RM) after Weight Pruning," 2019 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH), 2019, pp. 1-2, doi: 10.1109/NANOARCH47378.2019.181304.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
