Analysis on Lightweight Network Methods and Technologies
DOI:
https://doi.org/10.54097/hset.v4i.922Keywords:
Depthwise Separable Convolution, Grouped Convolution, Model Compression, NAS, AutoMLAbstract
There are currently two main schools of deep learning. One is academic. They pursue stronger performance through powerful, complex models. The other is the engineering school. Their purpose is to efficiently deploy models to various hardware platforms. Complex models have better performance. However, it also brings unavoidable consumption. With the increasing depth of convolutional neural networks, lightweighting has become a key research direction. There are currently four main methods for designing lightweight networks. This article will first introduce CNN model compression and basic convolution operations.This paper also introduces the model compression based on AutoML and the automatic animation design based on NAS. Finally, according to the above three points, this paper introduces the application of the above methods in artificially designed neural networks.This paper mainly introduces the step-by-step evolution of the existing methods. This paper analyzes aspects of current neural network improvements and emerging problems. The significance of this paper is to summarize and deepen the solved problems and key problems in the lightweight process through past experience.
Downloads
References
F, Chollet. (2017). Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1251-1258).
A. M., Saxe, Y, Bansal., j, Dapello, M, Advani., A., Kolchinsky, B. D., Tracey, & D. D., Cox (2019). On the information bottleneck theory of deep learning. Journal of Statistical Mechanics: Theory and Experiment, 2019(12), 124020.
S., Han, H., Mao, & W. J., Dally (2015). Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149.
Y., He, J., Lin, Z, Liu, H., Wang, L. J., Li, & S., Han (2018). Amc: Automl for model compression and acceleration on mobile devices. In Proceedings of the European conference on computer vision (ECCV) (pp. 784-800).
M., Tan, B., Chen, R., Pang, V., Vasudevan, M., Sandler, A., Howard, & Q. V., Le (2019). Mnasnet: Platform-aware neural architecture search for mobile. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 2820-2828).
F. N., Iandola, S. Han, M. W., Moskewicz, K., Ashraf, W. J., Dally, & K., Keutzer(2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 0.5 MB model size. arXiv preprint arXiv:1602.07360.
A., Gholami, K., Kwon, B., Wu, Z., Tai, X., Yue, P., Jin, ... & K., Keutzer (2018). Squeezenext: Hardware-aware neural network design. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 1638-1647).
X., Zhang, X., Zhou, M., Lin, & J., Sun (2018). Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 6848-6856).
C., Szegedy, S., Ioffe, V., Vanhoucke,, & A. A., Alemi (2017, February). Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence.
N., Ma, X., Zhang, H. T., Zheng, & J., Sun (2018). Shufflenet v2: Practical guidelines for efficient cnn architecture design. In Proceedings of the European conference on computer vision (ECCV) (pp. 116-131).
A. G., Howard, M., Zhu, B., Chen, D., Kalenichenko, W., Wang, T., Weyand, ... & H., Adam (2017). Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861.
K., Simonyan, & A., Zisserman (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
M., Sandler, A., Howard, M., Zhu, A.,Zhmoginov, & L. C., Chen (2018). Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4510-4520).
A., Howard, M., Sandler, G., Chu, L. C.,Chen, B., Chen, M., Tan, ... & H., Adam (2019). Searching for mobilenetv3. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 1314-1324).
S., Watanabe, T. Hori, S., Karita, T., Hayashi, J., Nishitoba, Y., Unno, ... & T., Ochiai (2018). Espnet: End-to-end speech processing toolkit. arXiv preprint arXiv:1804.00015.
Downloads
Published
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
