Research on Sketching Face Headshot Generation based on Improved CycleGAN

Zhen Liao; Guojun Lin

doi:10.54097/fcis.v4i1.9451

Authors

Zhen Liao
Guojun Lin

DOI:

https://doi.org/10.54097/fcis.v4i1.9451

Keywords:

Feature Extraction, UNet Self-encoder, Sketch Head, Supervised Learning, CycleGAN

Abstract

At present, sketch heads generated from realistic heads still has problems such as blurred contours and missing textures. For this reason, this work proposes a sketch head generation method based on CycleGAN. Firstly, the Self-Attention Mechanism (Squeeze-and-Excitation Networks (SENet) module is added to the UNet self-encoder; secondly, the base model is transformed into a supervised learning model so as to add constraints on the generated avatars and the real avatars. The experimental results show that the sketched avatar generated by the method in this paper has a better visual effect on the CUHK student test set with a 0.0274 improvement in SSIM value than the sketched avatar generated by the base model.

Downloads

Download data is not yet available.

References

Zhang S, Gao X, Wang N, et al. Face sketch synthesis via sparse representation-based greedy search [J]. IEEE transactions on image processing, 2015, 24(8): 2466-2477.

Chang L, Zhou M, Deng X, et al. Face sketch synthesis via multivariate output regression [C] // Human-Computer Interaction. Design and Development Approaches: 14th International Conference, HCI International 2011, Orlando, FL, USA, July 9-14, 2011, Proceedings, Part I 14. Springer Berlin Heidelberg, 2011: 555-561.

Wang X, Tang X. Face photo-sketch synthesis and recognition [J]. IEEE transactions on pattern analysis and machine intelligence, 2008, 31(11): 1955-1967.

Roweis S T, Saul L K. Nonlinear dimensionality reduction by locally linear embedding[J]. science, 2000, 290(5500): 2323-2326.

Zhou H Q, Cao L, Du C N. Multi-discriminator recurrent generative adversarial network for sketch face synthesis[J]. Computer Engineering and Applications,2021,57(03):231-238.

Creswell A, White T, Dumoulin V, et al. Generative adversarial networks: An overview[J]. IEEE signal processing magazine, 2018, 35(1): 53-65.

Yan B., Zhang J. Lin. A study on the status of image translation based on generative adversarial networks[J]. Foreign Electronic Measurement Technology,2019,38(06): 130-134. DOI: 10.19652/j.cnki.femt.1801330.

Xia Guangyou. Research on colorization method of grayscale images of farmers' paintings based on generative adversarial network [D]. Qinghai Normal University, 2022. DOI: 10. 27778 / d.cnki.gqhzy.2022.000550.

Isola P, Zhu J Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1125-1134.

Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]// Proceedings of the IEEE international conference on computer vision. 2017: 2223-2232.

Chu C, Zhmoginov A, Sandler M. Cyclegan, a master of steganography [J]. arXiv preprint arXiv:1712.02950, 2017.

Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer International Publishing, 2015: 234-241.

Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]// Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 7132-7141.

Mao X, Li Q, Xie H, et al. Least squares generative adversarial networks [C]// Proceedings of the IEEE international conference on computer vision. 2017: 2794-2802.

Wang L T, Hoover N E, Porter E H, et al. SSIM: A software levelized compiled-code simulator[C]//Proceedings of the 24th ACM/IEEE Design Automation Conference. 1987: 2-8.

Roweis S T, Saul L K. Nonlinear dimensionality reduction by locally linear embedding[J]. science, 2000, 290(5500): 2323-2326.

Anoosheh A, Agustsson E, Timofte R, et al. Combogan: Unrestrained scalability for image domain translation [C]// Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 2018: 783-79.