Full-body Texture Reconstruction of Clothed Human Body
DOI:
https://doi.org/10.54097/5macyr68Keywords:
Human reconstruction, Normal mapping, Texture, Implicit function.Abstract
In order to solve the distortion problem in monocular reconstruction, a new solution is proposed in this paper. In particular, the space occupancy field of the human body can be obtained by combining normal maps and symbol distance field. Meanwhile, transformer encoder and decoder are used to extract feature vectors from input images and feed them into MLP, and finally the whole texture including invisible areas can be inferred. Experiments show that the proposed method makes good progress in the reconstruction of human body based on monocular RGB, even with good robustness for invisible regions.
Downloads
References
Loper M, Mahmood N, Romero J, et al. SMPL: A Skinned Multi-Person Linear Model[J]. ACM Transactions on Graphics, 2015, 34(6).
Osman A A A, Bolkart T, Black M J. Star: Sparse trained articulated human body regressor[C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16. Springer International Publishing, 2020: 598-613.
Pavlakos G, Choutas V, Ghorbani N, et al. Expressive body capture: 3d hands, face, and body from a single image[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 10975-10985.
Corona E, Pumarola A, Alenya G, et al. Smplicit: Topology-aware generative model for clothed people[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 11875-11885.
Saito S, Huang Z, Natsume R, et al. Pifu: Pixel-aligned implicit function for high-resolution clothed human digitization[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2019: 2304-2314.
Saito S, Simon T, Saragih J, et al. Pifuhd: Multi-level pixel-aligned implicit function for high-resolution 3d human digitization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 84-93.
Xiu Y, Yang J, Tzionas D, et al. Icon: Implicit clothed humans obtained from normals[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022: 13286-13296.
Isola P, Zhu J Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 1125-1134.
Mildenhall B, Srinivasan P P, Tancik M, et al. Nerf: Representing scenes as neural radiance fields for view synthesis[J]. Communications of the ACM, 2021, 65(1): 99-106.
Peng S, Zhang Y, Xu Y, et al. Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021: 9054-9063.
Liu L, Habermann M, Rudnev V, et al. Neural actor: Neural free-view synthesis of human actors with pose control[J]. ACM transactions on graphics (TOG), 2021, 40(6): 1-16.
Weng C Y, Curless B, Srinivasan P P, et al. Humannerf: Free-viewpoint rendering of moving people from monocular video[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern Recognition. 2022: 16210-16220
Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. Accelerating 3D deep learning with PyTorch3D. arXiv:2007.08501, 2020. 3
Yi Wang, Xin Tao, Xiaojuan Qi, Xiaoyong Shen, and JiayaJia. Image inpainting via generative multi-column convolutional neural networks. Conference on Neural Information Processing Systems (NeurIPS), 2018.3
Yao Feng, Vasileios Choutas, Timo Bolkart, Dimitrios Tzionas, and Michael J. Black. Collaborative regression of expressive bodies using moderation. In International Conference on 3D Vision (3DV), 2021. 3,4
Jiang B, Hong Y, Bao H, et al. Selfrecon: Self reconstruction your digital avatar from monocular video[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2022: 5605-5615.
Shunsuke Saito, Zeng Huang, Ryota Natsume, Shigeo Morishima, Hao Li, and Angjoo Kanazawa. PIFu: Pixel-aligned implicit function for high-resolution clothed human digitization. In ICCV, pages 2304–2314, 2019. 2, 3, 5, 6, 7, 13, 14, 16
Tao Yu, Zerong Zheng, Kaiwen Guo, Pengpeng Liu, Qionghai Dai, and Yebin Liu. Function4D: Real-time human volumetric capture from very sparse consumer RGBD sensors. In Computer Vision and Pattern Recognition (CVPR), 2021. 6, 7, 10
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Academic Journal of Science and Technology
This work is licensed under a Creative Commons Attribution 4.0 International License.
