“Mesh Density Adaptation for Template-based Shape Reconstruction” by Jung, Kim, Hwang, Baek and Lee
Conference:
Type:
Title:
- Mesh Density Adaptation for Template-based Shape Reconstruction
Session/Category Title: All About Meshes
Presenter(s)/Author(s):
Moderator(s):
Abstract:
In 3D shape reconstruction based on template mesh deformation, a regularization, such as smoothness energy, is employed to guide the reconstruction into a desirable direction. In this paper, we highlight an often overlooked property in the regularization: the vertex density in the mesh. Without careful control on the density, the reconstruction may suffer from under-sampling of vertices near shape details. We propose a novel mesh density adaptation method to resolve the under-sampling problem. Our mesh density adaptation energy increases the density of vertices near complex structures via deformation to help reconstruction of shape details. We demonstrate the usability and performance of mesh density adaptation with two tasks, inverse rendering and non-rigid surface registration. Our method produces more accurate reconstruction results compared to the cases without mesh density adaptation. Our code is available at https://github.com/ycjungSubhuman/density-adaptation.
References:
1. Brett Allen, Brian Curless, and Zoran Popović. 2003. The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans. Graph. 22, 3 (2003), 587–594.
2. Pierre Alliez, David Cohen-Steiner, Olivier Devillers, Bruno Lévy, and Mathieu Desbrun. 2003. Anisotropic polygonal remeshing. In Proc. SIGGRAPH. 485–493.
3. Brian Amberg, Sami Romdhani, and Thomas Vetter. 2007. Optimal step nonrigid ICP algorithms for surface registration. In Proc. CVPR. IEEE, 1–8.
4. David Baraff and Andrew Witkin. 1998. Large steps in cloth simulation. In Proc. SIGGRAPH. 43–54.
5. Volker Blanz and Thomas Vetter. 1999. A morphable model for the synthesis of 3D faces. In Proc. SIGGRAPH. 187–194.
6. Federica Bogo, Javier Romero, Matthew Loper, and Michael J Black. 2014. FAUST: Dataset and evaluation for 3D mesh registration. In Proc. CVPR. 3794–3801.
7. Federica Bogo, Javier Romero, Gerard Pons-Moll, and Michael J Black. 2017. Dynamic FAUST: Registering human bodies in motion. In Proc. CVPR. 6233–6242.
8. James Booth, Anastasios Roussos, Allan Ponniah, David Dunaway, and Stefanos Zafeiriou. 2018. Large scale 3d morphable models. International Journal of Computer Vision 126, 2 (2018), 233–254.
9. Mario Botsch, Leif Kobbelt, Mark Pauly, Pierre Alliez, and Bruno Lévy. 2010a. Polygon mesh processing. A K Peteres, Ltd., Natick, Massachusetts, US, 85–110.
10. Mario Botsch, Leif Kobbelt, Mark Pauly, Pierre Alliez, and Bruno Lévy. 2010b. Polygon mesh processing. A K Peteres, Ltd., Natick, Massachusetts, US, 37–40.
11. Frédéric Cazals and Joachim Giesen. 2006. Delaunay triangulation based surface reconstruction. In Effective computational geometry for curves and surfaces. Springer, 231–276.
12. Long Chen and Michael Holst. 2011. Efficient mesh optimization schemes based on optimal Delaunay triangulations. Comput. Methods Appl. Mech. Eng. 200, 9-12 (2011), 967–984.
13. Rongshan Chen, Xiang Yin, Yuancheng Yang, and Chao Tong. 2022. Multi-view Pixel2Mesh++: 3D reconstruction via Pixel2Mesh with more images. The Visual Computer (2022), 1–14.
14. Zhonggui Chen, Juan Cao, and Wenping Wang. 2012. Isotropic surface remeshing using constrained centroidal Delaunay mesh. In Computer Graphics Forum, Vol. 31. Wiley Online Library, 2077–2085.
15. Hang Dai, Nick Pears, and William Smith. 2018. Non-rigid 3D shape registration using an adaptive template. In Proc. ECCVW. 0–0.
16. Tamal K Dey and Tathagata Ray. 2010. Polygonal surface remeshing with Delaunay refinement. Engineering with computers 26, 3 (2010), 289–301.
17. Qiang Du, Vance Faber, and Max Gunzburger. 1999. Centroidal Voronoi tessellations: Applications and algorithms. SIAM review 41, 4 (1999), 637–676.
18. Rana Hanocka, Gal Metzer, Raja Giryes, and Daniel Cohen-Or. 2020. Point2Mesh: a self-prior for deformable meshes. ACM Trans. Graph. 39, 4 (2020), 126–1.
19. David A Hirshberg, Matthew Loper, Eric Rachlin, and Michael J Black. 2012. Coregistration: Simultaneous alignment and modeling of articulated 3D shape. In Proc. ECCV. Springer, 242–255.
20. Wenzel Jakob, Marco Tarini, Daniele Panozzo, and Olga Sorkine-Hornung. 2015. Instant field-aligned meshes. ACM Trans. Graph. 34, 6 (2015), 1–15.
21. Hiroharu Kato, Yoshitaka Ushiku, and Tatsuya Harada. 2018. Neural 3d mesh renderer. In Proc. CVPR. 3907–3916.
22. Michael Kazhdan and Hugues Hoppe. 2013. Screened poisson surface reconstruction. ACM Trans. Graph. 32, 3 (2013), 1–13.
23. Dawar Khan, Alexander Plopski, Yuichiro Fujimoto, Masayuki Kanbara, Gul Jabeen, Yongjie Jessica Zhang, Xiaopeng Zhang, and Hirokazu Kato. 2020. Surface remeshing: A systematic literature review of methods and research directions. IEEE Trans. Vis. Comput. Graph. 28, 3 (2020), 1680–1713.
24. Diederik P Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Proc. ICLR.
25. Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. 2020. Modular primitives for high-performance differentiable rendering. ACM Trans. Graph. 39, 6 (2020), 1–14.
26. Chang Ha Lee, Amitabh Varshney, and David W. Jacobs. 2005. Mesh Saliency. In Proc. SIGGRAPH (Los Angeles, California) (SIGGRAPH ’05). Association for Computing Machinery, New York, NY, USA, 659–666. https://doi.org/10.1145/1186822.1073244
27. Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. 2017. Learning a model of facial shape and expression from 4D scans.ACM Trans. Graph. 36, 6 (2017), 194–1.
28. Selena Zihan Ling, Nicholas Sharp, and Alec Jacobson. 2022. VectorAdam for Rotation Equivariant Geometry Optimization. NeurIPS 35 (2022), 4111–4122.
29. Fujun Luan, Shuang Zhao, Kavita Bala, and Zhao Dong. 2021. Unified shape and svbrdf recovery using differentiable monte carlo rendering. In Computer Graphics Forum, Vol. 40. Wiley Online Library, 101–113.
30. Jiahui Lyu, Bojian Wu, Dani Lischinski, Daniel Cohen-Or, and Hui Huang. 2020. Differentiable refraction-tracing for mesh reconstruction of transparent objects. ACM Trans. Graph. 39, 6 (2020), 1–13.
31. Andrew Nealen, Takeo Igarashi, Olga Sorkine, and Marc Alexa. 2006. Laplacian mesh optimization. In Proc. GRAPHITE. 381–389.
32. Baptiste Nicolet, Alec Jacobson, and Wenzel Jakob. 2021. Large steps in inverse rendering of geometry. ACM Trans. Graph. 40, 6 (2021), 1–13.
33. Werner Palfinger. 2022. Continuous remeshing for inverse rendering. Computer Animation and Virtual Worlds 33, 5 (2022), e2101.
34. Pascal Paysan, Reinhard Knothe, Brian Amberg, Sami Romdhani, and Thomas Vetter. 2009. A 3D face model for pose and illumination invariant face recognition. In Proc. AVSS. Ieee, 296–301.
35. Yuda Qiu, Xiaojie Xu, Lingteng Qiu, Yan Pan, Yushuang Wu, Weikai Chen, and Xiaoguang Han. 2021. 3dcaricshop: A dataset and a baseline method for single-view 3d caricature face reconstruction. In Proc. CVPR. 10236–10245.
36. David C Schneider and Peter Eisert. 2009. Fast nonrigid mesh registration with a data-driven deformation prior. In Proc. ICCVW. IEEE, 304–311.
37. Matan Sela, Elad Richardson, and Ron Kimmel. 2017. Unrestricted facial geometry reconstruction using image-to-image translation. In Proc. ICCV. 1576–1585.
38. Olga Sorkine. 2009. Least-squares rigid motion using svd. Technical notes 120, 3 (2009), 52.
39. Olga Sorkine, Daniel Cohen-Or, Yaron Lipman, Marc Alexa, Christian Rössl, and H-P Seidel. 2004. Laplacian surface editing. In Proc. EG. 175–184.
40. Gabriel Taubin. 1995. Curve and surface smoothing without shrinkage. In Proc. ICCV. IEEE, 852–857.
41. Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. 2018. Pixel2mesh: Generating 3d mesh models from single rgb images. In Proc. ECCV. 52–67.
42. Chao Wen, Yinda Zhang, Zhuwen Li, and Yanwei Fu. 2019. Pixel2mesh++: Multi-view 3d mesh generation via deformation. In Proc. ICCV. 1042–1051.
43. Jingyu Yang, Ke Li, Kun Li, and Yu-Kun Lai. 2015. Sparse Non-rigid Registration of 3D Shapes. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 89–99.
44. Yizhou Yu, Kun Zhou, Dong Xu, Xiaohan Shi, Hujun Bao, Baining Guo, and Heung-Yeung Shum. 2004. Mesh editing with poisson-based gradient field manipulation. In Proc. SIGGRAPH. 644–651.
45. Silvia Zuffi, Angjoo Kanazawa, David W Jacobs, and Michael J Black. 2017. 3D menagerie: Modeling the 3D shape and pose of animals. In Proc. CVPR. 6365–6373.
