“Appearance capture and modeling of human teeth” – ACM SIGGRAPH HISTORY ARCHIVES

“Appearance capture and modeling of human teeth”

  • 2018 SA Technical Papers_Velinov_Appearance capture and modeling of human teeth

Conference:


Type(s):


Title:

    Appearance capture and modeling of human teeth

Session/Category Title:   Modeling things on (and in) your head

Presenter(s)/Author(s):


Moderator(s):


Abstract:


    Recreating the appearance of humans in virtual environments for the purpose of movie, video game, or other types of production involves the acquisition of a geometric representation of the human body and its scattering parameters which express the interaction between the geometry and light propagated throughout the scene. Teeth appearance is defined not only by the light and surface interaction, but also by its internal geometry and the intra-oral environment, posing its own unique set of challenges. Therefore, we present a system specifically designed for capturing the optical properties of live human teeth such that they can be realistically re-rendered in computer graphics. We acquire our data in vivo in a conventional multiple camera and light source setup and use exact geometry segmented from intra-oral scans. To simulate the complex interaction of light in the oral cavity during inverse rendering we employ a novel pipeline based on derivative path tracing with respect to both optical properties and geometry of the inner dentin surface. The resulting estimates of the global derivatives are used to extract parameters in a joint numerical optimization. The final appearance faithfully recreates the acquired data and can be directly used in conventional path tracing frameworks for rendering virtual humans.

References:


    1. Jae Sung Ahn, Anjin Park, Ju Wan Kim, Byeong Ha Lee, and Joo Beom Eom. 2017. Development of Three-Dimensional Dental Scanning Apparatus Using Structured Illumination. In Sensors.Google Scholar
    2. F. O. Bartell, E. L. Dereniak, and W. L. Wolfe. 1981. The theory and measurement of bidirectional reflectance distribution function BRDF and bidirectional transmittance distribution function BTDF. In Radiation scattering in optical systems, G. H. Hunt (Ed.), Vol. 257. 154–160.Google Scholar
    3. Thabo Beeler. 2012. Passive Spatio-Temporal Geometry Reconstruction of Human Faces at Very High Fidelity. Ph.D. Dissertation. ETH Zurich / Disney Research Zurich. http://graphics.ethz.ch/~dbeeler/research/bee12c/PassiveSpatioTemporalGeometryReconstructionOfHumanFacesAtVeryHighFidelity.pdfGoogle Scholar
    4. T. Brox, A. Bruhn, N. Papenberg, and J. Weickert. 2004. High accuracy optical flow estimation based on a theory for warping. In ECCV. 25–36.Google Scholar
    5. Stéphanie I. Buchaillard, S. H. Ong, Yohan Payan, and Kelvin Foong. 2007. 3D Statistical Models for Tooth Surface Reconstruction. Comput. Biol. Med. 37, 10 (2007), 1461–1471. Google ScholarDigital Library
    6. C.N. Carter, R.J. Pusateri, Dongqing Chen, A.H. Ahmed, and A.A. Farag. 2010. Shape from shading for hybrid surfaces as applied to tooth reconstruction. In IEEE ICIP. 4049–4052.Google Scholar
    7. S. Chandrasekhar. 1960. Radiative Transfer. Dover Publications.Google Scholar
    8. Paul Debevec, Tim Hawkins, Chris Tchou, Haarm-Pieter Duiker, Westley Sarokin, and Mark Sagar. 2000. Acquiring the Reflectance Field of a Human Face. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH ’00). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 145–156. Google ScholarDigital Library
    9. Eugene d’Eon and Geoffrey Irving. 2011. A quantized-diffusion model for rendering translucent materials. 30, 4 (July 2011), 56:1–56:14. Google ScholarDigital Library
    10. Eustace L. Dereniak, Langford G. Brod, and John E. Hubbs. 1982. Bidirectional transmittance distribution function measurements on ZnSe. Appl. Opt. 21, 24 (Dec 1982), 4421–4425.Google ScholarCross Ref
    11. Craig Donner and Henrik Wann Jensen. 2005. Light Diffusion in Multi-layered Translucent Materials. In ACM SIGGRAPH 2005 Papers (SIGGRAPH ’05). ACM, New York, NY, USA, 1032–1039. Google ScholarDigital Library
    12. Craig Donner and Henrik Wann Jensen. 2008. Rendering Translucent Materials Using Photon Diffusion. In ACM SIGGRAPH 2008 Classes (SIGGRAPH ’08). ACM, New York, NY, USA, Article 4, 9 pages. Google ScholarDigital Library
    13. Craig Donner, Tim Weyrich, Eugene d’Eon, Ravi Ramamoorthi, and Szymon Rusinkiewicz. 2008. A Layered, Heterogeneous Reflectance Model for Acquiring and Rendering Human Skin. In ACM SIGGRAPH Asia 2008 Papers (SIGGRAPH Asia ’08). ACM, New York, NY, USA, Article 140, 12 pages. Google ScholarDigital Library
    14. David Eberly. 2015. Distance Between Point and Triangle in 3D. Technical Report. https://www.geometrictools.com/Documentation/DistancePoint3Triangle3.pdfGoogle Scholar
    15. Aly Farag, Shireen Elhabian, Aly Abdelrehim, Wael Aboelmaaty, Allan Farman, and David Tasman. 2013. Model-Based Human Teeth Shape Recovery from a Single Optical Image with Unknown Illumination. In Medical Computer Vision: Recognition Techniques and Applications in Medical Imaging (MCV ’12). 263–272. Google ScholarDigital Library
    16. Roald Frederickx and Philip Dutré. 2017. A Forward Scattering Dipole Model from a Functional Integral Approximation. ACM Trans. Graph. 36, 4, Article 109 (July 2017), 13 pages. Google ScholarDigital Library
    17. Daniel Fried, Richard E. Glena, John D. B. Featherstone, and Wolf Seka. 1995. Nature of light scattering in dental enamel and dentin at visible and near-infrared wavelengths. Journal of Applied Optics 34, 7 (1995), 1278–1285.Google ScholarCross Ref
    18. Jeppe Revall Frisvad, Toshiya Hachisuka, and Thomas Kim Kjeldsen. 2014. Directional Dipole Model for Subsurface Scattering. ACM Trans. Graph. 34, 1, Article 5 (Dec. 2014), 12 pages. Google ScholarDigital Library
    19. S. Garrido-Jurado, R. Mu noz Salinas, F.J. Madrid-Cuevas, and M.J. Marín-Jiménez. 2014. Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognition 47, 6 (2014), 2280 — 2292. Google ScholarDigital Library
    20. Abhijeet Ghosh, Tim Hawkins, Pieter Peers, Sune Frederiksen, and Paul Debevec. 2008. Practical modeling and acquisition of layered facial reflectance. In ACM Transactions on Graphics (TOG), Vol. 27. ACM, 139. Google ScholarDigital Library
    21. Ioannis Gkioulekas, Anat Levin, and Todd Zickler. 2016. An Evaluation of Computational Imaging Techniques for Heterogeneous Inverse Scattering., 685–701 pages.Google Scholar
    22. Ioannis Gkioulekas, Shuang Zhao, Kavita Bala, Todd Zickler, and Anat Levin. 2013. Inverse Volume Rendering with Material Dictionaries. ACM Trans. Graph. 32, 6, Article 162 (Nov. 2013), 13 pages. Google ScholarDigital Library
    23. D. Guarnera, G.C. Guarnera, A. Ghosh, C. Denk, and M. Glencross. 2016. BRDF Representation and Acquisition. Comput. Graph. Forum 35, 2 (May 2016), 625–650.Google ScholarCross Ref
    24. Ralf Habel, Per H. Christensen, and Wojciech Jarosz. 2013. Photon Beam Diffusion: A Hybrid Monte Carlo Method for Subsurface Scattering. 32, 4 (June 2013).Google Scholar
    25. Milovš Hašan and Ravi Ramamoorthi. 2013. Interactive Albedo Editing in Path-traced Volumetric Materials. ACM Trans. Graph. 32, 2, Article 11 (April 2013), 11 pages. Google ScholarDigital Library
    26. L. G. Henyey and J. L. Greenstein. 1941. Diffuse radiation in the Galaxy. Astrophys. J. 93 (1941), 70–83.Google ScholarCross Ref
    27. Nicolas Holzschuch. 2015. Accurate computation of single scattering in participating media with refractive boundaries. Computer Graphics Forum 34, 6 (Sept. 2015), 48–59. Google ScholarDigital Library
    28. A. Ilie and G. Welch. 2005. Ensuring color consistency across multiple cameras. In Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1, Vol. 2. 1268–1275 Vol. 2. Google ScholarDigital Library
    29. Wenzel Jakob. 2010. Mitsuba renderer. http://mitsuba-renderer.org.Google Scholar
    30. Wenzel Jakob, Adam Arbree, Jonathan T. Moon, Kavita Bala, and Steve Marschner. 2010. A Radiative Transfer Framework for Rendering Materials with Anisotropic Structure. In ACM SIGGRAPH 2010 Papers (SIGGRAPH ’10). ACM, New York, NY, USA, Article 53, 13 pages. Google ScholarDigital Library
    31. Wenzel Jakob and Steve Marschner. 2012. Manifold exploration: a Markov Chain Monte Carlo technique for rendering scenes with difficult specular transport. 31, 4 (July 2012), 58:1–58:13. Google ScholarDigital Library
    32. Henrik Wann Jensen and Per H. Christensen. 1998. Efficient simulation of light transport in scenes with participating media using photon maps. In 98 (Annual Conference Series). ACM, New York, NY, USA, 311–320. Google ScholarDigital Library
    33. Henrik Wann Jensen, Stephen R Marschner, Marc Levoy, and Pat Hanrahan. 2001. A practical model for subsurface light transport. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques. ACM, 511–518. Google ScholarDigital Library
    34. James T. Kajiya and Brian P Von Herzen. 1984. Ray tracing volume densities. 18, 3 (Jan. 1984), 165–174. Google ScholarDigital Library
    35. Masahide Kawai, Tomoyori Iwao, Akinobu Maejima, and Shigeo Morishima. 2014. Automatic Photorealistic 3D Inner Mouth Restoration from Frontal Images., 51–62 pages.Google Scholar
    36. Pramook Khungurn, Daniel Schroeder, Shuang Zhao, Kavita Bala, and Steve Marschner. 2015. Matching Real Fabrics with Micro-Appearance Models. ACM Trans. Graph. 35, 1, Article 1 (Dec. 2015), 26 pages. Google ScholarDigital Library
    37. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2014). http://arxiv.org/abs/1412.6980Google Scholar
    38. Oliver Klehm, Fabrice Rousselle, Marios Papas, Derek Bradley, Christophe Hery, Bernd Bickel, Wojciech Jarosz, and Thabo Beeler. 2015. Recent Advances in Facial Appearance Capture. Comput. Graph. Forum 34, 2 (May 2015), 709–733. Google ScholarDigital Library
    39. Eric P. Lafortune and Yves D. Willems. 1996. Rendering participating media with bidirectional path tracing. Springer-Verlag, London, UK, 91–100. Google ScholarDigital Library
    40. Christian Thode Larsen, Jeppe Revall Frisvad, Peter Dahl Ejby Jensen, and Jakob Andreas Bærentzen. 2012. Real-Time Rendering of Teeth with No Preprocessing. In Advances in Visual Computing, George Bebis, Richard Boyle, Bahram Parvin, Darko Koracin, Charless Fowlkes, Sen Wang, Min-Hyung Choi, Stephan Mantler, Jürgen Schulze, Daniel Acevedo, Klaus Mueller, and Michael Papka (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 334–345.Google Scholar
    41. Hongsong Li, Fabio Pellacini, and Kenneth E. Torrance. 2005. A Hybrid Monte Carlo Method for Accurate and Efficient Subsurface Scattering. In Proceedings of the Sixteenth Eurographics Conference on Rendering Techniques (EGSR ’05). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 283–290. Google ScholarDigital Library
    42. Satya P Mallick, Todd E Zickler, David J Kriegman, and Peter N Belhumeur. 2005. Beyond Lambert: Reconstructing specular surfaces using color. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, Vol. 2. Ieee, 619–626. Google ScholarDigital Library
    43. Francesco Guido Mangano, Andrea Gandolfi, Giuseppe Luongo, and Silvia Logozzo. 2017. Intraoral scanners in dentistry: a review of the current literature. In BMC oral health.Google Scholar
    44. Albert Mehl and Volker Blanz. 2005. A new approach for automatic reconstruction of occlusal surfaces with the biogeneric tooth model. Int. J. Comput. Dent. 8 (2005), 13–25.Google Scholar
    45. Zhuo Meng, Steve Yao, Hui Yao, Yan Liang, Tiegen Liu, Yanni Li, Guanhua Wang, and Shoufeng Lan. 2009. Measurement of the refractive index of human teeth by optical coherence tomography. Journal of Biomedical Optics 14 (2009), 14 – 14 – 4.Google ScholarCross Ref
    46. E. Mostafa, S. Elhabian, A. Abdelrahim, S. Elshazly, and A. Farag. 2014. Statistical morphable model for human teeth restoration. In IEEE ICIP. 4285–4288.Google Scholar
    47. Adolfo Munoz, Jose I. Echevarria, Francisco J. Seron, Jorge Lopez-Moreno, Mashhuda Glencross, and Diego Gutierrez. 2011. BSSRDF Estimation from Single Images. Computer Graphics Forum 30, 2 (2011), 455–464. 2011.01873.xGoogle ScholarCross Ref
    48. S. Omachi, K. Saito, H. Aso, S. Kasahara, S. Yamada, and K. Kimura. 2007. Tooth shape reconstruction from ct images using spline Curves. In Wavelet Analysis and Pattern Recognition, Vol. 1. 393–396.Google Scholar
    49. Marios Papas, Christian Regg, Wojciech Jarosz, Bernd Bickel, Philip Jackson, Wojciech Matusik, Steve Marschner, and Markus Gross. 2013. Fabricating Translucent Materials Using Continuous Pigment Mixtures. ACM Trans. Graph. 32, 4, Article 146 (July 2013), 12 pages. Google ScholarDigital Library
    50. Mark Pauly, Thomas Kollig, and Alexander Keller. 2000. Metropolis light transport for participating media. Springer-Verlag, London, UK, 11–22. Google ScholarDigital Library
    51. Matt Pharr, Wenzel Jakob, and Greg Humphreys. 2016. Physically Based Rendering: From Theory to Implementation (3rd ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. Google ScholarDigital Library
    52. Ioana-Sofia Pop-Ciutrila, Razvan Ghinea, Maria del Mar Perez Gomez, Horatiu Alexandru Colosi, Diana Dudea, and Mandra Badea. 2015. Dentine scattering, absorption, transmittance and light reflectivity in human incisors, canines and molars. Journal of Dentistry 43 (2015), 1116–1124.Google ScholarCross Ref
    53. D. Spitzer and J. J. Ten Bosch. 1975. The absorption and scattering of light in bovine and human dental enamel. Calcified Tissue Research 17, 2 (1975), 129–137.Google ScholarCross Ref
    54. Jos Stam. 1995. Multiple scattering as a diffusion process. (1995), 41–50.Google Scholar
    55. Sarah Tariq, Andrew Gardner, Ignacio Llamas, Andrew Jones, Paul Debevec, and Greg Turk. 2006. Efficient estimation of spatially varying subsurface scattering parameters. Vision, Modeling, and Visualization (VMV2006) (2006), 129–136.Google Scholar
    56. Xin Tong, Jiaping Wang, Stephen Lin, Baining Guo, and Heung-Yeung Shum. 2005. Modeling and Rendering of Quasi-homogeneous Materials. ACM Trans. Graph. 24, 3 (July 2005), 1054–1061. Google ScholarDigital Library
    57. Bruce Walter, Stephen R. Marschner, Hongsong Li, and Kenneth E. Torrance. 2007. Microfacet Models for Refraction Through Rough Surfaces. In Proceedings of the 18th Eurographics Conference on Rendering Techniques (EGSR’07). Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, 195–206. Google ScholarDigital Library
    58. Bruce Walter, Shuang Zhao, Nicolas Holzschuch, and Kavita Bala. 2009. Single Scattering in Refractive Media with Triangle Mesh Boundaries. ACM Transactions on Graphics 28, 3 (Aug. 2009), 92:1–8. Google ScholarDigital Library
    59. Jiaping Wang, Shuang Zhao, Xin Tong, Stephen Lin, Zhouchen Lin, Yue Dong, Baining Guo, and Heung-Yeung Shum. 2008. Modeling and Rendering of Heterogeneous Translucent Materials Using the Diffusion Equation. ACM Trans. Graph. 27, 1, Article 9 (March 2008), 18 pages. Google ScholarDigital Library
    60. Tim Weyrich, Wojciech Matusik, Hanspeter Pfister, Bernd Bickel, Craig Donner, Chien Tu, Janet McAndless, Jinho Lee, Addy Ngan, Henrik Wann Jensen, et al. 2006. Analysis of human faces using a measurement-based skin reflectance model. In ACM Transactions on Graphics (TOG), Vol. 25. ACM, 1013–1024. Google ScholarDigital Library
    61. Chenglei Wu, Derek Bradley, Pablo Garrido, Michael Zollhöfer, Christian Theobalt, Markus Gross, and Thabo Beeler. 2016. Model-based Teeth Reconstruction. ACM Trans. Graph. 35, 6, Article 220 (2016), 220:1–220:13 pages. Google ScholarDigital Library
    62. R Yanagisawa, Y Sugaya, S Kasahara, and S Omachi. 2014. Tooth shape reconstruction from dental CT images with the region-growing method. Dentomaxillofacial Radiology 43, 6 (2014), 20140080.Google ScholarCross Ref
    63. M. D. Zeiler. 2012. ADADELTA: An Adaptive Learning Rate Method. ArXiv e-prints (Dec. 2012). arXiv:cs.LG/1212.5701Google Scholar
    64. Shu-Xian Zheng, Jia Li, and Qing-Feng Sun. 2011. A Novel 3D Morphing Approach for Tooth Occlusal Surface Reconstruction. Comput. Aided Des. 43, 3 (2011), 293–302. Google ScholarDigital Library

ACM Digital Library Publication:


Overview Page:


Submit a story:

If you would like to submit a story about this presentation, please contact us: historyarchives@siggraph.org