“Parametric directional coding for precomputed sound propagation” by Raghuvanshi and Snyder
Conference:
Type:
Entry Number: 108
Title:
- Parametric directional coding for precomputed sound propagation
Session/Category Title: Sounds Good!
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Convincing audio for games and virtual reality requires modeling directional propagation effects. The initial sound’s arrival direction is particularly salient and derives from multiply-diffracted paths in complex scenes. When source and listener straddle occluders, the initial sound and multiply-scattered reverberation stream through gaps and portals, helping the listener navigate. Geometry near the source and/or listener reveals its presence through anisotropic reflections. We propose the first precomputed wave technique to capture such directional effects in general scenes comprising millions of polygons. These effects are formally represented with the 9D directional response function of 3D source and listener location, time, and direction at the listener, making memory use the major concern. We propose a novel parametric encoder that compresses this function within a budget of ~100MB for large scenes, while capturing many salient acoustic effects indoors and outdoors. The encoder is complemented with a lightweight signal processing algorithm whose filtering cost is largely insensitive to the number of sound sources, resulting in an immediately practical system.
References:
1. Lakulish Antani, Anish Chandak, Micah Taylor, and Dinesh Manocha. 2012. Direct-to-indirect Acoustic Radiance Transfer. IEEE Transactions on Visualization and Computer Graphics 18, 2 (Feb. 2012), 261–269. Google ScholarDigital Library
2. P. Bilinski, Ahrens J., Thomas M. R. P., Tashev I. J., and Platt J. C. 2014. HRTF magnitude synthesis via sparse representation of anthropometric features. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence. 4468–4472. http://ieeexplore.ieee.org/stamp/stamp-jsp?tp=&arnumber=6854447&isnumber=6853544Google Scholar
3. J. Blauert. 1997. An introduction to binaural technology. In Binaural and Spatial Hearing in Real and Virtual Environments, R. Gilkey and T. R. Anderson (Eds.). Lawrence Erlbaum, USA.Google Scholar
4. Jeroen Breebaart, Sascha Disch, Christof Faller, Jürgen Herre, Gerard Hotho, Kristofer Kjörling, Francois Myburg, Matthias Neusinger, Werner Oomen, Heiko Purnhagen, and Jonas Rödén. 2005. MPEG Spatial Audio Coding / MPEG Surround: Overview and Current Status. In Audio Engineering Society Convention 119. http://www.aes.org/e-lib/browse.cfm?elib=13333Google Scholar
5. Chunxiao Cao, Zhong Ren, Carl Schissler, Dinesh Manocha, and Kun Zhou. 2016. Interactive Sound Propagation with Bidirectional Path Tracing, to appear. ACM Transactions on Graphics (SIGGRAPH Asia 2016) (2016). Google ScholarDigital Library
6. Jeffrey N. Chadwick, Steven S. An, and Doug L. James. 2009. Harmonic shells: a practical nonlinear sound model for near-rigid thin shells. In SIGGRAPH Asia ’09: ACM SIGGRAPH Asia 2009 papers. ACM, New York, NY, USA, 1–10. Google ScholarDigital Library
7. Anish Chandak, Christian Lauterbach, Micah Taylor, Zhimin Ren, and Dinesh Manocha. 2008. AD-Frustum: Adaptive Frustum Tracing for Interactive Sound Propagation. IEEE Transactions on Visualization and Computer Graphics 14, 6 (2008), 1707–1722. Google ScholarDigital Library
8. Jean-Jacques Embrechts. 2016. Review on the applications of directional impulse responses in room acoustics. In Proceedings of CFA 2016. Société française d’acoustique (SFA). http://orbi.ulg.ac.be/handle/2268/193820Google Scholar
9. Kenji Fujii, Takuya Hotehama, Kosuke Kato, Ryota Shimokura, Yosuke Okamoto, Yukio Suzumura, and Yoichi Ando. 2004. Spatial Distribution of Acoustical Parameters in Concert Halls: Comparison of Different Scattered Reflections. 4 (01 2004).Google Scholar
10. Anders Gade. 2007. Acoustics in Halls for Speech and Music. In Springer Handbook of Acoustics (2007 ed.), Thomas Rossing (Ed.). Springer, Chapter 9. http://www.worldcat.org/isbn/0387304460Google ScholarCross Ref
11. Michael A. Gerzon. 1973. Periphony: With-Height Sound Reproduction. J. Audio Eng. Soc 21, 1 (1973), 2–10. http://www.aes.org/e-lib/browse.cfm?elib=2012Google Scholar
12. Nail A. Gumerov and Ramani Duraiswami. 2005. Fast Multipole Methods for the Helmholtz Equation in Three Dimensions (Elsevier Series in Electromagnetism) (1 ed.). Elsevier Science, http://www.worldcat.org/isbn/0080443710Google Scholar
13. Brian Hamilton, Stefan Bilbao, Brian Hamilton, and Stefan Bilbao. 2017. FDTD Methods for 3-D Room Acoustics Simulation With High-Order Accuracy in Space and Time. IEEE/ACM Trans. Audio, Speech and Lang. Proc. 25, 11 (Nov. 2017), 2112–2124. Google ScholarDigital Library
14. Jürgen Herre, Johannes Hilpert, Achim Kuntz, and Jan Plogsties. 2015. MPEG-H Audio – The New Standard for Universal Spatial/3D Audio Coding. J. Audio Eng. Soc 62, 12 (2015), 821–830. http://www.aes.org/e-lib/browse.cfm?elib=17556Google ScholarCross Ref
15. Doug L. James, Jernej Barbie, and Dinesh K. Pai. 2006. Precomputed acoustic transfer: output-sensitive, accurate sound generation for geometrically complex vibration sources. ACM Transactions on Graphics 25, 3 (July 2006), 987–995. Google ScholarDigital Library
16. Heinrich Kuttruff. 2000. Room Acoustics (4 ed.). Taylor & Francis. http://www.worldcat.org/isbn/0419245804Google Scholar
17. Mikko V. Laitinen, Tapani Pihlajamäki, Cumhur Erkut, and Ville Pulkki. 2012. Parametric Time-frequency Representation of Spatial Sound in Virtual Worlds. ACM Trans. Appl. Percept. 9, 2 (June 2012). Google ScholarDigital Library
18. Dingzeyu Li, Yun Fei, and Changxi Zheng. 2015. Interactive Acoustic Transfer Approximation for Modal Sound. ACM Trans. Graph. 35, 1 (Dec. 2015). Google ScholarDigital Library
19. Ruth Y. Litovsky, Steven H. Colburn, William A. Yost, and Sandra J. Guzman. 1999. The precedence effect. The Journal of the Acoustical Society of America 106, 4 (1999), 1633–1654.Google ScholarCross Ref
20. Ravish Mehra, Nikunj Raghuvanshi, Lakulish Antani, Anish Chandak, Sean Curtis, and Dinesh Manocha. 2013. Wave-based Sound Propagation in Large Open Scenes Using an Equivalent Source Formulation. ACM Trans. Graph. 32, 2 (April 2013). Google ScholarDigital Library
21. Ravish Mehra, Nikunj Raghuvanshi, Lauri Savioja, Ming C. Lin, and Dinesh Manocha. 2012. An efficient GPU-based time domain solver for the acoustic wave equation. Applied Acoustics 73, 2 (Feb. 2012), 83–94.Google ScholarCross Ref
22. Ravish Mehra, Atul Rungta, Abhinav Golas, Ming Lin, and Dinesh Manocha. 2015. WAVE: Interactive Wave-based Sound Propagation for Virtual Environments. IEEE transactions on visualization and computer graphics 21, 4 (April 2015), 434–442. http://view.ncbi.nlm.nih.gov/pubmed/26357093Google Scholar
23. Juha Merimaa and Ville Pulkki. 2005. Spatial Impulse Response Rendering I: Analysis and Synthesis. J. Audio Eng. Soc 53, 12 (2005), 1115–1127. http://www.aes.org/e-lib/browse.cfm?elib=13401Google Scholar
24. Brian Moore and Brian Glasberg. 1996. A Revision of Zwicker’s Loudness Model. 82 (03 1996), 335–345.Google Scholar
25. D. Murphy, A. Kelloniemi, J. Mullen, and S. Shelley. 2007. Acoustic Modeling Using the Digital Waveguide Mesh. IEEE Signal Processing Magazine 24, 2 (March 2007), 55–66.Google ScholarCross Ref
26. Juhani Paasonen, Aleksandr Karapetyan, Jan Plogsties, and Ville Pulkki. 2017. Proximity of Surfaces – Acoustic and Perceptual Effects. J. Audio Eng. Soc 65, 12 (2017), 997–1004. http://www.aes.org/e-lib/browse.cfm?elib=19365Google ScholarCross Ref
27. Allan D. Pierce. 1989. Acoustics: An Introduction to Its Physical Principles and Applications. Acoustical Society of America, http://www.worldcat.org/isbn/0883186128Google Scholar
28. Boaz Rafaely 2015. Fundamentals of Spherical Array Processing (Springer Topics in Signal Processing) (2015 ed.). Springer, http://www.worldcat.org/isbn/9783662456644Google ScholarCross Ref
29. Nikunj Raghuvanshi, Rahul Narain, and Ming C. Lin. 2009a. Efficient and Accurate Sound Propagation Using Adaptive Rectangular Decomposition. IEEE Transactions on Visualization and Computer Graphics 15, 5 (2009), 789–801. Google ScholarDigital Library
30. Nikunj Raghuvanshi, Rahul Narain, and Ming C. Lin. 2009b. Efficient and Accurate Sound Propagation Using Adaptive Rectangular Decomposition. IEEE Transactions on Visualization and Computer Graphics 15, 5 (2009), 789–801. Google ScholarDigital Library
31. Nikunj Raghuvanshi and John Snyder. 2014. Parametric Wave Field Coding for Precomputed Sound Propagation. ACM Trans. Graph. 33, 4 (July 2014). Google ScholarDigital Library
32. Nikunj Raghuvanshi, John Snyder, Ravish Mehra, Ming C. Lin, and Naga K. Govindaraju. 2010. Precomputed Wave Simulation for Real-Time Sound Propagation of Dynamic Sources in Complex Scenes. ACM Transactions on Graphics 29, 3 (July 2010). Google ScholarDigital Library
33. Jens H. Rindel and Claus L. Christensen. 2013. The use of colors, animations and auralizations in room acoustics. In Internoise 2013.Google Scholar
34. Lauri Savioja and U. Peter Svensson. 2015. Overview of geometrical room acoustic modeling techniques. The Journal of the Acoustical Society of America 138, 2 (01 Aug. 2015), 708–730.Google ScholarCross Ref
35. Carl Schissler, Ravish Mehra, and Dinesh Manocha. 2014. High-order Diffraction and Diffuse Reflections for Interactive Sound Propagation in Large Environments. ACM Trans. Graph. 33, 4 (July 2014). Google ScholarDigital Library
36. Dirk Schröder. 2011. Physically Based Real-Time Auralization of Interactive Virtual Environments. Logos Verlag, http://www.worldcat.org/isbn/3832530312Google Scholar
37. Jonathan Sheaffer, Maarten Van Walstijn, Boaz Rafaely, and Konrad Kowalczyk. 2015. Binaural Reproduction of Finite Difference Simulations Using Spherical Array Processing. IEEE/ACM Trans. Audio, Speech and Lang. Proc. 23, 12 (Dec. 2015), 2125–2135. Google ScholarDigital Library
38. S. Siltanen, T. Lokki, and L. Savioja. 2010a. Rays or Waves? Understanding the Strengths and Weaknesses of Computational Room Acoustics Modeling Techniques. In Proc. Int. Symposium on Room Acoustics. Melbourne, Australia.Google Scholar
39. Samuel Siltanen, Tapio Lokki, and Lauri Savioja. 2010b. Room acoustics modeling with acoustic radiance transfer. Proc. ISRA Melbourne (2010).Google Scholar
40. Julius O. III Smith. 2007. Introduction to Digital Filters with Audio Applications. (2007). https://ccrma.stanford.edu/~jos/filters/Google Scholar
41. Alex Southern, Damian T. Murphy, and Lauri Savioja. 2012. Spatial Encoding of Finite Difference Time Domain Acoustic Models for Auralization. Trans. Audio, Speech and Lang. Proc. 20, 9 (Nov. 2012), 2420–2432. Google ScholarDigital Library
42. Micah T. Taylor, Anish Chandak, Lakulish Antani, and Dinesh Manocha. 2009. RESound: interactive sound rendering for dynamic virtual environments. In Proceedings of ACM conference on Multimedia. ACM, New York, NY, USA, 271–280. Google ScholarDigital Library
43. Sakari Tervo, Jukka Pätynen, Antti Kuusinen, and Tapio Lokki. 2013. Spatial Decomposition Method for Room Impulse Responses. J. Audio Eng. Soc 61, 1/2 (2013), 17–28.Google Scholar
44. Nicolas Tsingos. 2009. Pre-computing geometry-based reverberation effects for games. In 35th AES Conference on Audio for Games.Google Scholar
45. Nicolas Tsingos, Carsten Dachsbacher, Sylvain Lefebvre, and Matteo Dellepiane. 2007. Instant Sound Scattering. In Rendering Techniques (Proceedings of the Eurographics Symposium on Rendering). http://www-sop.inria.fr/reves/Basilic/2007/TDLD07 Google ScholarDigital Library
46. Michael Vorländer. 2007. Auralization: Fundamentals of Acoustics, Modelling, Simulation, Algorithms and Acoustic Virtual Reality (RWTHedition) (1 ed.). Springer, http://www.worldcat.org/isbn/3540488294 Google ScholarDigital Library
47. Hengchin Yeh, Ravish Mehra, Zhimin Ren, Lakulish Antani, Dinesh Manocha, and Ming Lin. 2013. Wave-ray Coupling for Interactive Sound Propagation in Large Complex Scenes. ACM Trans. Graph. 32, 6 (Nov. 2013). Google ScholarDigital Library
48. Wen Zhang, Thushara D. Abhayapala, Rodney A. Kennedy, and Ramani Duraiswami. 2010. Insights into head-related transfer function: Spatial dimensionality and continuous representation. The Journal of the Acoustical Society of America 127, 4 (01 April 2010), 2347–2357.Google ScholarCross Ref
