“An Architecture and Implementation of Real-Time Sound Propagation Hardware for Mobile Devices” by Kim, Choi, Kim, Nah, Jung, et al. …
Conference:
Type(s):
Title:
- An Architecture and Implementation of Real-Time Sound Propagation Hardware for Mobile Devices
Session/Category Title: Multidisciplinary Fusion
Presenter(s)/Author(s):
Abstract:
This paper presents a high-performance and low-power hardware architecture for real-time sound rendering on mobile devices. Traditional sound rendering algorithms require high-performance CPUs or GPUs for processing because of its high computational complexities to realize ultra-realistic 3D audio. Thus, it has been hard to achieve real-time rates on low-power mobile devices. To overcome this limitation, we propose a hardware architecture that adopts hardware-friendly sound-propagation-path calculation algorithms. We verified the function and performance of our architecture through its implementation on an FPGA board. According to ASIC evaluation with the 8-nm process technology, it achieves high performance with 120 FPS, low power consumption with 50 mW, and a small silicon area with 0.31 mm^2, allowing real-time sound rendering on mobile devices.
References:
[1]
Jont B Allen and David A Berkley. 1979. Image method for efficiently simulating small-room acoustics. The Journal of the Acoustical Society of America 65, 4 (1979), 943–950.
[2]
Andrei Frumusanu 2021. The Apple A15 SoC Performance Review: Faster & More Efficient. https://www.anandtech.com/show/16983/the-apple-a15-soc-performance-review-faster-more-efficient/3.
[3]
Chunxiao Cao, Zhong Ren, Carl Schissler, Dinesh Manocha, and Kun Zhou. 2016. Interactive sound propagation with bidirectional path tracing. ACM Transactions on Graphics (TOG) 35, 6 (2016), 1–11.
[4]
Chakravarty R Alla Chaitanya, Nikunj Raghuvanshi, Keith W Godin, Zechen Zhang, Derek Nowrouzezahrai, and John M Snyder. 2020. Directional sources and listeners in interactive sound propagation using reciprocal wave field coding. ACM Transactions on Graphics (TOG) 39, 4 (2020), 44–1.
[5]
Anish Chandak. 2011. Efficient geometric sound propagation using visibility culling. Ph. D. Dissertation. University of North Carolina at Chapel Hill.
[6]
Carl F Eyring. 1930. Reverberation time in “dead” rooms. The Journal of the Acoustical Society of America 1, 2A (1930), 217–241.
[7]
Eunjae Kim, Sukwon Choi, Cheong-Ghil Kim, and Woo-Chan Park. 2023. Multi-Threaded Sound Propagation Algorithm to Improve Performance on Mobile Devices. Sensors 23, 2 (2023), 973–989.
[8]
Eftychia Lakka, Athanasios G Malamos, Konstantinos G Pavlakis, and J Andrew Ware. 2018. Spatial Sound Rendering–A Survey. IJIMAI 5, 3 (2018), 33–45.
[9]
Christian Lauterbach, Anish Chandak, and Dinesh Manocha. 2007. Interactive sound rendering in complex and dynamic scenes using frustum tracing. IEEE Transactions on Visualization and Computer Graphics 13, 6 (2007), 1672–1679.
[10]
Shiguang Liu and Jin Liu. 2020. Outdoor sound propagation based on adaptive FDTD-PE. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, Online, 859–867.
[11]
Shiguang Liu and Dinesh Manocha. 2022. Sound Synthesis, Propagation, and Rendering. Morgan & Claypool Publishers, San Rafael, California.
[12]
Ravish Mehra, Nikunj Raghuvanshi, Lakulish Antani, Anish Chandak, Sean Curtis, and Dinesh Manocha. 2013. Wave-based sound propagation in large open scenes using an equivalent source formulation. ACM Transactions on Graphics (TOG) 32, 2 (2013), 1–13.
[13]
Ravish Mehra, Atul Rungta, Abhinav Golas, Ming Lin, and Dinesh Manocha. 2015. Wave: Interactive wave-based sound propagation for virtual environments. IEEE transactions on visualization and computer graphics 21, 4 (2015), 434–442.
[14]
Jae-Ho Nah, Hyuck-Joo Kwon, Dong-Seok Kim, Cheol-Ho Jeong, Jinhong Park, Tack-Don Han, Dinesh Manocha, and Woo-Chan Park. 2014. RayCore: A ray-tracing hardware architecture for mobile devices. ACM Transactions on Graphics (TOG) 33, 5 (2014), 1–15.
[15]
Nefi Alarcon 2019. VRWorks Audio Dials up the Immersion with RTX Acceleration. https://developer.nvidia.com/blog/vrworks-audio-dials-up-the-immersion-with-rtx-acceleration/.
[16]
Sönke Pelzer, Lukas Aspöck, Dirk Schröder, and Michael Vorländer. 2014. Interactive real-time simulation and auralization for modifiable rooms. Building Acoustics 21, 1 (2014), 65–73.
[17]
Nikunj Raghuvanshi and Ming C Lin. 2006. Interactive sound synthesis for large scale environments. In Proceedings of the 2006 symposium on Interactive 3D graphics and games. ACM, New York, NY, 101–108.
[18]
Nikunj Raghuvanshi and Ming C Lin. 2007. Physically based sound synthesis for large-scale virtual environments. IEEE Computer Graphics and Applications 27, 1 (2007), 14–18.
[19]
Nikunj Raghuvanshi, Rahul Narain, and Ming C Lin. 2009. Efficient and accurate sound propagation using adaptive rectangular decomposition. IEEE Transactions on Visualization and Computer Graphics 15, 5 (2009), 789–801.
[20]
Nikunj Raghuvanshi and John Snyder. 2018. Parametric directional coding for precomputed sound propagation. ACM Transactions on Graphics (TOG) 37, 4 (2018), 1–14.
[21]
Nikunj Raghuvanshi, John Snyder, Ravish Mehra, Ming Lin, and Naga Govindaraju. 2010. Precomputed wave simulation for real-time sound propagation of dynamic sources in complex scenes. ACM Transactions on Graphics (TOG) 29, 68 (2010), 1–11.
[22]
Lauri Savioja and U Peter Svensson. 2015. Overview of geometrical room acoustic modeling techniques. The Journal of the Acoustical Society of America 138, 2 (2015), 708–730.
[23]
Carl Schissler and Dinesh Manocha. 2016. Interactive sound propagation and rendering for large multi-source scenes. ACM Transactions on Graphics (TOG) 36, 4 (2016), 1.
[24]
Carl Schissler, Ravish Mehra, and Dinesh Manocha. 2014. High-order diffraction and diffuse reflections for interactive sound propagation in large environments. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1–12.
[25]
Carl Schissler, Gregor Mückl, and Paul Calamia. 2021. Fast diffraction pathfinding for dynamic sound propagation. ACM Transactions on Graphics (TOG) 40, 4 (2021), 1–13.
[26]
Samuel Siltanen, Tapio Lokki, Lauri Savioja, and Claus Lynge Christensen. 2008. Geometry reduction in room acoustics modeling. Acta Acustica united with Acustica 94, 3 (2008), 410–418.
[27]
Alex Southern, Samuel Siltanen, Damian T Murphy, and Lauri Savioja. 2013. Room impulse response synthesis and validation using a hybrid acoustic model. IEEE Transactions on Audio, Speech, and Language Processing 21, 9 (2013), 1940–1952.
[28]
Auston Sterling and Ming C Lin. 2016. Interactive modal sound synthesis using generalized proportional damping. In Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, Redwood City California, 79–86.
[29]
Zhenyu Tang, Hsien-Yu Meng, and Dinesh Manocha. 2021. Learning acoustic scattering fields for dynamic interactive sound propagation. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). IEEE, IEEE, Lisboa, Portugal, 835–844.
[30]
Micah Taylor, Anish Chandak, Lakulish Antani, and Dinesh Manocha. 2009a. Resound: interactive sound rendering for dynamic virtual environments. In Proceedings of the 17th ACM international conference on Multimedia. ACM, Beijing, 271–280.
[31]
Micah Taylor, Anish Chandak, Qi Mo, Christian Lauterbach, Carl Schissler, and Dinesh Manocha. 2010. i-sound: Interactive gpu-based sound auralization in dynamic scenes. Technical Report. Tech. Rep. TR10-006.
[32]
Micah Taylor, Anish Chandak, Qi Mo, Christian Lauterbach, Carl Schissler, and Dinesh Manocha. 2011. Gsound: Interactive sound propagation for games. In Audio Engineering Society Conference: 41st International Conference: Audio for Games. Audio Engineering Society, Audio Engineering Society (AES), London.
[33]
Micah Taylor, Anish Chandak, Qi Mo, Christian Lauterbach, Carl Schissler, and Dinesh Manocha. 2012. Guided multiview ray tracing for fast auralization. IEEE Transactions on Visualization and Computer Graphics 18, 11 (2012), 1797–1810.
[34]
Micah Taylor, Anish Chandak, Zhimin Ren, Christian Lauterbach, and Dinesh Manocha. 2009b. Fast edge-diffraction for sound propagation in complex virtual environments. In EAA auralization symposium. Citeseer, European Acoustics Association (EAA), Espoo, 15–17.
[35]
Nicolas Tsingos, Thomas Funkhouser, Addy Ngan, and Ingrid Carlbom. 2001. Modeling acoustics in virtual environments using the uniform theory of diffraction. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques. Association for Computing Machinery, New Yourk, NY, 545–552.
[36]
Hengchin Yeh, Ravish Mehra, Zhimin Ren, Lakulish Antani, Dinesh Manocha, and Ming Lin. 2013. Wave-ray coupling for interactive sound propagation in large complex scenes. ACM Transactions on Graphics (TOG) 32, 6 (2013), 1–11.
