“Immersive Conversations with Digital Einstein: Linking a Physical System and AI”
Conference:
Experience Type(s):
Title:
- Immersive Conversations with Digital Einstein: Linking a Physical System and AI
Organizer(s)/Presenter(s):
Description:
We present Digital Einstein, an immersive platform that enables interactive conversations with a digital representation of Albert Einstein. The physical setup includes a screen, armchair, camera, speakers, microphone, and decorative elements. Utilizing advanced technologies such as natural language processing, machine learning, and dynamic rendering, the platform simulates realistic dialogs with Einstein on various topics about his life and research. The system features two versions: Version 1 uses a dialog tree, motion-captured animations, and pre-recorded speech for fluid, controllable interactions. In Version 2, users can choose between GPT-4o-based and Llama 3-based chatbots for more open and engaging conversations. Data-driven models synthesize Einstein’s voice and generate corresponding animations on the fly, while context-specific images are automatically generated to visually enrich the conversation. Digital Einstein offers a glimpse into the future of immersive, personalized dialog experiences with intelligent characters.
References:
[1]
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:https://arXiv.org/abs/2303.08774 (2023).
Google Scholar
[2]
Abdullah Bin Sawad, Bhuva Narayan, Ahlam Alnefaie, Ashwaq Maqbool, Indra McKie, Jemma Smith, Berkan Yuksel, Deepak Puthal, Mukesh Prasad, and A. Baki Kocaballi. 2022. A systematic review on healthcare artificial intelligent conversational agents for chronic conditions. Sensors 22, 7 (2022), 2625.
Google Scholar
[3]
Llogari Casas and Kenny Mitchell. 2023. Intermediated reality with an AI 3D printed character. In ACM SIGGRAPH 2023 Real-Time Live!1–2.
Digital Library
Google Scholar
[4]
Volker Helzle and Kai Goetz. 2018. Digital Albert Einstein, a case study. In ACM SIGGRAPH 2018 Talks. 1–2.
Digital Library
Google Scholar
[5]
Pat Pataranutaporn, Valdemar Danry, Joanne Leong, Parinya Punpongsanon, Dan Novy, Pattie Maes, and Misha Sra. 2021. AI-generated characters for supporting personalized learning and well-being. Nature Machine Intelligence 3, 12 (2021), 1013–1022.
Google Scholar
[6]
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:https://arXiv.org/abs/2302.13971 (2023).
Google Scholar
[7]
Florian Weidner, Gerd Boettcher, Stephanie Arevalo Arboleda, Chenyao Diao, Luljeta Sinani, Christian Kunert, Christoph Gerhardt, Wolfgang Broll, and Alexander Raake. 2023. A Systematic Review on the Visualization of Avatars and Agents in AR & VR displayed using Head-Mounted Displays. IEEE Transactions on Visualization and Computer Graphics (2023).
Google Scholar
