“GenAI and Immersive Theater: Moment Factory x Third Rail Projects” by Bigeault, Harvey, Tozier and Doyon
Conference:
Type(s):
Title:
- GenAI and Immersive Theater: Moment Factory x Third Rail Projects
Session/Category Title:
- Generative Intelligence
Presenter(s)/Author(s):
Moderator(s):
Abstract:
Moment Factory’s GenAI initiative seeks to harness the potential of artificial intelligence for creative storytelling. Through a collaborative venture with Third Rail Projects (TRP) – a New York-based company acclaimed for its immersive, site-specific performances – we explored how generative AI could seamlessly integrate into live theater to co-create experiences with and for audiences. This partnership combined TRP’s expertise in crafting participatory narrative journeys and Moment Factory’s experience in real-time interactive technologies. We will demonstrate how AI-driven content can heighten audience agency and emotional resonance, transforming viewers from passive spectators into active co-creators. By examining practical examples and prototypes, we showcase real-time generative processes: from capturing audience prompts to instantly projecting immersive visuals that shape theatrical scenes. These prototypes allowed us to test the boundaries of AI’s role in improvisation, design workflows for high-quality outputs under strict time constraints, and refine how performers engage with AI on stage. Key takeaways include: • Techniques for integrating generative AI with live performance to enhance audience immersion. • Approaches to balancing creative spontaneity with the technical requirements (e.g., data capture, prompt engineering, real-time rendering). • Lessons learned about preserving human-centric artistry when leveraging machine-driven content generation. By blending TRP’s immersive theater language with Moment Factory’s technical pipeline, we uncovered novel ways to engage participants and amplify their input. In doing so, this project highlights the potential of AI as a genuine creative partner rather than a mere tool. We also discuss future directions for large-scale performances and educational programming that further bridge human imagination with generative AI. Ultimately, our findings demonstrate how generative AI can expand both the aesthetics and emotional reach of live storytelling, offering a new paradigm for interactive, co-created experiences that place the audience at the heart of the performance.
References:
[1] Comfyanonymous. 2022. ComfyUI: A Web Interface for Stable Diffusion. https://github.com/comfyanonymous/ComfyUI. Accessed: July 19, 2023.
[2] OpenAI. 2022. Whisper: Speech Recognition Model. https://github.com/openai/whisper. Accessed: June 15, 2025.
[3] Zhang, Yiheng and Liu, Lianmin and Zhu, Junbo and Shi, Hang. 2023. StreamDiffusion: One-Step Diffusion for Fast and Consistent Latent Image Generation. https://arxiv.org/abs/2312.12491. Accessed: June 15, 2025.
