“PoeSpin: A Human-AI Dance to Poetry System for Movement-Based Verse Generation” by Li, Chen, Li and Xin
Conference:
Type(s):
Title:
- PoeSpin: A Human-AI Dance to Poetry System for Movement-Based Verse Generation
Session/Category Title:
- Agency and Performance
Presenter(s)/Author(s):
Moderator(s):
Abstract:
This paper presents PoeSpin, a human–AI co-writing system that transforms pole dance movements into poetry. Inspired by pole dance and computational linguistics, this project reimagines dance as a form of embodied poetic creation situated within the traditions of spatial and concrete poetry. Drawing from these traditions, PoeSpin treats physical motion as a generative force in the poetic process. We implemented three movement-to-poetry strategies. Among them, the second approach—embedding pole dance trajectories into a reduced-dimensional semantic space—served as the foundation for a real-time interactive installation. Building on this implementation and user feedback, we also proposed an integrated strategy for future interaction design. Our primary method employs the reduction of dimensionality of word vectors, mapping dance trajectories into a semantic space constructed upon Yeats’s poetic corpus. Rather than merely describing physical movement, the system generates evocative verse that extends beyond literal translation, unleashing creative possibilities that bridge the corporeal and the literary. This work offers both a technical framework for multimodal dance-poetry generation and a critical reframing of pole dance aesthetics, aiming to liberate this art form from prevailing societal prejudice.
References:
[1] Explosion AI. 2023. spaCy: Industrial-strength Natural Language Processing in Python. https://spacy.io Version 3.0 and later. Accessed: January 15, 2025.
[2] Rudolf Benesh and Joan Benesh. 1956. An Introduction to Benesh Movement Notation. A. & C. Black.
[3] James W. Cooley and John W. Tukey. 1965. An Algorithm for the Machine Calculation of Complex Fourier Series. Math. Comp. 19, 90 (1965), 297–301.
[4] Google Arts & Culture and Bill T. Jones. 2020. Body, Movement, Language. https://experiments.withgoogle.com/billtjonesai An interactive project exploring the intersection of movement and language using AI.
[5] Noa Eshkol and Abraham Wachman. 1958. Movement Notation. Weidenfeld and Nicolson.
[6] Katerina Fragkiadaki, Sergey Levine, Paul Felsen, and Jitendra Malik. 2015. Recurrent network models for human dynamics. Proceedings of the IEEE International Conference on Computer Vision (2015), 4346–4354. https://arxiv.org/abs/1506.01942
[7] Martha Graham. 1930. Lamentation: A Dance of Grief. Dance Quarterly 4 (1930), 12–18. Analysis of Martha Graham’s choreographic techniques.
[8] Ann Hutchinson. 1970. Labanotation: The System of Analyzing and Recording Movement. Theatre Arts Books.
[9] Susan Jones. 2023. 570Dance. In The Oxford Handbook of W.B. Yeats. Oxford University Press. arXiv:https://academic.oup.com/book/0/chapter/411323198/chapter-ag-pdf/57217582/book_46715_section_411323198.ag.pdf
[10] Junnan Li, Dongxu Li, Caiming Xiong, and Steven C.H. Hoi. 2022. BLIP: Bootstrapped Language-Image Pre-training for Unified Vision-Language Understanding and Generation. arXiv preprint arXiv:https://arXiv.org/abs/2201.12086 (2022). https://arxiv.org/abs/2201.12086
[11] Guiyu Liu. 2021. Body Poetry. https://ars.electronica.art/outofthebox/en/body-poetry/ An interactive installation exploring the relationship between body movements and poetic expression.
[12] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. arXiv preprint arXiv:https://arXiv.org/abs/1301.3781 (2013). https://arxiv.org/abs/1301.3781
[13] Mallakhamb Federation of India. [n. d.]. Mallakhamb: Ancient Indian Gymnastics. https://www.mallakhambindia.com/. Accessed: 2025-01-15.
[14] Online Pole Studio. 2025. Online Pole Studio: A Comprehensive Resource for Pole Dance Training and Techniques. https://www.onlinepolestudio.com/ Accessed: January 15, 2025.
[15] Mario Petrucci. 2021. Spatial Form. Online Resource. http://www.mariopetrucci.com/spatialform.htm Accessed April 2025.
[16] Mary Ellen Solt. 1970. Concrete Poetry: A World View. Indiana University Press, Bloomington and London.
[17] Sony Electronics. 2025. Mocopi: Motion Capture Device. https://electronics.sony.com/more/mocopi/all-mocopi/p/qmss1-uscx Accessed: January 15, 2025.
[18] Yuqian Sun, Yuying Tang, Ze Gao, Zhijun Pan, Chuyan Xu, Yurou Chen, Kejiang Qian, Zhigang Wang, Tristan Braud, Chang Hee Lee, and Ali Asadipour. 2023. AI Nüshu: An Exploration of Language Emergence in Sisterhood Through the Lens of Computational Linguistics. In SIGGRAPH Asia 2023 Art Papers (Sydney, NSW, Australia) (SA ’23). Association for Computing Machinery, New York, NY, USA, Article 4, 7 pages.
[19] Yulan Wang. 2021. The Art of Chinese Chain Poetry: Tradition and Innovation. Peking University Press, Beijing, China. An in-depth analysis of chain poetry in Chinese literary tradition.
[20] Zeyu Wang, Chengan He, Zhe Yan, Jiashun Wang, Yingke Wang, Junhua Liu, Angela Shen, Mengying Zeng, Holly Rushmeier, Huazhe Xu, Borou Yu, Chenchen Lu, and Eugene Y. Wang. 2024. Chang-E: A High-Quality Motion Capture Dataset of Chinese Classical Dunhuang Dance. J. Comput. Cult. Herit. (Dec. 2024). Just Accepted.
[21] Emily E. Wilcox. 2024. An Image that Resonates: Yang Liping and the Evolution of Contemporary Chinese Folk Dance. In The Choreography of Traditional Dance on Stage: Crises, Perspectives and Global Dialogues. Institute of Ethnography SASA and Ensemble of Traditional Dances and Songs of Serbia “Kolo”, 223–235. https://scholarworks.wm.edu/asbookchapters/157
[22] Weidi Zhang and Donghao Ren. 2021. Cangjie’s poetry. In ACM SIGGRAPH 2021 Art Gallery (Virtual Event, USA) (SIGGRAPH ’21). Association for Computing Machinery, New York, NY, USA, Article 3, 3 pages.
