Weidi Zhang, Donghao Ren: Cangjie’s Poetry
Artist(s):
Title:
- Cangjie’s Poetry
Exhibition:
- SIGGRAPH 2021: Art with an Impact
-
More artworks from SIGGRAPH 2021:
Creation Year:
- 2021
Medium:
- Artificial Intelligent
Category:
Artist Statement:
Cangjie’s Poetry is an interactive art installation that provides a conceptual response to the semantic human-machine reality. In the Cangjie’s Poetry art installation, an intelligent system constantly observes the surroundings through the lens of a camera, writes poetry using a symbolic system based on its interpretation of the surroundings, and explains the evolving poem in natural language to audiences in real-time. This work prioritizes the ambiguity and tension between machinic vision and human perception, the actual and the virtual, past and present.
Cangjie’s Poetry starts with a conceptual question: if artificial intelligence can imitate human beings to create a symbolic system to communicate based on their understanding of the universe and begin interacting with us actively, how will this recontextualize and redefine our coexistence in this intertwined reality?
Inspired by the legendary Chinese historian Cangjie (c.2650 BCE), who invented Chinese char- acters based on the characteristics of everything on the earth, we trained a neural network that we call Cangjie to learn the constructions and principles of over 9000 Chinese characters. After training, Cangjie can interpret images through the lens of Chinese characters and produce new symbols constructed by Chinese strokes. Those new symbols appear in the aesthetic of Chinese characters, but they are meaningless novel symbols. To create meanings for this symbolic system conceptually and aesthetically, we implement a pre-trained model (DenseCap) to simultaneously generate localized descriptive sentences of images in natural language, making those novel symbols understandable.
In the art installation, our Cangjie system surveys the real world surroundings using a camera, transforming them as it streams data into a cluster of ever-changing symbols in real-time. These novel symbols are made of Chinese strokes tangled with the captured imagery, and are visualized algorithmically as an abstract pixelated landscape. The landscape dynamically moves, evolves, and writes poetry based on the image data. For the first projection in the exhibition space, we visualize the semantic landscape on the wall. Cangjie also generates descriptive sentences of the surroundings based on its interpretation of the space. For the second projection, those sentences are designed as flowing poetry written with a visual simulation of ink, and assembled with imagery fragments captured in real-time.
Cangjie is not only a conceptual response to the tension and fragility in the coexistence of humans and machines, but also an artistic expression of a future language that reflects on ancient truths, evoking enchantment in this artificial intelligence era. From learning over 9000 Chinese characters, Cangjie creates its own symbolic system. It observes the real-world, writes poetry using its symbolic system, and explains it in natural language to audiences. Just like the legendary historian Cangjie did nearly 5000 years ago, it continuously writes poetry with humans collaboratively, never ending as long as the real world exists.
