Interpreter is an audiovisual installation that explores the possibility of communication beyond spoken language. Using MediaPipe-based gesture recognition, hand movements are translated into changes in sound and image, transforming physical gestures into a new audiovisual language. Everyday gestures and signs can convey meaning across cultural and linguistic boundaries, and this work investigates how nonverbal forms of communication can create new relationships between people and technology.
Presented across five screens with 360-degree spatial audio, the installation combines AI-generated visuals and layered visual compositions to create an immersive environment. The work follows an abstract narrative that begins in a world shaped by war and natural disasters and gradually evolves into a state where humanity, nature, and technology become interconnected. Through this process, Interpreter explores the concept of unity—not as the erasure of differences, but as a condition in which diverse forms of life, knowledge, and experience can coexist and connect.
Interactive effects and interactive sound were created by Stephanie Jantzen, 360-degree spatial audio was created by Yoonjae Choi, and AI-generated visuals were created by Atsunobu Takemoto.
Original Video :
