Riksteatern CREA and Humanities Lab collaboration on Sign Language Avatar on Stage awarded at first Deaf Tech Conference 

The conference paper and presentation was a non-scientific methodological paper (not based on a study) in competition with a mix of 60 scientific and other papers, 14 of which was selected for conference presentation. Awarded with the price as 2nd best and reactions on site was yet another approval of a Deaf led collaboration that profoundly improved the lab’s research workflows on motion capture and multimodal communication, including the understanding of assessment and well prepared production lines for lab users.

The Deaf Tech 2026 conference site and link to papers: 
https://www.deaftech26.eu
https://uvaauas.figshare.com/collections/Deaftech_2026_Conference_Proceedings/8440687

Here is a signed abstract of the presentation by Riksteatern Crea’s Artistic Director Mindy Drapsa with transcription below. Also Joel Kankkonen, Teckenbro, one of the masterminds behind this project, took part in the presentation at the conference, that was sign-language only.     

"Sign Language Avatar on Stage: Deaf-Led Motion Capture Innovation"

Riksteatern Crea, the national touring sign language theatre of Sweden, together with the research infrastructure Lund University Humanities Lab have developed a solution to create a real-time sign language avatar on stage.

The theatre performance was performed with a combination of deaf actors on stage and one deaf actor in a motion capture studio next to the stage, being projected as an avatar on stage and interacting with the actors on stage.

Several challenges arouse during the process. For example, to achieve precision in the linguistic features of sign language, such as eye gaze, eyebrow movements, details in finger movements, and body movements and to capture and transfer the avatar in real-time.

The technique developed did not reach perfection but was still good enough for the creation of the performance.

The performance “The Canterville Ghost” premiered and toured around Sweden during 2025.

The project was ongoing for many years and contained workshops, a pilot, and rehearsals and engaged both technical and artistic personnel. One important feature of the process was a deaf-led approach meaning that deaf perspectives, deaf culture, and the lived experiences of deaf individuals was included throughout the whole process, to ensure cultural authenticity and linguistic accuracy.

The technical rig consisted of motion capture sensors and cameras, an iPhone to read facial expressions, and stretch-tech gloves reading finger movements. The avatar was streamed thought the programs Unity and Unreal. This motion capture rig was used in synchronise with the production line of the touring performance.

The project contributed to a methodology development that can contribute to research infrastructure within other fields such as education, digital communication, etc, and inspire further development.

To conclude, a deaf-led approach can contribute, not only with sign language knowledge, but with new perspectives, new processes, and enhance the creativity within technical and artistic work.