Singing Machine
A browser-based singing instrument for people who have had a total laryngectomy. A camera reads the shape of the mouth, and the machine sings the vowel.
- With
- Ivica Ico Bukvic, technical collaborator
- Built with
- MediaPipe and WebPd-L2Ork, running in the browser
- Support
- Center for Future Work Places and Practices (CFWPP), Virginia Tech
- Shown at
- ICAT Playdates, Virginia Tech; CMMR 2025, London; DRS2026, Edinburgh
After a total laryngectomy, most voice technology aims at one thing: bringing speech back to a functional standard. The Singing Machine starts from a different question. What would it take to sing?
The instrument runs in a web browser, with nothing to install. A camera tracks the shape of the mouth, a classifier recognizes which vowel is being formed, and a synthesizer sings it in real time. It needs no airflow and no vocal folds, only the gestures that shape a vowel.
I treat it as a research object as much as an instrument: a way to ask what a voice is for, and to design for expression and identity rather than only for function.