Zappi, V., Pistillo, A., Calinon, S., Brogni, A. and Caldwell, D.G. (2012)
Music Expression with a Robot Manipulator Used as a Bidirectional Tangible Interface
EURASIP Journal on Audio, Speech, and Music Processing 2012:2, 1-11, Special Issue on Music Content Processing by and for Robots (Springer).
Abstract
The availability of haptic interfaces in music content processing offers interesting possibilities of performer-instrument interaction for musical expression. These new musical instruments can precisely modulate the haptic feedback, and map it to a sonic output, thus offering new artistic content creation possibilities. With this article, we investigate the use of a robotic arm as a bidirectional tangible interface for musical expression, actively modifying the compliant control strategy to create a bind between gestural input and music output. The user can define recursive modulations of music parameters by grasping and gradually refining periodic movements on a gravity-compensated robot manipulator. The robot learns on-line the new desired trajectory, increasing its stiffness as the modulation refinement proceeds. This article reports early results of an artistic performance that has been carried out with the collaboration of a musician, who played with the robot as part of his live stage setup.
Bibtex reference
@article{Zappi12,
author="Zappi, V. and Pistillo, A. and Calinon, S. and Brogni, A. and Caldwell, D. G.",
title="Music Expression with a Robot Manipulator Used as a Bidirectional Tangible Interface",
journal="EURASIP Journal on Audio, Speech, and Music Processing (SpringerOpen), Special Issue on Music Content Processing by and for Robots",
year="2012",
volume="2012",
number="2",
pages="1--11",
month="January"
}
Video
This video shows how the active compliant control capabilities of a robotic arm can be exploited as an artistic tool for music composition. Such haptic interfaces offers interesting possibilities of performer-instrument interaction for musical expression. These new musical instruments can precisely modulate the haptic feedback, and map it to a sonic output, thus offering new artistic content creation possibilities. The robotic arm is used here as a bidirectional interface for musical expression, actively modifying the compliant control strategy to create a bind between gestural input and music output. The user can define recursive modulations of music parameters by grasping and gradually refining periodic movements on a gravity-compensated robot manipulator. The robot learns on-line the new desired trajectory, increasing its stiffness as the modulation refinement proceeds.