Calinon, S. and Billard, A. (2005)
Recognition and Reproduction of Gestures using a Probabilistic Framework combining PCA, ICA and HMM
In Proc. of the Intl Conf. on Machine Learning (ICML), Bonn, Germany, pp. 105-112.
Abstract
This paper explores the issue of recognizing, generalizing and reproducing arbitrary gestures. We aim at extracting a representation that encapsulates only the key aspects of the gesture and discards the variability intrinsic to each person's motion. We compare a decomposition into principal components (PCA) and independent components (ICA) as a first step of preprocessing in order to decorrelate and denoise the data, as well as to reduce the dimensionality of the dataset to make this one tractable. In a second stage of processing, we explore the use of a probabilistic encoding through continuous Hidden Markov Models (HMMs), as a way to encapsulate the sequential nature and intrinsic variability of the motions in stochastic finite state automata. Finally, the method is validated in a humanoid robot to reproduce a variety of gestures performed by a human demonstrator.
Bibtex reference
@InProceedings{CalinonBillard05,
author="S. Calinon and A. Billard",
title="Recognition and Reproduction of Gestures using a Probabilistic Framework combining {PCA}, {ICA} and {HMM}",
booktitle="Proceedings of the International Conference on Machine Learning ({ICML})",
year="2005",
month="August",
location="Bonn, Germany",
pages="105--112"
}
Video
Observation of the user's gestures through stereoscopic vision and inertial motion sensors attached to the upper body of the user.
Learning and reproduction of a set of various gestures (waving goodbye, knocking on a door, dringking from a glass and writing of alphabet letters on a vertical board). For the last set of gestures, we see on the vidoe that the robot is able to generalize the motion by writing the alphabet letters on a different plane.
Source codes
Download
Â
Download GMM latent space sourcecode
Usage
Unzip the file and run 'demo1' in Matlab.
References
- Calinon, S. and Billard, A. (2005) Recognition and Reproduction of Gestures using a Probabilistic Framework combining PCA, ICA and HMM. In Proceedings of the International Conference on Machine Learning (ICML), pp. 105-112.
Demo 1 - Demonstration of a probabilistic encoding through Gaussian Mixture Model (GMM) in a latent space of motion extracted by Principal Component Analysis (PCA)
This programs loads a dataset, finds a latent space of lower dimensionality encapsulating the important characteristics of the motion using Principal Component Analysis (PCA), trains a Gaussian Mixture Model (GMM) using the data projected in this latent space, and projects back the Gaussian distributions in the original data space. Training a GMM with EM algorithm usually fails to find a good local optimum when data are high-dimensional. By projecting the original dataset in a latent space as a pre-processing step, GMM training can be performed in a robust way, and the Gaussian parameters can be projected back to the original data space.