Use of gazing/pointing direction and social feedback
First, the user plays an imitation game with the robot, where the user reproduces commmunicative gestures performed by the robot (attention scaffolding, turn taking, head nods and head shakes). The robot then creates a Hidden Markov Model (HMM) characterizing the specificities of the user's gestures. The user then directs the attention of the robot to an object on the table by looking and pointing at it. The robot points at the object of interest and ask the user for an evaluation of its selection.
In this video, the user points at the 3 objects one by one. The robot still points at the most probable object and ask for an evaluation. The user can then clarify its selection and focus the attention of the robot again to the object that he intended to show to the robot.