The project

PepperMint (Interacting with Pepper: mutual learning of turn-taking practices in HRI) is funded by the ASLAN Labex. It proposes an explorative study of embodied turn-taking practices in task-related Human-Robot Interaction (HRI) to improve the social abilities of robots and make HRI more natural to humans. The project initiates a cooperation between researchers in AI (Artificial Intelligence) (LIRIS) and CA (Conversation Analysis) (ICAR and GenZ Oulu - Finland). It investigates if and how CA findings on natural occurring interaction can be used to develop innovative and effective AI models for HRI. The project is grounded in a detailed multimodal analysis of turn-taking in naturally occurring HRI, putting forward the emergence of turn allocation as complex sequential and multimodal practices.

The project will build upon existing works on AI/ML (Machine Learning) algorithms of the state of the art to program an application for reception and orientation of people in an university library. This application will be used for the recording of human-robot interactions and turn-taking practices, that will be used in CA studies to identify successful interactions. The outcome of the CA studies will be used to develop a new version of the robot’s algorithms, with the objective to improve HRI.

Steps of the project

In a complex three-step-methodology, combining AI and CA, it proposes:

1. a first version of the robot with state of the art (SoA) algorithms, and ad hoc turn-taking practices, based on the setting and CA findings about turn-taking practices in human interaction and HRI,
2. the video-recording of non-elicited HRI in the targeted setting and the analysis of turn-taking practices, considering opening and closing sequences and
3. the proposal of new AI models for HRI, coupling developmenta learning and CA findings. The project adopts a cooperative perspective, considering humans and robots as teammates that learn to mutually adapt their turn-taking practices.

Expected results

The project will have a strong social impact with regard to an improvement of HRI. The multimodal analysis of HRI in natural occurring settings contributes to a better understanding of how humans accept robots in these settings, how they approach them, start an interaction, “use” them - or not. It will reveal interaction problems as well as their reasons from an interactional point of view, in order to improve technologically what can be improved or maybe to find other types of possible solutions. The project aims to draw especially on these problems for further research in AI. Moreover, our project is based on a conception of humans and robots as teammates, each one taking into account the strengths and the weaknesses of the other. This idea objects to the common anthropomorphic view of robots and will increase humans engagement in their interaction with robots, helping the progress of research and development in HRI and the deployment of HRI-based applications.