What Do You Want From Me? Adapting Systems to the Uncertainty of Human Preferences

Gavidia-Calderon, Carlos; Bennaceur, Amel; Kordoni, Anastasia; Levine, Mark and Nuseibeh, Bashar (2022). What Do You Want From Me? Adapting Systems to the Uncertainty of Human Preferences. In: 2022 IEEE/ACM 44th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), 21-29 May 2022, Pittsburgh, PA, USA, pp. 126–130.

DOI: https://doi.org/10.1109/ICSE-NIER55298.2022.9793539

Abstract

Autonomous systems, like drones and self-driving cars, are becoming part of our daily lives. Multiple people interact with them, each with their own expectations regarding system behaviour. To adapt system behaviour to human preferences, we propose and explore a game-theoretic approach. In our architecture, autonomous systems use sensor data to build game-theoretic models of their interaction with humans. In these models, we represent human preferences with types and a probability distribution over them. Game-theoretic analysis then outputs a strategy, that determines how the system should act to maximise utility, given its beliefs over human types. We showcase our approach in a search-and-rescue (SAR) scenario, with a robot in charge of locating victims. According to social psychology, depending on their identity some people are keen to help others, while some prioritise their personal safety. These social identities define what a person favours, so we can map them directly to game-theoretic types. We show that our approach enables a SAR robot to take advantage of human collaboration, outperforming non-adaptive configurations in average number of successful evacuations.

Viewing alternatives

Metrics

Item Actions

Export

You can export this page using these formats Digital Object Identifier - DOI