MURAL - Maynooth University Research Archive Library



    The Effect of Multimodal Emotional Expression and Agent Appearance on Trust in Human-Agent Interaction


    Torre, Ilaria and Carrigan, Emma and McDonnell, Rachel and Domijan, Katarina and McCabe, Killian and Harte, Naomi (2019) The Effect of Multimodal Emotional Expression and Agent Appearance on Trust in Human-Agent Interaction. In: Motion, Interaction and Games (MIG '19). Association for Computing Machinery, p. 14. ISBN 9781450369947

    [img]
    Preview
    Download (2MB) | Preview


    Share your research

    Twitter Facebook LinkedIn GooglePlus Email more...



    Add this article to your Mendeley library


    Abstract

    Emotional expressivity can boost trust in human-human and humanmachine interaction. As a multimodal phenomenon, previous research argued that a mismatch in the expressive channels provides evidence of joint audio-video emotional processing. However, while previous work studied this from the point of view of emotion recognition and processing, not much is known about what effect a multimodal agent would have on a human-agent interaction task. Also, agent appearance could influence this interaction too. Here we manipulated the agent’s multimodal emotional expression ("smiling face" and "smiling voice", or both) and agent type (photorealistic or cartoon-like virtual human) and assessed people’s trust toward this agent. We measured trust using a mixed-methods approach, combining behavioural data from a survival task, questionnaire ratings and qualitative comments. These methods gave different results: while people commented on the importance of emotional expressivity in the agent’s voice, this factor had limited influence on trusting behaviours; while people rated the cartoon-like agent on several traits higher than the photorealistic one, the agent’s style also was not the most influential feature on people’s trusting behaviour. These results highlight the contribution of a mixedmethods approach in human-machine interaction, as both explicit and implicit perception and behaviour will contribute to the success of the interaction.

    Item Type: Book Section
    Additional Information: Cite as: Ilaria Torre, Emma Carrigan, Rachel McDonnell, Katarina Domijan, Killian McCabe, and Naomi Harte. 2019. The Effect of Multimodal Emotional Expression and Agent Appearance on Trust in Human-Agent Interaction. In Motion, Interaction and Games (MIG '19). Association for Computing Machinery, New York, NY, USA, Article 14, 1–6. DOI: https://doi.org/10.1145/3359566.3360065
    Keywords: trust; virtual agents; multimodal emotional expression; mixed-methods;
    Academic Unit: Faculty of Science and Engineering > Mathematics and Statistics
    Faculty of Science and Engineering > Research Institutes > Hamilton Institute
    Item ID: 14362
    Identification Number: https://doi.org/10.1145/3359566.3360065
    Depositing User: Katarina Domijan
    Date Deposited: 21 Apr 2021 14:24
    Publisher: Association for Computing Machinery
    Refereed: Yes
    URI:

    Repository Staff Only(login required)

    View Item Item control page

    Downloads

    Downloads per month over past year