How Humans Perceive Human-like Behavior in Video Game Navigation

Evelyn Zuniga; Stephanie Milani; Guy Leroy; Jaroslaw Rzepecki; Raluca Georgescu; Ida Momennejad; Dave Bignell; Mingfei Sun; Alison Shaw; Gavin Costello; Mikhail Jacob; Sam Devlin; Katja Hofmann

doi:10.1145/3491101.3519735

How Humans Perceive Human-like Behavior in Video Game Navigation

Evelyn Zuniga
, Stephanie Milani
, Guy Leroy
, Jaroslaw Rzepecki
, Raluca Georgescu
, Ida Momennejad
, Dave Bignell
, Mingfei Sun
, Alison Shaw
, Gavin Costello
, Mikhail Jacob
, Sam Devlin
, Katja Hofmann

Machine Learning and Robotics

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

Abstract

The goal of this paper is to understand how people assess human-likeness in human-and AI-generated behavior. To this end, we present a qualitative study of hundreds of crowd-sourced assessments of human-likeness of behavior in a 3D video game navigation task. In particular, we focus on an AI agent that has passed a Turing Test, in the sense that human judges were not able to reliably distinguish between videos of a human and AI agent navigating on a quantitative level. Our insights shine a light on the characteristics that people consider as human-like. Understanding these characteristics is a key first step for improving AI agents in the future.

Original language	English
Title of host publication	CHI 2022
Subtitle of host publication	Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems
Editors	Simone Barbosa, Cliff Lampe, Caroline Appert, David A. Shamma
Place of Publication	New York, NY
Publisher	Association for Computing Machinery
Pages	1-11
Number of pages	11
ISBN (Electronic)	9781450391566
DOIs	https://doi.org/10.1145/3491101.3519735
Publication status	Published - 28 Apr 2022
Event	2022 CHI Conference on Human Factors in Computing Systems, CHI EA 2022 - Virtual, Online, United States Duration: 30 Apr 2022 → 5 May 2022

Conference

Conference	2022 CHI Conference on Human Factors in Computing Systems, CHI EA 2022
Country/Territory	United States
City	Virtual, Online
Period	30/04/22 → 5/05/22

Keywords

Believable AI
Human-AI Interaction
Human-subject Study

Access to Document

10.1145/3491101.3519735

Cite this

Zuniga, E., Milani, S., Leroy, G., Rzepecki, J., Georgescu, R., Momennejad, I., Bignell, D., Sun, M., Shaw, A., Costello, G., Jacob, M., Devlin, S., & Hofmann, K. (2022). How Humans Perceive Human-like Behavior in Video Game Navigation. In S. Barbosa, C. Lampe, C. Appert, & D. A. Shamma (Eds.), CHI 2022: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (pp. 1-11). Article 391 Association for Computing Machinery. https://doi.org/10.1145/3491101.3519735

@inproceedings{ca0157f9ec8744c58c8a82beab932db4,

title = "How Humans Perceive Human-like Behavior in Video Game Navigation",

abstract = "The goal of this paper is to understand how people assess human-likeness in human-and AI-generated behavior. To this end, we present a qualitative study of hundreds of crowd-sourced assessments of human-likeness of behavior in a 3D video game navigation task. In particular, we focus on an AI agent that has passed a Turing Test, in the sense that human judges were not able to reliably distinguish between videos of a human and AI agent navigating on a quantitative level. Our insights shine a light on the characteristics that people consider as human-like. Understanding these characteristics is a key first step for improving AI agents in the future.",

keywords = "Believable AI, Human-AI Interaction, Human-subject Study",

author = "Evelyn Zuniga and Stephanie Milani and Guy Leroy and Jaroslaw Rzepecki and Raluca Georgescu and Ida Momennejad and Dave Bignell and Mingfei Sun and Alison Shaw and Gavin Costello and Mikhail Jacob and Sam Devlin and Katja Hofmann",

note = "Publisher Copyright: {\textcopyright} 2022 ACM.; 2022 CHI Conference on Human Factors in Computing Systems, CHI EA 2022 ; Conference date: 30-04-2022 Through 05-05-2022",

year = "2022",

month = apr,

day = "28",

doi = "10.1145/3491101.3519735",

language = "English",

pages = "1--11",

editor = "Simone Barbosa and Cliff Lampe and Caroline Appert and Shamma, \{David A.\}",

booktitle = "CHI 2022",

publisher = "Association for Computing Machinery",

address = "United States",

}

Zuniga, E, Milani, S, Leroy, G, Rzepecki, J, Georgescu, R, Momennejad, I, Bignell, D, Sun, M, Shaw, A, Costello, G, Jacob, M, Devlin, S & Hofmann, K 2022, How Humans Perceive Human-like Behavior in Video Game Navigation. in S Barbosa, C Lampe, C Appert & DA Shamma (eds), CHI 2022: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems., 391, Association for Computing Machinery, New York, NY, pp. 1-11, 2022 CHI Conference on Human Factors in Computing Systems, CHI EA 2022, Virtual, Online, United States, 30/04/22. https://doi.org/10.1145/3491101.3519735

How Humans Perceive Human-like Behavior in Video Game Navigation. / Zuniga, Evelyn; Milani, Stephanie; Leroy, Guy et al.
CHI 2022: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems. ed. / Simone Barbosa; Cliff Lampe; Caroline Appert; David A. Shamma. New York, NY: Association for Computing Machinery, 2022. p. 1-11 391.

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - How Humans Perceive Human-like Behavior in Video Game Navigation

AU - Zuniga, Evelyn

AU - Milani, Stephanie

AU - Leroy, Guy

AU - Rzepecki, Jaroslaw

AU - Georgescu, Raluca

AU - Momennejad, Ida

AU - Bignell, Dave

AU - Sun, Mingfei

AU - Shaw, Alison

AU - Costello, Gavin

AU - Jacob, Mikhail

AU - Devlin, Sam

AU - Hofmann, Katja

PY - 2022/4/28

Y1 - 2022/4/28

N2 - The goal of this paper is to understand how people assess human-likeness in human-and AI-generated behavior. To this end, we present a qualitative study of hundreds of crowd-sourced assessments of human-likeness of behavior in a 3D video game navigation task. In particular, we focus on an AI agent that has passed a Turing Test, in the sense that human judges were not able to reliably distinguish between videos of a human and AI agent navigating on a quantitative level. Our insights shine a light on the characteristics that people consider as human-like. Understanding these characteristics is a key first step for improving AI agents in the future.

AB - The goal of this paper is to understand how people assess human-likeness in human-and AI-generated behavior. To this end, we present a qualitative study of hundreds of crowd-sourced assessments of human-likeness of behavior in a 3D video game navigation task. In particular, we focus on an AI agent that has passed a Turing Test, in the sense that human judges were not able to reliably distinguish between videos of a human and AI agent navigating on a quantitative level. Our insights shine a light on the characteristics that people consider as human-like. Understanding these characteristics is a key first step for improving AI agents in the future.

KW - Believable AI

KW - Human-AI Interaction

KW - Human-subject Study

UR - https://www.scopus.com/pages/publications/85129705349

U2 - 10.1145/3491101.3519735

DO - 10.1145/3491101.3519735

M3 - Conference contribution

AN - SCOPUS:85129705349

SP - 1

EP - 11

BT - CHI 2022

A2 - Barbosa, Simone

A2 - Lampe, Cliff

A2 - Appert, Caroline

A2 - Shamma, David A.

PB - Association for Computing Machinery

CY - New York, NY

T2 - 2022 CHI Conference on Human Factors in Computing Systems, CHI EA 2022

Y2 - 30 April 2022 through 5 May 2022

ER -

Zuniga E, Milani S, Leroy G, Rzepecki J, Georgescu R, Momennejad I et al. How Humans Perceive Human-like Behavior in Video Game Navigation. In Barbosa S, Lampe C, Appert C, Shamma DA, editors, CHI 2022: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems. New York, NY: Association for Computing Machinery. 2022. p. 1-11. 391 doi: 10.1145/3491101.3519735

How Humans Perceive Human-like Behavior in Video Game Navigation

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this