Towards Developing a Virtual Guitar Instructor through Biometrics Informed Human-Computer Interaction

Christopher Rhodes; Richard Allmendinger; Caroline Jay; Ricardo Climent

Towards Developing a Virtual Guitar Instructor through Biometrics Informed Human-Computer Interaction

Christopher Rhodes, Richard Allmendinger, Caroline Jay, Ricardo Climent

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

Abstract

Within the last few years, wearable sensor technologies have allowed us to access novel biometrics that give us the ability to connect musical gesture to computing systems. Doing this affords us to study how we perform musically and understand the process at data level. However, biometric information is complex and cannot be directly mapped to digital systems. In this work, we study how guitar performance techniques can be captured/analysed towards developing an AI which can provide real-time feedback to guitar students.
We do this by performing musical exercises on the guitar whilst acquiring and processing biometric (plus audiovisual) information during their performance. Our results show: there are notable differences within biometrics when playing a guitar scale in two different ways (legato and staccato) and this outcome can be used to motivate our intention to build an AI guitar tutor.

CCS Concepts: • Human-centered computing → Gestural input.

Additional Key Words and Phrases: Deep learning, Biometrics, Musical performance, Guitar, Multimodal data, Game engines, EMG, HCI

Original language	English
Title of host publication	CHI 2022: CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 April 2022- 5 May 2022
Publication status	Accepted/In press - 1 Feb 2023

Embargoed Document

paper
Accepted author manuscript, 7.97 MB
Licence: CC BY
Embargo ends: 1/01/30

1 Composition

Sonic Tales of a Windsurfer (2024) - 17:45 | 4-channel Fixed media
Climent, R. (Artist), 21 Jun 2024
Research output: Non-textual form › Composition

Cite this

@inproceedings{28dd4d925daf4d9c8d86e66456b16891,

title = "Towards Developing a Virtual Guitar Instructor through Biometrics Informed Human-Computer Interaction",

abstract = "Within the last few years, wearable sensor technologies have allowed us to access novel biometrics that give us the ability to connect musical gesture to computing systems. Doing this affords us to study how we perform musically and understand the process at data level. However, biometric information is complex and cannot be directly mapped to digital systems. In this work, we study how guitar performance techniques can be captured/analysed towards developing an AI which can provide real-time feedback to guitar students.We do this by performing musical exercises on the guitar whilst acquiring and processing biometric (plus audiovisual) information during their performance. Our results show: there are notable differences within biometrics when playing a guitar scale in two different ways (legato and staccato) and this outcome can be used to motivate our intention to build an AI guitar tutor.CCS Concepts: • Human-centered computing → Gestural input.Additional Key Words and Phrases: Deep learning, Biometrics, Musical performance, Guitar, Multimodal data, Game engines, EMG, HCI",

author = "Christopher Rhodes and Richard Allmendinger and Caroline Jay and Ricardo Climent",

year = "2023",

month = feb,

day = "1",

language = "English",

booktitle = "CHI 2022: CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 April 2022- 5 May 2022",

}

Towards Developing a Virtual Guitar Instructor through Biometrics Informed Human-Computer Interaction. / Rhodes, Christopher; Allmendinger, Richard ; Jay, Caroline et al.
CHI 2022: CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 April 2022- 5 May 2022. 2023.

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Towards Developing a Virtual Guitar Instructor through Biometrics Informed Human-Computer Interaction

AU - Rhodes, Christopher

AU - Allmendinger, Richard

AU - Jay, Caroline

AU - Climent, Ricardo

PY - 2023/2/1

Y1 - 2023/2/1

N2 - Within the last few years, wearable sensor technologies have allowed us to access novel biometrics that give us the ability to connect musical gesture to computing systems. Doing this affords us to study how we perform musically and understand the process at data level. However, biometric information is complex and cannot be directly mapped to digital systems. In this work, we study how guitar performance techniques can be captured/analysed towards developing an AI which can provide real-time feedback to guitar students.We do this by performing musical exercises on the guitar whilst acquiring and processing biometric (plus audiovisual) information during their performance. Our results show: there are notable differences within biometrics when playing a guitar scale in two different ways (legato and staccato) and this outcome can be used to motivate our intention to build an AI guitar tutor.CCS Concepts: • Human-centered computing → Gestural input.Additional Key Words and Phrases: Deep learning, Biometrics, Musical performance, Guitar, Multimodal data, Game engines, EMG, HCI

AB - Within the last few years, wearable sensor technologies have allowed us to access novel biometrics that give us the ability to connect musical gesture to computing systems. Doing this affords us to study how we perform musically and understand the process at data level. However, biometric information is complex and cannot be directly mapped to digital systems. In this work, we study how guitar performance techniques can be captured/analysed towards developing an AI which can provide real-time feedback to guitar students.We do this by performing musical exercises on the guitar whilst acquiring and processing biometric (plus audiovisual) information during their performance. Our results show: there are notable differences within biometrics when playing a guitar scale in two different ways (legato and staccato) and this outcome can be used to motivate our intention to build an AI guitar tutor.CCS Concepts: • Human-centered computing → Gestural input.Additional Key Words and Phrases: Deep learning, Biometrics, Musical performance, Guitar, Multimodal data, Game engines, EMG, HCI

M3 - Conference contribution

BT - CHI 2022: CHI Conference on Human Factors in Computing Systems, New Orleans, LA, USA, 29 April 2022- 5 May 2022

ER -

Towards Developing a Virtual Guitar Instructor through Biometrics Informed Human-Computer Interaction

Abstract

Embargoed Document

Fingerprint

Research output

Sonic Tales of a Windsurfer (2024) - 17:45 | 4-channel Fixed media

Cite this