ABC: Using Object Tracking to Automate Behavioural Coding

Aitor Apaolaza, Robert Haines, Amaia Aizpurua Aguirrezabal, Andy Brown, Michael Evans, Stephen Jolly, Simon Harper, Caroline Jay

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

347 Downloads (Pure)


Video data of people interacting with devices contains rich information about human behaviour that can be used to design or improve user experience. As a first step, it must be interpreted -- or coded -- into a form that can be analyzed systematically. The coding process is currently performed manually, and it can be slow and difficult, and biased by subjectivity. This is particularly problematic when trying to obtain data that should be objective, such as the movements of a user in relation to a device. We describe Automated Behavioural Coding (ABC), an open source object tracking technique designed to log user and device movements, and then output positional data that can be used to model interaction. We validate the technique in a study of dual screen TV viewing, and show that the ABC tool is able to correctly classify the direction of gaze to the TV or tablet up to 95\% of the time, in a fraction of the time it takes to capture this data manually.
Original languageEnglish
Title of host publication Extended Abstracts on Human Factors in Computing Systems: CHI'16; 07 May 2016-12 San Jose, CA, USA
PublisherAssociation for Computing Machinery
Publication statusPublished - May 2016
EventCHI'16 - San Jose, CA, USA
Duration: 7 May 201612 May 2016


CitySan Jose, CA, USA


  • Behavioural coding
  • object tracking
  • visual attention
  • television
  • reproducible methods


Dive into the research topics of 'ABC: Using Object Tracking to Automate Behavioural Coding'. Together they form a unique fingerprint.

Cite this