EntityBot: Actionable Entity Recommendations for Everyday Digital Task

Tung Vuong, Salvatore Andolina, Giulio Jacucci, Pedram Daee, Khalil Klouche, Mats Sjöberg, Tuukka Ruotsalo, Samuel Kaski

Research output: Chapter in Book/Conference proceedingConference contributionpeer-review

Abstract

Our everyday digital tasks require access to information from a wide range of applications and systems. Although traditional search systems can help find information, they usually operate within one application (e.g., email client or web browser) and require the user's cognitive effort and attention to formulate proper search queries. In this paper, we demonstrate EntityBot, a system that proactively provides useful and supporting entities across application boundaries without requiring explicit query formulation. Our methodology is to exploit the context from screen frames captured every 2 seconds to recommend relevant entities for the current task. Recommendations are not restricted to only documents but include various kinds of entities, such as applications, documents, contact persons, and keywords representing the tasks. Recommendations are actionable, that is, a user can perform actions on recommended entities, such as opening documents and applications. The EntityBot also includes support for interactivity, allowing the user to affect the recommendations by providing explicit feedback on the entities. The usefulness of entity recommendations and their impact on user behavior has been evaluated in a user study based on real-world tasks. Quantitative and qualitative results suggest that the system had an actual impact on the tasks and led to high user satisfaction.

Original languageEnglish
Title of host publicationCHI 2022
Subtitle of host publicationExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems
Place of PublicationNew York, NY
PublisherAssociation for Computing Machinery
Pages1-4
Number of pages4
ISBN (Electronic)9781450391566
DOIs
Publication statusPublished - 28 Apr 2022
EventACM SIGCHI Annual Conference on Human Factors in Computing Systems - Virtual, Online
Duration: 30 Apr 20225 May 2022

Conference

ConferenceACM SIGCHI Annual Conference on Human Factors in Computing Systems
Period30/04/225/05/22

Keywords

  • Proactive information retrieval
  • real-world tasks
  • user intent modeling

Research Beacons, Institutes and Platforms

  • Institute for Data Science and AI
  • Digital Futures
  • Sustainable Futures

Fingerprint

Dive into the research topics of 'EntityBot: Actionable Entity Recommendations for Everyday Digital Task'. Together they form a unique fingerprint.

Cite this