Interactive Clustering of Cooking Recipe Instructions: Towards the Automatic Detection of Events Involving Kitchen Devices

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Cooking recipes are a rich source of semantic information. They contain instructions for food preparation tasks, specifying the actions that should be carried out which typically involve various ingredients and kitchen devices. In an IoT scenario, instructions in cooking recipes can form the basis for automatically controlling kitchen devices without any programming. However, as these instructions are written in natural language, they first need to be transformed or parsed into machine-interpretable commands. As a step towards this, we investigate methods for identifying the various types of actions (events) that kitchen devices are involved in. We cast this problem as a clustering task, whereby recipe instructions involving a given device of interest, are automatically grouped according to the type of event described. Each sentence in every instruction is represented by its embedding vector which is computed using a BERT-based model, specifically one pre-trained using a Roberta architecture. We cluster these sentence embeddings using our newly proposed interactive machine learning (IML)-based framework underpinned by the HDBScan clustering technique. We demonstrate that our IML framework can detect events in sentences with satisfactory accuracy, reaching almost the same level as human performance.
Original languageEnglish
Title of host publicationIEEE 15th International Conference on Semantic Computing ICSC 2021
Subtitle of host publicationVirtual Conference 27 -29 January 2021
Place of PublicationCalifornia
PublisherIEEE Computer Society
Pages341-346
Number of pages6
ISBN (Print)9781728188997
DOIs
Publication statusPublished - 3 Mar 2021

Publication series

NameInternational Conference on Semantic Computing
PublisherIEEE

Keywords

  • Interactive Machine Learning
  • Clustering
  • Event Schema Induction
  • Instructional Text
  • Sentence Embeddings

Fingerprint

Dive into the research topics of 'Interactive Clustering of Cooking Recipe Instructions: Towards the Automatic Detection of Events Involving Kitchen Devices'. Together they form a unique fingerprint.

Cite this