An evaluation of AI auto-segmentation for Head & Neck cancer

Abstract

Abstract Purpose To investigate performance of multiple commercial AI auto-segmentation systems for head and neck (H&N) radiotherapy treatment planning, to inform on associated quality assurance (QA) requirements, and to investigate patient views on the use of such technology in the planning of their own radiotherapy treatment. Methods Four commercial AI auto-segmentation systems were used to generate contours for five commonly used H&N organs at risk (OAR) using 50 H&N patient datasets. Resulting contours were compared to gold standard contours using multiple similarity metrics. One commercial system was used to generate four common H&N OARs on 500 patient datasets. Auto-segmented OARs were compared to manually-created clinical contours using Dice Similarity Coefficient (DSC) and failure rates were identified using previously calculated expected DSC values. An existing standardised patient questionnaire was distributed to cancer patients who were receiving radiotherapy at the Clatterbridge Cancer Centre between November 2021 and March 2022. Results Overall performance differences between commercial systems were found to be statistically insignificant for all comparison metrics. For the 500 patient study, true failure rates for the four OARs investigated were 0.4% for brainstem, 2.2% for mandible, 1.4% for left parotid and 0.8% for right parotid. The patient questionnaire results showed that there was a moderately negative patient view towards the use of AI in radiotherapy. Conclusions Comparable levels of performance were observed between all systems. This indicates that AI-based auto-segmentation products are developing at a similar pace in terms of the quality of contours produced. The true failure rate for AI auto-segmentation systems in the H&N region for the OARs investigated is extremely low and it is therefore advised that QA of resulting auto-segmented OARs should utilise automated methods. There are clear patient concerns around the use of AI in radiotherapy and therefore both staff and patient education is required.

Date of Award	31 Dec 2022
Original language	English
Awarding Institution	The University of Manchester
Supervisor	Julia-Claire Handley (Supervisor) & Robert Chuter (Supervisor)