Towards robust needle segmentation and tracking in pediatric endoscopic surgery

Neonatal tracheoesophageal fistula surgery poses technical challenges to surgeons, given the limited workspace and fragile tissues. In previous studies from our collaborators, a neonatal chest model was developed to allow surgeons to enhance their performance, such as suturing ability, before conducting actual surgery. Endoscopic images are recorded while the model is used, and surgeon skill can be manually assessed by using a 29-point checklist. However, that is a time-consuming process. In the checklist, there are 15 points that regard needle position and angle that could be automatized if the needle could be efficiently tracked. This paper is a first step towards the goal of tracking the needle. Pixel HSV color space channels, opponent color space channels, and pixel oriented gradient information are used as features to train a random forest model. Three methods are compared in the segmentation stage: single pixel features, pixel and its immediate 10-by-10 square window features, and the features of randomly offset pixels in a larger 169-by-169 window. Our analysis using 9-fold cross-validation shows that using randomly offset pixels increases needle segmentation f-measure by 385 times when comparing with single pixel color, and by 3 times when comparing with the immediate square window even though the same amount of memory is used. The output in the segmentation step is fed into a particle filter to track the full state of the needle.