Agent with Tangent-Based Formulation and Anatomical Perception for Standard Plane Localization in 3D Ultrasound

Yuxin Zou; Haoran Dou; Yuhao Huang; Xin Yang; Jikuan Qian; Chaojiong Zhen; Xiaodan Ji; Nishant Ravikumar; Guoqiang Chen; Weijun Huang; Alejandro F. Frangi; Dong Ni

doi:10.1007/978-3-031-16440-8_29

Agent with Tangent-Based Formulation and Anatomical Perception for Standard Plane Localization in 3D Ultrasound

Yuxin Zou, Haoran Dou, Yuhao Huang, Xin Yang, Jikuan Qian, Chaojiong Zhen, Xiaodan Ji, Nishant Ravikumar, Guoqiang Chen, Weijun Huang, Alejandro F. Frangi, Dong Ni^*

^*Corresponding author for this work

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

Abstract

Standard plane (SP) localization is essential in routine clinical ultrasound (US) diagnosis. Compared to 2D US, 3D US can acquire multiple view planes in one scan and provide complete anatomy with the addition of coronal plane. However, manually navigating SPs in 3D US is laborious and biased due to the orientation variability and huge search space. In this study, we introduce a novel reinforcement learning (RL) framework for automatic SP localization in 3D US. Our contribution is three-fold. First, we formulate SP localization in 3D US as a tangent-point-based problem in RL to restructure the action space and significantly reduce the search space. Second, we design an auxiliary task learning strategy to enhance the model’s ability to recognize subtle differences crossing Non-SPs and SPs in plane search. Finally, we propose a spatial-anatomical reward to effectively guide learning trajectories by exploiting spatial and anatomical information simultaneously. We explore the efficacy of our approach on localizing four SPs on uterus and fetal brain datasets. The experiments indicate that our approach achieves a high localization accuracy as well as robust performance.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2022 - 25th International Conference, Proceedings
Editors	Linwei Wang, Qi Dou, P. Thomas Fletcher, Stefanie Speidel, Shuo Li
Publisher	Springer Nature
Pages	300-309
Number of pages	10
ISBN (Print)	9783031164392
DOIs	https://doi.org/10.1007/978-3-031-16440-8_29
Publication status	Published - 2022
Event	25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022 - Singapore, Singapore Duration: 18 Sept 2022 → 22 Sept 2022

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13434 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022
Country/Territory	Singapore
City	Singapore
Period	18/09/22 → 22/09/22

Keywords

Reinforcement learning
Standard plane localization
Ultrasound

Access to Document

10.1007/978-3-031-16440-8_29

Cite this

Zou, Y., Dou, H., Huang, Y., Yang, X., Qian, J., Zhen, C., Ji, X., Ravikumar, N., Chen, G., Huang, W., Frangi, A. F., & Ni, D. (2022). Agent with Tangent-Based Formulation and Anatomical Perception for Standard Plane Localization in 3D Ultrasound. In L. Wang, Q. Dou, P. T. Fletcher, S. Speidel, & S. Li (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2022 - 25th International Conference, Proceedings (pp. 300-309). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13434 LNCS). Springer Nature. https://doi.org/10.1007/978-3-031-16440-8_29

Zou, Yuxin ; Dou, Haoran ; Huang, Yuhao et al. / Agent with Tangent-Based Formulation and Anatomical Perception for Standard Plane Localization in 3D Ultrasound. Medical Image Computing and Computer Assisted Intervention – MICCAI 2022 - 25th International Conference, Proceedings. editor / Linwei Wang ; Qi Dou ; P. Thomas Fletcher ; Stefanie Speidel ; Shuo Li. Springer Nature, 2022. pp. 300-309 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{3367470657d348df999d22e7a1bad082,

title = "Agent with Tangent-Based Formulation and Anatomical Perception for Standard Plane Localization in 3D Ultrasound",

abstract = "Standard plane (SP) localization is essential in routine clinical ultrasound (US) diagnosis. Compared to 2D US, 3D US can acquire multiple view planes in one scan and provide complete anatomy with the addition of coronal plane. However, manually navigating SPs in 3D US is laborious and biased due to the orientation variability and huge search space. In this study, we introduce a novel reinforcement learning (RL) framework for automatic SP localization in 3D US. Our contribution is three-fold. First, we formulate SP localization in 3D US as a tangent-point-based problem in RL to restructure the action space and significantly reduce the search space. Second, we design an auxiliary task learning strategy to enhance the model{\textquoteright}s ability to recognize subtle differences crossing Non-SPs and SPs in plane search. Finally, we propose a spatial-anatomical reward to effectively guide learning trajectories by exploiting spatial and anatomical information simultaneously. We explore the efficacy of our approach on localizing four SPs on uterus and fetal brain datasets. The experiments indicate that our approach achieves a high localization accuracy as well as robust performance.",

keywords = "Reinforcement learning, Standard plane localization, Ultrasound",

author = "Yuxin Zou and Haoran Dou and Yuhao Huang and Xin Yang and Jikuan Qian and Chaojiong Zhen and Xiaodan Ji and Nishant Ravikumar and Guoqiang Chen and Weijun Huang and Frangi, {Alejandro F.} and Dong Ni",

note = "Funding Information: Acknowledgement. This work was supported by the grant from National Natural Science Foundation of China (Nos. 62171290, 62101343), Shenzhen-Hong Kong Joint Research Program (No. SGDX20201103095613036), Shenzhen Science and Technology Innovations Committee (No. 20200812143441001), the Royal Academy of Engineering (INSILEX CiET1819/19), the Royal Society Exchange Programme CROSSLINK IES\NSFC\201380, and Engineering and Physical Sciences Research Council (EPSRC) programs TUSCA EP/V04799X/1. Publisher Copyright: {\textcopyright} 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022 ; Conference date: 18-09-2022 Through 22-09-2022",

year = "2022",

doi = "10.1007/978-3-031-16440-8_29",

language = "English",

isbn = "9783031164392",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Nature",

pages = "300--309",

editor = "Linwei Wang and Qi Dou and Fletcher, {P. Thomas} and Stefanie Speidel and Shuo Li",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2022 - 25th International Conference, Proceedings",

address = "United States",

}

Zou, Y, Dou, H, Huang, Y, Yang, X, Qian, J, Zhen, C, Ji, X, Ravikumar, N, Chen, G, Huang, W, Frangi, AF & Ni, D 2022, Agent with Tangent-Based Formulation and Anatomical Perception for Standard Plane Localization in 3D Ultrasound. in L Wang, Q Dou, PT Fletcher, S Speidel & S Li (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2022 - 25th International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13434 LNCS, Springer Nature, pp. 300-309, 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022, Singapore, Singapore, 18/09/22. https://doi.org/10.1007/978-3-031-16440-8_29

Agent with Tangent-Based Formulation and Anatomical Perception for Standard Plane Localization in 3D Ultrasound. / Zou, Yuxin; Dou, Haoran; Huang, Yuhao et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2022 - 25th International Conference, Proceedings. ed. / Linwei Wang; Qi Dou; P. Thomas Fletcher; Stefanie Speidel; Shuo Li. Springer Nature, 2022. p. 300-309 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13434 LNCS).

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Agent with Tangent-Based Formulation and Anatomical Perception for Standard Plane Localization in 3D Ultrasound

AU - Zou, Yuxin

AU - Dou, Haoran

AU - Huang, Yuhao

AU - Yang, Xin

AU - Qian, Jikuan

AU - Zhen, Chaojiong

AU - Ji, Xiaodan

AU - Ravikumar, Nishant

AU - Chen, Guoqiang

AU - Huang, Weijun

AU - Frangi, Alejandro F.

AU - Ni, Dong

N1 - Funding Information: Acknowledgement. This work was supported by the grant from National Natural Science Foundation of China (Nos. 62171290, 62101343), Shenzhen-Hong Kong Joint Research Program (No. SGDX20201103095613036), Shenzhen Science and Technology Innovations Committee (No. 20200812143441001), the Royal Academy of Engineering (INSILEX CiET1819/19), the Royal Society Exchange Programme CROSSLINK IES\NSFC\201380, and Engineering and Physical Sciences Research Council (EPSRC) programs TUSCA EP/V04799X/1. Publisher Copyright: © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

PY - 2022

Y1 - 2022

N2 - Standard plane (SP) localization is essential in routine clinical ultrasound (US) diagnosis. Compared to 2D US, 3D US can acquire multiple view planes in one scan and provide complete anatomy with the addition of coronal plane. However, manually navigating SPs in 3D US is laborious and biased due to the orientation variability and huge search space. In this study, we introduce a novel reinforcement learning (RL) framework for automatic SP localization in 3D US. Our contribution is three-fold. First, we formulate SP localization in 3D US as a tangent-point-based problem in RL to restructure the action space and significantly reduce the search space. Second, we design an auxiliary task learning strategy to enhance the model’s ability to recognize subtle differences crossing Non-SPs and SPs in plane search. Finally, we propose a spatial-anatomical reward to effectively guide learning trajectories by exploiting spatial and anatomical information simultaneously. We explore the efficacy of our approach on localizing four SPs on uterus and fetal brain datasets. The experiments indicate that our approach achieves a high localization accuracy as well as robust performance.

AB - Standard plane (SP) localization is essential in routine clinical ultrasound (US) diagnosis. Compared to 2D US, 3D US can acquire multiple view planes in one scan and provide complete anatomy with the addition of coronal plane. However, manually navigating SPs in 3D US is laborious and biased due to the orientation variability and huge search space. In this study, we introduce a novel reinforcement learning (RL) framework for automatic SP localization in 3D US. Our contribution is three-fold. First, we formulate SP localization in 3D US as a tangent-point-based problem in RL to restructure the action space and significantly reduce the search space. Second, we design an auxiliary task learning strategy to enhance the model’s ability to recognize subtle differences crossing Non-SPs and SPs in plane search. Finally, we propose a spatial-anatomical reward to effectively guide learning trajectories by exploiting spatial and anatomical information simultaneously. We explore the efficacy of our approach on localizing four SPs on uterus and fetal brain datasets. The experiments indicate that our approach achieves a high localization accuracy as well as robust performance.

KW - Reinforcement learning

KW - Standard plane localization

KW - Ultrasound

UR - http://www.scopus.com/inward/record.url?scp=85139066457&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-16440-8_29

DO - 10.1007/978-3-031-16440-8_29

M3 - Conference contribution

AN - SCOPUS:85139066457

SN - 9783031164392

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 300

EP - 309

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2022 - 25th International Conference, Proceedings

A2 - Wang, Linwei

A2 - Dou, Qi

A2 - Fletcher, P. Thomas

A2 - Speidel, Stefanie

A2 - Li, Shuo

PB - Springer Nature

T2 - 25th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2022

Y2 - 18 September 2022 through 22 September 2022

ER -

Zou Y, Dou H, Huang Y, Yang X, Qian J, Zhen C et al. Agent with Tangent-Based Formulation and Anatomical Perception for Standard Plane Localization in 3D Ultrasound. In Wang L, Dou Q, Fletcher PT, Speidel S, Li S, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2022 - 25th International Conference, Proceedings. Springer Nature. 2022. p. 300-309. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-16440-8_29