Searching Collaborative Agents for Multi-plane Localization in 3D Ultrasound

Yuhao Huang; Xin Yang; Rui Li; Jikuan Qian; Xiaoqiong Huang; Wenlong Shi; Haoran Dou; Chaoyu Chen; Yuanji Zhang; Huanjia Luo; Alejandro F Frangi; Yi Xiong; Dong Ni

doi:10.1007/978-3-030-59716-0_53

Searching Collaborative Agents for Multi-plane Localization in 3D Ultrasound

Yuhao Huang, Xin Yang, Rui Li, Jikuan Qian, Xiaoqiong Huang, Wenlong Shi, Haoran Dou, Chaoyu Chen, Yuanji Zhang, Huanjia Luo, Alejandro F Frangi, Yi Xiong, Dong Ni^*

^*Corresponding author for this work

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

Abstract

3D ultrasound (US) is widely used due to its rich diagnostic information, portability and low cost. Automated standard plane (SP) localization in US volume not only improves efficiency and reduces user-dependence, but also boosts 3D US interpretation. In this study, we propose a novel Multi-Agent Reinforcement Learning (MARL) framework to localize multiple uterine SPs in 3D US simultaneously. Our contribution is two-fold. First, we equip the MARL with a one-shot neural architecture search (NAS) module to obtain the optimal agent for each plane. Specifically, Gradient-based search using Differentiable Architecture Sampler (GDAS) is employed to accelerate and stabilize the training process. Second, we propose a novel collaborative strategy to strengthen agents’ communication. Our strategy uses recurrent neural network (RNN) to learn the spatial relationship among SPs effectively. Extensively validated on a large dataset, our approach achieves the accuracy of 7.05^∘/2.21 mm, 8.62^∘/2.36 mm and 5.93^∘/0.89 mm for the mid-sagittal, transverse and coronal plane localization, respectively. The proposed MARL framework can significantly increase the plane localization accuracy and reduce the computational cost and model size.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings
Editors	Anne L. Martel, Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, Maria A. Zuluaga, S. Kevin Zhou, Daniel Racoceanu, Leo Joskowicz
Publisher	Springer Nature
Pages	553-562
Number of pages	10
ISBN (Print)	9783030597153
DOIs	https://doi.org/10.1007/978-3-030-59716-0_53
Publication status	Published - 4 Oct 2020
Event	23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020 - Lima, Peru Duration: 4 Oct 2020 → 8 Oct 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12263 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020
Country/Territory	Peru
City	Lima
Period	4/10/20 → 8/10/20

Keywords

3D ultrasound
NAS
Reinforcement learning

Access to Document

10.1007/978-3-030-59716-0_53

Cite this

Huang, Y., Yang, X., Li, R., Qian, J., Huang, X., Shi, W., Dou, H., Chen, C., Zhang, Y., Luo, H., Frangi, A. F., Xiong, Y., & Ni, D. (2020). Searching Collaborative Agents for Multi-plane Localization in 3D Ultrasound. In A. L. Martel, P. Abolmaesumi, D. Stoyanov, D. Mateus, M. A. Zuluaga, S. K. Zhou, D. Racoceanu, & L. Joskowicz (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings (pp. 553-562). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12263 LNCS). Springer Nature. https://doi.org/10.1007/978-3-030-59716-0_53

Huang, Yuhao ; Yang, Xin ; Li, Rui et al. / Searching Collaborative Agents for Multi-plane Localization in 3D Ultrasound. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings. editor / Anne L. Martel ; Purang Abolmaesumi ; Danail Stoyanov ; Diana Mateus ; Maria A. Zuluaga ; S. Kevin Zhou ; Daniel Racoceanu ; Leo Joskowicz. Springer Nature, 2020. pp. 553-562 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{cf55d3735ffb44439544b453eba7153e,

title = "Searching Collaborative Agents for Multi-plane Localization in 3D Ultrasound",

abstract = "3D ultrasound (US) is widely used due to its rich diagnostic information, portability and low cost. Automated standard plane (SP) localization in US volume not only improves efficiency and reduces user-dependence, but also boosts 3D US interpretation. In this study, we propose a novel Multi-Agent Reinforcement Learning (MARL) framework to localize multiple uterine SPs in 3D US simultaneously. Our contribution is two-fold. First, we equip the MARL with a one-shot neural architecture search (NAS) module to obtain the optimal agent for each plane. Specifically, Gradient-based search using Differentiable Architecture Sampler (GDAS) is employed to accelerate and stabilize the training process. Second, we propose a novel collaborative strategy to strengthen agents{\textquoteright} communication. Our strategy uses recurrent neural network (RNN) to learn the spatial relationship among SPs effectively. Extensively validated on a large dataset, our approach achieves the accuracy of 7.05∘/2.21 mm, 8.62∘/2.36 mm and 5.93∘/0.89 mm for the mid-sagittal, transverse and coronal plane localization, respectively. The proposed MARL framework can significantly increase the plane localization accuracy and reduce the computational cost and model size.",

keywords = "3D ultrasound, NAS, Reinforcement learning",

author = "Yuhao Huang and Xin Yang and Rui Li and Jikuan Qian and Xiaoqiong Huang and Wenlong Shi and Haoran Dou and Chaoyu Chen and Yuanji Zhang and Huanjia Luo and Frangi, {Alejandro F} and Yi Xiong and Dong Ni",

note = "Funding Information: Acknowledgement. This work was supported by the grant from National Key R&D Program of China (No. 2019YFC0118300), Shenzhen Peacock Plan (No. KQTD2016053112051497, KQJSCX20180328095606003) and Medical Scientific Research Foundation of Guangdong Province, China (No. B2018031). Funding Information: This work was supported by the grant from National Key R&D Program of China (No. 2019YFC0118300), Shenzhen Peacock Plan (No. KQTD2016053112051497, KQJSCX20180328095606003) and Medical Scientific Research Foundation of Guangdong Province, China (No. B2018031). Publisher Copyright: {\textcopyright} 2020, Springer Nature Switzerland AG.; 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020 ; Conference date: 04-10-2020 Through 08-10-2020",

year = "2020",

month = oct,

day = "4",

doi = "10.1007/978-3-030-59716-0_53",

language = "English",

isbn = "9783030597153",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Nature",

pages = "553--562",

editor = "Martel, {Anne L.} and Purang Abolmaesumi and Danail Stoyanov and Diana Mateus and Zuluaga, {Maria A.} and Zhou, {S. Kevin} and Daniel Racoceanu and Leo Joskowicz",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings",

address = "United States",

}

Huang, Y, Yang, X, Li, R, Qian, J, Huang, X, Shi, W, Dou, H, Chen, C, Zhang, Y, Luo, H, Frangi, AF, Xiong, Y & Ni, D 2020, Searching Collaborative Agents for Multi-plane Localization in 3D Ultrasound. in AL Martel, P Abolmaesumi, D Stoyanov, D Mateus, MA Zuluaga, SK Zhou, D Racoceanu & L Joskowicz (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12263 LNCS, Springer Nature, pp. 553-562, 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020, Lima, Peru, 4/10/20. https://doi.org/10.1007/978-3-030-59716-0_53

Searching Collaborative Agents for Multi-plane Localization in 3D Ultrasound. / Huang, Yuhao; Yang, Xin; Li, Rui et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings. ed. / Anne L. Martel; Purang Abolmaesumi; Danail Stoyanov; Diana Mateus; Maria A. Zuluaga; S. Kevin Zhou; Daniel Racoceanu; Leo Joskowicz. Springer Nature, 2020. p. 553-562 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12263 LNCS).

Research output: Chapter in Book/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Searching Collaborative Agents for Multi-plane Localization in 3D Ultrasound

AU - Huang, Yuhao

AU - Yang, Xin

AU - Li, Rui

AU - Qian, Jikuan

AU - Huang, Xiaoqiong

AU - Shi, Wenlong

AU - Dou, Haoran

AU - Chen, Chaoyu

AU - Zhang, Yuanji

AU - Luo, Huanjia

AU - Frangi, Alejandro F

AU - Xiong, Yi

AU - Ni, Dong

N1 - Funding Information: Acknowledgement. This work was supported by the grant from National Key R&D Program of China (No. 2019YFC0118300), Shenzhen Peacock Plan (No. KQTD2016053112051497, KQJSCX20180328095606003) and Medical Scientific Research Foundation of Guangdong Province, China (No. B2018031). Funding Information: This work was supported by the grant from National Key R&D Program of China (No. 2019YFC0118300), Shenzhen Peacock Plan (No. KQTD2016053112051497, KQJSCX20180328095606003) and Medical Scientific Research Foundation of Guangdong Province, China (No. B2018031). Publisher Copyright: © 2020, Springer Nature Switzerland AG.

PY - 2020/10/4

Y1 - 2020/10/4

N2 - 3D ultrasound (US) is widely used due to its rich diagnostic information, portability and low cost. Automated standard plane (SP) localization in US volume not only improves efficiency and reduces user-dependence, but also boosts 3D US interpretation. In this study, we propose a novel Multi-Agent Reinforcement Learning (MARL) framework to localize multiple uterine SPs in 3D US simultaneously. Our contribution is two-fold. First, we equip the MARL with a one-shot neural architecture search (NAS) module to obtain the optimal agent for each plane. Specifically, Gradient-based search using Differentiable Architecture Sampler (GDAS) is employed to accelerate and stabilize the training process. Second, we propose a novel collaborative strategy to strengthen agents’ communication. Our strategy uses recurrent neural network (RNN) to learn the spatial relationship among SPs effectively. Extensively validated on a large dataset, our approach achieves the accuracy of 7.05∘/2.21 mm, 8.62∘/2.36 mm and 5.93∘/0.89 mm for the mid-sagittal, transverse and coronal plane localization, respectively. The proposed MARL framework can significantly increase the plane localization accuracy and reduce the computational cost and model size.

AB - 3D ultrasound (US) is widely used due to its rich diagnostic information, portability and low cost. Automated standard plane (SP) localization in US volume not only improves efficiency and reduces user-dependence, but also boosts 3D US interpretation. In this study, we propose a novel Multi-Agent Reinforcement Learning (MARL) framework to localize multiple uterine SPs in 3D US simultaneously. Our contribution is two-fold. First, we equip the MARL with a one-shot neural architecture search (NAS) module to obtain the optimal agent for each plane. Specifically, Gradient-based search using Differentiable Architecture Sampler (GDAS) is employed to accelerate and stabilize the training process. Second, we propose a novel collaborative strategy to strengthen agents’ communication. Our strategy uses recurrent neural network (RNN) to learn the spatial relationship among SPs effectively. Extensively validated on a large dataset, our approach achieves the accuracy of 7.05∘/2.21 mm, 8.62∘/2.36 mm and 5.93∘/0.89 mm for the mid-sagittal, transverse and coronal plane localization, respectively. The proposed MARL framework can significantly increase the plane localization accuracy and reduce the computational cost and model size.

KW - 3D ultrasound

KW - NAS

KW - Reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=85092724789&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-59716-0_53

DO - 10.1007/978-3-030-59716-0_53

M3 - Conference contribution

SN - 9783030597153

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 553

EP - 562

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings

A2 - Martel, Anne L.

A2 - Abolmaesumi, Purang

A2 - Stoyanov, Danail

A2 - Mateus, Diana

A2 - Zuluaga, Maria A.

A2 - Zhou, S. Kevin

A2 - Racoceanu, Daniel

A2 - Joskowicz, Leo

PB - Springer Nature

T2 - 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020

Y2 - 4 October 2020 through 8 October 2020

ER -

Huang Y, Yang X, Li R, Qian J, Huang X, Shi W et al. Searching Collaborative Agents for Multi-plane Localization in 3D Ultrasound. In Martel AL, Abolmaesumi P, Stoyanov D, Mateus D, Zuluaga MA, Zhou SK, Racoceanu D, Joskowicz L, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings. Springer Nature. 2020. p. 553-562. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-59716-0_53

Searching Collaborative Agents for Multi-plane Localization in 3D Ultrasound

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this