Multi-task SonoEyenet: detection of fetal standardized planes assisted by generated sonographer attention maps

Conference item

We present a novel multi-task convolutional neural network called Multi-task SonoEyeNet (M-SEN ) that learns to generate clinically relevant visual attention maps using sonographer gaze tracking data on input ultrasound (US) video frames so as to assist standardized abdominal circumference (AC) plane detection. Our architecture consists of a generator and a discriminator, which are trained in an adversarial scheme. The generator learns sonographer attention on a given US video frame to predict the frame label (standardized AC plane / background). The discriminator further fine-tunes the predicted attention map by encouraging it to mimick the ground-truth sonographer attention map. The novel model expands the potential clinical usefulness of a previous model by eliminating the requirement of input gaze tracking data during inference without compromising its plane detection performance (Precision: 96.8, Recall: 96.2, F-1 score: 96.5).

Authors: Cai, Y; Sharma, H; Chatelain, P; Noble, J

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Springer
• Host title: Medical Image Computing and Computer Assisted Intervention – MICCAI 2018
• Journal: Lecture Notes in Computer Science
• Series: Lecture Notes in Computer Science
• Publication date: 2018-09-16
• Acceptance date: 2018-05-25
• DOI: 10.1007/978-3-030-00928-1_98
• EISSN: 1611-3349
• ISSN: 0302-9743
• ISBN: 9783030009281
• Keywords: gaze tracking; fetal ultrasound; generative adversarial network; standardized plane detection; multi-task learning; fetal ultrasound, saliency prediction; saliency prediction; generative adversarial network