eduzhai > Applied Sciences > Engineering >

M2Net Multi-modal Multi-channel Network for Overall Survival Time Prediction of Brain Tumor Patients

  • Save

... pages left unread,continue reading

Document pages: 11 pages

Abstract: Early and accurate prediction of overall survival (OS) time can help toobtain better treatment planning for brain tumor patients. Although many OStime prediction methods have been developed and obtain promising results, thereare still several issues. First, conventional prediction methods rely onradiomic features at the local lesion area of a magnetic resonance (MR) volume,which may not represent the full image or model complex tumor patterns. Second,different types of scanners (i.e., multi-modal data) are sensitive to differentbrain regions, which makes it challenging to effectively exploit thecomplementary information across multiple modalities and also preserve themodality-specific properties. Third, existing methods focus on predictionmodels, ignoring complex data-to-label relationships. To address the aboveissues, we propose an end-to-end OS time prediction model; namely, Multi-modalMulti-channel Network (M2Net). Specifically, we first project the 3D MR volumeonto 2D images in different directions, which reduces computational costs,while preserving important information and enabling pre-trained models to betransferred from other tasks. Then, we use a modality-specific network toextract implicit and high-level features from different MR scans. A multi-modalshared network is built to fuse these features using a bilinear pooling model,exploiting their correlations to provide complementary information. Finally, weintegrate the outputs from each modality-specific network and the multi-modalshared network to generate the final prediction result. Experimental resultsdemonstrate the superiority of our M2Net model over other methods.

Please select stars to rate!

         

0 comments Sign in to leave a comment.

    Data loading, please wait...
×