eduzhai > Applied Sciences > Engineering >

Are pathologist-defined labels reproducible? Comparison of the TUPAC16 mitotic figure dataset with an alternative set of labels

  • king
  • (0) Download
  • 20210505
  • Save

... pages left unread,continue reading

Document pages: 10 pages

Abstract: Pathologist-defined labels are the gold standard for histopathological datasets, regardless of well-known limitations in consistency for some tasks. Todate, some datasets on mitotic figures are available and were used fordevelopment of promising deep learning-based algorithms. In order to assessrobustness of those algorithms and reproducibility of their methods it isnecessary to test on several independent datasets. The influence of differentlabeling methods of these available datasets is currently unknown. To tacklethis, we present an alternative set of labels for the images of the auxiliarymitosis dataset of the TUPAC16 challenge. Additional to manual mitotic figurescreening, we used a novel, algorithm-aided labeling process, that allowed tominimize the risk of missing rare mitotic figures in the images. All potentialmitotic figures were independently assessed by two pathologists. The novel,publicly available set of labels contains 1,999 mitotic figures (+28.80 ) andadditionally includes 10,483 labels of cells with high similarities to mitoticfigures (hard examples). We found significant difference comparing F 1 scoresbetween the original label set (0.549) and the new alternative label set(0.735) using a standard deep learning object detection architecture. Themodels trained on the alternative set showed higher overall confidence values,suggesting a higher overall label consistency. Findings of the present studyshow that pathologists-defined labels may vary significantly resulting innotable difference in the model performance. Comparison of deep learning-basedalgorithms between independent datasets with different labeling methods shouldbe done with caution.

Please select stars to rate!


0 comments Sign in to leave a comment.

    Data loading, please wait...