Stress Testing Deep Learning Models for Prostate Cancer Detection on Biopsies and Surgical Specimens
Document Type
Article
Publication Date
12-11-2024
Abstract
The presence, location, and extent of prostate cancer is assessed by pathologists using H&E-stained tissue slides. Machine learning approaches can accomplish these tasks for both biopsies and radical prostatectomies. Deep learning approaches using convolutional neural networks (CNNs) have been shown to identify cancer in pathologic slides, some securing regulatory approval for clinical use. However, differences in sample processing can subtly alter the morphology between sample types, making it unclear whether deep learning algorithms will consistently work on both types of slide images. Our goal was to investigate whether morphological differences between sample types affected the performance of biopsy-trained cancer detection CNN models when applied to radical prostatectomies and vice versa using multiple cohorts (N = 1,000). Radical prostatectomies (N = 100) and biopsies (N = 50) were acquired from The University of Pennsylvania to train (80%) and validate (20%) a DenseNet CNN for biopsies Mᴮ radical prostatectomies Mᴿ and a combined dataset Mᴮ⁺ᴿ. On a tile level, Mᴮ and Mᴿ achieved F1 scores greater than 0.88 when applied to their own sample type but less than 0.65 when applied across sample types. On a whole-slide level, models achieved significantly better performance on their own sample type compared to the alternative model (p < 0.05) for all metrics. This was confirmed by external validation using digitized biopsy slide images from a clinical trial [NRG Radiation Therapy Oncology Group (RTOG)] (NRG/RTOG 0521, N = 750) via both qualitative and quantitative analyses (p < 0.05). A comprehensive review of model outputs revealed morphologically driven decision making that adversely affected model performance. Mᴮ appeared to be challenged with the analysis of open gland structures, whereas Mᴿ appeared to be challenged with closed gland structures, indicating potential morphological variation between the training sets. These findings suggest that differences in morphology and heterogeneity necessitate the need for more tailored, sample-specific (i.e. biopsy and surgical) machine learning models.
Keywords
biopsy, convolutional neural networks, deep learning, generalizability, interpretability, machine learning, morphology, prostate cancer, radical prostatectomy
Language
English
Publication Title
Journal of Pathology
Grant
1U01CA248226‐01
Rights
© 2024 The Author(s). This is an open access work distributed under the terms of the Creative Commons Attribution-Non-Commercial (https://creativecommons.org/licenses/by-nc/4.0/) which permits non-commercial reuse, distribution, and reproduction in any medium, provided the original work is properly cited.
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Recommended Citation
Flannery, B.T., Sandler, H.M., Lal, P., Feldman, M.D., Santa-Rosario, J.C., Pathak, T., Mirtti, T., Farre, X., Correa, R., Chafe, S., Shah, A., Efstathiou, J.A., Hoffman, K., Hallman, M.A., Straza, M., Jordan, R., Pugh, S.L., Feng, F. and Madabhushi, A. (2025), Stress testing deep learning models for prostate cancer detection on biopsies and surgical specimens. J. Pathol., 265: 146-157. https://doi.org/10.1002/path.6373
Manuscript Version
Final Publisher Version