EN
登录

从全玻片图像到生物标志物预测:计算病理学中的端到端弱监督深度学习

From whole-slide image to biomarker prediction: end-to-end weakly supervised deep learning in computational pathology

Nature 等信源发布 2024-09-16 21:24

可切换为仅中文


AbstractHematoxylin- and eosin-stained whole-slide images (WSIs) are the foundation of diagnosis of cancer. In recent years, development of deep learning-based methods in computational pathology has enabled the prediction of biomarkers directly from WSIs. However, accurately linking tissue phenotype to biomarkers at scale remains a crucial challenge for democratizing complex biomarkers in precision oncology.

摘要苏木精和伊红染色的全幻灯片图像(WSI)是诊断癌症的基础。近年来,计算病理学中基于深度学习的方法的发展使得能够直接从WSI预测生物标志物。然而,在精确肿瘤学中,准确地将组织表型与生物标志物联系起来仍然是使复杂生物标志物民主化的关键挑战。

This protocol describes a practical workflow for solid tumor associative modeling in pathology (STAMP), enabling prediction of biomarkers directly from WSIs by using deep learning. The STAMP workflow is biomarker agnostic and allows for genetic and clinicopathologic tabular data to be included as an additional input, together with histopathology images.

该协议描述了病理学实体瘤关联建模(STAMP)的实用工作流程,可以通过深度学习直接从WSIs预测生物标志物。STAMP工作流程与生物标志物无关,可以将遗传和临床病理表格数据与组织病理学图像一起作为附加输入。

The protocol consists of five main stages that have been successfully applied to various research problems: formal problem definition, data preprocessing, modeling, evaluation and clinical translation. The STAMP workflow differentiates itself through its focus on serving as a collaborative framework that can be used by clinicians and engineers alike for setting up research projects in the field of computational pathology.

该协议由五个主要阶段组成,这些阶段已成功应用于各种研究问题:形式化问题定义,数据预处理,建模,评估和临床翻译。STAMP工作流程与众不同,它专注于作为一个协作框架,临床医生和工程师都可以使用它来建立计算病理学领域的研究项目。

As an example task, we applied STAMP to the prediction of microsatellite instability (MSI) status in colorectal cancer, showing accurate performance for the identification of tumors high in MSI. Moreover, we provide an open-source code base, which has been deployed at several hospitals across the globe to set up computational pathology workflows.

作为一个示例任务,我们将STAMP应用于预测结直肠癌中的微卫星不稳定性(MSI)状态,显示出识别MSI高的肿瘤的准确性能。此外,我们提供了一个开源代码库,已在全球多家医院部署,以建立计算病理工作流程。

The STAMP workflow requires one workday of hands-on computational execution and basic command line knowledge.Key points.

STAMP工作流需要一个工作日的动手计算执行和基本命令行知识。关键点。

STAMP (solid tumor associative modeling in pathology) is a practical workflow for end-to-end weakly supervised deep learning in computational pathology, enabling prediction of biomarkers directly from whole-slide images.

STAMP(病理学中的实体瘤关联建模)是计算病理学中端到端弱监督深度学习的实用工作流程,可以直接从整个幻灯片图像预测生物标志物。

This protocol differentiates itself from others by providing a collaborative framework through which clinical researchers can work with engineers to set up a complete computational pathology project.

该协议通过提供一个合作框架将自身与其他协议区分开来,通过该框架,临床研究人员可以与工程师合作建立一个完整的计算病理学项目。

Access through your institution

通过您的机构访问

Buy or subscribe

购买或订阅

This is a preview of subscription content, access via your institution

这是订阅内容的预览,可通过您的机构访问

Access options

访问选项

Access through your institution

通过您的机构访问

Access through your institution

通过您的机构访问

Change institution

Buy or subscribe

购买或订阅

Access Nature and 54 other Nature Portfolio journalsGet Nature+, our best-value online-access subscription24,99 € / 30 dayscancel any timeLearn moreSubscription info for Chinese customersWe have a dedicated website for our Chinese customers. Please go to naturechina.com to subscribe to this journal.Go to naturechina.comBuy this articlePurchase on SpringerLinkInstant access to full article PDFBuy nowPrices may be subject to local taxes which are calculated during checkout.

Access Nature和54篇其他Nature Portfolio journalsGet Nature+,我们最有价值的在线订阅24,99欧元/30天,随时为中国客户获取更多订阅信息我们为中国客户提供了一个专门的网站。请访问naturechina.com订阅本期刊。访问naturechina.comBuy本文在Springerlink上购买即时访问完整文章PDFBuy now价格可能需要缴纳结帐时计算的地方税。

Additional access options:

其他访问选项:

Log in

登录

Learn about institutional subscriptions

了解机构订阅

Read our FAQs

阅读我们的常见问题

Contact customer support

联系客户支持

Fig. 1: Conceptual overview of the protocol.Fig. 2: Computational workflow from WSI to patient-level biomarker prediction.Fig. 3: Factors influencing the required sample size in computational pathology projects.Fig. 4: Positioning of the STAMP software.Fig. 5: Anticipated results of the evaluation phase of the protocol for the analysis of CRC from TCGA and the CPTAC.Fig.

图1:协议的概念概述。图2:从WSI到患者水平生物标志物预测的计算工作流程。图3:影响计算病理学项目中所需样本量的因素。图4:STAMP软件的定位。图5:TCGA和CPTAC分析CRC的协议评估阶段的预期结果。图。

6: Anticipated results of the translation phase of the protocol for the analysis of CRC from the CPTAC..

6: 。。

Data availability

数据可用性

Histopathology slides and genomics data from TCGA and CPTAC were used to train and validate the models. The slides for TCGA are available at https://portal.gdc.cancer.gov/. The slides for CPTAC are available at https://proteomics.cancer.gov/data-portal. The molecular and clinical data for TCGA and CPTAC used in the experiments are available at https://github.com/KatherLab/cancer-metadata.

来自TCGA和CPTAC的组织病理学载玻片和基因组学数据用于训练和验证模型。TCGA的幻灯片可在https://portal.gdc.cancer.gov/.CPTAC的幻灯片可在https://proteomics.cancer.gov/data-portal.实验中使用的TCGA和CPTAC的分子和临床数据可在https://github.com/KatherLab/cancer-metadata.

Source data are provided with this paper..

本文提供了源数据。。

Code availability

代码可用性

The open-source STAMP software for the implementation of the MSI experiments is available on GitHub (https://github.com/KatherLab/STAMP).

GitHub上提供了用于实现MSI实验的开源STAMP软件(https://github.com/KatherLab/STAMP)。

ReferencesShmatko, A., Ghaffari Laleh, N., Gerstung, M. & Kather, J. N. Artificial intelligence in histopathology: enhancing cancer research and clinical oncology. Nat. Cancer 3, 1026–1038 (2022).Article

Referenceshmatko,A.,Ghaffari-Laleh,N.,Gerstung,M。&Kather,J.N。组织病理学中的人工智能:增强癌症研究和临床肿瘤学。《自然癌症》31026-1038(2022)。文章

PubMed

PubMed

Google Scholar

谷歌学者

Ghaffari Laleh, N. et al. Benchmarking weakly-supervised deep learning pipelines for whole slide classification in computational pathology. Med. Image Anal. 79, 102474 (2022).Article

Ghaffari-Laleh,N.等人。在计算病理学中对整个幻灯片分类的弱监督深度学习管道进行基准测试。医学图像肛门。79102474(2022)。文章

PubMed

PubMed

Google Scholar

谷歌学者

Foersch, S. et al. Deep learning for diagnosis and survival prediction in soft tissue sarcoma. Ann. Oncol. 32, 1178–1187 (2021).Article

。安科。321178-1187(2021)。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Klein, C. et al. Artificial intelligence for solid tumour diagnosis in digital pathology. Br. J. Pharmacol. 178, 4291–4315 (2021).Article

Klein,C.等人,《数字病理学中实体瘤诊断的人工智能》。Br.J.药理学。。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Woerl, A.-C. et al. Deep learning predicts molecular subtype of muscle-invasive bladder cancer from conventional histopathological slides. Eur. Urol. 78, 256–264 (2020).Article

Woerl,A.-C.等人,《深度学习》从常规组织病理学载玻片预测肌肉浸润性膀胱癌的分子亚型。。78256-264(2020)。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Hong, R., Liu, W., DeLair, D., Razavian, N. & Fenyö, D. Predicting endometrial cancer subtypes and molecular features from histopathology images using multi-resolution deep learning models. Cell Rep. Med. 2, 100400 (2021).Article

Hong,R.,Liu,W.,DeLair,D.,Razavian,N。&Fenyö,D。使用多分辨率深度学习模型从组织病理学图像预测子宫内膜癌亚型和分子特征。细胞代表医学2100400(2021)。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Kather, J. N. et al. Predicting survival from colorectal cancer histology slides using deep learning: a retrospective multicenter study. PLoS Med. 16, e1002730 (2019).Article

Kather,J.N.等人。使用深度学习预测结直肠癌组织学载玻片的存活率:一项回顾性多中心研究。《公共科学图书馆·医学》16,e1002730(2019)。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Ghaffari Laleh, N. et al. Deep Learning for interpretable end-to-end survival (E-ESurv) prediction in gastrointestinal cancer histopathology. Proceedings of the MICCAI Workshop on Computational Pathology. PMLR 156, 81–93 (2021).Foersch, S. et al. Multistain deep learning for prediction of prognosis and therapy response in colorectal cancer.

Ghaffari Laleh,N。等人。胃肠道癌症组织病理学中可解释的端到端生存(E-ESurv)预测的深度学习。MICCAI计算病理学研讨会论文集。PMLR 156,81-93(2021)。Foersch,S.等人。多阶段深度学习用于预测结直肠癌的预后和治疗反应。

Nat. Med. 29, 430–439 (2023).Article .

《自然医学》29430-439(2023)。文章。

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Wang, C.-W. et al. Weakly supervised deep learning for prediction of treatment effectiveness on ovarian cancer from histopathology images. Comput. Med. Imaging Graph. 99, 102093 (2022).Article

Wang,C.-W.等人。从组织病理学图像预测卵巢癌治疗效果的弱监督深度学习。计算机。医学成像图。99102093(2022)。文章

PubMed

PubMed

Google Scholar

谷歌学者

Ghaffari Laleh, N., Ligero, M., Perez-Lopez, R. & Kather, J. N. Facts and hopes on the use of artificial intelligence for predictive immunotherapy biomarkers in cancer. Clin. Cancer Res. 29, 316–323 (2023).Article

Ghaffari-Laleh,N.,Ligero,M.,Perez-Lopez,R。&Kather,J.N。关于使用人工智能预测癌症免疫治疗生物标志物的事实和希望。临床。癌症研究29316-323(2023)。文章

PubMed

PubMed

Google Scholar

谷歌学者

Kather, J. N. et al. Pan-cancer image-based detection of clinically actionable genetic alterations. Nat. Cancer 1, 789–799 (2020).Article

Kather,J.N.等人。基于泛癌图像的临床可行基因改变检测。《自然癌症》1789-799(2020)。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Kanavati, F. et al. Weakly-supervised learning for lung carcinoma classification using deep learning. Sci. Rep. 10, 9297 (2020).Article

Kanavati,F。等人。使用深度学习进行肺癌分类的弱监督学习。科学。代表109297(2020)。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Wang, X. et al. Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE Trans. Cybern. 50, 3950–3962 (2020).Article

Wang,X。等人。用于全幻灯片肺癌图像分析的弱监督深度学习。IEEE Trans。赛博。503950–3962(2020)。文章

PubMed

PubMed

Google Scholar

谷歌学者

Kather, J. N. et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat. Med. 25, 1054–1056 (2019).Article

。《自然医学》第251054-1056页(2019年)。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Bilal, M. et al. Development and validation of a weakly supervised deep learning framework to predict the status of molecular pathways and key mutations in colorectal cancer from routine histology images: a retrospective study. Lancet Digit. Health 3, e763–e772 (2021).Article

Bilal,M.等人。开发和验证弱监督深度学习框架,以从常规组织学图像预测结直肠癌中分子途径和关键突变的状态:一项回顾性研究。柳叶刀数字。健康3,e763–e772(2021)。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Schrammen, P. L. et al. Weakly supervised annotation-free cancer detection and prediction of genotype in routine histopathology. J. Pathol. 256, 50–60 (2022).Article

Schrammen,P.L.等人。常规组织病理学中无注释癌症检测和基因型预测的弱监督。J、 病理学。。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Echle, A. et al. Clinical-grade detection of microsatellite instability in colorectal tumors by deep learning. Gastroenterology 159, 1406–1416.e11 (2020).Article

Echle,A。等人。通过深度学习对结直肠肿瘤中微卫星不稳定性的临床等级检测。胃肠病学1591406-1416.e11(2020)。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Zeng, Q. et al. Artificial intelligence predicts immune and inflammatory gene signatures directly from hepatocellular carcinoma histology. J. Hepatol. 77, 116–127 (2022).Article

Zeng,Q。等人。人工智能直接从肝细胞癌组织学预测免疫和炎症基因特征。J、 肝脏。77116-127(2022)。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Jaroensri, R. et al. Deep learning models for histologic grading of breast cancer and association with disease prognosis. NPJ Breast Cancer 8, 113 (2022).Article

Jaroensri,R。等人。乳腺癌组织学分级的深度学习模型及其与疾病预后的关系。NPJ乳腺癌8113(2022)。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Li, C. et al. Weakly supervised mitosis detection in breast histopathology images using concentric loss. Med. Image Anal. 53, 165–178 (2019).Article

Li,C.等人。使用同心丢失对乳腺组织病理学图像中的有丝分裂检测进行弱监督。医学图像肛门。53165-178(2019)。文章

PubMed

PubMed

Google Scholar

谷歌学者

Zheng, Q. et al. A weakly supervised deep learning model and human-machine fusion for accurate grading of renal cell carcinoma from histopathology slides. Cancers (Basel) 15, 3198 (2023).Article

Zheng,Q。等人。一种弱监督深度学习模型和人机融合,用于从组织病理学载玻片中准确分级肾细胞癌。癌症(巴塞尔)153198(2023)。文章

PubMed

PubMed

Google Scholar

谷歌学者

Muti, H. S. et al. The Aachen Protocol for Deep Learning Histopathology: A Hands-on Guide for Data Preprocessing. Available at https://oa.mg/work/10.5281/zenodo.3694994 (2020).Graziani, M. et al. Attention-based interpretable regression of gene expression in histology. Interpretability of Machine Intelligence in Medical Image Computing: 5th International Workshop, iMIMIC 2022, Held in Conjunction with MICCAI 2022, Singapore, Singapore, September 22, 2022, Proceedings 44–60 (Springer-Verlag, 2022).Campanella, G.

Muti,H.S.等人,《亚琛深度学习组织病理学协议:数据预处理实践指南》。可在https://oa.mg/work/10.5281/zenodo.3694994(2020年)。Graziani,M。等人。组织学中基因表达的基于注意力的可解释回归。医学图像计算中机器智能的可解释性:第五届国际研讨会,iMIMIC 2022,与MICCAI 2022联合举办,新加坡,新加坡,2022年9月22日,会议记录44-60(Springer-Verlag,2022)。坎帕内拉,G。

et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25, 1301–1309 (2019).Article .

等。临床级计算病理学,在整个幻灯片图像上使用弱监督深度学习。《自然医学》251301-1309(2019)。文章。

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Schmauch, B. et al. A deep learning model to predict RNA-Seq expression of tumours from whole slide images. Nat. Commun. 11, 1–15 (2020).Article

Schmauch,B。等人。一种深度学习模型,用于从整个幻灯片图像预测肿瘤的RNA-Seq表达。国家公社。11,1-15(2020)。文章

Google Scholar

谷歌学者

Lu, M. Y. et al. AI-based pathology predicts origins for cancers of unknown primary. Nature 594, 106–110 (2021).Article

Lu,M.Y.等人,基于AI的病理学预测未知原发性癌症的起源。自然594106-110(2021)。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Wagner, S. J. et al. Built to last? Reproducibility and reusability of deep learning algorithms in computational pathology. Mod. Pathol. 37, 100350 (2023).Article

Wagner,S.J.等人,建造到最后?计算病理学中深度学习算法的可重复性和可重用性。国防部。病理学。37100350(2023)。文章

PubMed

PubMed

Google Scholar

谷歌学者

Veldhuizen, G. P. et al. Deep learning-based subtyping of gastric cancer histology predicts clinical outcome: a multi-institutional retrospective study. Gastric Cancer 26, 708–720 (2023).Article

Veldhuizen,G.P.等人。基于深度学习的胃癌组织学亚型预测临床结果:一项多机构回顾性研究。胃癌26708-720(2023)。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Muti, H. S. et al. Deep learning trained on lymph node status predicts outcome from gastric cancer histopathology: a retrospective multicentric study. Eur. J. Cancer 194, 113335 (2023).Article

Muti,H.S.等人。接受淋巴结状态培训的深度学习可预测胃癌组织病理学的结果:一项回顾性多中心研究。《欧洲癌症杂志》194113335(2023)。文章

PubMed

PubMed

Google Scholar

谷歌学者

Saldanha, O. L. et al. Direct prediction of genetic aberrations from pathology images in gastric cancer with swarm learning. Gastric Cancer 26, 264–274 (2023).Article

Saldanha,O.L.等人。使用群体学习从胃癌病理图像直接预测遗传畸变。胃癌26264-274(2023)。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Niehues, J. M. et al. Generalizable biomarker prediction from cancer pathology slides with self-supervised deep learning: a retrospective multi-centric study. Cell Rep. Med. 4, 100980 (2023).Article

Niehues,J.M.等人。通过自我监督的深度学习从癌症病理学幻灯片中进行可推广的生物标志物预测:一项回顾性多中心研究。细胞代表医学4100980(2023)。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Wagner, S. J. et al. Transformer-based biomarker prediction from colorectal cancer histology: a large-scale multicentric study. Cancer Cell 41, 1650–1661.e4 (2023).Jiang, X. et al. End-to-end prognostication in colorectal cancer by deep learning: a retrospective, multicentre study. Lancet Digit.

Wagner,S.J.等人。基于变压器的结直肠癌组织学生物标志物预测:一项大规模多中心研究。癌细胞411650-1661.e4(2023)。。柳叶刀数字。

Health 6, e33–e43 (2024).Article .

健康6,e33-e43(2024)。文章。

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Chatterji, S. et al. Prediction models for hormone receptor status in female breast cancer do not extend to males: further evidence of sex-based disparity in breast cancer. NPJ Breast Cancer 9, 91 (2023).Article

Chatterji,S.等人,《女性乳腺癌中激素受体状态的预测模型不适用于男性:乳腺癌中基于性别差异的进一步证据》。NPJ乳腺癌9,91(2023)。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Hewitt, K. J. et al. Direct image to subtype prediction for brain tumors using deep learning. Neurooncol. Adv. 5, vdad139 (2023).PubMed

休伊特(Hewitt,K.J.)等人,《利用深度学习对脑肿瘤进行图像到亚型的直接预测》。神经肿瘤学。Adv.5,vdad139(2023)。PubMed出版社

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Saldanha, O. L. et al. Self-supervised attention-based deep learning for pan-cancer mutation prediction from histopathology. NPJ Precis. Oncol. 7, 35 (2023).Article

Saldanha,O.L.等人。基于自我监督的注意力深度学习,用于从组织病理学预测泛癌突变。NPJ精度。Oncol公司。7,35(2023)。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Loeffler, C. M. L. et al. Direct prediction of Homologous Recombination Deficiency from routine histology in ten different tumor types with attention-based Multiple Instance Learning: a development and validation study. Preprint at https://www.medrxiv.org/content/10.1101/2023.03.08.23286975v1 (2023).El Nahhas, O.

Loeffler,C.M.L.等人。通过基于注意力的多实例学习,从十种不同肿瘤类型的常规组织学直接预测同源重组缺陷:开发和验证研究。预印于https://www.medrxiv.org/content/10.1101/2023.03.08.23286975v1(2023年)。埃尔纳哈斯,O。

S. M. et al. Regression-based Deep-Learning predicts molecular biomarkers from pathology slides. Nat. Commun. 15, 1–253 (2024)..

S、 基于回归的深度学习可以从病理切片中预测分子生物标志物。国家公社。15,1–253(2024年)。。

Google Scholar

谷歌学者

Wang, X. et al. Transformer-based unsupervised contrastive learning for histopathological image classification. Med. Image Anal. 81, 102559 (2022).Article

Wang,X。等。基于变压器的无监督对比学习用于组织病理学图像分类。医学图像肛门。81102559(2022)。文章

PubMed

PubMed

Google Scholar

谷歌学者

Causality in digital medicine. Nat. Commun. 12, 5471 (2021).Wölflein, G. et al. Benchmarking pathology feature extractors for whole slide image classification. Preprint at https://arxiv.org/abs/2311.11772 (2023).Goode, A., Gilbert, B., Harkes, J., Jukic, D. & Satyanarayanan, M. OpenSlide: a vendor-neutral software foundation for digital pathology.

数字医学中的因果关系。国家公社。125471(2021)。Wölflein,G.等人。用于全幻灯片图像分类的基准病理特征提取器。预印于https://arxiv.org/abs/2311.11772(2023年)。。

J. Pathol. Inform. 4, 27 (2013).Article .

J、 病理学。。4,27(2013)。文章。

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Bankhead, P. et al. QuPath: open source software for digital pathology image analysis. Sci. Rep. 7, 16878 (2017).Article

Bankhead,P。等。QuPath:用于数字病理图像分析的开源软件。科学。代表716878(2017)。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Ghassemi, M., Oakden-Rayner, L. & Beam, A. L. The false hope of current approaches to explainable artificial intelligence in health care. Lancet Digit. Health 3, e745–e750 (2021).Article

Ghassemi,M.,Oakden Rayner,L。和Beam,A.L。目前在医疗保健中解释人工智能的方法的错误希望。柳叶刀数字。健康3,e745–e750(2021)。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Paszke, A. et al. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems (eds Wallach, H. et al.) Vol. 32 (Curran Associates, Inc., 2019).Jorge Cardoso, M. et al. MONAI: an open-source framework for deep learning in healthcare.

Paszke,A。et al。PyTorch:一个命令式、高性能的深度学习库。《神经信息处理系统的进展》(eds Wallach,H.et al。)第32卷(Curran Associates,Inc.,2019)。Jorge Cardoso,M.等人,《MONAI:医疗保健深度学习的开源框架》。

Preprint at https://arxiv.org/abs/2211.02701 (2022).Martinez, K. & Cupitt, J. VIPS - a highly tuned image processing software architecture. In IEEE International Conference on Image Processing 2005. Genova, Italy II–574 (IEEE, 2005).Janowczyk, A., Zuo, R., Gilmore, H., Feldman, M. & Madabhushi, A. HistoQC: an open-source quality control tool for digital pathology slides.

预印于https://arxiv.org/abs/2211.02701(2022年)。Martinez,K.&Cupitt,J.VIPS-一种高度优化的图像处理软件体系结构。在2005年IEEE国际图像处理会议上。意大利热那亚II–574(IEEE,2005)。Janowczyk,A.,Zuo,R.,Gilmore,H.,Feldman,M。&Madabhushi,A。HistoQC:数字病理切片的开源质量控制工具。

JCO Clin. Cancer Inform. 3, 1–7 (2019).Article .

JCO临床。癌症通报。3,1-7(2019)。文章。

PubMed

PubMed

Google Scholar

谷歌学者

Pedersen, A. et al. FastPathology: an open-source platform for deep learning-based research and decision support in digital pathology. IEEE Access 9, 58216–58229 (2021).Article

Pedersen,A.等人,《快速病理学:数字病理学中基于深度学习的研究和决策支持的开源平台》。IEEE Access 958216–58229(2021)。文章

Google Scholar

谷歌学者

Pocock, J. et al. TIAToolbox as an end-to-end library for advanced tissue image analytics. Commun. Med. (Lond.) 2, 120 (2022).Article

Pocock,J。等人。TIAToolbox作为先进组织图像分析的端到端库。Commun公司。医学(Lond。)2120(2022)。文章

PubMed

PubMed

Google Scholar

谷歌学者

Lu, M. Y. et al. Data-efficient and weakly supervised computational pathology on whole-slide images. Nat. Biomed. Eng. 5, 555–570 (2021).Article

Lu,M.Y.等人。全幻灯片图像上的数据有效且监督较弱的计算病理学。自然生物医学。工程5555-570(2021)。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Verghese, G. et al. Computational pathology in cancer diagnosis, prognosis, and prediction—present day and prospects. J. Pathol. 260, 551–563 (2023).Article

Verghese,G.等人,《计算病理学在癌症诊断、预后和预测中的现状和前景》。J、 病理学。260551-563(2023)。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Saillard, C. et al. Validation of MSIntuit as an AI-based pre-screening tool for MSI detection from colorectal cancer histology slides. Nat. Commun. 14, 6695 (2023).Article

Saillard,C。等人。验证MSIntuit作为基于AI的预筛选工具,用于从结直肠癌组织学载玻片中检测MSI。国家公社。146695(2023)。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Greenson, J. K. et al. Pathologic predictors of microsatellite instability in colorectal cancer. Am. J. Surg. Pathol. 33, 126–133 (2009).Article

Greenson,J.K.等。结直肠癌微卫星不稳定性的病理预测因子。美国外科病理学杂志。33126-133(2009)。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Cancer Genome Atlas Network. Comprehensive molecular characterization of human colon and rectal cancer. Nature 487, 330–337 (2012).Ellis, M. J. et al. Connecting genomic alterations to cancer biology with proteomics: the NCI Clinical Proteomic Tumor Analysis Consortium. Cancer Discov.

。《自然》487330-337(2012)。Ellis,M.J.等人。将基因组改变与癌症生物学与蛋白质组学联系起来:NCI临床蛋白质组学肿瘤分析联盟。癌症发现。

3, 1108–1112 (2013).Article .

31108-1112(2013)。文章。

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Macenko, M. et al. A method for normalizing histology slides for quantitative analysis. In 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro 1107–1110 (IEEE, 2009).Howard, F. M. et al. The impact of site-specific digital histology signatures on deep learning model accuracy and bias.

Macenko,M.等人。用于定量分析的组织学载玻片标准化方法。2009年IEEE生物医学成像国际研讨会:从纳米到宏观1107-1110(IEEE,2009)。Howard,F.M.等人。特定地点数字组织学特征对深度学习模型准确性和偏倚的影响。

Nat. Commun. 12, 4423 (2021).Article .

Nat.普通。124423(2021)。文章。

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Canny, J. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 679–698 (1986).Article

Canny,J。边缘检测的计算方法。IEEE Trans。模式肛门。马赫。因特尔。8679-698(1986)。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Comes, M. C. et al. A deep learning model based on whole slide images to predict disease-free survival in cutaneous melanoma patients. Sci. Rep. 12, 20366 (2022).Article

Comes,M.C.等人提出了一种基于全幻灯片图像的深度学习模型,用于预测皮肤黑色素瘤患者的无病生存率。科学。代表120366(2022)。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Jiang, S., Suriawinata, A. A. & Hassanpour, S. MHAttnSurv: multi-head attention for survival prediction using whole-slide pathology images. Comput. Biol. Med. 158, 106883 (2023).Article

Jiang,S.,Suriawinata,A.A。和Hassanpour,S.MHAttnSurv:使用全幻灯片病理图像进行生存预测的多头注意。计算机。生物医学158106883(2023)。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Sounderajah, V. et al. Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol. BMJ Open 11, e047709 (2021).Article

Sounderajah,V。等人。为以人工智能为中心的诊断测试准确性研究制定报告指南:STARD-AI协议。BMJ公开赛11,e047709(2021)。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Collins, G. S. et al. Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ Open 11, e048008 (2021).Article

Collins,G.S.等人,《基于人工智能的诊断和预后预测模型研究报告指南(TRIPOD-AI)和偏倚风险工具(PROBAST-AI)的制定方案》。。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Trautmann, K. et al. Chromosomal instability in microsatellite-unstable and stable colon cancer. Clin. Cancer Res. 12, 6379–6385 (2006).Article

Trautmann,K。等人。微卫星不稳定和稳定结肠癌的染色体不稳定性。临床。癌症研究126379-6385(2006)。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Lin, E. I. et al. Mutational profiling of colorectal cancers with microsatellite instability. Oncotarget 6, 42334–42344 (2015).Article

Lin,E.I.等人。具有微卫星不稳定性的结直肠癌的突变分析。Oncotarget 642334–42344(2015)。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Boland, C. R. & Goel, A. Microsatellite instability in colorectal cancer. Gastroenterology 138, 2073–2087.e3 (2010).Article

Boland,C.R。&Goel,A。结直肠癌中的微卫星不稳定性。胃肠病学1382073-2087.e3(2010)。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Battaglin, F., Naseem, M., Lenz, H.-J. & Salem, M. E. Microsatellite instability in colorectal cancer: overview of its clinical significance and novel perspectives. Clin. Adv. Hematol. Oncol. 16, 735–745 (2018).PubMed

Battaglin,F.,Naseem,M.,Lenz,H.-J.&Salem,M.E。结直肠癌中的微卫星不稳定性:其临床意义和新观点的概述。临床。高级血液学。Oncol公司。16735-745(2018)。PubMed出版社

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Selvaraju, R. R. et al. Grad-CAM: visual explanations from deep networks via gradient-based localization. In 2017 IEEE International Conference on Computer Vision (ICCV) 618–626 (IEEE, 2017).Pataki, B. Á. et al. HunCRC: annotated pathological slides to enhance deep learning applications in colorectal cancer screening.

Selvaraju,R.R.等人,Grad CAM:通过基于梯度的定位从深度网络进行视觉解释。2017年IEEE国际计算机视觉会议(ICCV)618–626(IEEE,2017)。巴塔基,B。HunCRC等人:注释病理幻灯片,以增强结直肠癌筛查中的深度学习应用。

Sci. Data 9, 370 (2022).Article .

科学。数据9370(2022)。文章。

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Cheng, J. et al. Computational analysis of pathological images enables a better diagnosis of TFE3 Xp11.2 translocation renal cell carcinoma. Nat. Commun. 11, 1778 (2020).Article

Cheng,J。等人。病理图像的计算分析可以更好地诊断TFE3 Xp11.2易位肾细胞癌。国家公社。。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Download referencesAcknowledgementsWe thank the testers of the protocol, S. Sainath, O. L. Saldanha, L. Žigutytė, C. Kummer, G. Serna, K. Boehm and L. Shaktah, who executed the STAMP protocol on various systems at cancer centers around the world. O.S.M.E.N. is supported by the German Federal Ministry of Education and Research (BMBF) through grant 1IS23070, Software Campus 3.0 (TU Dresden), as part of the Software Campus project ’MIRACLE-AI’.

下载参考文献致谢我们感谢该协议的测试人员S.Sainath,O.L.Saldanha,L.igutyt,C.Kummer,G.Serna,K.Boehm和L.Shaktah,他们在世界各地的癌症中心的各种系统上执行了STAMP协议。O、 S.M.E.N.由德国联邦教育和研究部(BMBF)通过拨款1IS23070,软件校园3.0(德累斯顿大学)提供支持,作为软件校园项目“MIRACLE-AI”的一部分。

J.N.K. is supported by the German Federal Ministry of Health (DEEP LIVER, ZMVI1-2520DAT111), the German Cancer Aid (DECADE, 70115166), the German Federal Ministry of Education and Research (PEARL, 01KD2104C; CAMINO, 01EO2101; SWAG, 01KD2215A; TRANSFORM LIVER, 031L0312A; TANGERINE, 01KT2302 through ERA-NET Transcan), the German Academic Exchange Service (SECAI, 57616814), the German Federal Joint Committee (TransplantKI, 01VSF21048), the European Union’s Horizon Europe and innovation programme (ODELIA, 101057091; GENIAL, 101096312) and the National Institute for Health and Care Research (NIHR, NIHR213331) Leeds Biomedical Research Centre.

J、 N.K.得到了德国联邦卫生部(DEEP LIVER,ZMVI1-2520DAT111),德国癌症援助(DECADE,70115166),德国联邦教育和研究部(PEARL,01KD2104C;CAMINO,01EO2101;SWAG,01KD2215A;TRANSFORM LIVER,031L0312A;TANGERINE,01KT2302,通过ERA-NET Transcan),德国学术交流服务(SECAI,57616814),德国联邦联合委员会(Transplaki,01VSF21048),欧盟地平线欧洲与创新计划(ODELIA,101057091;GENIAL,10109638)的支持12)和国家卫生与保健研究所(NIHR,NIHR21331)利兹生物医学研究中心。

G.W. is supported by Lothian NHS. D.T. is supported by the German Federal Ministry of Education and Research (SWAG, 01KD2215A; TRANSFORM LIVER) and the European Union’s Horizon Europe and innovation programme (ODELIA, 101057091). S.F. is supported by the German Federal Ministry of Education and Research (SWAG, 01KD2215A), the German Cancer Aid (DECADE, 70115166) and the German Research Foundation (504101714).

G、 W.由Lothian NHS支持。D、 T.得到了德国联邦教育与研究部(SWAG,01KD2215A;TRANSFORM LIVER)和欧盟地平线欧洲与创新计划(ODELIA,101057091)的支持。S、 F.得到了德国联邦教育和研究部(SWAG,01KD2215A),德国癌症援助(TEADE,70115166)和德国研究基金会(504101714)的支持。

S.J.W. was supported by the Helmholtz Association under the joint research school ‘Munich School for Data Science – MUDS’ and the Add-on Fellowship of the Joachim Herz Foundation. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or th.

S、 J.W.得到了亥姆霍兹协会联合研究学院“慕尼黑数据科学学院-MUDS”和约阿希姆·赫兹基金会附加奖学金的支持。表达的观点是作者的观点,不一定是NHS,NIHR或th的观点。

PubMed Google ScholarMarko van TreeckView author publicationsYou can also search for this author in

PubMed Google ScholarMarko van TreeckView作者出版物您也可以在

PubMed Google ScholarGeorg WölfleinView author publicationsYou can also search for this author in

PubMed Google ScholarGeorg WölfleinView作者出版物您也可以在

PubMed Google ScholarMichaela UngerView author publicationsYou can also search for this author in

PubMed Google Scholarmamichaela UngerView作者出版物您也可以在

PubMed Google ScholarMarta LigeroView author publicationsYou can also search for this author in

PubMed Google ScholarMarta LigeroView作者出版物您也可以在

PubMed Google ScholarTim LenzView author publicationsYou can also search for this author in

PubMed Google ScholarTim LenzView作者出版物您也可以在

PubMed Google ScholarSophia J. WagnerView author publicationsYou can also search for this author in

PubMed Google ScholarSophia J.WagnerView作者出版物您也可以在

PubMed Google ScholarKatherine J. HewittView author publicationsYou can also search for this author in

PubMed Google ScholarKatherine J.HewittView作者出版物您也可以在

PubMed Google ScholarFiras KhaderView author publicationsYou can also search for this author in

PubMed Google ScholarFiras KhaderView作者出版物您也可以在

PubMed Google ScholarSebastian FoerschView author publicationsYou can also search for this author in

PubMed Google ScholarSebastian FoerschView作者出版物您也可以在

PubMed Google ScholarDaniel TruhnView author publicationsYou can also search for this author in

PubMed Google ScholarDaniel TruhnView作者出版物您也可以在

PubMed Google ScholarJakob Nikolas KatherView author publicationsYou can also search for this author in

PubMed Google ScholarJakob Nikolas KatherView作者出版物您也可以在

PubMed Google ScholarContributionsO.S.M.E.N. and J.N.K. designed the protocol. O.S.M.E.N., M.v.T., G.W. and T.L. developed the software and wrote technical documentation. O.S.M.E.N., M.v.T., G.W., T.L., M.L., M.U., S.J.W., F.K., S.F. and D.T. tested the software. O.S.M.E.N., J.N.K.

PubMed谷歌学术贡献。S、 M.E.N.和J.N.K.设计了该协议。O、 S.M.E.N.,M.v.T.,G.W.和T.L.开发了该软件并撰写了技术文档。O、 S.M.E.N.,M.v.T.,G.W.,T.L.,M.L.,M.U.,S.J.W.,F.K.,S.F.和D.T.测试了该软件。O、 S.M.E.N.,J.N.K。

and K.J.H. interpreted and analyzed the data. All authors wrote and reviewed the protocol and approved the final version for submission.Corresponding authorCorrespondence to.

。所有作者都编写并审查了该协议,并批准了提交的最终版本。。

Jakob Nikolas Kather.Ethics declarations

雅各布·尼古拉斯·凯瑟。道德宣言

Competing interests

相互竞争的利益

O.S.M.E.N., F.K. and D.T. hold shares in StratifAI GmbH. J.N.K. declares consulting services for Owkin, France; DoMore Diagnostics, Norway; Panakeia, UK,; Scailyte, Switzerland; Mindpeak, Germany; and Histofy, UK; furthermore, he holds shares in StratifAI GmbH, Germany, and has received honoraria for lectures by AstraZeneca, Bayer, Eisai, MSD, BMS, Roche, Pfizer and Fresenius.

O、 S.M.E.N.,F.K.和D.T.持有StratifAI GmbH的股份。J.N.K.宣布为法国奥金提供咨询服务;挪威多莫尔诊断公司;英国Panakea,;瑞士斯凯尔特;Mindpeak,德国;和英国Histofy;此外,他持有德国StratifAI GmbH的股份,并因阿斯利康,拜耳,卫材,MSD,BMS,罗氏,辉瑞和费森尤斯的讲座而获得酬金。

D.T. received honoraria for lectures by Bayer and holds shares in StratifAI GmbH, Germany. S.F. has received honoraria from MSD and BMS..

D、 T.因拜耳公司的讲座而获得酬金,并持有德国StratifAI GmbH的股份。S、 F.已收到MSD和BMS的酬金。。

Peer review

同行评审

Peer review information

同行评审信息

Nature Protocols thanks the anonymous reviewers for their contribution to the peer review of this work.

《自然协议》感谢匿名审稿人对这项工作的同行评审做出的贡献。

Additional informationPublisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.Related linksKey references using this protocolWagner, S. J. et al. Cancer Cell 41, 1650–1661.e4 (2023): https://doi.org/10.1016/j.ccell.2023.08.002El Nahhas, O.

Additional informationPublisher的注释Springer Nature在已发布的地图和机构隶属关系中的管辖权主张方面保持中立。使用此协议的相关linksKey参考文献Wagner,S.J.等人,《癌细胞》411650–1661.e4(2023):https://doi.org/10.1016/j.ccell.2023.08.002ElO.纳哈斯。

S. M. et al. Nat. Commun. 15, 1253 (2024): https://doi.org/10.1038/s41467-024-45589-1Jiang, X. et al. Lancet Digit. Health 6, e33–e43 (2024): https://doi.org/10.1016/S2589-7500(23)00208-XHewitt, K. J. et al. Neurooncol. Adv. 5, vdad139 (2023): https://doi.org/10.1093/noajnl/vdad139Saldanha, O. L. et al.

S.M.等人,《国家通讯》。15, 1253 (2024):https://doi.org/10.1038/s41467-024-45589-1JiangX.等人,《柳叶刀数字》。健康6,e33-e43(2024):https://doi.org/10.1016/S2589-7500(23)00208 XHewitt,K.J.等人,神经肿瘤学。辩护律师5,vdad139(2023):https://doi.org/10.1093/noajnl/vdad139SaldanhaO.L.等人。

npj Precis. Onc. 7, 35 (2023): https://doi.org/10.1038/s41698-023-00365-0Supplementary informationSupplementary InformationSupplementary Text 1, Table 1 and Fig. 1Source dataSource dataSource dataRights and permissionsSpringer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.Reprints and permissionsAbout this articleCite this articleEl Nahhas, O.S.M., van Treeck, M., Wölflein, G.

npj精度。Onc。7、35(2023年):https://doi.org/10.1038/s41698-023-00365-0Supplementary信息补充信息补充文本1,表1和图1源数据源数据源数据权限和许可Pringer Nature或其许可方(例如社会或其他合作伙伴)根据与作者或其他权利持有人的出版协议对本文拥有专有权;本文接受稿件版本的作者自行存档仅受此类出版协议和适用法律的条款管辖。转载和许可本文引用本文El Nahhas,O.S.M.,van Treeck,M.,Wölflein,G。

et al. From whole-slide image to biomarker prediction: end-to-end weakly supervised deep learning in computational pathology..

等。从整个幻灯片图像到生物标志物预测:计算病理学中的端到端弱监督深度学习。。

Nat Protoc (2024). https://doi.org/10.1038/s41596-024-01047-2Download citationReceived: 17 December 2023Accepted: 04 July 2024Published: 16 September 2024DOI: https://doi.org/10.1038/s41596-024-01047-2Share this articleAnyone you share the following link with will be able to read this content:Get shareable linkSorry, a shareable link is not currently available for this article.Copy to clipboard.

Nat Protoc(2024年)。https://doi.org/10.1038/s41596-024-01047-2Download引文收到日期:2023年12月17日接受日期:2024年7月4日发布日期:2024年9月16日OI:https://doi.org/10.1038/s41596-024-01047-2Share本文与您共享以下链接的任何人都可以阅读此内容:获取可共享链接对不起,本文目前没有可共享的链接。复制到剪贴板。

Provided by the Springer Nature SharedIt content-sharing initiative

由Springer Nature SharedIt内容共享计划提供

CommentsBy submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

。如果您发现有虐待行为或不符合我们的条款或准则,请将其标记为不合适。