AbstractThe application of machine learning to tasks involving volumetric biomedical imaging is constrained by the limited availability of annotated datasets of three-dimensional (3D) scans for model training. Here we report a deep-learning model pre-trained on 2D scans (for which annotated data are relatively abundant) that accurately predicts disease-risk factors from 3D medical-scan modalities.

摘要机器学习在涉及体积生物医学成像的任务中的应用受到用于模型训练的三维（3D）扫描注释数据集的有限可用性的限制。在这里，我们报告了一个在2D扫描（注释数据相对丰富）上预先训练的深度学习模型，该模型可以从3D医学扫描模式中准确预测疾病风险因素。

The model, which we named SLIViT (for ‘slice integration by vision transformer’), preprocesses a given volumetric scan into 2D images, extracts their feature map and integrates it into a single prediction. We evaluated the model in eight different learning tasks, including classification and regression for six datasets involving four volumetric imaging modalities (computed tomography, magnetic resonance imaging, optical coherence tomography and ultrasound).

我们将该模型命名为SLIViT（用于“视觉变压器的切片集成”），它将给定的体积扫描预处理为2D图像，提取其特征图并将其集成到单个预测中。我们在八个不同的学习任务中评估了该模型，包括涉及四种体积成像模式（计算机断层扫描，磁共振成像，光学相干断层扫描和超声波）的六个数据集的分类和回归。

SLIViT consistently outperformed domain-specific state-of-the-art models and was typically as accurate as clinical specialists who had spent considerable time manually annotating the analysed scans. Automating diagnosis tasks involving volumetric scans may save valuable clinician hours, reduce data acquisition costs and duration, and help expedite medical research and clinical applications..

SLIViT始终优于特定领域的最先进模型，并且通常与花费大量时间手动注释分析扫描的临床专家一样准确。自动化涉及体积扫描的诊断任务可以节省宝贵的临床医生时间，减少数据采集成本和持续时间，并有助于加快医学研究和临床应用。。

Access through your institution

通过您的机构访问

Buy or subscribe

购买或订阅

This is a preview of subscription content, access via your institution

这是订阅内容的预览，可通过您的机构访问

Access options

访问选项

Access through your institution

通过您的机构访问

Access through your institution

通过您的机构访问

Change institution

变革机构

Buy or subscribe

购买或订阅

Access Nature and 54 other Nature Portfolio journalsGet Nature+, our best-value online-access subscription24,99 € / 30 dayscancel any timeLearn moreSubscribe to this journalReceive 12 digital issues and online access to articles111,21 € per yearonly 9,27 € per issueLearn moreBuy this articlePurchase on SpringerLinkInstant access to full article PDFBuy nowPrices may be subject to local taxes which are calculated during checkout.

Access Nature和54本其他Nature Portfolio journalsGet Nature+，我们最有价值的在线订阅24,99欧元/30天浏览所有时间更多订阅本期刊每年收到12期数字期刊和在线访问文章111,21欧元每期仅9,27欧元更多了解更多购买本文在SpringerLink上购买即时访问完整文章PDFBuy Now价格可能需要缴纳结帐期间计算的当地税费。

Additional access options:

其他访问选项：

Log in

登录

Learn about institutional subscriptions

了解机构订阅

Read our FAQs

阅读我们的常见问题

Contact customer support

联系客户支持

Fig. 1: The SLIViT framework.Fig. 2: Overview of SLIViT’s performance across 3D imaging modalities.Fig. 3: Performance comparison on four tasks of AMD-biomarker classification when trained on less than 700 OCT volumes.Fig. 4: Performance comparison on cardiac function prediction tasks when trained on echocardiograms.Fig.

图1:SLIViT框架。图2:SLIViT在3D成像模式中的性能概述。图3:AMD生物标志物分类的四项任务在少于700个OCT体积的训练中的性能比较。。图。

5: SLIViT’s performance compared with manual assessment by retina clinical specialists..

5： SLIViT的表现与视网膜临床专家的手动评估相比较。。

Data availability

数据可用性

The 2D OCT dataset was downloaded from https://data.mendeley.com/datasets/rscbjbr9sj/3. The 3D OCT datasets are not publicly available owing to institutional data-use policy and to concerns about patient privacy. However, they are available from the authors upon reasonable request and with permission of the IRB.

2D OCT数据集下载自https://data.mendeley.com/datasets/rscbjbr9sj/3.由于机构数据使用政策以及对患者隐私的担忧，3D OCT数据集无法公开获得。然而，在合理的要求下，并经IRB许可，作者可以获得它们。

The echocardiogram dataset was downloaded from https://stanfordaimi.azurewebsites.net/datasets/834e1cd1-92f7-4268-9daa-d359198b310a. The MRI dataset was downloaded from https://www.ukbiobank.ac.uk under application number 33127. The 3D CT, the 2D CT and the 2D X-ray datasets were downloaded from https://medmnist.com..

超声心动图数据集下载自https://stanfordaimi.azurewebsites.net/datasets/834e1cd1-92f7-4268-9daa-d359198b310a.MRI数据集下载自https://www.ukbiobank.ac.uk根据申请号33127。3D CT，2D CT和2D X射线数据集下载自https://medmnist.com..

Code availability

代码可用性

The code of SLIViT is available via the project’s GitHub repository at https://github.com/cozygene/SLIViT.

SLIViT的代码可通过项目的GitHub存储库获得，网址为https://github.com/cozygene/SLIViT.

ReferencesChiang, J. N. et al. Automated identification of incomplete and complete retinal epithelial pigment and outer retinal atrophy using machine learning. Ophthalmol. Retina 7, 118–126 (2023).Article

参考Chiang，J.N.等人。使用机器学习自动识别不完全和完全的视网膜上皮色素和外部视网膜萎缩。眼科。视网膜7118-126（2023）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Wong, T. Y., Liew, G. & Mitchell, P. Clinical update: new treatments for age-related macular degeneration. Lancet 370, 204–206 (2007).Article

Wong，T.Y.，Liew，G。＆Mitchell，P。临床更新：年龄相关性黄斑变性的新疗法。柳叶刀370204-206（2007）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Gandhi, S. K. et al. The pathogenesis of acute pulmonary edema associated with hypertension. N. Engl. J. Med. 344, 17–22 (2001).Article

甘地，S.K。等。与高血压相关的急性肺水肿的发病机制。N、 英语。J、 医学杂志344,17-22（2001）。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Bloom, M. W. et al. Heart failure with reduced ejection fraction. Nat. Rev. Dis. Primers 3, 17058 (2017).Article

Bloom，M.W。等人。射血分数降低的心力衰竭。自然版本Dis。引物317058（2017）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Guindalini, R. S. C. et al. Intensive surveillance with biannual dynamic contrast-enhanced magnetic resonance imaging downstages breast cancer in BRCA1 mutation carriers. Clin. Cancer Res. 25, 1786–1794 (2019).Article

Guindalini，R.S.C.等人。通过两年一次的动态对比增强磁共振成像进行强化监测，可降低BRCA1突变携带者的乳腺癌分期。。癌症研究251786-1794（2019）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Mann, R. M., Kuhl, C. K. & Moy, L. Contrast-enhanced MRI for breast cancer screening. J. Magn. Reson. Imaging 50, 377–390 (2019).Article

Mann，R.M.，Kuhl，C.K。＆Moy，L。对比增强MRI用于乳腺癌筛查。J、 马格纳。雷森。成像50377-390（2019）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Azizi, S. et al. Robust and data-efficient generalization of self-supervised machine learning for diagnostic imaging. Nat. Biomed. Eng. 7, 756–779 (2023).Article

Azizi，S.等人。用于诊断成像的自监督机器学习的稳健且数据有效的泛化。自然生物医学。工程7756-779（2023）。文章

PubMed

PubMed

Google Scholar

谷歌学者

O’Shea, K. & Nash, R. An introduction to convolutional neural networks. Preprint at arXiv https://doi.org/10.48550/arXiv.1511.08458 (2015).Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Commun. ACM 60, 84–90 (2017).Article .

O'Shea，K。＆Nash，R。卷积神经网络简介。arXiv预印本https://doi.org/10.48550/arXiv.1511.08458（2015年）。Krizhevsky，A.，Sutskever，I。＆Hinton，G.E。使用深度卷积神经网络进行ImageNet分类。Commun公司。ACM 60,84-90（2017）。文章。

Google Scholar

谷歌学者

Esteva, A. et al. A guide to deep learning in healthcare. Nat. Med. 25, 24–29 (2019).Article

Esteva，A.等人，《医疗保健深度学习指南》。《自然医学杂志》25,24-29（2019）。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Zhuang, F. et al. A comprehensive survey on transfer learning. In Proc. IEEE (ed. Setti, G.) 43–76 (IEEE, 2021).McKinney, S. M. et al. International evaluation of an AI system for breast cancer screening. Nature 577, 89–94 (2020).Article

庄，F。等。关于迁移学习的综合调查。在过程中。IEEE（ed.Setti，G.）43–76（IEEE，2021）。McKinney，S.M.等人，《人工智能乳腺癌筛查系统的国际评估》。《自然》577,89-94（2020）。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Hannun, A. Y. et al. Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nat. Med. 25, 65–69 (2019).Article

Hannun，A.Y.等人，《使用深度神经网络在动态心电图中进行心脏病专家级心律失常检测和分类》，《自然医学》25，65-69（2019）。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Rajpurkar, P. et al. Deep learning for chest radiograph diagnosis: a retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLoS Med. 15, e1002686 (2018).Article

Rajpurkar，P。等人，《胸片诊断的深度学习：CheXNeXt算法与执业放射科医生的回顾性比较》。PLoS Med.15，e1002686（2018）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402–2410 (2016).Article

Gulshan，V。等人。在视网膜眼底照片中检测糖尿病视网膜病变的深度学习算法的开发和验证。JAMA 3162402–2410（2016）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Deng, J. et al. ImageNet: a large-scale hierarchical image database. In Proc. 2009 IEEE Conference on Computer Vision and Pattern Recognition 248–255 (IEEE, 2009).Tiu, E. et al. Expert-level detection of pathologies from unannotated chest X-ray images via self-supervised learning. Nat.

Deng，J。等人。ImageNet：一个大规模的分层图像数据库。在过程中。2009年IEEE计算机视觉和模式识别会议248–255（IEEE，2009）。Tiu，E.等人。通过自我监督学习从未注释的胸部X射线图像中检测病理的专家级检测。纳特。

Biomed. Eng. 6, 1399–1406 (2022).Article .

生物医学。工程61399-1406（2022）。文章。

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Zhang, Y., Jiang, H., Miura, Y., Manning, C. D. & Langlotz, C. P. Contrastive learning of medical visual representations from paired images and text. Preprint at arXiv https://doi.org/10.48550/arXiv.2010.00747 (2020).Xie, Y., Zhang, J., Xia, Y. & Wu, Q. UniMiSS: Universal Medical Self-Supervised learning via breaking dimensionality barrier.

Zhang，Y.，Jiang，H.，Miura，Y.，Manning，C.D。和Langlotz，C.P。从成对的图像和文本对比学习医学视觉表征。arXiv预印本https://doi.org/10.48550/arXiv.2010.00747（2020年）。Xie，Y.，Zhang，J.，Xia，Y。＆Wu，Q。UniMiSS：通过打破维度障碍进行通用医学自我监督学习。

In Proc. European Conference on Computer Vision (eds. Avidan, S. et al.) 558–575 (Springer, 2022).Azizi, S. et al. Big self-supervised models advance medical image classification. Preprint at arXiv https://doi.org/10.48550/arXiv.2101.05224 (2021).Wu, Z. et al. OCT signs of early atrophy in age-related macular degeneration: interreader agreement: classification of atrophy meetings report 6.

在过程中。欧洲计算机视觉会议（编辑Avidan，S。等人）558-575（Springer，2022）。Azizi，S.等人。大型自监督模型推进了医学图像分类。arXiv预印本https://doi.org/10.48550/arXiv.2101.05224（2021年）。Wu，Z.等。OCT年龄相关性黄斑变性早期萎缩的迹象：读者间协议：萎缩分类会议报告6。

Ophthalmol. Retina 6, 4–14 (2022).Article .

眼科。。文章。

PubMed

PubMed

Google Scholar

谷歌学者

Huang, S.-C. et al. Self-supervised learning for medical image classification: a systematic review and implementation guidelines. npj Digit. Med. 6, 74 (2023).Article

Huang，S.-C.等人。医学图像分类的自我监督学习：系统综述和实施指南。npj数字。医学杂志6，74（2023）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Bycroft, C. et al. The UK Biobank resource with deep phenotyping and genomic data. Nature 562, 203–209 (2018).Article

Bycroft，C。等人。具有深度表型和基因组数据的英国生物库资源。自然562203-209（2018）。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Willemink, M. J. et al. Preparing medical imaging data for machine learning. Radiology 295, 4–15 (2020).Article

Willemink，M.J.等人。为机器学习准备医学成像数据。放射学295,4-15（2020）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Tran, D. et al. A closer look at spatiotemporal convolutions for action recognition. In Proc. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 6450–6459 (IEEE, 2018).Arnab, A. et al. ViViT: a video vision transformer. In Proc. 2021 IEEE/CVF International Conference on Computer Vision (ICVV) 6816–6826 (IEEE, 2021).Zhu, H., Chen, B.

Tran，D.等人，《动作识别的时空卷积研究》。在过程中。2018年IEEE/CVF计算机视觉和模式识别会议6450-6459（IEEE，2018）。Arnab，A.等人，《ViViT：视频视觉转换器》。在过程中。2021年IEEE/CVF国际计算机视觉会议（ICVV）6816–6826（IEEE，2021）。朱，陈，B。

& Yang, C. Understanding why ViT trains badly on small datasets: an intuitive perspective. Preprint at arXiv https://doi.org/10.48550/arXiv.2302.03751 (2023).Rakocz, N. et al. Automated identification of clinical features from sparsely annotated 3-dimensional medical imaging. npj Digit. Med. 4, 44 (2021).Article .

&Yang，C.理解为什么ViT在小数据集上训练不佳：一个直观的观点。arXiv预印本https://doi.org/10.48550/arXiv.2302.03751（2023年）。Rakocz，N.等人。从稀疏注释的三维医学成像中自动识别临床特征。npj数字。医学杂志4，44（2021）。文章。

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Ghorbani, A. et al. Deep learning interpretation of echocardiograms. npj Digit. Med. 3, 10 (2020).Article

Ghorbani，A。等人。超声心动图的深度学习解释。npj数字。医学3，10（2020）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Gupta, U. et al. Transferring models trained on natural images to 3D MRI via position encoded slice models. Preprint at arXiv https://doi.org/10.48550/arXiv.2303.01491 (2023).Witowski, J. et al. Improving breast cancer diagnostics with deep learning for MRI. Sci. Transl. Med. 14, eabo4802 (2022).Article .

Gupta，U.等人。通过位置编码切片模型将自然图像上训练的模型转移到3D MRI。arXiv预印本https://doi.org/10.48550/arXiv.2303.01491（2023年）。Witowski，J.等人。通过MRI的深度学习改进乳腺癌诊断。科学。翻译。医学14，eabo4802（2022）。文章。

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Yang, M., Huang, X., Huang, L. & Cai, G. Diagnosis of Parkinson’s disease based on 3D ResNet: the frontal lobe is crucial. Biomed. Signal Process. Control 85, 104904 (2023).Article

Yang，M.，Huang，X.，Huang，L。＆Cai，G。基于3D ResNet的帕金森病诊断：额叶至关重要。生物医学。信号处理。控制85104904（2023）。文章

Google Scholar

谷歌学者

Zou, Q. et al. Three-dimensional ultrasound image reconstruction based on 3D-ResNet in the musculoskeletal system using a 1D probe: ex vivo and in vivo feasibility studies. Phys. Med. Biol. 68, 165003 (2023).Article

Zou，Q.等人。使用一维探针在肌肉骨骼系统中基于3D ResNet的三维超声图像重建：离体和体内可行性研究。物理。医学生物学。68165003（2023）。文章

Google Scholar

谷歌学者

Turnbull, R. Using a 3D ResNet for detecting the presence and severity of COVID-19 from CT scans. In Proc. Computer Vision – ECCV 2022 Workshops (eds Karlinsky, L. et al.) 663–676 (Springer, 2023).Caron, M. et al. Emerging properties in self-supervised vision transformers. In Proc. 2021 IEEE/CVF International Conference on Computer Vision (ICVV) 9630–9640 (IEEE, 2021).Zhou, H.-Y., Lu, C., Yang, S., Han, X.

Turnbull，R。使用3D ResNet从CT扫描中检测COVID-19的存在和严重程度。在过程中。计算机视觉-ECCV 2022研讨会（eds Karlinsky，L.等人）663-676（Springer，2023）。Caron，M.等人，《自监督视觉变压器的新兴特性》。在过程中。2021年IEEE/CVF国际计算机视觉会议（ICVV）9630–9640（IEEE，2021）。周，陆，杨，韩，X。

& Yu, Y. Preservational learning improves self-supervised medical image models by reconstructing diverse contexts. In Proc. 2021 IEEE/CVF International Conference on Computer Vision (ICVV) 3479–3489 (IEEE, 2021).Xie, Y., Zhang, J., Liao, Z., Xia, Y. & Shen, C. PGL: prior-guided local self-supervised learning for 3D medical image segmentation.

&Yu，Y。保存学习通过重建不同的背景来改进自我监督的医学图像模型。在过程中。2021年IEEE/CVF国际计算机视觉会议（ICVV）3479–3489（IEEE，2021）。Xie，Y.，Zhang，J.，Liao，Z.，Xia，Y。＆Shen，C。PGL：用于3D医学图像分割的先验引导局部自监督学习。

Preprint at arXiv https://doi.org/10.48550/arXiv.2011.12640 (2020).Chen, X., Fan, H., Girshick, R. & He, K. Improved baselines with momentum contrastive learning. Preprint at arXiv https://doi.org/10.48550/arXiv.2003.04297 (2020).Chen, X., Xie, S. & He, K. An empirical study of training self-supervised vision transformers.

arXiv预印本https://doi.org/10.48550/arXiv.2011.12640（2020年）。Chen，X.，Fan，H.，Girshick，R。＆He，K。通过动量对比学习改进了基线。arXiv预印本https://doi.org/10.48550/arXiv.2003.04297（2020年）。Chen，X.，Xie，S。和He，K。训练自我监督视觉变形金刚的实证研究。

In Proc. 2021 IEEE/CVF International Conference on Computer Vision (ICVV) 9620–9629 (IEEE, 2021).Liu, Z. et al. A ConvNet for the 2020s. In Proc. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 11966–11976 (IEEE, 2022).Dosovitskiy, A. et al. An image is worth 16x16 words: transformers for image recognition at scale.

在过程中。2021年IEEE/CVF国际计算机视觉会议（ICVV）9620–9629（IEEE，2021）。Liu，Z.等人，《20世纪20年代的变革》。在过程中。2022年IEEE/CVF计算机视觉和模式识别会议（CVPR）11966–11976（IEEE，2022）。Dosovitskiy，A。等人。一幅图像相当于16x16个单词：用于大规模图像识别的变压器。

Preprint at arXiv https://doi.org/10.48550/arXiv.2010.11929 (2021)Gupta, U., Lam, P. K., Ver Steeg, G. & Thompson, P. M. Improved brain age estimation with slice-based set networks. In Proc. 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI) 840–844 (IEEE, 2021).Zeng, Y. et al. A 2.5D deep learning-based method for drowning diagnosis using .

arXiv预印本https://doi.org/10.48550/arXiv.2010.11929（2021）Gupta，U.，Lam，P.K.，Ver Steeg，G。＆Thompson，P.M。使用基于切片的集合网络改进了大脑年龄估计。在过程中。2021 IEEE第18届生物医学成像国际研讨会（ISBI）840-844（IEEE，2021）。Zeng，Y。等人。一种基于2.5D深度学习的溺水诊断方法。

PubMed

PubMed

Google Scholar

谷歌学者

Schlemper, J. et al. Attention gated networks: learning to leverage salient regions in medical images. Med. Image Anal. 53, 197–207 (2019).Article

Schlemper，J.等人，《注意力门控网络：学习利用医学图像中的显着区域》。医学图像肛门。53197-207（2019）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Bertasius, G., Wang, H. & Torresani, L. Is space-time attention all you need for video understanding? In Proc. 38th International Conference on Machine Learning (ICML) (2021).Neimark, D., Bar, O., Zohar, M. & Asselmann, D. Video transformer network. In Proc. IEEE/CVF International Conference on Computer Vision (ICVV) 3156–3165 (2021).Kermany, D.

Bertasius，G.，Wang，H。＆Torresani，L。时空注意力是视频理解所需要的全部吗？在过程中。第38届国际机器学习会议（ICML）（2021）。Neimark，D.，Bar，O.，Zohar，M。和Asselmann，D。视频变压器网络。IEEE/CVF国际计算机视觉会议（ICVV）3156–3165（2021）。。

S. et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172, 1122–1131.e9 (2018).Article .

S、 等人。通过基于图像的深度学习识别医学诊断和可治疗疾病。细胞1721122-1131.e9（2018）。文章。

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Wong, W. L. et al. Global prevalence of age-related macular degeneration and disease burden projection for 2020 and 2040: a systematic review and meta-analysis. Lancet Glob. Health 2, e106–e116 (2014).Article

Wong，W.L.等人，《2020年和2040年全球年龄相关性黄斑变性患病率和疾病负担预测：系统评价和荟萃分析》。柳叶刀球。健康2，e106-e116（2014）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Hirabayashi, K. et al. OCT risk factors for development of atrophy in eyes with intermediate age-related macular degeneration. Ophthalmol. Retina 7, 253–260 (2023).Article

Hirabayashi，K。等。OCT中老年黄斑变性眼萎缩发生的危险因素。眼科。视网膜7253-260（2023）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Ouyang, D. et al. EchoNet-Dynamic: a large new cardiac motion video data resource for medical machine learning. In Proc. 33rd International Conference on Neural Information Processing Systems (eds Wallach, H. M.) (Curran Associates Inc., 2019).Ziaeian, B. & Fonarow, G. C. Epidemiology and aetiology of heart failure.

欧阳，D。等。EchoNet Dynamic：一种用于医学机器学习的大型新心脏运动视频数据资源。在过程中。第33届神经信息处理系统国际会议（eds Wallach，H.M.）（Curran Associates Inc.，2019）。Ziaeian，B。＆Fonarow，G.C。心力衰竭的流行病学和病因学。

Nat. Rev. Cardiol. 13, 368–378 (2016).Article .

国家心脏病修订版。13368-378（2016）。文章。

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Klapholz, M. et al. Hospitalization for heart failure in the presence of a normal left ventricular ejection fraction: results of the New York Heart Failure Registry. J. Am. Coll. Cardiol. 43, 1432–1438 (2004).Article

Klapholz，M.等人。在左心室射血分数正常的情况下住院治疗心力衰竭：纽约心力衰竭登记处的结果。J、 美国科罗拉多州。心脏病。431432-1438（2004）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Dunlay, S. M., Roger, V. L. & Redfield, M. M. Epidemiology of heart failure with preserved ejection fraction. Nat. Rev. Cardiol. 14, 591–602 (2017).Article

Dunlay，S.M.，Roger，V.L。和Redfield，M.M。保留射血分数的心力衰竭流行病学。国家心脏病修订版。14591-602（2017）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Idilman, I. S. et al. Hepatic steatosis: quantification by proton density fat fraction with MR imaging versus liver biopsy. Radiology 267, 767–775 (2013).Article

Idilman，I.S.等。肝脂肪变性：通过MR成像与肝活检的质子密度脂肪分数定量。放射学267767–775（2013）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Jung, J. et al. Direct comparison of quantitative US versus controlled attenuation parameter for liver fat assessment using MRI proton density fat fraction as the reference standard in patients suspected of having NAFLD. Radiology 304, 75–82 (2022).Article

Jung，J.等人。在疑似患有NAFLD的患者中，使用MRI质子密度脂肪分数作为参考标准，直接比较定量美国与受控衰减参数对肝脏脂肪评估的影响。放射学304,75-82（2022）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Runge, J. H. et al. MR spectroscopy-derived proton density fat fraction is superior to controlled attenuation parameter for detecting and grading hepatic steatosis. Radiology 286, 547–556 (2018).Article

Runge，J.H.等人。MR光谱衍生的质子密度脂肪分数在检测和分级肝脂肪变性方面优于受控衰减参数。放射学286547-556（2018）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Schawkat, K. et al. Preoperative evaluation of pancreatic fibrosis and lipomatosis: correlation of magnetic resonance findings with histology using magnetization transfer imaging and multigradient echo magnetic resonance imaging. Invest. Radiol. 53, 720–727 (2018).Article

Schawkat，K。等人。胰腺纤维化和脂肪瘤病的术前评估：使用磁化转移成像和多梯度回波磁共振成像将磁共振发现与组织学的相关性。投资。放射性。53720-727（2018）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Kühn, J.-P. et al. Pancreatic steatosis demonstrated at MR imaging in the general population: clinical relevance. Radiology 276, 129–136 (2015).Article

Kühn，J.-P.等人。普通人群的MR成像显示胰腺脂肪变性：临床相关性。放射学276129-136（2015）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Patel, N. S. et al. Insulin resistance increases MRI-estimated pancreatic fat in nonalcoholic fatty liver disease and normal controls. Gastroenterol. Res. Pract. 2013, 498296 (2013).Article

Patel，N.S.等人。胰岛素抵抗会增加非酒精性脂肪肝病和正常对照组中MRI估计的胰腺脂肪。胃肠道。实践研究。2013498296（2013）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Trout, A. T. et al. Relationship between abdominal fat stores and liver fat, pancreatic fat, and metabolic comorbidities in a pediatric population with non-alcoholic fatty liver disease. Abdom. Radiol. 44, 3107–3114 (2019).Article

Trout，A.T.等人。非酒精性脂肪性肝病患儿腹部脂肪储存与肝脏脂肪、胰腺脂肪和代谢合并症之间的关系。阿布多姆。放射性。443107-3114（2019）。文章

Google Scholar

谷歌学者

Covarrubias, Y. et al. Pilot study on longitudinal change in pancreatic proton density fat fraction during a weight-loss surgery program in adults with obesity. J. Magn. Reson. Imaging 50, 1092–1102 (2019).Article

。J、 马格纳。雷森。成像501092-1102（2019）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Yang, J. et al. MedMNIST v2 – a large-scale lightweight benchmark for 2D and 3D biomedical image classification. Sci. Data 10, 41 (2023).Article

Yang，J.等人。MedMNIST v2–用于2D和3D生物医学图像分类的大规模轻量级基准。科学。数据10,41（2023）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Halder, A., Dey, D. & Sadhu, A. K. Lung nodule detection from feature engineering to deep learning in thoracic CT images: a comprehensive review. J. Digit. Imaging 33, 655–677 (2020).Article

Halder，A.，Dey，D。＆Sadhu，A.K。从胸部CT图像的特征工程到深度学习的肺结节检测：全面综述。J、 数字。成像33655-677（2020）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Mazzone, P. J. & Lam, L. Evaluating the patient with a pulmonary nodule: a review. JAMA 327, 264–273 (2022).Article

Mazzone，P.J。＆Lam，L。评估肺结节患者：综述。JAMA 327264-273（2022）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).Article

Esteva，A。等人。皮肤科医生用深度神经网络对皮肤癌进行分类。自然542115-118（2017）。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Huang, Z., Bianchi, F., Yuksekgonul, M., Montine, T. J. & Zou, J. A visual-language foundation model for pathology image analysis using medical Twitter. Nat. Med. https://doi.org/10.1038/s41591-023-02504-3 (2023).Article

Huang，Z.，Bianchi，F.，Yuksekgonul，M.，Montine，T.J。＆Zou，J。使用医学推特进行病理图像分析的视觉语言基础模型。自然医学。https://doi.org/10.1038/s41591-023-02504-3（2023年）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Liu, Y. et al. A deep learning system for differential diagnosis of skin diseases. Nat. Med. 26, 900–908 (2020).Article

Liu，Y。等人。皮肤病鉴别诊断的深度学习系统。《自然医学》26900-908（2020）。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Guan, H., Wang, L., Yao, D., Bozoki, A. & Liu, M. Learning transferable 3D-CNN for MRI-based brain disorder classification from scratch: an empirical study. In Proc. Machine Learning in Medical Imaging (eds. Lian, C. et al.) 10–19 (Springer, 2021).Mustafa, B. et al. Supervised transfer learning at scale for medical imaging.

Guan，H.，Wang，L.，Yao，D.，Bozoki，A。＆Liu，M。从零开始学习可转移的3D-CNN用于基于MRI的脑部疾病分类：一项实证研究。在过程中。医学成像中的机器学习（eds。Lian，C。et al。）10-19（Springer，2021）。Mustafa，B.等人在医学成像规模上监督转移学习。

Preprint at arXiv https://doi.org/10.48550/arXiv.2101.05913 (2021).Raghu, M., Zhang, C., Kleinberg, J. & Bengio, S. Transfusion: understanding transfer learning for medical imaging. In Proc. 33rd International Conference on Neural Information Processing Systems (eds Wallach, H. M.) (Curran Associates Inc., 2019).Zhou, Y.

arXiv预印本https://doi.org/10.48550/arXiv.2101.05913（2021年）。Raghu，M.，Zhang，C.，Kleinberg，J。＆Bengio，S。输血：理解医学成像的转移学习。在过程中。第33届神经信息处理系统国际会议（eds Wallach，H.M.）（Curran Associates Inc.，2019）。周，Y。

et al. A foundation model for generalizable disease detection from retinal images. Nature 622, 156–163 (2023).Article .

等人。从视网膜图像中检测普遍性疾病的基础模型。。文章。

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Newell, A. & Deng, J. How useful is self-supervised pretraining for visual tasks? In Proc. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 7343–7352 (IEEE, 2020).Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. A simple framework for contrastive learning of visual representations.

Newell，A。＆Deng，J。自我监督预训练对视觉任务有多有用？在过程中。2020 IEEE/CVF计算机视觉和模式识别会议（CVPR）7343–7352（IEEE，2020）。Chen，T.，Kornblith，S.，Norouzi，M。＆Hinton，G。视觉表征对比学习的简单框架。

In Proc. 37th International Conference on Machine Learning (eds Daumé, H. & Singh, A.) (JMLR, 2020).He, K., Fan, H., Wu, Y., Xie, S. & Girshick, R. Momentum contrast for unsupervised visual representation learning. In Proc. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition 9726–9735 (IEEE, 2020).Mitrovic, J., McWilliams, B., Walker, J., Buesing, L.

在过程中。第37届国际机器学习会议（eds Daumé，H.＆Singh，A.）（JMLR，2020）。He，K.，Fan，H.，Wu，Y.，Xie，S。＆Girshick，R。无监督视觉表征学习的动量对比。在过程中。2020 IEEE/CVF计算机视觉和模式识别会议9726–9735（IEEE，2020）。米特罗维奇，J.，麦克威廉姆斯，B.，沃克，J.，布埃辛，L。

& Blundell, C. Representation learning via invariant causal mechanisms. In Proc. International Conference on Learning Representations (2020).Zbontar, J., Jing, L., Misra, I., LeCun, Y. & Deny, S. Barlow twins: self-supervised learning via redundancy reduction. In Proc. International Conference on Machine Learning (2021).Kornblith, S., Norouzi, M., Lee, H.

&Blundell，C。通过不变的因果机制进行表征学习。在过程中。国际学习表征会议（2020年）。Zbontar，J.，Jing，L.，Misra，I.，LeCun，Y。＆Deny，S。Barlow twins：通过冗余减少进行自我监督学习。在过程中。国际机器学习会议（2021）。科恩布里斯，S.，诺鲁兹，M.，李，H。

& Hinton, G. Similarity of neural network representations revisited. In Proc. 36th International Conference on Machine Learning (eds Chaudhuri, K. & Salakhutdinov, R.) (Curran Associates, Inc., 2019).Taleb, A. et al. 3D self-supervised methods for medical imaging. In Proc. 34th International Conference on Neural Information Processing Systems (Curran Associates, Inc., 2020).Tang, Y.

&Hinton，G。重新审视神经网络表示的相似性。在过程中。第36届国际机器学习会议（eds Chaudhuri，K.＆Salakhutdinov，R.）（Curran Associates，Inc.，2019）。Taleb，A。等人。医学成像的3D自我监督方法。在过程中。第34届神经信息处理系统国际会议（Curran Associates，Inc.，2020）。唐，Y。

et al. Self-supervised pre-training of Swin transformers for 3D medical image analysis. In Proc. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition 20698–20708 (IEEE, 2022).He, K. et al. Masked autoencoders are scalable vision learners. In Proc. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition 15979–15988 (IEEE, 2022).Woo, .

等人。用于3D医学图像分析的Swin变压器的自我监督预训练。在过程中。2022年IEEE/CVF计算机视觉和模式识别会议20698-20708（IEEE，2022）。他，K。等人。蒙面自动编码器是可伸缩的视觉学习者。在过程中。2022年IEEE/CVF计算机视觉和模式识别会议15979–15988（IEEE，2022）。呜呜。

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Chen, R. J. et al. Algorithmic fairness in artificial intelligence for medicine and healthcare. Nat. Biomed. Eng. 7, 719–742 (2023).Article

Chen，R.J.等人。用于医学和医疗保健的人工智能中的算法公平性。自然生物医学。工程7719-742（2023）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Paszke, A. et al. PyTorch: an imperative style, high-performance deep learning library. In Proc. 33rd International Conference on Neural Information Processing Systems (eds Wallach, H. M. et al.) (Curran Associates Inc., 2019).Howard, J. & Gugger, S. fastai: a layered API for deep learning.

Paszke，A。et al。PyTorch：一个命令式、高性能的深度学习库。在过程中。第33届神经信息处理系统国际会议（eds Wallach，H.M.等人）（Curran Associates Inc.，2019）。Howard，J.＆Gugger，S.fastai：用于深度学习的分层API。

Information, 11, 108 (2020).Pedregosa, F. et al. Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).Hendrycks, D. & Gimpel, K. Gaussian error linear units (GELUs). Preprint at arXiv https://doi.org/10.48550/arXiv.1606.08415 (2016).Smith, L. N. Cyclical learning rates for training neural networks.

信息，11108（2020）。Pedregosa，F.等人，《Scikit learn：Python中的机器学习》。J、 马赫。学习。第122825-2830号决议（2011年）。Hendrycks，D。和Gimpel，K。高斯误差线性单位（GELUs）。arXiv预印本https://doi.org/10.48550/arXiv.1606.08415（2016年）。Smith，L.N。训练神经网络的循环学习率。

In Proc. 2017 IEEE Winter Conference on Applications of Computer Vision (WACV) 464–472 (IEEE, 2017).Smith, L. N. & Topin, N. Super-convergence: very fast training of neural networks using large learning rates. Preprint at arXiv https://doi.org/10.48550/arXiv.1708.07120 (2018).Rajkomar, A. et al. Scalable and accurate deep learning with electronic health records.

在过程中。2017 IEEE计算机视觉应用冬季会议（WACV）464–472（IEEE，2017）。Smith，L.N。＆Topin，N。Super convergence：使用大学习率对神经网络进行非常快速的训练。arXiv预印本https://doi.org/10.48550/arXiv.1708.07120（2018年）。Rajkomar，A。等人。利用电子健康记录进行可扩展和准确的深度学习。

npj Digit. Med. 1, 18 (2018).Article .

数字。医学1，18（2018）。第条。

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Kay, W. et al. The kinetics human action video dataset. Preprint at arXiv https://doi.org/10.48550/arXiv.1705.06950 (2017).Ferris, F. L. et al. Clinical classification of age-related macular degeneration. Ophthalmology 120, 844–851 (2013).Article

Kay，W.等人，《动力学人体动作视频数据集》。arXiv预印本https://doi.org/10.48550/arXiv.1705.06950（2017年）。Ferris，F.L.等人。年龄相关性黄斑变性的临床分类。眼科120844-851（2013）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Nassisi, M. et al. OCT risk factors for development of late age-related macular degeneration in the fellow eyes of patients enrolled in the HARBOR study. Ophthalmology 126, 1667–1674 (2019).Article

Nassisi，M.等人，参加HARBOR研究的患者同侧眼睛中发生晚期年龄相关性黄斑变性的OCT危险因素。眼科1261667-1674（2019）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Lei, J., Balasubramanian, S., Abdelfattah, N. S., Nittala, M. G. & Sadda, S. R. Proposal of a simple optical coherence tomography-based scoring system for progression of age-related macular degeneration. Graefes Arch. Clin. Exp. Ophthalmol. 255, 1551–1558 (2017).Article

Lei，J.，Balasubramanian，S.，Abdelfattah，N.S.，Nittala，M.G。＆Sadda，S.R。提出了一种简单的基于光学相干断层扫描的年龄相关性黄斑变性进展评分系统。格雷夫斯拱门。。实验眼科。2551551-1558（2017）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Nittala, M. G. et al. Amis Eye Study: baseline spectral domain optical coherence tomography characteristics of age-related macular degeneration. Retina 39, 1540–1550 (2019).Article

Nittala，M.G.等。Amis眼科研究：年龄相关性黄斑变性的基线光谱域光学相干断层扫描特征。视网膜391540-1550（2019）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Armato, S. G. et al. The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): a completed reference database of lung nodules on CT scans. Med. Phys. 38, 915–931 (2011).Article

Armato，S.G.等人。肺图像数据库联盟（LIDC）和图像数据库资源倡议（IDRI）：一个完整的CT扫描肺结节参考数据库。医学物理。38915-931（2011）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Download referencesAcknowledgementsThis work was supported by a National Institutes of Health (NIH)/National Institute of General Medical Sciences grant 5R25GM135043, an NIH/National Institute of Biomedical Imaging and Bioengineering grant R01EB035028, NIH/National Eye Institute grants R01EY023164 and 1R01EY030614, and an Unrestricted Grant from Research to Prevent Blindness.

下载参考文献致谢这项工作得到了美国国立卫生研究院（NIH）/国家普通医学科学研究所5R25GM135043，NIH/国家生物医学成像和生物工程研究所R01EB035028，NIH/国家眼科研究所R01EY023164和1R01EY030614的支持，以及来自预防失明研究的无限制拨款。

This research was conducted using the UK Biobank Resource under application number 33127. We also acknowledge the participants of the UCLA Computational Genomics Summer Institute (supported by the aforementioned NIH/National Institute of General Medical Sciences grant) for stimulating discussions.Author informationAuthor notesThese authors contributed equally: Oren Avram, Berkin Durmus.Authors and AffiliationsDepartment of Computational Medicine, University of California, Los Angeles, Los Angeles, CA, USAOren Avram, Ulzee An, Prerit Terway, Akos Rudas, Elior Rahmani, Bolei Zhou, Sriram Sankararaman & Jeffrey N.

。我们还感谢加州大学洛杉矶分校计算基因组学夏季研究所（由上述NIH/国家普通医学科学研究所资助）的参与者激发了讨论。作者信息作者注意到这些作者做出了同样的贡献：Oren Avram，Berkin Durmus。作者和附属机构加利福尼亚大学洛杉矶分校计算医学系，加利福尼亚州洛杉矶，USAOren Avram，Ulzee An，Prerit Terway，Akos Rudas，Elior Rahmani，Bolei Zhou，Sriram Sankararaman＆Jeffrey N。

ChiangDepartment of Computer Science, University of California, Los Angeles, Los Angeles, CA, USAOren Avram, Berkin Durmus, Nadav Rakocz, Ulzee An, Prerit Terway, Zeyuan Johnson Chen, Bolei Zhou, Sriram Sankararaman & Eran HalperinDepartment of Anesthesiology and Perioperative Medicine, University of California, Los Angeles, Los Angeles, CA, USAOren Avram & Maxime CannessonDoheny Eye Institute, University of California, Los Angeles, Pasadena, CA, USAGiulia Corradetti, Muneeswar G.

Chiang加利福尼亚大学洛杉矶分校计算机科学系，加利福尼亚州洛杉矶，USAOren Avram，Berkin Durmus，Nadav Rakocz，Ulzee An，Prerit Terway，Zeyuan Johnson Chen，Bolei Zhou，Sriram Sankararaman＆Eran Halperin加利福尼亚大学洛杉矶分校麻醉学和围手术期医学系，加利福尼亚大学洛杉矶分校USAOren Avram＆Maxime CannessonDoheny眼科研究所，加利福尼亚州帕萨迪纳，USAGiulia Corradetti，Muneeswar G。

Nittala, Yu Wakatsuki, Kazutaka Hirabayashi, Swetha Velaga, Liran Tiosano, Federico Corvi, Aditya Verma, Ayesha Karamat, Sophiana Lindenberg, Deniz Oncel, Louay Almidani, Victoria Hull, Sohaib Fasih-Ahmad, Houri Esmaeilkhanian & Srinivas R. SaddaDepartment of Ophthalmology, University of California,.

，。

PubMed Google ScholarBerkin DurmusView author publicationsYou can also search for this author in

PubMed Google ScholarBerkin DurmusView作者出版物您也可以在

PubMed Google ScholarNadav RakoczView author publicationsYou can also search for this author in

PubMed Google ScholarNadav RakoczView作者出版物您也可以在

PubMed Google ScholarGiulia CorradettiView author publicationsYou can also search for this author in

PubMed Google ScholarGiulia CorradettiView作者出版物您也可以在

PubMed Google ScholarUlzee AnView author publicationsYou can also search for this author in

PubMed Google ScholarUlzee AnView作者出版物您也可以在

PubMed Google ScholarMuneeswar G. NittalaView author publicationsYou can also search for this author in

PubMed Google Scholarmanueeswar G.NittalaView作者出版物您也可以在

PubMed Google ScholarPrerit TerwayView author publicationsYou can also search for this author in

PubMed Google ScholarPrerit TerwayView作者出版物您也可以在

PubMed Google ScholarAkos RudasView author publicationsYou can also search for this author in

PubMed Google ScholarAkos RudasView作者出版物您也可以在

PubMed Google ScholarZeyuan Johnson ChenView author publicationsYou can also search for this author in

PubMed Google ScholarZeyuan Johnson ChenView作者出版物您也可以在

PubMed Google ScholarYu WakatsukiView author publicationsYou can also search for this author in

PubMed Google ScholarYu WakatsukiView作者出版物您也可以在

PubMed Google ScholarKazutaka HirabayashiView author publicationsYou can also search for this author in

PubMed Google ScholarSwetha VelagaView author publicationsYou can also search for this author in

PubMed Google ScholarSwetha VelagaView作者出版物您也可以在

PubMed Google ScholarLiran TiosanoView author publicationsYou can also search for this author in

PubMed谷歌学术评论TiosanoView作者出版物您也可以在

PubMed Google ScholarFederico CorviView author publicationsYou can also search for this author in

PubMed Google ScholarFederico CorviView作者出版物您也可以在

PubMed Google ScholarAditya VermaView author publicationsYou can also search for this author in

PubMed谷歌学术评论作者出版物您也可以在

PubMed Google ScholarAyesha KaramatView author publicationsYou can also search for this author in

PubMed Google ScholarAyesha KaramatView作者出版物您也可以在

PubMed Google ScholarSophiana LindenbergView author publicationsYou can also search for this author in

PubMed Google ScholarSophiana LindenbergView作者出版物您也可以在

PubMed Google ScholarDeniz OncelView author publicationsYou can also search for this author in

PubMed Google ScholarDeniz OncelView作者出版物您也可以在

PubMed Google ScholarLouay AlmidaniView author publicationsYou can also search for this author in

PubMed Google ScholarLouay AlmidaniView作者出版物您也可以在

PubMed Google ScholarVictoria HullView author publicationsYou can also search for this author in

PubMed Google ScholarVictoria HullView作者出版物您也可以在

PubMed Google ScholarSohaib Fasih-AhmadView author publicationsYou can also search for this author in

PubMed Google ScholarSohaib Fasih AhmadView作者出版物您也可以在

PubMed Google ScholarHouri EsmaeilkhanianView author publicationsYou can also search for this author in

PubMed Google ScholarHouri EsmaeilkhanianView作者出版物您也可以在

PubMed Google ScholarMaxime CannessonView author publicationsYou can also search for this author in

PubMed Google ScholarMaxime CannessonView作者出版物您也可以在

PubMed Google ScholarCharles C. WykoffView author publicationsYou can also search for this author in

PubMed Google ScholarCharles C.WykoffView作者出版物您也可以在

PubMed Google ScholarElior RahmaniView author publicationsYou can also search for this author in

PubMed Google ScholarElior RahmaniView作者出版物您也可以在

PubMed Google ScholarCorey W. ArnoldView author publicationsYou can also search for this author in

PubMed Google ScholarBolei ZhouView author publicationsYou can also search for this author in

PubMed Google ScholarBolei Zhou查看作者出版物您也可以在

PubMed Google ScholarNoah ZaitlenView author publicationsYou can also search for this author in

PubMed Google ScholarNoah ZaitlenView作者出版物您也可以在

PubMed Google ScholarIlan GronauView author publicationsYou can also search for this author in

PubMed Google ScholarIlan GronauView作者出版物您也可以在

PubMed Google ScholarSriram SankararamanView author publicationsYou can also search for this author in

PubMed谷歌学术评论SankararamanView作者出版物您也可以在

PubMed Google ScholarJeffrey N. ChiangView author publicationsYou can also search for this author in

PubMed谷歌学者Jeffrey N.ChiangView作者出版物您也可以在

PubMed Google ScholarSrinivas R. SaddaView author publicationsYou can also search for this author in

PubMed Google ScholarEran HalperinView author publicationsYou can also search for this author in

PubMed Google ScholarEran HalperinView作者出版物您也可以在

PubMed Google ScholarContributionsO.A., B.D., N.R., G.C., U.A., M.G.N., B.Z., S.S., J.N.C., S.R.S. and E.H. contributed to the conception and design of the work. O.A., B.D., G.C., U.A., M.G.N., A.R., Z.J.C., Y.W., K.H., S.V., L.T., F.C., A.V., A.K., S.L., D.O., L.A., V.H., S.F.-A., H.E., C.C.W., S.R.S.

PubMed谷歌学术贡献。A、 。

and E.H. contributed to data acquisition. O.A., B.D., N.R., G.C., B.Z., N.Z., I.G., J.N.C., S.R.S. and E.H. contributed to the evaluation of the work. O.A., B.D., N.R., G.C., U.A., M.G.N., P.T., I.G., S.S., J.N.C., S.R.S. and E.H. contributed to the analysis and interpretation of the data. O.A., B.D., N.R., G.C., U.A., A.R., M.C., E.R., C.W.A., N.Z., I.G., S.S., J.N.C., S.R.S.

E.H.为数据采集做出了贡献。O、 A.，B.D.，N.R.，G.C.，B.Z.，N.Z.，I.G.，J.N.C.，S.R.S.和E.H.为这项工作的评估做出了贡献。O.A.，B.D.，N.R.，G.C.，U.A.，M.G.N.，P.T.，I.G.，S.S.，J.N.C.，S.R.S.和E.H.为数据的分析和解释做出了贡献。O、 A.，B.D.，N.R.，G.C.，U.A.，A.R.，M.C.，E.R.，C.W.A.，N.Z.，I.G.，S.S.，J.N.C.，S.R.S。

and E.H. contributed to drafting and revising the paper. S.R.S. and E.H. contributed equally as co-advisers. All authors read and approved the final version of the paper.Corresponding authorsCorrespondence to.

E.H.为起草和修订该文件做出了贡献。S、 R.S.和E.H.作为联合顾问的贡献相同。所有作者都阅读并批准了论文的最终版本。通讯作者通讯。

Oren Avram, Srinivas R. Sadda or Eran Halperin.Ethics declarations

Oren Avram，Srinivas R.Sadda或Eran Halperin。道德宣言

Competing interests

相互竞争的利益

E.H. has an affiliation with Optum. S.R.S. has affiliations with Abbvie/Allergan, Alexion, Amgen, Apellis, ARVO, Astellas, Bayer, Biogen, Boerhinger Ingelheim, Carl Zeiss Meditec, Centervue, Character, Eyepoint, Heidelberg, iCare, IvericBio, Jannsen, Macula Society, Nanoscope, Nidek, NotalVision, Novartis, Optos, OTx, Pfizer, Regeneron, Roche, Samsung Bioepis and Topcon.

E、 H.与Optum有关联。S、 R.S.与Abbvie/Allergan、Alexion、Amgen、Apellis、ARVO、Astellas、Bayer、Biogen、Boerhinger Ingelheim、Carl Zeiss Meditec、Centervue、Character、Eyepoint、Heidelberg、iCare、IvericBio、Jannsen、Macula Society、Nanoscope、Nidek、NotalVision、Novartis、Optos、OTx、Pfizer、Regeneron、Roche、Samsung BioPis和Topcon有关联。

The other authors declare no competing interests..

其他作者声明没有利益冲突。。

Peer review

同行评审

Peer review information

同行评审信息

Nature Biomedical Engineering thanks Tianyu Zhang, Yukun Zhou and the two other, anonymous, reviewers for their contribution to the peer review of this work.

《自然生物医学工程》感谢张天宇、周玉坤和另外两位匿名审稿人为这项工作的同行评审做出的贡献。

Additional informationPublisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.Extended dataExtended Data Fig. 1 PR AUC comparison of five models in four single-task AMD-biomarker classification problems when trained on less than 700 OCT volumes.Shown are the PR AUC as an alternative scoring metric for the OCT experiments shown in Fig.

Additional informationPublisher的注释Springer Nature在已发布的地图和机构隶属关系中的管辖权主张方面保持中立。扩展数据扩展数据图1当在少于700个OCT体积上训练时，四个单任务AMD生物标志物分类问题中五个模型的PR AUC比较。显示的是PR AUC作为OCT实验的替代评分指标，如图所示。

3. The left panel shows the performance when trained and tested on the Houston Dataset (see Supplementary Table 1). The right panel shows the performance when trained on the Houston Dataset and tested on the SLIVER-net Dataset (see Supplementary Table 2). The dashed lines represent the corresponding biomarker’s positive-label prevalence, which is the expected performance of a random model.

3.左侧面板显示了在休斯顿数据集上训练和测试时的性能（见补充表1）。右侧面板显示了在休斯顿数据集上训练并在银网数据集上测试时的性能（请参见补充表2）。虚线表示相应生物标志物的阳性标记流行率，这是随机模型的预期性能。

Box plot whiskers represent a 90% CI.Extended Data Fig. 2 Precision-recall performance compared to clinical retina specialists’ assessment.Shown are the PR curves (blue) of SLIViT as an alternative scoring metric for the OCT experiments shown in Fig. 5. SLIViT was trained using less than 700 OCT volumes (Houston Dataset) and tested on an independent dataset (Pasadena Dataset).

箱形图胡须代表90%的置信区间。扩展数据图2与临床视网膜专家的评估相比，精确回忆性能。所示为SLIViT的PR曲线（蓝色），作为图5所示OCT实验的替代评分指标。SLIViT使用少于700个OCT体积（休斯顿数据集）进行训练，并在独立数据集（帕萨迪纳数据集）上进行测试。

In each panel, the light-blue shaded area represents a 90% CI for SLIViT’s performance, the red dot represents the retina clinical specialists’ average performance, and the green asterisks correspond to the retina clinical specialists’ assessments. Two of the clinical specialists obtained the exact same performance score for IHRF classification.Extended Data Fig.

在每个小组中，浅蓝色阴影区域代表SLIViT表现的90%置信区间，红点代表视网膜临床专家的平均表现，绿色星号对应于视网膜临床专家的评估。两名临床专家获得了与IHRF分类完全相同的表现评分。扩展数据图。

3 SLIViT’s performance in a frame-shuffling experiment.Shown are the ROC AUC scores distribution of 101 SLIViT models in four single-task classification problems of AMD high-risk factors (DV, IHRF, SDD, and hDC) trained on volumetric-OCT dataset.

3 SLIViT在帧改组实验中的表现。显示了在体积OCT数据集上训练的AMD高危因素（DV，IHRF，SDD和hDC）的四个单任务分类问题中101个SLIViT模型的ROC AUC分数分布。

Nat. Biomed. Eng (2024). https://doi.org/10.1038/s41551-024-01257-9Download citationReceived: 12 June 2023Accepted: 23 August 2024Published: 01 October 2024DOI: https://doi.org/10.1038/s41551-024-01257-9Share this articleAnyone you share the following link with will be able to read this content:Get shareable linkSorry, a shareable link is not currently available for this article.Copy to clipboard.

自然生物医学。Eng（2024年）。https://doi.org/10.1038/s41551-024-01257-9Download引文接收日期：2023年6月12日接收日期：2024年8月23日发布日期：2024年10月1日OI：https://doi.org/10.1038/s41551-024-01257-9Share本文与您共享以下链接的任何人都可以阅读此内容：获取可共享链接对不起，本文目前没有可共享的链接。复制到剪贴板。

Provided by the Springer Nature SharedIt content-sharing initiative

由Springer Nature SharedIt内容共享计划提供