AbstractPathologists’ assessment of sentinel lymph nodes (SNs) for breast cancer (BC) metastases is a treatment-guiding yet labor-intensive and costly task because of the performance of immunohistochemistry (IHC) in morphologically negative cases. This non-randomized, single-center clinical trial (International Standard Randomized Controlled Trial Number:14323711) assessed the efficacy of an artificial intelligence (AI)-assisted workflow for detecting BC metastases in SNs while maintaining diagnostic safety standards.

摘要病理学家对乳腺癌（BC）转移的前哨淋巴结（SNs）的评估是一项治疗指导但劳动密集且昂贵的任务，因为免疫组织化学（IHC）在形态学阴性病例中的表现。这项非随机单中心临床试验（国际标准随机对照试验编号：14323711）评估了人工智能（AI）辅助工作流程在检测SNs中BC转移同时维持诊断安全标准的功效。

From September 2022 to May 2023, 190 SN specimens were consecutively enrolled and allocated biweekly to the intervention arm (n = 100) or control arm (n = 90). In both arms, digital whole-slide images of hematoxylin–eosin sections of SN specimens were assessed by an expert pathologist, who was assisted by the ‘Metastasis Detection’ app (Visiopharm) in the intervention arm.

从2022年9月到2023年5月，连续招募190个SN标本，每两周分配给干预组（n=100）或对照组（n=90）。在两组中，由专家病理学家评估了SN标本苏木精-伊红切片的数字全幻灯片图像，该专家在干预组的“转移检测”应用程序（Visiopharm）的帮助下进行了评估。

Our primary endpoint showed a significantly reduced adjusted relative risk of IHC use (0.680, 95% confidence interval: 0.347–0.878) for AI-assisted pathologists, with subsequent cost savings of ~3,000 €. Secondary endpoints showed significant time reductions and up to 30% improved sensitivity for AI-assisted pathologists.

我们的主要终点显示，人工智能辅助病理学家使用IHC的调整后相对风险显着降低（0.680，95%置信区间：0.347-0.878），随后节省了约3000欧元的成本。次要终点显示AI辅助病理学家的时间显着减少，敏感性提高了30%。

This trial demonstrates the safety and potential for cost and time savings of AI assistance..

该试验证明了人工智能援助的安全性以及节省成本和时间的潜力。。

MainWith an incidence of 2.3 million in 2020, breast cancer remains the most common type of cancer in women worldwide1. In the Netherlands, approximately 18,000 women and more than 100 men are diagnosed with breast cancer annually, translating into the development of breast cancer during life in about one in every seven women2.

乳腺癌在2020年的发病率为230万，仍然是全球女性中最常见的癌症类型1。在荷兰，每年约有18000名女性和100多名男性被诊断出患有乳腺癌，这意味着大约每七名女性中就有一名在一生中患上乳腺癌2。

An important prognostic factor in breast cancer is the presence of (axillary) lymph node metastases3. Sentinel lymph nodes (SNs) are the first lymph nodes to drain lymphatic flow from the tumor; thus, the SN status predicts the likelihood of further axillary lymph node metastases, without the need for removing all (axillary) lymph nodes, thereby preventing major morbidity.

乳腺癌的一个重要预后因素是（腋窝）淋巴结转移的存在3。前哨淋巴结清扫器淋巴结清扫器淋巴管，前哨淋巴结清扫器淋巴管，前哨淋巴结清扫器淋巴管，前哨淋巴结清扫器淋巴管，前哨淋巴结清扫器淋巴管，前哨淋巴结清扫器淋巴管，前哨淋巴结清扫器；因此，SN状态预测了进一步腋窝淋巴结转移的可能性，而不需要切除所有（腋窝）淋巴结，从而预防了严重的发病率。

The SN procedure is, therefore, performed in persons with breast cancer in whom diagnostic imaging is negative for involved axillary lymph nodes, which is the case in the majority of persons with breast cancer3,4. The SN procedure itself entails a combination of intratumor injections with radiocolloid and a perioperative injection of patent blue to detect and resect the SN(s)3,4.

因此，SN手术是在乳腺癌患者中进行的，其中诊断成像对腋窝淋巴结阴性，这在大多数乳腺癌患者中都是如此3,4。SN程序本身需要结合肿瘤内注射放射性胶体和围手术期注射专利蓝来检测和切除SN（s）3,4。

The presence of metastases in the SNs is strongly associated with worse survival5,6,7,8 and consequently guides treatment according to the size of the metastases (that is, macrometastases (≥2 mm), micrometastases (<2 mm) or isolated tumor cells (ITCs; single tumor cells or tumor cell clusters with a maximum diameter of ≤0.2 mm and a maximum number of 200 cells per section))3.

SNs中转移瘤的存在与较差的生存率密切相关[5,6,7,8]，因此根据转移瘤的大小（即大转移瘤（≥2mm），微转移瘤（<2mm）或分离的肿瘤细胞（ITC；最大直径≤0.2mm的单个肿瘤细胞或肿瘤细胞簇，每节最多200个细胞））3。

In general, persons with an SN containing (micro)metastases, without prior neoadjuvant treatment, require adjuvant treatment, whereas those with a negative SN or only ITCs do not3,4.For pathologists, the assessment of these SNs is a tedious and labor-intensive task with a dichotomous answer: the presence or absence of SN metastase.

一般来说，患有含SN（微）转移的人，没有事先的新辅助治疗，需要辅助治疗，而那些SN阴性或只有ITC的人则不需要3,4。对于病理学家来说，对这些SN的评估是一项繁琐且劳动密集的任务，有一个二分的答案：SN转移的存在与否。

1.

1.

Workflow improvements:

工作流改进：

a.

答：。

Differences between both arms in time spent per SN specimen, measured using a stopwatch by a researcher (C.v.D.) sitting next to the pathologist assessing the slides. For practical reasons, these measurements were only performed during a few weeks within the third and fourth months of the trial.

坐在病理学家旁边评估载玻片的研究人员（C.v.D.）使用秒表测量每个SN标本的双臂时间差异。出于实际原因，这些测量仅在试验的第三个月和第四个月的几周内进行。

b.

b。

Difference in absolute number of IHC stains and subsequent costs (indicative cost of ~25 € per section) between both study arms, stratified for type of metastasis (ITC, micrometastasis or macrometastasis).

两个研究组之间IHC染色的绝对数量和随后的成本（每节约25欧元的指示性成本）的差异，按转移类型（ITC，微转移或大转移）分层。

2.

2.

Pathologist performance in both arms:

病理学家在双臂上的表现：

a.

答：。

Sensitivity and NPV of the pathologist on the HE slides, stratified for type of metastasis (ITC, micrometastasis or macrometastasis).

病理学家在HE载玻片上的敏感性和NPV，根据转移类型（ITC，微转移或大转移）进行分层。

b.

b。

AI user experience (questionnaire) of the participating pathologists (Extended Data Table 1).

参与病理学家的AI用户体验（问卷）（扩展数据表1）。

3.

3.

AI performance:

AI性能：

Standalone performance of the algorithm was assessed by one of the researchers (C.v.D.).This assessment consisted of checking whether the annotated metastases (by the pathologist) on the HE slide or the IHC slide were also annotated by the algorithm. This outcome was binary; metastases were either annotated (regardless of color—red, orange or yellow) or not.

该算法的独立性能由一位研究人员（C.v.D.）评估。该评估包括检查HE载玻片或IHC载玻片上注释的转移灶（由病理学家）是否也由算法注释。这个结果是二元的；转移瘤要么被注释（无论红色，橙色或黄色如何）。

In cases of doubt, the researcher consulted a pathologist (P.J.v.D.)..

如果有疑问，研究人员咨询了病理学家（P.J.v.D.）。。

a.

答：。

Retrospective standalone performance of the algorithm for cases with metastases in the control arm (sensitivity), stratified for type of metastasis (ITC, micrometastasis or macrometastasis).

该算法对对照组转移病例（敏感性）的回顾性独立表现，按转移类型（ITC，微转移或大转移）分层。

b.

b。

Standalone performance (sensitivity) of the algorithm in the intervention arm.

干预组中算法的独立性能（灵敏度）。

c.

c。

Overall combined performance in both arms.

双臂整体综合表现。

Lastly, from the obtained parameters (distribution of SN outcome, average number of tissue blocks and slides, sensitivity of AI-assisted pathologists and laboratory SN workflow), we calculated potential cost savings in different scenarios (Methods). Parameters in this file are adjustable, thereby enabling individualized calculations of potential cost savings.False-positive interpretations and false-positive alertsA crucial distinction was made between false-positive interpretations by the pathologist(s) (either AI-assisted or not) and false alerts by the algorithm itself (being yellow, orange or red outlines).

最后，根据获得的参数（SN结果的分布，组织块和载玻片的平均数量，AI辅助病理学家的敏感性和实验室SN工作流程），我们计算了不同场景（方法）下的潜在成本节约。此文件中的参数是可调整的，因此可以对潜在的成本节约进行个性化计算。假阳性解释和假阳性警报病理学家（人工智能辅助或不辅助）的假阳性解释与算法本身的假警报（黄色，橙色或红色轮廓）之间有着至关重要的区别。

The first type of false positive cannot be confirmed in this study as, by design, like in clinical practice, no confirmatory stains were performed when the pathologist (either AI-assisted or not) concluded that tumor cells were present on the HE slides. However, a retrospective study by Challa et al.33 with the same algorithm by Visiopharm did report on this type of false-positive pathologist interpretation and showed that they are extremely rare.

在这项研究中无法确认第一种假阳性，因为根据设计，就像在临床实践中一样，当病理学家（AI辅助或不辅助）得出结论认为HE载玻片上存在肿瘤细胞时，没有进行确认染色。。

The authors reported similarly high rates of concordance between the ground truth and the interpretation of three subspecialized breast pathologists, either assisted by AI or not (98–100%). Moreover, false positives occurred slightly more when pathologists interpreted IHC results (two of three pathologists, 1–2 of 102 cases) versus when pathologists interpreted AI results (one of three pathologists, 1 of 102 cases).

。此外，当病理学家解释IHC结果时（三位病理学家中的两位，102例中的1-2例），与病理学家解释AI结果时（三位病理学家中的一位，102例中的1例），假阳性发生率略高。

In addition, we firmly believe that no pathologist is biased toward making more cancer diagnoses (either AI-assisted or not) because these dedicated pathologists are fully aware of the diagnostic pitfalls and clinical consequences of their conclusions for patients. Therefore, although false-positive interpret.

。因此，虽然是假阳性解释。

Data availability

数据可用性

The data within this trial were derived from the structured pathology reports and information in PACS from all persons with consecutive breast cancer or DCIS with an SN. These data were securely stored in Castor EDC31. All relevant data supporting the findings of this study are available within the paper and its Supplementary Information.

该试验中的数据来自结构化病理报告和PACS中来自所有连续乳腺癌患者或患有SN的DCIS的信息。这些数据安全地存储在Castor EDC31中。本文及其补充信息中提供了支持本研究结果的所有相关数据。

The raw data that support the findings of this study are not openly available because of reasons of patient privacy but are available from the corresponding author upon reasonable request. Data are located in controlled access data storage at UMC Utrecht..

由于患者隐私的原因，支持本研究结果的原始数据无法公开获得，但可根据合理要求从通讯作者处获得。数据位于UMC乌得勒支的受控访问数据存储器中。。

Code availability

代码可用性

No specific code was designed for this trial. We used the CE-IVD-approved (certified under IVDR) deep-learning Metastasis Detection app by Visiopharm (Hoersholm, Denmark). This is a commercially available AI app, which was purchased from Visiopharm by our pathology department. All relevant information regarding the app can be obtained from Visiopharm (see also https://visiopharm.com/app-center/app/metastasis-detection-ai/)..

没有为该试验设计特定代码。我们使用了Visiopharm（Hoersholm，丹麦）批准的CE IVD（IVDR认证）深度学习转移检测应用程序。这是一个商业上可用的AI应用程序，由我们的病理科从Visiopharm购买。有关该应用程序的所有相关信息均可从Visiopharm获得（另请参见https://visiopharm.com/app-center/app/metastasis-detection-ai/)。。

ReferencesWorld Health Organization Breast Cancer (WHO, accessed 11 May 2023); https://www.who.int/news-room/fact-sheets/detail/breast-cancerThe Netherlands Comprehensive Cancer Organization Breast Cancer in The Netherlands: Key Figures from the Dutch Cancer Registry (IKNL, accessed 11 May 2023); https://iknl.nl/borstkankercijfersDutch Federation of Medical Specialist Breast Cancer Clinical Practice Guideline (NABON/NIV, accessed 11 May 2023); https://www.nabon.nl/wp-content/uploads/2022/10/Dutch-Breast-Cancer-Guideline-2012.pdfKrag, D.

参考世界卫生组织乳腺癌（世卫组织，2023年5月11日访问）；https://www.who.int/news-room/fact-sheets/detail/breast-cancerThe荷兰综合癌症组织荷兰乳腺癌：荷兰癌症登记处（IKNL，2023年5月11日访问）的关键数字；https://iknl.nl/borstkankercijfersDutch医学专家乳腺癌临床实践指南联合会（NABON/NIV，2023年5月11日访问）；https://www.nabon.nl/wp-content/uploads/2022/10/Dutch-Breast-Cancer-Guideline-2012.pdfKrag，D。

N. et al. Sentinel-lymph-node resection compared with conventional axillary-lymph-node dissection in clinically node-negative patients with breast cancer: overall survival findings from the NSABP B-32 randomised phase 3 trial. Lancet Oncol. 11, 927–933 (2010).Article .

N、 等。临床淋巴结阴性乳腺癌患者前哨淋巴结切除术与常规腋窝淋巴结清扫术的比较：NSABP B-32随机3期试验的总体生存结果。柳叶刀Oncol。11927-933（2010）。文章。

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Andersson, Y., Frisell, J., Sylvan, M., de Boniface, J. & Bergkvist, L. Breast cancer survival in relation to the metastatic tumor burden in axillary lymph nodes. J. Clin. Oncol. 28, 2868–2873 (2010).Article

Andersson，Y.，Frisell，J.，Sylvan，M.，de Boniface，J。＆Bergkvist，L。乳腺癌生存与腋窝淋巴结转移性肿瘤负荷的关系。J、 临床。Oncol公司。282868-2873（2010）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Weaver, D. L. et al. Effect of occult metastases on survival in node-negative breast cancer. N. Engl. J. Med. 364, 412–421 (2011).Article

Weaver，D.L.等人。隐匿性转移对淋巴结阴性乳腺癌生存的影响。N、 英语。J、 医学364412-421（2011）。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

de Boer, M. et al. Micrometastases or isolated tumor cells and the outcome of breast cancer. N. Engl. J. Med. 361, 653–663 (2009).Article

de Boer，M.等人。微转移或分离的肿瘤细胞与乳腺癌的结果。N、 英语。J、 医学361653-663（2009）。文章

PubMed

PubMed

Google Scholar

谷歌学者

van der Heiden-van der Loo, M. et al. Outcomes of a population-based series of early breast cancer patients with micrometastases and isolated tumour cells in axillary lymph nodes. Ann. Oncol. 24, 2794–2801 (2013).Article

van der Heiden van der Loo，M.等人。基于人群的一系列早期乳腺癌患者腋窝淋巴结微转移和孤立肿瘤细胞的结果。安科。242794-2801（2013）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Jiang, Y., Yang, M., Wang, S., Li, X. & Sun, Y. Emerging role of deep learning-based artificial intelligence in tumor pathology. Cancer Commun. (Lond.) 40, 154–166 (2020).Article

Jiang，Y.，Yang，M.，Wang，S.，Li，X。＆Sun，Y。基于深度学习的人工智能在肿瘤病理学中的新兴作用。癌症社区。（伦敦）40154-166（2020）。文章

PubMed

PubMed

Google Scholar

谷歌学者

van der Laak, J., Litjens, G. & Ciompi, F. Deep learning in histopathology: the path to the clinic. Nat. Med. 27, 775–784 (2021).Article

van der Laak，J.，Litjens，G。＆Ciompi，F。组织病理学的深度学习：通往临床的道路。《自然医学》27775-784（2021）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Steiner, D. F. et al. Evaluation of the use of combined artificial intelligence and pathologist assessment to review and grade prostate biopsies. JAMA Netw. Open 3, e2023267 (2020).Article

Steiner，D.F.等人。使用人工智能和病理学家联合评估对前列腺活检进行审查和分级的评估。JAMA网络。开放3，e2023267（2020）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Muehlematter, U. J., Daniore, P. & Vokinger, K. N. Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015–20): a comparative analysis. Lancet Digit. Health 3, e195–e203 (2021).Article

Muehlematter，U.J.，Daniore，P。＆Vokinger，K.N。批准美国和欧洲基于人工智能和机器学习的医疗设备（2015-20）：比较分析。柳叶刀数字。健康3，e195–e203（2021）。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Ehteshami Bejnordi, B. et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318, 2199–2210 (2017).Article

Ehteshami Bejnordi，B.等人。用于检测乳腺癌女性淋巴结转移的深度学习算法的诊断评估。JAMA 3182199-2210（2017）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Harrison, J. H. et al. Introduction to artificial intelligence and machine learning for pathology. Arch. Pathol. Lab. Med. 145, 1228–1254 (2021).Article

Harrison，J.H.等人，《病理学人工智能和机器学习简介》。拱门。病理学。实验室医学1451228-1254（2021）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Donker, M. et al. Marking axillary lymph nodes with radioactive iodine seeds for axillary staging after neoadjuvant systemic treatment in breast cancer patients: the MARI procedure. Ann. Surg. 261, 378–382 (2015).Article

Donker，M.等人。乳腺癌患者新辅助全身治疗后用放射性碘种子标记腋窝淋巴结进行腋窝分期：MARI手术。《外科杂志》261378-382（2015）。文章

PubMed

PubMed

Google Scholar

谷歌学者

van Bergeijk, S. A. et al. Deep learning supported mitoses counting on whole slide images: a pilot study for validating breast cancer grading in the clinical workflow. J. Pathol. Inform. 14, 100316 (2023).Article

van Bergeijk，S.A.等人，《深度学习支持对整个幻灯片图像进行有丝分裂计数：一项在临床工作流程中验证乳腺癌分级的初步研究》。J、 病理学。通知。14100316（2023）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Vestjens, J. et al. Relevant impact of central pathology review on nodal classification in individual breast cancer patients. Ann. Oncol. 23, 2561–2566 (2012).Article

Vestjens，J.等。中央病理学检查对个体乳腺癌患者淋巴结分类的相关影响。安科。232561-2566（2012）。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

Ho, J. et al. Can digital pathology result in cost savings? A financial projection for digital pathology implementation at a large integrated health care organization. J. Pathol. Inform. 5, 33 (2014).Article

Ho，J.等人。数字病理学可以节省成本吗？大型综合医疗保健组织数字病理学实施的财务预测。J、 病理学。通知。5，33（2014）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Steiner, D. F., Chen, P.-H. C. & Mermel, C. H. Closing the translation gap: AI applications in digital pathology. Biochim. Biophys. Acta Rev. Cancer 1875, 188452 (2021).Article

Steiner，D.F.，Chen，P.-H.C.＆Mermel，C.H。缩小翻译差距：人工智能在数字病理学中的应用。生物化学。生物物理。《癌症学报》1875188452（2021）。文章

CAS

中科院

PubMed

PubMed

Google Scholar

谷歌学者

van Dooijeweert, C., van Diest, P. J., Baas, I. O., van der Wall, E. & Deckers, I. A. Variation in breast cancer grading: the effect of creating awareness through laboratory-specific and pathologist-specific feedback reports in 16 734 patients with breast cancer. J. Clin. Pathol. 73, 793–799 (2020).Article .

van Dooijewert，C.，van Diest，P.J.，Baas，I.O.，van der Wall，E。＆Deckers，I.A。乳腺癌分级的变化：通过实验室特异性和病理学家特异性反馈报告在16734名乳腺癌患者中产生意识的效果。J、 临床。病理学。73793-799（2020）。文章。

PubMed

PubMed

Google Scholar

谷歌学者

McNutt, L. A., Wu, C., Xue, X. & Hafner, J. P. Estimating the relative risk in cohort studies and clinical trials of common outcomes. Am. J. Epidemiol. 157, 940–943 (2003).Article

。美国流行病学杂志。157940-943（2003）。文章

PubMed

PubMed

Google Scholar

谷歌学者

Lumley, T., Kronmal, R. & Ma, S. Relative risk regression in medical research: models, contrasts, estimators, and algorithms. In UW Biostatistics Working Paper Series. Working Paper 293 (2006).Zou, G. A modified poisson regression approach to prospective studies with binary data. Am.

Lumley，T.，Kronmal，R。＆Ma，S。医学研究中的相对风险回归：模型，对比，估计量和算法。在UW生物统计学工作论文系列中。工作文件293（2006）。Zou，G。使用二元数据进行前瞻性研究的改进泊松回归方法。上午。

J. Epidemiol. 159, 702–706 (2004).Article .

J、流行病。159702-706（2004）。文章。

PubMed

PubMed

Google Scholar

谷歌学者

Knol, M. J., Duijnhoven, R. G., Grobbee, D. E., Moons, K. G. & Groenwold, R. H. Potential misinterpretation of treatment effects due to use of odds ratios and logistic regression in randomized controlled trials. PLoS ONE 6, e21248 (2011).Article

。PLoS ONE 6，e21248（2011）。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Mittinty, M. N. & Lynch, J. Reflection on modern methods: risk ratio regression—simple concept yet complex computation. Int. J. Epidemiol. 52, 309–314 (2022).Article

Mittinty，M.N。＆Lynch，J。对现代方法的反思：风险比回归简单的概念但复杂的计算。国际流行病学杂志。52309-314（2022）。文章

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Knol, M. J. Down with odds ratios: risk ratios in cohort studies and randomised clinical trials. Ned. Tijdschr. Geneeskd. 156, A4775 (2012).PubMed

Knol，M.J。下降优势比：队列研究和随机临床试验中的风险比。内德·蒂伊德施尔。Geneeskd。156，A4775（2012）。PubMed出版社

Google Scholar

谷歌学者

Schwendinger, F., Grün, B. & Hornik, K. A comparison of optimization solvers for log binomial regression including conic programming. Comput. Stat. 36, 1721–1754 (2021).Article

Schwendinger，F.，Grün，B。＆Hornik，K。对数二项回归优化求解器的比较，包括圆锥规划。计算机。Stat.361721–1754（2021）。文章

Google Scholar

谷歌学者

Schwendinger, F. Detecting separation and infinite estimates in log binomial regression. CRAN https://cran.r-project.org/web/packages/detectseparation/vignettes/infinite_estimates.html (2022).Stathonikos, N., Nguyen, T. Q., Spoto, C. P., Verdaasdonk, M. A. M. & van Diest, P. J. Being fully digital: perspective of a Dutch academic pathology laboratory.

。起重机https://cran.r-project.org/web/packages/detectseparation/vignettes/infinite_estimates.html（2022年）。Stathonikos，N.，Nguyen，T.Q.，Spoto，C.P.，Verdaasdonk，M.A.M。和van Diest，P.J。完全数字化：荷兰学术病理实验室的观点。

Histopathology 75, 621–635 (2019).Article .

。文章。

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Cruz Rivera, S. et al. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nat. Med. 26, 1351–1363 (2020).Article

Cruz-Rivera，S.等人，《涉及人工智能干预的临床试验方案指南：SPIRIT-AI扩展》。《自然医学》261351-1363（2020）。文章

CAS

中科院

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

Capture CED. Castor https://castoredc.com (2019).Brooke, J. SUS: A quick and dirty usability scale. In Usability Evaluation in Industry 1st edn (eds Jordan, P. W. et al.) (CRC Press, 1996).Challa, B. et al. Artificial intelligence-aided diagnosis of breast cancer lymph node metastasis on histologic slides in a digital workflow.

捕获CED。脚轮https://castoredc.com（2019年）。布鲁克（Brooke，J.SUS）：一个快速而肮脏的可用性量表。《工业第一版可用性评估》（eds Jordan，P.W.等人）（CRC出版社，1996年）。Challa，B.等人。在数字工作流程中，人工智能辅助诊断组织学载玻片上的乳腺癌淋巴结转移。

Mod. Pathol. 36, 100216 (2023).Article .

国防部。病理学。36100216（2023）。文章。

PubMed

PubMed

Google Scholar

谷歌学者

Petersen, M. R. & Deddens, J. A. A comparison of two methods for estimating prevalence ratios. BMC Med. Res. Methodol. 8, 9 (2008).Article

Petersen，M.R。＆Deddens，J.A。两种估计患病率方法的比较。BMC医学研究方法。8、9（2008年）。文章

PubMed

PubMed

PubMed Central

公共医学中心

Google Scholar

谷歌学者

R: a language and environment for statistical computing. CRAN https://www.R-project.org/ (2018).Download referencesAcknowledgementsWe sincerely thank R. Stellato for her statistical advice. Funding for this study was obtained from the Hanarth Fund by P.J.v.D. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

R： 用于统计计算的语言和环境。起重机https://www.R-project.org/（2018年）。。这项研究的资金来自P.J.v.D.的Hanarth基金。资助者在研究设计，数据收集和分析，决定出版或准备手稿方面没有任何作用。

The AI implementation program at UMC Utrecht was supported by an unrestricted educational grant from Pfizer, Inc. The AI algorithm used was purchased at a reasonable market price and the company providing the algorithm (Visiopharm, Denmark) did not have any role in this study nor were the data (both the data in Castor EDC31 and the participant files) accessible to the companies or funding source at any point.Author informationAuthors and AffiliationsDepartment of Pathology, University Medical Center Utrecht, Utrecht, The NetherlandsC.

UMC乌得勒支的人工智能实施计划得到了辉瑞公司（Pfizer，Inc.）不受限制的教育资助。所使用的人工智能算法是以合理的市场价格购买的，提供该算法的公司（丹麦Visiopharm）在这项研究中没有任何作用，公司或资金来源在任何时候都无法访问数据（Castor EDC31中的数据和参与者文件）。作者信息作者和附属机构荷兰乌得勒支大学医学中心病理学系。

van Dooijeweert, R. N. Flach, N. D. ter Hoeve, C. P. H. Vreuls, R. Goldschmeding, J. E. Freund, P. Pham, T. Q. Nguyen, N. Stathonikos & P. J. van DiestDepartment of Medical Oncology, University Medical Center Utrecht, Utrecht, The NetherlandsE. van der WallDepartment of Epidemiology and Health Economics, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The NetherlandsG.

van Dooijewert，R。N、 Flach，N。D、 特霍夫，C。P、 H.Vreuls，R。戈德施梅丁，J。E、 弗劳德，P。范，T。Q、 阮，N。Stathonikos公司。J、 荷兰乌得勒支乌得勒支大学医学中心医学肿瘤学系。荷兰乌得勒支乌得勒支大学医学中心朱利叶斯健康科学和初级保健中心流行病学和卫生经济学系。

W. J. FrederixAuthorsC. van DooijeweertView author publicationsYou can also search for this author in.

W、 弗雷德里克萨瑟斯。van DooijewertView作者出版物您也可以在中搜索此作者。

PubMed Google ScholarR. N. FlachView author publicationsYou can also search for this author in

PubMed Google ScholarR.N.FlachView作者出版物您也可以在

PubMed Google ScholarN. D. ter HoeveView author publicationsYou can also search for this author in

PubMed Google ScholarN.D.ter HoeveView作者出版物您也可以在

PubMed Google ScholarC. P. H. VreulsView author publicationsYou can also search for this author in

PubMed谷歌ScholarC。P、 H.VreulsView作者出版物您也可以在

PubMed Google ScholarR. GoldschmedingView author publicationsYou can also search for this author in

PubMed Google ScholarR。GoldschmedingView作者出版物您也可以在

PubMed Google ScholarJ. E. FreundView author publicationsYou can also search for this author in

PubMed Google ScholarJ.E.FreundView作者出版物您也可以在

PubMed Google ScholarP. PhamView author publicationsYou can also search for this author in

PubMed谷歌ScholarP。PhamView作者出版物您也可以在

PubMed Google ScholarT. Q. NguyenView author publicationsYou can also search for this author in

PubMed谷歌ScholarT。Q、 NguyenView作者出版物您也可以在

PubMed Google ScholarE. van der WallView author publicationsYou can also search for this author in

PubMed谷歌ScholarE。van der WallView作者出版物您也可以在

PubMed Google ScholarG. W. J. FrederixView author publicationsYou can also search for this author in

PubMed谷歌学术博客。W、 J.FrederixView作者出版物您也可以在

PubMed Google ScholarN. StathonikosView author publicationsYou can also search for this author in

PubMed Google ScholarN.StathonikosView作者出版物您也可以在

PubMed Google ScholarP. J. van DiestView author publicationsYou can also search for this author in

PubMed谷歌ScholarP。J、 van DiestView作者出版物您也可以在

PubMed Google ScholarContributionsC.v.D., R.N.F. and P.J.v.D. developed the concept and designed the study. C.P.H.V., R.G., J.E.F. and P.J.v.D participated in the study as expert breast pathologists. N.S. and P.P. provided IT support throughout the study. N.D.t.H. and C.v.D. collected the data.

PubMed Google ScholarContributionsC.v.D.，R.N.F.和P.J.v.D.开发了这一概念并设计了这项研究。C、 P.H.V.，R.G.，J.E.F.和P.J.V.D作为专家乳腺病理学家参与了这项研究。N、 在整个研究过程中，S.和P.P.提供了IT支持。N、 D.t.H.和C.v.D.收集了数据。

G.W.J.F. advised on the health technology assessment methodology. T.Q.N. and E.v.d.W. provided (non-participating) clinical perspective as a pathologist and medical oncologist. C.v.D. and R.N.F. performed the analysis. C.v.D. wrote the original draft and all authors reviewed and edited the paper. All authors read and agreed to the published version of the article.Corresponding authorsCorrespondence to.

G、 W.J.F.为卫生技术评估方法提供咨询。T、 Q.N.和E.v.d.W.作为病理学家和医学肿瘤学家提供了（非参与的）临床观点。C、 v.D.和R.N.F.进行了分析。C、 v.D.撰写了原稿，所有作者都对论文进行了审查和编辑。所有作者都阅读并同意文章的发布版本。通讯作者通讯。

C. van Dooijeweert or P. J. van Diest.Ethics declarations

C.van Dooijewert或P.J.van Diest。道德宣言

Competing interests

相互竞争的利益

P.J.v.D. is a member of the advisory boards of Visiopharm, Paige and Sectra. The other authors declare no competing interests.

P、 J.v.D.是Visiopharm、Paige和Sectra咨询委员会的成员。其他作者声明没有利益冲突。

Additional informationPublisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.Extended dataExtended Data Fig. 1 Example images of two cases of initially undetected micro-metastases (false negatives) in the AI-assisted arm (A-B & C-D) and false positive alerts of the Metastasis Detection App by Visiopharm (E-J).a/b: Sentinel node with micro-metastases on the regular HE-slide partly highlighted (in yellow and orange) by the algorithm, that was overlooked by the AI-assisted pathologist on the HE slide (A) and the detected micro-metastasis on the IHC-stained slide (B).

Additional informationPublisher的注释Springer Nature在已发布的地图和机构隶属关系中的管辖权主张方面保持中立。扩展数据扩展数据图1 AI辅助组（A-B＆C-D）中两例最初未检测到的微转移（假阴性）和Visiopharm（E-J）转移检测应用程序的假阳性警报的示例图像。A/B：常规HE载玻片上具有微转移的前哨淋巴结通过算法部分突出显示（黄色和橙色），这被AI辅助病理学家在HE载玻片上忽略（A），并且在IHC染色的载玻片上检测到微转移（B）。

c/d: Sentinel node with micro-metastases located in a heavily cauterized area on the HE-section (C), which therefore could only be detected in the IHC-section (D). e-j: False positive alerts by AI: blood vessels highlighted in red, yellow and orange (A), sinus histiocytosis highlighted in red (B), nerves highlighted in red (C), a follicle center highlighted in red (D), a pigment laden macrophage highlighted in red and orange (E) and a capsular naevus (F) in red.

c/d：前哨淋巴结微转移位于HE切片（c）的严重烧灼区域，因此只能在IHC切片（d）中检测到。e-j：AI的假阳性警报：以红色，黄色和橙色突出显示的血管（A），以红色突出显示的窦组织细胞增多症（B），以红色突出显示的神经（C），以红色突出显示的卵泡中心（D），以红色和橙色突出显示的富含色素的巨噬细胞（e）和红色的囊泡痣（F）。

The example images are snapshots derived from whole slide images of sentinel lymph nodes included in the CONFIDENT-B trial.Extended Data Table 1 User experience survey among participating pathologists that used the algorithm (n = 4)Full size tableSupplementary informationSupplementary InformationStudy protocol, statistical analysis plan and SPIRIT-AI checklist.Reporting SummarySupplementary Data 1Explainer scenario ‘maintaining current safety standards’ and explainer scenario ‘personalized IHC use’.

示例图像是从CONFIDENT-B试验中包含的前哨淋巴结的全幻灯片图像中获得的快照。扩展数据表1使用算法的参与病理学家的用户体验调查（n=4）全尺寸表补充信息补充信息研究协议，统计分析计划和SPIRIT-AI清单。报告摘要补充数据1解释者情景“维持当前安全标准”和解释者情景“个性化IHC使用”。

Parameters in this file are adjustable, thereby enabling individualized calculations of potential cost savings.Rights and permissions.

此文件中的参数是可调整的，因此可以对潜在的成本节约进行个性化计算。权限和权限。

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

开放获取本文是根据知识共享署名4.0国际许可证授权的，该许可证允许以任何媒体或格式使用，共享，改编，分发和复制，只要您对原始作者和来源给予适当的信任，提供知识共享许可证的链接，并指出是否进行了更改。

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

本文中的图像或其他第三方材料包含在文章的知识共享许可中，除非在材料的信用额度中另有说明。如果材料未包含在文章的知识共享许可证中，并且您的预期用途未被法律法规允许或超出允许的用途，则您需要直接获得版权所有者的许可。

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/..

要查看此许可证的副本，请访问http://creativecommons.org/licenses/by/4.0/..

Reprints and permissionsAbout this articleCite this articlevan Dooijeweert, C., Flach, R.N., ter Hoeve, N.D. et al. Clinical implementation of artificial-intelligence-assisted detection of breast cancer metastases in sentinel lymph nodes: the CONFIDENT-B single-center, non-randomized clinical trial..

转载和许可本文引用本文van Dooijewert，C.，Flach，R.N.，ter Hoeve，N.D.等人。人工智能辅助检测前哨淋巴结乳腺癌转移的临床实施：CONFIDE-B单中心非随机临床试验。。

Nat Cancer (2024). https://doi.org/10.1038/s43018-024-00788-zDownload citationReceived: 15 January 2024Accepted: 29 May 2024Published: 27 June 2024DOI: https://doi.org/10.1038/s43018-024-00788-zShare this articleAnyone you share the following link with will be able to read this content:Get shareable linkSorry, a shareable link is not currently available for this article.Copy to clipboard.

Nat Cancer（2024）。https://doi.org/10.1038/s43018-024-00788-zDownload引文接收日期：2024年1月15日接受日期：2024年5月29日发布日期：2024年6月27日OI：https://doi.org/10.1038/s43018-024-00788-zShare本文与您共享以下链接的任何人都可以阅读此内容：获取可共享链接对不起，本文目前没有可共享的链接。复制到剪贴板。

Provided by the Springer Nature SharedIt content-sharing initiative

由Springer Nature SharedIt内容共享计划提供