NEW YORK – Researchers from Harvard Medical School have developed a new artificial intelligence-based model for systematically diagnosing multiple cancer types and predicting patients' prognoses from digital pathology images.

纽约——哈佛医学院的研究人员开发了一种新的基于人工智能的模型，用于系统诊断多种癌症类型，并根据数字病理图像预测患者的预后。

Described in a paper published this week in Nature,  the Clinical Histopathology Imaging Evaluation Foundation (CHIEF) model was developed to help clinicians evaluate pathology samples routinely collected from cancer patients, according to Kun-Hsing Yu, an assistant professor of biomedical informatics at Harvard and the lead author on the paper.

哈佛大学生物医学信息学助理教授、论文主要作者余坤兴表示，临床组织病理学成像评估基金会（CHIEF）模型在本周《自然》杂志上发表的一篇论文中有描述，该模型旨在帮助临床医生评估常规从癌症患者身上采集的病理样本。

The foundation model was developed using two different types of pretraining: self-supervised machine learning that used 15 million unlabeled pathology image tiles cropped from whole-slide images to learn tile-level microscopic features, and additional pretraining on the more than 60,000 whole-slide images to understand the context of the whole tissue.

基础模型是使用两种不同类型的预训练开发的：自我监督机器学习，使用从整个幻灯片图像中裁剪的1500万个未标记的病理图像块来学习瓷砖级别的微观特征，以及对60000多个整个幻灯片图像进行额外的预训练以了解整个组织的背景。

.

.

The self-supervised pretraining required the model to identify repeating signals in the pathology images to determine what a typical pathology sample would look like and establish a general framework for cancer-related microscopic features, Yu said.

Yu说，自我监督的预训练要求该模型识别病理图像中的重复信号，以确定典型病理样本的外观，并建立与癌症相关的微观特征的一般框架。

Meanwhile, the weakly-supervised training then required the model to pick out those features in much larger, more complex and higher resolution whole-slide images and characterize the similarities and differences between cancer types. The model continued to update its 'knowledge about what typical pathology manifestations would look like across different cancer types and across samples collected from different hospitals,' Yu added.

同时，弱监督训练要求模型在更大，更复杂和更高分辨率的全幻灯片图像中挑选出这些特征，并表征癌症类型之间的相似性和差异性。Yu补充说，该模型继续更新其“关于不同癌症类型和不同医院采集的样本中典型病理表现的知识”。

By aggregating tissue signals from different regions within the same pathology sample, the model better understands visual context and can provide a 'holistic evaluation for each patient,' Yu said..

Yu说，通过聚集同一病理样本中不同区域的组织信号，该模型可以更好地理解视觉环境，并可以为每个患者提供“整体评估”。。

Foundation models are large, general-purpose AI models that can be tailored for different functions, Yu said, and the CHIEF model can be applied to a variety of different tasks, including cancer detection, tumor origin prediction, genomic profile identification, and survival prediction. Yu and his colleagues externally validated the model's ability to detect cancer with more than 13,000 whole-slide images, including those from public databases like the Clinical Proteomic Tumor Analysis Consortium and those from specific hospitals.

Yu说，基础模型是大型通用人工智能模型，可以针对不同的功能进行定制，主要模型可以应用于各种不同的任务，包括癌症检测、肿瘤起源预测、基因组图谱识别和生存预测。Yu和他的同事通过13000多张全幻灯片图像（包括来自临床蛋白质组学肿瘤分析协会等公共数据库的图像和来自特定医院的图像）从外部验证了该模型检测癌症的能力。

The whole-slide images contained biopsy and surgical resection slides and encompassed 11 different primary cancer sites, such as the breast, skin, prostate, kidneys, and lungs, the researchers noted in the Nature paper. .

研究人员在《自然》杂志的论文中指出，整个幻灯片图像包含活检和手术切除幻灯片，涵盖了11个不同的原发癌部位，如乳腺癌、皮肤癌、前列腺癌、肾癌和肺癌。

They compared CHIEF to three other weakly-supervised whole-slide image classification methods and CHIEF outperformed all three with an average area under the receiver operating characteristic curve (AUROC) of .94, 10 to 14 percent higher than the comparator methods. Yu noted that the model had near-perfect performance in many cancer types, such as colorectal and esophageal cancers, but that its performance in some cancers, such as kidney cancer, in some cohorts was lower, possibly due to the quality of the scanned slides and the different techniques used for collecting samples.

他们将CHIEF与其他三种弱监督的全幻灯片图像分类方法进行了比较，CHIEF优于所有三种方法，接收器工作特征曲线（AUROC）下的平均面积为0.94，比比较方法高10%至14%。Yu指出，该模型在许多癌症类型（如结直肠癌和食道癌）中具有近乎完美的表现，但在某些人群中，其在某些癌症（如肾癌）中的表现较低，这可能是由于扫描载玻片的质量和用于收集样本的不同技术。

.

.

The team focused on common cancers because of the larger amount of data available to train and validate the model. As a result, the performance of CHIEF for detecting particularly rare cancers has not been validated. However, the researchers are considering approaches for diagnosing rarer cancers that rely on using fewer samples, Yu said.

该团队专注于常见癌症，因为有大量数据可用于训练和验证模型。因此，CHIEF在检测特别罕见癌症方面的表现尚未得到验证。然而，Yu说，研究人员正在考虑使用较少样本来诊断罕见癌症的方法。

.

.

Beyond cancer detection, the model was also tested on its ability to predict molecular profiles of cancer samples. The researchers focused on four specific tasks: systematic prediction of prevalent genetic mutations across cancer types, identification of mutations related to targeted therapies, isocitrate dehydrogenase status prediction for the classification of glioma, and microsatellite instability prediction for determining the benefits of immune checkpoint blockade in patients with colorectal cancer..

除了癌症检测之外，还测试了该模型预测癌症样品分子谱的能力。研究人员专注于四项具体任务：系统预测癌症类型中普遍存在的基因突变，鉴定与靶向治疗相关的突变，异柠檬酸脱氢酶状态预测用于神经胶质瘤分类，以及微卫星不稳定性预测用于确定结直肠癌患者免疫检查点阻断的益处。。

The team conducted the analysis using more than 13,000 whole-slide images across 30 types of cancer and 53 genes with the top-five highest mutation rates in each cancer type, the researchers noted in the paper. CHIEF was able to predict the mutation status of nine genes with AUROCs above .80. The researchers also used CHIEF to predict genes associated with US Food and Drug Administration-approved targeted therapies across 18 genes and 15 cancer types, and the model was able to predict the mutation status of all 18 genes with AUROCs above .60, with one as high as .96..

研究人员在论文中指出，该团队使用了30种癌症和53种基因的13000多张全幻灯片图像进行了分析，每种癌症的前五位突变率最高。CHIEF能够预测AUROC高于0.80的9个基因的突变状态。研究人员还使用CHIEF预测了与美国食品和药物管理局批准的针对18种基因和15种癌症类型的靶向治疗相关的基因，该模型能够预测AUROC高于0.60的所有18个基因的突变状态，其中一个高达0.96。。

In addition, the researchers used CHIEF to establish stage-stratified survival prediction models, utilizing more than 9,000 whole-slide images in 17 datasets for seven cancer types. In all cancer types and study cohorts, CHIEF was able to distinguish patients with longer-term survival from those with shorter-term survival, the researchers wrote..

此外，研究人员使用CHIEF建立了阶段分层生存预测模型，利用17个数据集中的7种癌症类型的9000多张全幻灯片图像。研究人员写道，在所有癌症类型和研究队列中，CHIEF能够区分长期生存的患者和短期生存的患者。。

The team is also interested in investigating if the model can be used to predict patients who will respond to immunotherapies and predict potential side effects, Yu said.

Yu说，该团队还对研究该模型是否可用于预测对免疫疗法有反应的患者以及预测潜在的副作用感兴趣。

Bias can be a significant problem in artificial intelligence models, which the Harvard team wanted to address. The researchers accounted for bias in their model by using data from multiple hospitals in multiple countries in pretraining and validation, and by evaluating CHIEF's performance without adjusting or fine-tuning the model when applied to different datasets, which ensures the model is not discriminatory or impacted by different types of sample preparation or collection procedures, Yu added.

偏见可能是人工智能模型中的一个重要问题，哈佛团队希望解决这个问题。Yu补充道，研究人员通过使用来自多个国家多家医院的数据进行预训练和验证，并通过评估CHIEF的表现来解释他们模型中的偏差，而不需要在应用于不同数据集时调整或微调模型，从而确保模型不具歧视性或不受不同类型样本制备或采集程序的影响。

However, the researchers are investigating the model's performance in patients with different ancestry and have developed a method to address that potential bias that they plan to submit for publication soon, he said..

然而，研究人员正在调查该模型在不同血统患者中的表现，并开发了一种方法来解决潜在的偏见，他们计划很快提交出版，他说。。

Now that CHIEF has been validated for multiple cancers and for multiple diagnostic tasks, Yu said the team is initiating a study to build clinical data for regulatory approval in the US. It also plans to partner with biotechnology companies to collect more samples and conduct further clinical trials, eventually packaging the model into a product or system that clinicians can use.

现在，CHIEF已经被验证用于多种癌症和多种诊断任务，Yu说，该团队正在启动一项研究，以建立临床数据，供美国监管部门批准。它还计划与生物技术公司合作，收集更多样本并进行进一步的临床试验，最终将模型包装成临床医生可以使用的产品或系统。

The team is also focused on developing additional techniques to further improve the performance and reliability of CHIEF, Yu said..

Yu说，该团队还专注于开发其他技术，以进一步提高CHIEF的性能和可靠性。。

Nigam Shah, a professor of medicine at Stanford University and chief data scientist for Stanford Health Care who was not involved in CHIEF's development, said that while foundation models have been around for a couple of years, what Yu's group has done is 'pretty innovative.'

斯坦福大学（Stanford University）医学教授、斯坦福医疗保健（Stanford Health Care）首席数据科学家尼加姆·沙阿（Nigam Shah）没有参与奇普的开发，他说，虽然基金会模型已经存在了几年，但余的团队所做的是“相当创新的”

However, he noted that evaluating AI models is not always the main issue when determining clinical utility. Instead, it is determining where they fit in a clinical workflow and the resource constraints of using them, as well as their sustainability.

然而，他指出，在确定临床效用时，评估AI模型并不总是主要问题。相反，它决定了它们在临床工作流程中的位置以及使用它们的资源限制，以及它们的可持续性。

To that end, other researchers, including a group from the University of Washington and Microsoft Research that published a paper in Nature in May, are also working on foundation models for digital pathology, and according to Shah, as more groups come up with foundation models or other AI-based tools, and clinical use of these tools becomes more widespread, sustainability and reimbursement concerns will become an even bigger area of focus..

为此，其他研究人员，包括华盛顿大学和微软研究院（Microsoft Research）的一个小组，在5月的《自然》杂志上发表了一篇论文，也正在研究数字病理学的基础模型，据沙阿（Shah）称，随着越来越多的小组提出基础模型或其他基于人工智能的工具，以及这些工具的临床使用变得越来越广泛，可持续性和报销问题将成为更大的关注领域。。

Addressing where the CHIEF model will fit in the clinical workflow, Yu said he sees the model 'initially serving as an aid to pathologists, providing second opinions to support specialists in their final evaluations, thereby reducing the time needed to diagnose each case.' However, in the long term CHIEF could evolve into an 'autonomous pathology evaluation' system for routine cases, which would reduce pathologists' workload.

谈到主要模型在临床工作流程中的适用性，余说，他认为该模型“最初是为了帮助病理学家，提供第二意见以支持专家进行最终评估，从而减少了诊断每个病例所需的时间。”然而，从长远来看，首席可以演变为常规病例的“自主病理评估”系统，这将减少病理学家的工作量。

.

.

He added that resource and economic constraints 'could actually facilitate the adoption of open-source models like CHIEF.'  As long as a hospital has digital pathology scanners and electronic medical record systems for pathology images, incorporating CHIEF or another similar model 'would involve minimal costs, primarily for hiring one to two developers to integrate our tools into the existing medical record system and covering computer server expenses,' he added.

他补充说，只要医院拥有用于病理图像的数字病理扫描仪和电子病历系统，合并CHIEF或其他类似模型“将涉及最低成本，主要是雇用一到两名开发人员将我们的工具集成到现有的病历系统中，并支付计算机服务器费用”。

'These costs would be significantly lower than onboarding additional medical specialists and technicians to manage the growing clinical workload.' .

“这些费用将大大低于聘请额外的医学专家和技术人员来管理日益增长的临床工作量。”。

Yu also mentioned that his team is working on a separate project to enable low-cost digital pathology imaging for hospitals and clinics without standard pathology scanners.

Yu还提到，他的团队正在进行一个单独的项目，以便在没有标准病理扫描仪的情况下为医院和诊所实现低成本的数字病理成像。