Epic launched an open source tool on Wednesday to enable healthcare organizations to test and monitor artificial intelligence models.

Epic于周三推出了一个开源工具，使医疗保健组织能够测试和监控人工智能模型。

The AI validation software suite is free and available to the public on GitHub, Corey Miller, vice president of research and development at Epic, told Fierce Healthcare. Health systems can download the code to their electronic health record systems, he said.

Epic研究与开发副总裁科里·米勒（CoreyMiller）告诉Fierce Healthcare，AI验证软件套件在GitHub上是免费的，公众可以使用。他说，卫生系统可以将代码下载到电子健康记录系统。

Health systems can use the tool to validate AI models that integrate with EHR systems, including models developed by Epic as well as other those developed by other organizations. As AI best practices are developed, the open-source framework will enable organizations to bring in those standards and practices alongside the AI validation capabilities, Epic executives said..

卫生系统可以使用该工具验证与EHR系统集成的人工智能模型，包括Epic开发的模型以及其他组织开发的模型。Epic高管表示，随着人工智能最佳实践的发展，开源框架将使组织能够将这些标准和实践与人工智能验证功能结合起来。。

Miller said it marks Epic's first open-source tool.

米勒说，它标志着Epic的第一个开源工具。

'By publishing on GitHub, it's truly available to everyone; it's not behind any lock or key that we control. We're excited to dive into this world. It's fitting that a tool intended to ensure the equity of health AI is going to be publicly available and open to contributors from around the globe,' he said in an interview..

“通过在GitHub上发布，每个人都可以真正使用它；它不在我们控制的任何锁或钥匙后面。我们很兴奋能深入这个世界。他在一次采访中说，一种旨在确保健康人工智能公平性的工具将向全球各地的贡献者公开，这是合适的。。

In early April, the EHR giant announced plans to release an AI validation software suite to enable healthcare organizations to evaluate AI models at the local level and monitor those systems over time.

4月初，这家EHR巨头宣布计划发布AI验证软件套件，以使医疗保健组织能够在当地评估AI模型，并随着时间的推移监测这些系统。

The AI software suite, what Epic calls an 'AI trust and assurance software suite,' automates data collection and mapping to provide near real-time metrics and analysis on AI models, Seth Hain, Epic senior vice president of R&D, said back in April. The automation creates consistency and eliminates the need for healthcare organization data scientists to do their own data mapping—the most time-consuming aspect of validation, according to Hain..

Epic研发高级副总裁塞思·海恩（SethHain）在4月份表示，人工智能软件套件（Epic称之为“人工智能信任和保证软件套件”）自动化了数据收集和映射，以提供人工智能模型的近实时度量和分析。海恩说，自动化可以创建一致性，消除了医疗机构数据科学家自己进行数据映射的需要，这是验证中最耗时的方面。。

Related

相关

Epic plans to launch AI validation software for healthcare organizations to test, monitor models

Epic计划推出AI验证软件，供医疗保健组织测试、监控模型

The key is to enable AI testing and validation at a local level and allow ongoing monitoring at scale, Hain noted.

Hain指出，关键是在地方层面上实现人工智能测试和验证，并允许大规模持续监测。

The current version of the open-source tool does not validate the performance of generative AI models, Miller said this week, but Epic plans to expand it to more AI models in the future.

Miller本周表示，目前版本的开源工具并不能验证生成型人工智能模型的性能，但Epic计划在未来将其扩展到更多的人工智能模型。

The Health AI Partnership (HAIP), a collaboration of organizations including Duke Health, Mayo Clinic and Kaiser Permanente, plans to use Epic's AI trust and assurance software suite to validate AI models locally for its health system members, Miller said.

米勒说，由杜克健康（Duke Health）、梅奥诊所（Mayo Clinic）和凯撒永久医疗（Kaiser Permanente）等组织合作的健康人工智能伙伴关系（HAIP）计划使用Epic的人工智能信托和保证软件套件，在当地为其卫生系统成员验证人工智能模型。

Epic also plans to join HAIP and the University of Wisconsin to study the use of a predictive model using the validation tool, he noted.

他指出，Epic还计划加入HAIP和威斯康星大学，研究使用验证工具的预测模型的使用。

'We will be able to learn quickly how the tool needs to evolve and develop,' he said. 'We are excited that HAIP decided to use the tool because that is exactly why we created it so that individual health systems and these third-party collaboratives could use the tool and adopt it to further the use of responsible AI to benefit the entire healthcare system.'.

他说，我们将能够快速了解该工具需要如何发展和发展我们很高兴HAIP决定使用该工具，因为这正是我们创建该工具的原因，以便个人卫生系统和这些第三方合作伙伴可以使用该工具并采用它来进一步使用负责任的人工智能，从而使整个医疗保健系统受益。”。

Healthcare organizations have been using predictive AI models and machine learning for almost a decade. But large language models (LLMs) and generative AI tools present a different challenge.

近十年来，医疗保健组织一直在使用预测性人工智能模型和机器学习。但大型语言模型（LLM）和生成性AI工具提出了不同的挑战。

Health systems are moving quickly to deploy LLMs and gen AI to tackle tasks like summarizing medical records and automating clinical notetaking. But these early adopters are still working through the best methods to validate AI models to feel confident about the technology's accuracy, performance and safety..

卫生系统正在迅速部署LLM和gen AI，以解决总结病历和自动化临床记录等任务。但这些早期采用者仍在通过最佳方法验证人工智能模型，以对该技术的准确性、性能和安全性充满信心。。

Hain said in April the AI software suite includes intuitive reporting dashboards that are updated automatically. Users get analysis broken down by age, sex, race/ethnicity and other demographics. The software features a common monitoring template and data schema to make it easier to extend the suite to new AI models in the future, he noted..

Hain在4月份表示，人工智能软件套件包括自动更新的直观报告仪表板。用户可以按年龄、性别、种族/民族和其他人口统计数据进行分析。他指出，该软件具有一个通用的监控模板和数据模式，以便将来更容易将套件扩展到新的AI模型。。

The open source tool enables healthcare organizations run AI validation in their EHR system on their own patient populations and workflows, Epic executives said.

Epic高管表示，开源工具使医疗机构能够在其EHR系统中对自己的患者群体和工作流程进行AI验证。

'It will pull in downstream outcomes and interventions. You can slice and dice the data across different patient cohorts. When you're thinking about AI equity, you could filter to certain protected classes, like age, sex, race or ethnicity, to ensure that your AI model is working appropriately across all the different patient cohorts.

“它将吸引下游成果和干预措施。您可以在不同的患者队列中对数据进行切片和切分。当你考虑人工智能公平性时，你可以过滤到某些受保护的类别，如年龄、性别、种族或种族，以确保你的人工智能模型在所有不同的患者群体中都能正常工作。

Looking at these downstream workflows is going to be critical to ensure fairness and equity,' Miller said. 'The tool is like a funnel of all this data and it's putting it into visualizations that are easy to understand. With single points and clicks, you can dig deeper into any certain piece of data.'.

米勒说，查看这些下游工作流程对于确保公平和公正至关重要这个工具就像一个收集所有这些数据的漏斗，它将这些数据转化为易于理解的可视化效果。通过单点和点击，您可以深入挖掘任何特定的数据。”。

He added, 'The tool is targeted primarily at data scientists and clinicians, but we hope that it will be easy enough to understand that a clinician without a data science background can also dig in and learn where the tools are fair and equitable, and where they might need to change the way they are delivering care to improve equity.'.

他补充道，“该工具主要针对数据科学家和临床医生，但我们希望很容易理解，没有数据科学背景的临床医生也可以深入了解这些工具在哪里是公平和公平的，以及他们可能需要改变他们提供护理的方式以提高公平性。”。

Some stakeholders have raised concerns that Epic, a health IT vendor with a massive influence in the market, developing tools to validate AI models represents a conflict of interest.

一些利益相关者担心，Epic是一家在市场上具有巨大影响力的健康IT供应商，开发用于验证AI模型的工具代表了利益冲突。

“This is completely agnostic to any of our code. It’s designed to work with predictive models that we created as well as predictive models we had no hand in creating,” Miller said. “At Epic, the motto we like to live by is ‘Do good, have fun, make money’ and we feel like this one really fits in that ‘Do good’ part.

米勒说：“这对我们的任何代码都是完全不可知的。它旨在与我们创建的预测模型以及我们没有参与创建的预测模型一起工作。”。“在Epic，我们喜欢遵循的座右铭是‘行善、娱乐、赚钱’，我们觉得这句话真的很适合‘行善’这一部分。

It feels like an opportunity to leverage our expertise in this space to further the global community's ability to evaluate AI and use it safely.”.

这感觉像是一个机会，可以利用我们在这一领域的专业知识，进一步提高全球社区评估人工智能和安全使用人工智能的能力。”。