急诊复诊质控筛查：临床决策逻辑与大语言模型知识图谱辅助

My Analysis of an Emergency Department Revisit Study

Okay, so here's what I've been thinking about as I reviewed this abstract. The usual quality assurance checks for Emergency Department (ED) revisits are often limited, for practical reasons, say to those within 48-72 hours. The goal is to catch issues that can be fixed while not drowning everyone in paperwork. This seemed like it might miss crucial opportunities.

So, I devised a plan. I conducted an exploratory, retrospective study looking at a multi-hospital system, specifically at ED visits that were followed by another ED visit to the same system within 1-14 days. I focused on the primary diagnosis for each visit, and I had raters – a few clinicians and the GPT-4 large language model – assess pairs of diagnoses. I was looking for pairs that "warranted further assessment."

We took a good look at how the raters responded. Then, I created an algorithm that used a knowledge graph populated by an LLM – think of it as "KGA" – to automatically screen for potentially concerning diagnosis pairs, which I then preliminarily assessed.

In the analysis, I included 99 diagnosis pairs. The results from GPT-4 and those of the clinicians didn't line up very well. GPT-4 wanted follow-up on nearly all the diagnosis pairs. But I realized the prompts I gave GPT-4 were pretty basic. From the clinician's point of view, when the revisit was more medically serious, that always predicted the need for further assessment. While differential diagnosis/complication combined was significant in an unadjusted analysis, it disappeared when adjusted. The KGA really shined, scoring high for accurately finding pairs that at least one clinician thought needed more investigation.

Looking ahead, these results provide insight for making the ED revisit screening work better with tools like ChatGPT. I need to validate this preliminary finding that the KGA, which has showed promise, can boost the yield and coverage of ED revisit screening without making life harder for reviewers.

背景：通常会对急诊科（ED）返诊情况进行质控审查，但此类审查往往受到限制（例如仅限于48至72小时内的返诊），以便在尽量减轻病历审查负担的同时，提高获得可操作性发现的产出率。这些限制可能会导致错失改进医疗质量的机会。

方法：我们对某多院区医疗系统开展了一项探索性、回顾性研究，随机抽取了在1至14天内于该系统内再次发生急诊返诊的病例。在仅获知每次就诊主要诊断的情况下，评估人员（2至3名临床医生及GPT-4大语言模型[LLM]）对诊断对的特征进行了评估，其中包括“目标”项：即该诊断对是否值得进一步评估。基于对评估人员评估结果的分析，我们构建了一种利用大语言模型填充的知识图谱算法（“KGA”），用于自动筛查潜在令人担忧的诊断对，并对其进行了初步评估。

结果：共纳入99个诊断对。GPT-4的评估结果与临床医生相关性较差，几乎将所有（94%）诊断对均判定为需要跟进（是临床医生的4.4至13.3倍）。然而，研究中仅进行了极少的提示词工程优化。在临床医生评估中，返诊病情的严重程度与目标项始终显著相关；而鉴别诊断/并发症复合指标在未调整分析中呈显著相关，但在调整后分析中则不显著（尽管其统计效能较低）。针对至少有一名临床医生判定基于诊断对需作进一步评估的情况，KGA实现了83%至100%的阳性预测值。

结论：这些结果可为利用ChatGPT等大语言模型改进筛查的后续工作提供参考。有必要开展进一步研究，以验证这项初步工作的发现，即KGA或能在不大幅增加审查人员工作负担的情况下，扩大筛查范围并提高筛查产出。