Joseph Breda, Student Researcher, and Jake Sunshine, Research Scientist, Google Research We present
a first-of-its-kind research of AI for differential diagnosis and symptom checking through a national-scale study. A large proportion of clinical diagnoses：https://www.bmj.com/content/bmj/2/5969/486.full.pdf? oken=QpPOmSNFUfkAAAAA:gnHAoVZt4KYlF9HFCqW77DGodmpifcB_n-Ea3AVn5QSXAQ5Ih8ejajCwyYMhHSNDyUvspHk7MNuZnw can be derived from language-based interviews alone. These diagnostic interviews are typically conducted by clinicians through doctor-patient interactions during in-person or remote visits. While these interactions are the gold standard for symptom assessment, they can often suffer from financial , geographic , and systemic barriers that limit their accessibility. Current language models (LMs) have demonstrated strong differential diagnosis assessment capabilities：https://www.natu
约瑟夫·布雷达（Joseph Breda），学生研究员，杰克·阳光（Jake Sunshine），研究科学家，谷歌研究院
我们首次提出了一项针对差异诊断和症状检查的人工智能研究，并通过全国范围的研究进行验证。
在评估SymptomAI差异诊断（DDx）准确性之后，我们进一步将SymptomAI的诊断结果与参与者在使用SymptomAI对话前的Fitbit可穿戴设备生物信号进行比较。我们发现，SymptomAI得出的感染性疾病诊断与参与者可能显示免疫反应的生理趋势相一致，这进一步表明了SymptomAI的性能。
在本研究中，我们使用了一系列对话式人工智能代理来进行端到端的患者访谈和差异诊断评估。参与者可以与代理进行症状对话，接收候选差异诊断列表，并输入随后的诊断结果。
我们招募了13,917名同意参与研究的受试者，每位受试者向五个随机分配的SymptomAI代理之一描述他们的症状，这些代理在进行症状访谈时灵活程度各不相同。在对话过程中，参与者描述他们的症状，SymptomAI提出后续问题，对话最终形成最终的差异诊断：https://my.clevelandclinic.org/health/diagnostics/22327-differential-diagnosis （DDx，可疑诊断列表）及下一步建议。参与者随后可以去看医疗提供者，并被要求在两周后通过问卷分享就诊结果。为了评估并建立SymptomAI评估的基线，我们进行了临床专家注释研究，一组由三名注册临床医生组成的小组审查了对话记录并提供他们的评估（即差异诊断）。然后，每位医生以盲法方式：https://en.wikipedia.org/wiki/Blinded_experiment，对SymptomAI提供的DDx及其他医生提供的DDx进行排名。
我们发现，在超过50%的病例中，临床医生更倾向于选择由SymptomAI生成的DDx，而不是其他临床医生提供的DDx。这表明，SymptomAI生成的DDx与我们临床医生的医疗评估一致的频率与其他临床医生的结果一样高甚至更高。
我们的临床评估人员更有可能将SymptomAI生成的DDx评为整体质量最佳的DDx。
同样，我们通过Top-5准确率（即参与者的个人医疗服务提供者提供的真实诊断是否出现在DDx的五个可能诊断之一）来比较SymptomAI生成的DDx与真实临床医生提供的DDx的准确性。我们让每位临床医生识别在每个DDx中是否包含提供的诊断，包括SymptomAI生成的DDx和临床医生提供的DDx。我们发现，临床医生认为SymptomAI生成的DDx比其他临床医生提供的DDx更常准确。
SymptomAI生成的DDx更可能包含由医疗提供者提供的自我报告诊断。
作为本研究的一部分，我们评估了进行病史采集访谈的不同方法。参与者被随机分配到五个研究组，每组采用不同的提示策略。其中两组（动态实时和动态最终）获得完全自主权，可以提出无限制的追问；另外两组（固定规范和灵活规范）则从医学院教授的一套标准病史采集问题中提出问题；最后一组为基础无提示语言模型，代表当前使用语言模型聊天机器人时的完全用户驱动体验。我们发现，所有由代理驱动的提示策略（即SymptomAI主动提出追问的情况）都显著优于基础条件，这表明从参与者获取信息对于提高鉴别诊断准确性具有重要价值。
按SymptomAI实验组划分的SymptomAI和临床医生的准确性。
按SymptomAI实验组划分的总用户字数。
我们发现，对于临床医生对自身DDx信心最低的病例，SymptomAI的表现相比临床基线提升最大。
临床医生根据SymptomAI和基线临床医生生成的鉴别诊断（DDx）的前五名准确率，并按基线临床医生对其自身DDx的信心进行分层。
鉴于SymptomAI相对于临床基线的准确率，我们还可以探索其在大规模应用中的潜力。目前，临床标签的成本阻碍了对人口规模数据集的实际分析。像SymptomAI这样准确的症状检查系统有潜力实现临床质量诊断的自动参考标签，这可以开启对生理数据的大规模分析——这是目前在大规模上不可能完成的任务。
其中一个例子是将可穿戴生物信号与不同类别的疾病相关联。可穿戴生物信号最显著的变化在急性呼吸道感染患者中观察到。为了在人口规模上研究这一点，我们收集了受试者在与SymptomAI交互前最多30天的每日生物测量数据。我们发现，在用户报告症状的前几天，生物信号出现了明显的变化，表明症状的出现。重要的是，通过对SymptomAI的第一候选诊断进行分类并将被归类为呼吸道感染的诊断分组，从而确定了队列的区分。这一队列排除了如过敏性鼻炎（https://my.clevelandclinic.org/health/diseases/8622-allergic-rhinitis-hay-fever）或慢性阻塞性肺疾病（https://my.clevelandclinic.org/health/diseases/8709-chronic-obstructive-pulmonary-disease-copd）等非感染性呼吸疾病。可穿戴生物信号变化与这些参与者报告症状日期的峰值相关性，提供了与其自报症状一致的观察性生理证据。
在与SymptomAI对话前的几天内，可穿戴生物信号相对于基线时期第-30天至第-15天的历史平均值变化情况，其中红色代表感染队列，灰色代表基线队列。感染队列包括SymptomAI诊断为呼吸道感染的参与者，而基线队列包括我们数据集中所有其他参与者。第0天表示与SymptomAI对话的日期。
基于人工智能的症状表现评估为新的研究打开了大门。通过使用 SymptomAI 分析大量症状报告，并将这些报告与实时 Fitbit 数据配对，我们可以探索各种疾病的数字生物信号表型。我们的分析显示，在用户与 SymptomAI 进行症状对话的前几天，生理指标——包括心血管功能、呼吸、皮肤温度和睡眠质量——出现了明显变化。这些客观变化与症状对话的时间紧密对应，提供了一种潜在方式来验证患者自报的症状，或提供被动数据以帮助在症状对话中辅助鉴别诊断。此外，这种实时可访问性突显了 AI 症状检查器的核心优势。与可能因排班延迟而受限的传统临床预约不同，参与者能够同时参与 SymptomAI 研究，而此时症状仍然新鲜。这有可能提高患者自报症状出现时间线的准确性——这是进行大规模健康分析时的关键细节。
SymptomAI 是一项探索性研究工作，可能代表 AI 基于症状评估的重大研究进展，并展示了其为一般公众提供症状理解潜力。虽然通过远程患者访谈进行的人群部署评估显示了症状评估的准确性，但在与临床医生评估进行比较时仍存在一些细微的局限性。
首先，鉴别诊断本身就是一项模糊的任务，即使是已报告的诊断也可能随时间变化和发展。症状评估是某一时间点的快照，捕捉的是症状在当时的表现。由于我们部署规模的原因，我们无法控制症状报告的频率和时间。因此，一些参与者可能在更具代表性的指标出现之前就报告了他们的症状，而另一些则可能在经过多年慢性病经验后从已有知识背景报告明显的指标。未来的工作可能会关注特定症状发展阶段的特定疾病，例如早发代谢综合征或呼吸道感染初期讨论的症状。研究中生成的所有诊断、标签和疾病关联均为人工智能生成，仅用于研究分析，不构成已确认的临床诊断或官方医学评估。
其次，在我们的评估中，临床医生仅审阅了静态聊天记录，没有权限提出自己的后续问题。如果临床医生自己主导症状访谈，他们可能会直觉上获取不同的信息。此外，尽管近期研究：https://www.nature.com/articles/s41586-025-08866-7 表明，对话式人工智能系统能够以临床医生水平的细致和准确获取临床数据，但此类系统可能会遗漏其他信号，如肢体语言、视觉评估、病历记录，或在初级护理环境中，与患者已有的信任关系。
总之，我们介绍了SymptomAI，一种用于进行真实世界患者访谈和症状评估的实验性对话式人工智能代理。我们通过对人群样本的鉴别诊断准确性展示了SymptomAI的端到端真实表现，并展示了SymptomAI诊断如何促进分析人口规模信号，例如可穿戴生物信号，以识别生理信号与报告疾病之间的关联。
这项工作是Joe Breda、Jake Sunshine 和 Daniel McDuff 平等贡献的结果。我们要感谢来自Google Research和Google DeepMind的合著者和合作伙伴对这项工作的贡献。
Joseph Breda, Student Researcher, and Jake Sunshine, Research Scientist, Google Research
We present a first-of-its-kind research of AI for differential diagnosis and symptom checking through a national-scale study.
After assessing the accuracy of SymptomAI’s differential diagnoses (DDx), we further compare SymptomAI’s diagnoses against biosignals from participants’ Fitbit wearable devices in the time leading up to their conversation with SymptomAI. We show that SymptomAI conversations that led to diagnosis with an infectious disease etiology coincide with physiological trends that may indicate an immune response, suggesting further evidence of SymptomAI’s performance.
For this research, we administered a set of conversational AI agents for end-to-end patient interviewing and differential diagnostic assessment. A participant could converse with an agent about their symptoms, receive a candidate differential list and enter a subsequent diagnosis.
We enrolled 13,917 consenting research study participants who each describe their symptoms to one of five randomized SymptomAI agents, each with varying degrees of flexibility in how they conducted the symptom interview. During these conversations, participants described their symptoms and SymptomAI asked follow-up questions, with conversations culminating in a final differential diagnosis：https://my.clevelandclinic.org/health/diagnostics/22327-differential-diagnosis (DDx, a list of plausible diagnoses) and recommendations for next steps. Participants could then go on to see a healthcare provider and were asked to share the outcome of that visit via a survey two-weeks later. To evaluate and baseline SymptomAI’s assessment, we conducted a clinical-expert annotation study in which a panel of three board-certified clinicians reviewed the conversation transcripts and provided their own assessment (i.e., differential diagnosis). Then each clinician, in a blinded fashion：https://en.wikipedia.org/wiki/Blinded_experiment, ranked the DDx provided by SymptomAI and those provided by the remaining clinicians.
We found that the clinicians preferred the DDx generated by SymptomAI over those provided by the other clinicians in over 50% of the cases. This indicates that SymptomAI DDx aligned with our clinicians’ medical assessments just as often or more often than that of other clinicians.
The SymptomAI generated DDx were more likely to be ranked as the best DDx in overall quality by our clinical raters.
Similarly, we compare the accuracy of the DDx generated by SymptomAI and provided by real clinicians via top-5 Accuracy (i.e., whether the true diagnosis provided by our participants' personal healthcare provider appears as one of the five possible diagnoses in the DDx). We had our clinicians each identify whether the provided diagnosis was in each DDx, including both the DDx generated by SymptomAI and those provided by clinicians. We found that the clinicians ranked the DDx generated by SymptomAI to be accurate more often than the DDx provided by other clinicians.
The SymptomAI generated DDx were more likely to contain the self-reported diagnosis provided by a healthcare provider.
As part of this research, we assessed different approaches for conducting history taking interviews. Participants were randomly assigned to five study arms, each employing different prompting strategies. Two ( Dynamic Live and Dynamic Final ) were given total agency to ask unrestricted follow up questions, two more ( Fixed Canonical and Flexible Canonical ) each asked questions from a set of standard history taking questions taught in medical school, and finally a Base unprompted LM, representing the fully user-driven experience that is the current status quo when querying LM chatbots. We found that all agent-driven prompting strategies (i.e., where SymptomAI actively asked follow up questions) significantly outperformed the Base condition, demonstrating the value of eliciting information from participants for improving differential diagnostic accuracy.
Accuracy of Symptom AI and clinicians by SymptomAI experiment arm.
Total user word count by SymptomAI experiment arm.
We found that SymptomAI’s performance above clinical baselines was greatest for cases where the clinician’s felt least confident in their own DDx.
The top-5 accuracy assigned by clinicians to SymptomAI and baseline clinician-generated DDx stratified by the baseline clinician’s confidence in their own DDx.
Given SymptomAI's accuracy against clinical baselines, we can also explore its potential at scale. Currently, the cost of clinical labels prohibits real-world analyses of population-scale datasets. Accurate symptom checking systems like SymptomAI have the potential to enable automated reference labeling of clinical quality diagnosis, which can open up large-scale analyses of physiological data — a task that is currently impossible at scale.
One such example is correlating wearable biosignals with different categories of illness. The most notable changes in wearable biosignals are observed for acute respiratory infections. To study this at population scale, we collected daily biometric data from our consenting participants for up to 30 days prior to their interaction with SymptomAI. We find clear biosignal shifts indicating symptom onset in the days approaching the user's symptom reporting. Importantly, the separation between cohorts was derived through categorizing SymptomAI's top-1 candidate diagnosis and grouping diagnoses that were classified as respiratory infections. This cohort excludes non-infectious respiratory illnesses like allergic rhinitis：https://my.clevelandclinic.org/health/diseases/8622-allergic-rhinitis-hay-fever or chronic obstructive pulmonary disease：https://my.clevelandclinic.org/health/diseases/8709-chronic-obstructive-pulmonary-disease-copd. The correlation of wearable biosignals shift peaks aligning with the date of symptom reporting for these participants serves as observational physiological evidence that align with their reported symptoms.
Wearable biosignals in the days leading up to a SymptomAI conversation relative to a historic average from a baseline period across day -30 to -15 for the infected ( red ) and baseline ( gray ) cohorts. The infected cohort includes participants which SymptomAI diagnosed with a respiratory infection while the baseline includes all other participants in our dataset. Day 0 denotes the date of the SymptomAI conversation.
AI-based assessment of symptom presentations opens the door to new research. By using SymptomAI to analyze a large volume of symptom reports and pairing those with real-time Fitbit data, we can explore digital biosignal phenotypes across a wide range of diseases. Our analysis revealed distinct shifts in physiological metrics — including cardiovascular function, respiration, skin temperature, and sleep quality — in the days leading up to a user's SymptomAI conversation. These objective changes align closely with the timing of the symptom conversation, offering a potential way to validate patient-reported symptoms or provide passive data to help inform a differential diagnosis alongside their symptom conversation. Additionally, this real-time accessibility highlights a core benefit of AI symptom checkers. Unlike traditional clinical appointments that can suffer from scheduling delays, participants could take part on the SymptomAI research study contemporaneously while symptoms are fresh. This potentially could improve the accuracy of patient-reported onset timelines — a crucial detail for population-scale health analysis.
SymptomAI is an exploratory research effort that could represent a significant research advancement in AI-based symptom assessment and demonstrates the potential it could provide for the general public seeking understanding of their symptoms. While a population deployment evaluation reveals the accuracy of symptom assessment through remote patient interviews, there are nuanced limitations when comparing against clinician’s assessments.
Firstly, differential diagnosis itself is an ambiguous task and even reported diagnoses may change and develop longitudinally. A symptom assessment is a snapshot in time and captures the symptoms as they present in that moment. Due to the scale of our deployment, we were unable to control for frequency and timing of symptom reporting. As a result, some participants may have reported their symptoms well before more representative indicators developed, while others may have reported obvious indicators from an informed context after years of experience with chronic illness. Future work may focus on specific illnesses at specific points during symptom development such as early-onset metabolic syndrome or symptoms discussed at the start of respiratory infections. All diagnoses, labels, and disease associations generated during the study are AI-derived for research analysis only and do not constitute confirmed clinical diagnoses or official medical assessments.
Secondly, in our evaluation the clinicians reviewed static chat transcripts and were not given agency to ask their own follow-up questions. Clinicians may have intuitively sourced different information had they directed the symptom interview. Moreover, while recent research：https://www.nature.com/articles/s41586-025-08866-7 has shown that conversational AI systems can source clinical data with a clinician-level of detail and accuracy, such systems may miss alternative signals like body language, visual assessment, medical records, or in the context of primary care, existing rapport with the patient.
In conclusion, we introduce SymptomAI, an investigational conversational AI agent for conducting real-world patient interviews and symptom assessments. We demonstrate SymptomAI’s end-to-end real-world performance through DDx accuracy on a population sample, and show how SymptomAI diagnoses can enable analysis of population-scale signals like wearable biosignals for identifying associations in physiological signals with reported illness.
This work is the result of equal contributions from Joe Breda, Jake Sunshine and Daniel McDuff. We would like to thank our co-authors and collaborators from Google Research and Google DeepMind for their contributions to this work.
扫码直达文章详情