Towards Conversational Diagnostic AI

1 Introduction

The dialogue between the physician and the patient is fundamental to effective and compassionate care. The medical interview has been termed “the most powerful, sensitive, and most versatile instrument available to the physician” [engel1973interviewing]. In some settings, it is believed that 60-80% of diagnoses are made through clinical history-taking alone [peterson1992contributions, hampton1975relative, kassirer1983teaching, roshan2000study, sandler1980importance]. The physician-patient dialogue extends beyond history-taking and diagnosis; it is a complex interaction which establishes rapport and trust, serves as a tool for addressing health needs and can empower patients to make informed decisions that account for their preferences, expectations, and concerns [silverman2016skills]. Clinicians wield considerable skills in clinical history-taking and the wider “diagnostic dialogue”, but access to this expertise remains episodic and globally scarce [rennie2014global].

Recent progress in general-purpose large language models (LLMs) [openai2023gpt4, google2023palm2, google2023gemini] has shown that artificial intelligence (AI) systems have capabilities to plan, reason, and incorporate relevant context to hold naturalistic conversations. This progress affords an opportunity to rethink the possibilities of AI in medicine towards the development of fully interactive conversational AI. Such medical AI systems would understand clinical language, intelligently acquire information under uncertainty, and engage in natural, diagnostically useful medical conversations with patients and those who care for them. The potential real-world utility of AI systems capable of clinical and diagnostic dialogue is broad, as the development of such capabilities might improve access to diagnostic and prognostic expertise, to improved quality, consistency, availability, and affordability of care, and to help realize better health outcomes (particularly for populations facing healthcare disparities).

Refer to caption — Figure 1: Overview of contributions. AMIE is a conversational medical AI optimised for diagnostic dialogue. AMIE is instruction fine-tuned with a combination of real-world and simulated medical dialogues, alongside a diverse set of medical reasoning, question answering, and summarization datasets. Notably, we designed a self-play based simulated dialogue environment with automated feedback mechanisms to scale AMIE’s capabilities across various medical contexts and specialities. Specifically, this iterative self-improvement process consisted of two self-play loops: (1) An “inner” self-play loop, where AMIE leveraged in-context critic feedback to refine its behavior on simulated conversations with an AI patient agent; (2) An “outer” self-play loop where the set of refined simulated dialogues were incorporated into subsequent fine-tuning iterations. During online inference, AMIE used a chain-of-reasoning strategy to progressively refine its response conditioned on the current conversation to arrive at an accurate and grounded reply to the patient in each dialogue turn. We designed and conducted a blinded remote Objective Structured Clinical Examination (OSCE) with validated simulated patient actors interacting with AMIE or Primary Care Physicians (PCPs) via a text interface. Across multiple axes corresponding to both specialist physician (28 out of 32) and patient actor (24 out of 26) perspective, AMIE was rated as superior to PCPs while being non-inferior on the rest.

However, while LLMs have been shown to encode clinical knowledge and proven capable of highly accurate single-turn medical question-answering [singhal2022large, singhal2023towards, nori2023can], their conversational capabilities have been tailored to domains outside clinical medicine [thoppilan2022lamda, openai2022chatgpt]. Prior work in LLMs for health [singhal2022large, singhal2023towards, nori2023can, toma2023clinical, chen2023meditron] has not yet rigorously examined the clinical history-taking and diagnostic dialogue capabilities of AI systems or contextualized this by comparison to the extensive capabilities of expert clinicians.

Clinical history-taking and diagnostic dialogue through which clinicians derive diagnosis and management plans represent a complex skill [levine2017history] whose optimal conduct is highly dependent on context. Thus, multiple evaluation axes are needed to assess the quality of a diagnostic dialogue, including the structure and completeness of the elicited history, diagnostic accuracy, the appropriateness of management plans and their rationale, and patient-centred considerations such as relationship-building, respect for the individual and communication efficacy [king2013best]. If the conversational potential of LLMs is to be realized in medicine, there is a significant unmet need to better optimize development and evaluation of medical AI systems for characteristics such as these, which are unique to history-taking and diagnostic dialogue between clinicians and patients.

In this work, we detail our progress towards a conversational medical AI system for clinical history-taking and diagnostic reasoning.

Our key contributions are summarized as:

•

We introduced AMIE (Articulate Medical Intelligence Explorer), an LLM based AI system optimized for clinical history-taking and diagnostic dialogue.
•

To scale AMIE across a multitude of specialties and scenarios, we developed a novel self-play based simulated diagnostic dialogue environment with automated feedback mechanisms to enrich and accelerate its learning process. We also introduced an inference time chain-of-reasoning strategy to improve AMIE’s diagnostic accuracy and conversation quality.
•

We developed a pilot evaluation rubric to assess the history-taking, diagnostic reasoning, communication skills and empathy of diagnostic conversational medical AI, encompassing both clinician-centred and patient-centred metrics.
•

We designed and conducted a blinded remote OSCE study with 149 case scenarios from clinical providers in Canada, the UK, and India, enabling randomized and counterbalanced comparison of AMIE to PCPs when performing consultations with validated patient actors. AMIE exhibited superior diagnostic accuracy compared to PCPs as assessed by various measures (e.g., top-1 and top-3 accuracy of the differential diagnosis list). Across 28 out of 32 evaluation axes from the specialist physician perspective and 24 out of 26 evaluation axes from the patient actor perspective, AMIE was rated superior to PCPs while being non-inferior on the rest.
•

We performed a range of ablations to further understand and characterize the capabilities of AMIE, highlighted important limitations, and proposed key next steps for real-world clinical translation of AMIE.

Our research has important limitations, most notably that we utilized a text-chat interface, which although enabling potentially large-scale interaction between patients and LLMs specialized for diagnostic dialogue, was unfamiliar to PCPs for remote consultation. Thus our study should not be regarded as representative of usual practice in (tele)medicine.

2 AMIE: An LLM based AI System for Diagnostic Dialogue

In the following sections, we describe the real-world datasets, simulated self-play environment, fine-tuning process, and inference time chain-of-reasoning that we designed to optimize AMIE for diagnostic conversation capabilities and clinical communication skills.

2.1 Real-world Datasets for AMIE

AMIE was developed using a diverse suite of real-world datasets including multiple-choice medical question-answering, expert-curated long-form medical reasoning, electronic health record (EHR) note summaries, and large-scale transcribed medical conversation interactions. As described in detail below, in addition to dialogue generation tasks, the training task mixture for AMIE consisted of medical question-answering, reasoning, and summarization tasks.

Medical Reasoning.

We used the MedQA (multiple-choice) dataset consisting of US Medical Licensing Examination (USMLE) multiple-choice style open domain questions with four or five possible answers [jin2021disease]. The training set consisted of 11,450 questions and the test set had 1,273 questions. We also curated 191 MedQA questions from the training set where clinical experts crafted step-by-step reasoning leading to the correct answer [singhal2023towards].

Long-form Medical Question Answering.

The dataset used here consisted of expert-crafted long-form responses to 64 questions from HealthSearchQA, LiveQA, and Medication QA in MultiMedBench [singhal2022large].

Medical Summarization.

A dataset consisting of 65 clinician-written summaries of medical notes from MIMIC-III, a large, publicly available database containing medical records of intensive care unit patients [johnson2016mimic], was used as additional training data for AMIE. MIMIC-III contains approximately 2 million notes spanning 13 types including cardiology, respiratory, radiology, physician, general, discharge, case management, consult, nursing, pharmacy, nutrition, rehabilitation and social work. 5 notes from each category were selected, with a minimum total length of 400 tokens and at least one nursing note per patient. Clinicians were instructed to write abstractive summaries of individual medical notes, capturing key information while also permitting the inclusion of new informative and clarifying phrases and sentences not present in the original note.

Real-world Dialogue.

Here, we used a de-identified dataset licensed from a dialogue research organisation comprising 98,919 audio transcripts of medical conversations during in-person clinical visits from over 1,000 clinicians over a 10-year period in the United States [chiu2017speech]. It covered 51 medical specialties (primary care, rheumatology, hematology, oncology, internal medicine and psychiatry among others) and 168 medical conditions and visit reasons (type II diabetes, rheumatoid arthritis, asthma, depression among the common conditions). Audio transcripts contained utterances from different speaker roles such as doctors, patients, and nurses. On average a conversation had 149.8 turns ( $P_{0.25}=75.0$ , $P_{0.75}=196.0$ ). For each conversation, the metadata contained information about patient demographics, reason for the visit (follow-up for pre-existing condition, acute needs, annual exam and more), and diagnosis type (new, existing or other unrelated). We refer to (chiu2017speech) for more details.

For this study, we selected dialogues involving only doctors and patients, but not other roles such as nurses. During preprocessing, we removed paraverbal annotations such as “[LAUGHING]” and “[INAUDIBLE]” from the transcripts. We then divided the dataset into training (90%) and validation (10%) sets using stratified sampling based on condition categories and reasons for visits, resulting in 89,027 conversations for training and 9,892 for validation.

2.2 Simulated Dialogue Learning Environment and Self-play for AMIE

While passively collecting and transcribing real-world dialogues from in-person clinical visits is feasible, two substantial challenges limit its effectiveness in training LLMs for medical conversations: (1) existing real-world data often fails to capture the vast range of medical conditions and scenarios, hindering its scalability and comprehensiveness; (2) the data derived from real-world dialogue transcripts tends to be noisy, containing ambiguous language (including slang, jargon, and sarcasm), interruptions, ungrammatical utterances, and implicit references. This in turn, may limit AMIE’s knowledge, capabilities, and applicability.

To address these limitations, we designed a self-play based simulated learning environment for diagnostic medical dialogues in a virtual care setting, enabling us to scale AMIE’s knowledge and capabilities across a multitude of medical conditions and contexts. We used this environment to iteratively fine-tune AMIE with an evolving set of simulated dialogues in addition to the static corpus of medical QA, reasoning, summarization, and real-world dialogue data described above (see Figure 1).

This process consisted of two self-play loops:

•

An “inner” self-play loop where AMIE leveraged in-context critic feedback to refine its behavior on simulated conversations with an AI patient agent.
•

An “outer” self-play loop where the set of refined simulated dialogues were incorporated into subsequent fine-tuning iterations. The resulting new version of AMIE could then participate in the inner loop again, creating a continuous learning cycle.

Simulated Dialogues.

At each iteration of fine-tuning, we produced 11,686 dialogues, stemming from 5,230 different medical conditions. Conditions were selected from three datasets:

•

Health QA dataset singhal2022large which contained 613 common medical conditions.
•

MalaCards Human Disease Database¹¹1https://github.com/Shivanshu-Gupta/web-scrapers/blob/master/medical_ner/malacards-diseases.json which contained 18,455 less common disease conditions.
•

MedicineNet Diseases & Conditions Index²²2https://github.com/Shivanshu-Gupta/web-scrapers/blob/master/medical_ner/medicinenet-diseases.json which contained 4,617 less common conditions.

At each self-play iteration, four conversations were generated from each of the 613 common conditions, while two conversations were generated from each of the 4,617 less common conditions randomly chosen from MedicineNet and MalaCards. The average simulated dialogue conversation length was 21.28 turns ( $P_{0.25}=19.0$ , $P_{0.75}=25.0$ ).

Using simulated dialogues allowed us to address the limited availability of high-quality, labelled real-world conversation data and improved the model’s generalization and adaptability to diverse medical contexts. By leveraging this self-play paradigm, AMIE could continuously learn and refine its conversational and diagnostic capabilities during patient interactions.

2.2.1 Simulated Dialogue Data Curation

In order to produce high-quality simulated dialogues at scale, we developed a novel multi-agent framework which comprised three key components:

•

Vignette Generator: AMIE leverages web searches to craft unique patient vignettes given a specific medical condition.
•

Simulated Dialogue Generator: Three LLM agents play the roles of patient agent, doctor agent, and moderator, engaging in a turn-by-turn dialogue simulating realistic diagnostic interactions.
•

Self-play Critic: A fourth LLM agent acts as a critic to give feedback to the doctor agent for self-improvement. Notably, AMIE acted as all agents in this framework. We describe each component in detail below.

Vignette Generator.

The vignette generator aimed to create varied and realistic patient scenarios at scale, which could be subsequently used as context for generating simulated doctor-patient dialogues thereby allowing AMIE to undergo a training process emulating exposure to a greater number of conditions and patient backgrounds. The patient vignette (scenario) included essential background information such as patient demographics, symptoms, past medical history, past surgical history, past social history, and patient questions, as well as an associated diagnosis and management plan.

For a given condition, patient vignettes were constructed using the following process. First, we retrieved 60 passages (20 each) on the range of demographics, symptoms, and management plans associated with the condition from using an internet search engine. To ensure these passages were relevant to the given condition, we used the general-purpose LLM, PaLM-2 google2023palm2, to filter these retrieved passages, removing any passages deemed unrelated to the given condition. We then prompted AMIE to generate plausible patient vignettes aligned with the demographics, symptoms, and management plans retrieved from the filtered passages, by providing a one-shot exemplar to enforce a particular vignette format. The prompts for each of these steps are as follows:

Simulated Dialogue Generator.

Given a patient vignette detailing a specific medical condition, the simulated dialogue generator was designed to simulate a realistic dialogue between a patient and a doctor in an online chat setting where in-person physical examination may not be feasible.

Three specific LLM agents (patient agent, doctor agent, and moderator), each played by AMIE, were tasked with communicating amongst each other to generate the simulated dialogues. Each agent had distinct instructions. The patient agent embodied the individual experiencing the medical condition outlined in the vignette. Their role involved truthfully responding to the doctor agent’s inquiries as well as raising any additional questions or concerns they may have had. The doctor agent played the role of an empathetic clinician seeking to comprehend the patient’s medical history within the online chat environment sharma2020computational. Their objective was to formulate questions that could effectively reveal the patient’s symptoms and background, leading to an accurate diagnosis and an effective treatment plan. The moderator continually assessed the ongoing dialogue between the patient agent and doctor agent, determining when the conversation had reached a natural conclusion.

The turn-by-turn dialogue simulation started with the doctor agent initiating the conversation: “Doctor: So, how can I help you today?”. Following this, the patient agent responded, and their answer was incorporated into the ongoing dialogue history. Subsequently, the doctor agent formulated a response based on the updated dialogue history. This response was then appended to the conversation history. The conversation progressed until the moderator detected the dialogue had reached a natural conclusion, when the doctor agent had provided a differential diagnosis, treatment plan, and adequately addressed any remaining patient agent questions, or if either agent initiated a farewell.

Self-play Critic.

To ensure high-quality dialogues, we implemented a tailored self-play fu2023improving framework specifically for self-improvement of diagnostic conversations. This framework introduced a fourth LLM agent, acting as a “critic” which was also played by AMIE and aware of the ground truth diagnosis, to provide in-context feedback to the doctor agent and enhance its performance in subsequent conversations. The critic agent evaluated the doctor agent’s responses based on the following criteria:

•

The doctor agent exhibits empathy and professionalism while addressing the patient agent’s latest questions or comments in a concise manner.
•

The doctor agent avoids asking too many or repetitive questions (about information already acquired), focusing on a maximum of one or two per response.
•

The responses should not reveal that the doctor agent is an AI chatbot. They should flow naturally, maintain factual accuracy, and facilitate further engagement from the patient.
•

The doctor agent asks sufficient questions to identify at least two of the most likely differential diagnoses. They further refine their understanding through targeted questions towards the ground truth diagnosis and offer the corresponding treatment.

Following the critic’s feedback, the doctor agent incorporated the suggestions to improve its responses in subsequent rounds of dialogue with the same patient agent from scratch. Notably, the doctor agent retained access to its previous dialogue history at each new round. This self-improvement process was repeated twice to generate the dialogues used for each iteration of fine-tuning.

2.3 Instruction Fine-tuning

AMIE, built upon the base LLM PaLM 2 google2023palm2, was instruction fine-tuned to enhance its capabilities for medical dialogue and reasoning. We refer to the PaLM-2 technical report for more details on the base LLM architecture.

We employed task-specific instructions to fine-tune AMIE in playing either the patient or doctor role within medical dialogues, performing medical question answering and reasoning, and summarizing EHR notes. While the first round of fine-tuning from the base LLM only used the static datasets, subsequent rounds of fine-tuning leveraged the simulated dialogues generated through the self-play inner loop as described in Section 2.2.1.

For dialogue generation tasks, AMIE was trained to predict the next conversational turn based on all previous interactions, assuming either the doctor or patient role. When playing the patient agent, AMIE was prompted to reply to the doctor agent’s questions about their symptoms, drawing upon information provided in patient scenarios. These scenarios included patient vignettes (see Section 2.2.1) for simulated dialogues or metadata such as demographics, visit reason, and diagnosis type for the real-world dialogue dataset. In the doctor agent role, AMIE was prompted to act as an empathetic clinician, interviewing patients about their medical history and symptoms to ultimately arrive at an accurate diagnosis. From each dialogue, we sampled on average 3 turns for each the doctor and patient roles as the target turns to predict based on the conversation leading up to that target turn. Target turns were randomly sampled from all turns in the dialogue that had a minimum length of 30 characters.

Similarly, for the EHR note summarization task, AMIE was provided with a clinical note and prompted to generate a summary of the note. Medical reasoning/QA and long-form response generation tasks followed the same setup as in singhal2023towards. Notably, all tasks except dialogue generation and long-form response generation incorporated few-shot (1-5) exemplars in addition to task-specific instructions for additional context.

2.4 Chain-of-reasoning for Online Inference

To address the core challenge in diagnostic dialogue - effectively acquiring information under uncertainty to enhance diagnostic accuracy and confidence while maintaining positive rapport with the patient - AMIE employed a chain-of-reasoning strategy before generating a response in each dialogue turn. Here, “chain-of-reasoning” refers to a series of sequential model calls, each dependent on the outputs of prior steps. Specifically, we used a three-step reasoning process, described as follows:

1.

Analyzing patient information: Given the current conversation history, AMIE was instructed to 1) summarize the positive and negative symptoms of the patient as well as any relevant medical/family/social history and demographic information, 2) produce a current differential diagnosis, 3) note missing information needed for a more accurate diagnosis and 4) assess confidence in the current differential and highlight its urgency.
2.

Formulating response and action: Building upon the conversation history and the output of step 1, AMIE performed the following: 1) Generate a response to the patient’s last message and formulate further questions to acquire missing information and refine the differential diagnosis. 2) If necessary, recommend immediate action, such as an emergency room visit. If confident in the diagnosis based on available information, present the differential.
3.

Refining the response: AMIE revises its previous output to meet specific criteria based on the conversation history and outputs from earlier steps. The criteria are primarily related to factuality and formatting of the response (e.g., avoid factual inaccuracies on patient facts and unnecessary repetition, show empathy, and display in a clear format).

This chain-of-reasoning strategy enabled AMIE to progressively refine its response conditioned on the current conversation to arrive at an informed and grounded reply.

3 Evaluation

Prior works developing models for clinical dialogue have focused on metrics such as the accuracy of note-to-dialogue or dialogue-to-note generations abacha2023overview; ionescu2023overview, or natural language generation metrics such as BLEU or ROUGE scores that fail to capture the clinical quality of a consultation he2022dialmed; naseem2022incorporating.

In contrast to these prior works we sought to anchor our human evaluation in criteria more commonly used for evaluating the quality of physicians’ expertise in history-taking, including their communication skills in consultation. We derived a framework from principles published in reviews of the consensus for best practices for patient-centered communication (PCCBP) in medical interviews king2013best, criteria examined for history-taking skills by the Royal College of Physicians in the UK as part of their Practical Assessment of Clinical Examination Skills (PACES)³³3https://www.mrcpuk.org/mrcpuk-examinations/paces/marksheets dacre2003mrcp, and criteria proposed by the UK General Medical Council Patient Questionnaire (GMCPQ)⁴⁴4https://www.ed.ac.uk/sites/default/files/imports/fileManager/patient_questionnaire%20pdf_48210488.pdf for doctors seeking patient feedback as part of professional re-validation⁵⁵5https://www.gmc-uk.org/registration-and-licensing/managing-your-registration/revalidation/revalidation-resources/collecting-colleague-and-patient-feedback-for-revalidation. We iterated upon these criteria to refine items for inclusion and derived pilot scales and instructions for assessment by using focus groups and interviews with clinicians and OSCE examiners based in the UK, Canada, US, and India. Our resulting pilot framework enabled assessment from two perspectives: clinician (board-certified physicians) and lay raters (patient actors). The framework included consideration of consultation quality, structure and completeness, the roles, responsibilities, and skills of the interviewer (Tables A.1, A.2, A.3, and A.4).

3.1 Objective Structured Clinical Examination

Objective Structured Clinical Examination (OSCE) is a practical assessment format used in healthcare to assess clinical skills and competencies in a standardized and objective fashion sloan1995objective; carraccio2000objective; epstein2002defining. It differs from traditional written or oral exams that focus primarily on theoretical knowledge and instead aims to provide an environment in which the skills of real-world clinical practice might be assessed.

The OSCE is typically divided into multiple stations (often 8-12), each simulating a real-life clinical scenario enacted by standardized patient actors trained to portray specific symptoms or conditions based on pre-defined scenario descriptions. At each station, students are given specific tasks to perform, such as taking a clinical history, or making a diagnosis. Each station has a set time limit, ensuring fairness and efficient assessment. Trained examiners observe students’ performance at each station using a pre-defined checklist or marking scheme. They assess clinical skills like communication, history-taking, physical examination techniques, clinical reasoning, and decision-making.

3.2 Remote OSCE Study Design

To compare AMIE’s performance to that of real clinicians, we conducted a randomized crossover study of blinded consultations in the style of a remote OSCE. Our OSCE study involved 20 board-certified primary care physicians (PCPs) and 20 validated patient actors, 10 each from India and Canada, respectively, to partake in online text-based consultations. PCPs had between 3 and 25 years of post-residency experience (median 7 years). Patient actors comprised of a mix of medical students, residents, and nurse practitioners with experience in OSCE participation. We sourced 149 scenario packs from India (75), Canada (60), and the UK (14).

The scenario packs and simulated patients in our study were prepared by two OSCE laboratories (one each in Canada and India), each affiliated to a medical school and with extensive experience in preparing scenario packs and simulated patients for OSCE examinations. UK scenario packs were sourced from the samples provided on the MRCPUK website. Each scenario pack was associated with a ground truth diagnosis and a set of acceptable diagnoses. The scenario packs covered conditions from cardiovascular (29), respiratory (30), gastroenterology (31), neurology (30), urology, obstetric, and gynecology domains (15), and internal medicine (14). Pediatric or psychiatry domains were excluded from this study, as were intensive care or inpatient case management scenarios.

Indian patient actors played the roles in all India scenario packs and 7 of the 14 UK scenario packs. Canadian patient actors participated in scenario packs for both Canada and the other half of UK-based scenario packs. This assignment process resulted in 149 distinct simulated patients (“scenarios”). Below, we use the term “OSCE agent” to refer to the conversational counterpart interviewing the patient actor, i.e., either PCP or AMIE. Table 1 summarizes the OSCE assignment information across three geographical locations. Each of the 149 simulated patients completed the three-step study flow depicted in Figure 2.

Table 1: OSCE study summary. Number of scenario packs, patient actors, simulated patients, and primary care physicians (PCPs) in each of the three locations (Canada, India, and the UK) in the remote OSCE study. 20 board-certified PCPs participated in the study as OSCE agents in comparison with AMIE, 10 each from India and Canada. 20 trained patient actors were involved, with 10 each from India and Canada. Indian patient actors played the roles in both India and UK scenario packs. Canadian patient actors participated in scenario packs for both Canada and the UK. This process resulted in 149 distinct simulated patients.

Location	# of Scenario Packs	# of Simulated Patients	# of Patient Actors	# of PCPs
Canada	60	67	10	10
India	75	82	10	10
UK	14	0	0	0
Total	149	149	20	20

3.2.1 Online Text-based Consultation

PCPs and patient actors were primed with sample scenarios and instructions, and participated in pilot consultations prior to the study commencing in order to familiarize themselves with the interface and experiment requirements.

For the experiment, each simulated patient completed two online text-based consultations via a synchronous text chat interface (Figure A.2), one with a PCP (control) and one with AMIE (intervention). The ordering of PCP and AMIE was randomized and patient actors were not informed as to which they were talking to in each consultation. PCPs were located in the same country as patient actors, and were randomly drawn based on availability at the specified time slot for the consultation. Patient actors role-played the scenario and were instructed to conclude the conversation after no more than 20 minutes. Both OSCE agents were asked (PCPs via study-specific instructions, and AMIE as part of the prompt template) to not reveal their identity, or whether they were human, under any circumstances.

3.2.2 Post-questionnaires

Upon conclusion of the consultation, the patient actor and OSCE agent each filled in a post-questionnaire in light of the resulting consultation transcript (Figure A.3). The post-questionnaire for patient actors consisted of the complete GMCPQ (Table A.1), the PACES components for “Managing Patient Concerns” and “Maintaining Patient Welfare” (Table A.2), and a checklist representation of the PCCBP category for “Fostering the Relationship” (Table A.3). Responses patient actors provided to the post-questionnaire are referred to as “patient actor ratings” below. The post-questionnaire for the OSCE agent asked for a ranked differential diagnosis (DDx) list with a minimum of 3 and no more than 10 conditions, as well as recommendations for escalation to in-person or video-based consultation, investigations, treatments, management plan, and the need for a follow-up.

3.2.3 Specialist Physician Evaluation

Finally, a pool of 23 specialist physicians from India (14), North America (6), and the UK (3) evaluated PCPs and AMIE with respect to the quality of their consultation, and their responses to the post-questionnaire. During evaluation, specialist physicians also had access to the full scenario pack along with its associated ground truth differential and additional accepted differentials. All of the data the specialist physicians had access to during evaluation are collectively referred to as “OSCE data” below. Specialist physicians were sourced to match the specialties and geographic regions corresponding to the scenario packs included in our study, and had between 1 and 36 years of post-residency experience (median 5 years). Each set of OSCE data was evaluated by one specialist physician randomly assigned to match the specialty and geographic region of the underlying scenario (e.g., Canadian pulmonologist evaluated OSCE data from Canada-sourced respiratory medicine scenario). Each specialist evaluated OSCE data from both PCP and AMIE for a given scenario. Evaluations for PCP and AMIE were conducted by the same specialist in a randomized and blinded sequence.

Evaluation criteria included the accuracy, appropriateness and comprehensiveness of the provided DDx list, appropriateness of recommendations regarding escalation, investigation, treatment, management plan and follow-up (Table A.4), and all PACES (Table A.2) and PCCBP (Table A.3) rating items. We also asked specialist physicians to highlight confabulations in the consultations and questionnaire responses, i.e., text passages that were non-factual or referred to information not provided in the conversation. Each OSCE scenario pack additionally supplied specialists with scenario-specific clinical information to assist with rating the clinical quality of the consultation, such as the ideal investigation or management plans; or important aspects of the clinical history that would ideally have been elucidated for the highest quality of consultation possible.

3.3 Auto-evaluation

In addition to human evaluations, we implemented model-based auto-evaluation methods as economical consistent alternatives to specialist assessments. These techniques were employed to evaluate both dialogue quality and diagnostic accuracy of the OSCE agent. To establish the validity of our auto-evaluation methods for assessing dialogue quality, we initially focused on a subset of four evaluation axes from the PACES rubric (Table A.2) that were assessed by both the patient actors and the specialist physicians. The auto-evaluation, which uses a self-CoT strategy (details described in Section A.9) with AMIE to rate dialogues, was in good alignment with human raters and comparable to the inter-specialist agreement on these criteria. For the auto-evaluation of differential diagnoses, we leveraged another LLM, Med-PaLM 2 singhal2023towards as a surrogate for a specialist rater to grade the predicted diagnoses against the ground truth diagnoses (more details in Section A.7). Our auto-evaluation on DDx accuracy showed a similar trend for AMIE and OSCE agents compared to the specialist ratings. Overall, auto-evaluation trends aligned with human ratings for both dialogue quality and diagnostic accuracy.

We also conducted additional auto-evaluation analyses for the following purposes:

•

To compare the performance of the DDx accuracy derived from AMIE or PCP consultations;
•

To compare the DDx accuracy between simulated patients performed in Canada and India and determine if there is systematic differences between the two locations;
•

To isolate the effects of information acquisition and information interpretation by analyzing the DDx accuracy of AMIE when provided the PCP consultation instead of its own;
•

To evaluate the efficiency of information acquisition between AMIE and PCPs by analyzing the DDx accuracy as the number of conversation turns increases;
•

To evaluate the benefit of inner-loop self-play on dialogue quality before and after critic feedback.

3.4 Statistical Analysis

We evaluated the top-k accuracy of the DDx lists generated by AMIE and PCPs across all 149 simulated patients. Top-k accuracy was defined as the percentage of cases where the correct diagnosis appeared within the top-k positions of the DDx list. Specifically, a candidate diagnosis was considered a match if the specialist rater marked it as either an exact match with, very close to or closely related to the ground truth diagnosis (or accepted differential). Statistical significance for DDx accuracy was determined using bootstrap tests horowitz2001bootstrap with 10,000 samples and false discovery rate (FDR) correction benjamini1995controlling across all k. Statistical significance for patient actor and specialist ratings was determined using Wilcoxon signed-rank tests woolson2007wilcoxon FDR correction. Cases where either agent received “Cannot rate / Does not apply” were excluded from the test. Results below refer to $p$ -values after FDR correction.

4 Results

4.1 Diagnostic Accuracy

4.1.1 AMIE showed higher DDx accuracy than PCPs under specialist physician evaluation.

AMIE’s diagnostic accuracy was assessed as higher than that of PCPs. Figure 3 shows the top-k accuracy for AMIE and PCPs, considering matches with the ground truth diagnosis (a) and matches with any item on the accepted differential (b). AMIE showed significantly higher top-k accuracy than that of PCPs across all values of k ( $p<0.05$ ). Note that unlike AMIE, PCPs did not always provide 10 diagnoses in their differential diagnoses (min: 3, mean: 5.39). Additionally, we performed a comparison of DDx accuracy between AMIE and PCP by varying the matching criteria for determining a match. Results depicted in Figure A.7 further substantiate AMIE’s superior DDx performance across various matching criteria.

Accuracy by Specialty.

Figure A.8 illustrates the DDx accuracy achieved by AMIE and PCPs across the six medical specialties covered by scenarios in our study. We observed that AMIE’s performance matched or surpassed PCP performance for all specialties with the most pronounced improvements in the respiratory and cardiovascular specialities.

4.1.2 Auto-evaluation suggested AMIE matched PCPs’ efficiency in acquiring information.

Auto-evaluation Accuracy.

We reproduced the DDx accuracy analysis with our model-based auto-evaluator instead of the specialist raters using the same procedure as in Figure 3. The overall performance trends obtained through the auto-evaluator align well with specialist assessments despite marginal differences in the computed accuracy values, as shown in Figure A.9.

Isolating the Source of Performance Gains.

To investigate whether AMIE’s superior DDx performance observed in Figure 3 stemmed from improved information acquisition or from better diagnostic reasoning capability, we compared AMIE’s diagnoses based on its own consultations with AMIE’s diagnoses generated from the corresponding PCP consultations, using the DDx auto-evaluator. Results depicted in Figure A.10 revealed markedly similar DDx performance, indicating that the diagnostic performance remained consistent regardless of whether AMIE processed information from its own dialogue or from the PCP’s conversation. Both methods significantly outperformed the differential diagnoses produced by PCPs. These results suggest that AMIE was approximately equivalent to PCPs at information acquisition but better than PCPs at interpreting that information to produce an accurate/complete differential diagnosis.

Efficiency of Information Acquisition.

Although AMIE displayed greater verbosity compared to PCPs in terms of total number of words generated in their responses during the consultation, the number of conversational turns and the number of words elicited from the patient actors were similar across both OSCE agents, as illustrated in Figure A.11. This suggests that both AMIE and PCPs acquired a similar amount of information from the patients during the encounter. To investigate how efficient AMIE or PCPs were at gathering sufficient information to formulate a correct diagnosis, we truncated the conversations at various turn counts and used AMIE to generate differential diagnoses based on these partial conversations. Figure A.12 depicts the top-3 DDx accuracy as a function of the number of turns provided to the model. The observed accuracies plateaued within the initial 10 conversational turns for both AMIE and PCPs. This suggests that both AMIE and PCPs were able to acquire the information necessary for formulating a diagnosis within the early stages of the conversation. Additionally, the comparable performance at every turn indicates that neither AMIE nor PCPs had a significant advantage in the efficiency or quality of information acquisition.

4.2 Conversation Quality

4.2.1 AMIE surpassed PCPs in conversation quality, per specialists and patient actors.

Conversation quality was assessed using patient actor ratings, specialist ratings, and outputs from auto-evaluation. Figure A.5 and A.6 show two example consultations for the same simulated patient from AMIE and PCP, respectively.

Patient Actor Ratings.

Figure 4 presents the various conversation qualities patient actors assessed following their consultations with the OSCE agents. Overall, AMIE’s consultations were rated significantly better ( $p<0.05$ ) by patient actors than those from PCPs across 24 of 26 axes. No significant differences in ratings were detected for the two PCCBP axes “Respecting Patient’s Privacy” (N=108) and “Acknowledging Mistakes” (N=41). For the latter criterion, the number of exclusions was substantially higher since the question applied only when mistakes were made by the OSCE agent and pointed out in the conversation.

Specialist Physician Ratings.

Specialist physicians evaluated both the conversational quality as well as the responses to the post-questionnaire for scenarios within their domain expertise (see Figure 5). Again, AMIE’s responses were rated significantly better by specialists than those from PCPs on 28 of 32 evaluation axes; Specialists preferred AMIE’s consultation, diagnoses, and management plan over those from PCPs. For this set of evaluations, differences in specialist ratings between AMIE and PCPs were statistically significant ( $p<0.05$ ). No significant differences in ratings were detected for four of the axes in the Diagnosis & Management rubric, namely, “Escalation Recommendation Appropriate”, “Treatment Inappropriate Avoided”, “Followup Recommendation Appropriate” and “Confabulation Absent”, despite no exclusions (N=149).

4.2.2 Auto-evaluations demonstrated the effectiveness of inner self-play for AMIE.

Auto-evaluation of Conversation Ratings.

We leveraged the model-based self-CoT auto-evaluation strategy to rate conversations on four evaluation axes from the PACES rubric, and validated that these auto-evaluation ratings were accurate and well aligned with the specialist ratings (Figures A.17 and A.18). Furthermore, to demonstrate that the inner self-play loop improved simulated dialogue quality, we applied the auto-evaluation method to the simulated dialogues generated before and after the self-play procedure. Results in Figure A.19 revealed that the simulated dialogues after self-play were preferred more often than the baseline dialogues without self-critique.

5 Related Work

5.1 Clinical History-taking and the Diagnostic Dialogue

History-taking and the clinical interview are widely taught in both medical schools’ and postgraduate curricula keifenheim2015teaching; yedidia2003effect; makoul2003communication; tan2021teaching; raper2015improving; von2008uk. Consensus on physician-patient communication has evolved to embrace patient-centred communication practices, with recommendations that communication in clinical encounters should address six core functions: fostering the relationship, gathering information, providing information, making decisions, responding to emotions and enabling disease- and treatment-related behavior king2013best; de2009endpoints; epstein2007patient. Specific skills and behaviours for meeting these goals have also been described, taught and assessed schirmer2005assessing; king2013best with validated tools schirmer2005assessing. Medical conventions consistently cite that certain categories of information should be gathered during a clinical interview, comprising topics such as the presenting complaint, past medical history and medication history, social and family history, and systems review nichol2018medical; denness2013consultation. Clinicians’ ability to meet these goals is commonly assessed using the framework of an objective structured clinical examination (OSCE) sloan1995objective; carraccio2000objective; epstein2002defining. Such assessments vary in their reproducibility or implementation and have even been adapted for remote practice as virtual OSCEs (vOSCEs) with telemedical scenarios, an issue of particular relevance during the COVID-19 pandemic chan2023implementation.

5.2 Conversational AI and Goal-oriented Dialogue

Conversational AI systems for goal-oriented dialogue and task completion have a rich history budzianowski2018multiwoz; wei2018airdialogue; lin2023decisionoriented. The emergence of transformers vaswani2017attention and large language models thoppilan2022lamda have led to renewed interest in this direction. The development of strategies for alignment ouyang2022training, self-improvement zhao2021ethical; saunders2022self; scheurer2023training; glaese2022improving and scalable oversight mechanisms bai2022constitutional have enabled large scale deployment of such conversational systems in the real world openai2022chatgpt; askell2021general. However, the rigorous evaluation and exploration of conversational and task-completion capabilities of such AI systems remains limited for clinical applications, where studies have largely focused on single-turn interaction use cases such as question-answering or summarization.

5.3 AI for Medical Consultations and Diagnostic Dialogue

The majority of explorations of AI as tools for conducting medical consultations have focused on “symptom checker” applications rather than a full natural dialogue, or on topics such as transcription of medical audio or the generation of plausible dialogue given clinical notes or summaries shor2023clinical; abacha2017overview; wallace2022diagnostic; zeltzer2023diagnostic. Language models have been trained using clinical dialogue datasets but not comprehensively evaluated johri2023testing. Studies have been grounded in messages between doctors and patients in commercial chat platforms (which may have altered doctor-patient engagement compared to 1:1 medical consultations) zeng2020meddialog; liu2022meddg; he2022dialmed. Many focused largely on predicting next turns in the recorded exchanges rather than clinically meaningful metrics. And to date, there have been no reported studies that have examined the quality of AI models for diagnostic dialogue using the same criteria that are used to examine and train human physicians in dialogue and communication skills; nor evaluating AI systems in common frameworks such as the OSCE.

5.4 Evaluation of Diagnostic Dialogue

Prior frameworks for human evaluation of AI systems’ performance in diagnostic dialogue have been limited in detail. They have not been anchored in established criteria for assessing communication skills and the quality of history-taking. For example, naseem2022incorporating reported a 5-point scale describing overall “human evaluation”, zeng2020meddialog reported “relevance, informativeness and human likeness”, liu2022meddg reported “fluency, expertise and relevance”, varshney2022cdialog “fluency and adequacy” and yan2022remedi “fluency”. These criteria are far less comprehensive and specific than those taught and practiced by medical professionals. A multi-agent framework for assessing conversational capabilities of LLMs is introduced in johri2023testing, however, the study was performed in the restricted setting of dermatology, used AI models to emulate both doctor and patient sides of simulated interactions, and performed limited expert evaluation of history-taking as “complete” or not.

6 Discussion

In this study, we introduced AMIE, an LLM based AI system optimised for clinical dialogue with diagnostic reasoning capabilities. We compared AMIE consultations to those performed by PCPs using a randomized, double-blind crossover study with human simulated patients in the style of an Objective Structured Clinical Examination (OSCE). Notably, our study was not designed to be representative of clinical conventions either for traditional OSCE evaluations, for remote- or tele-medical consultation practices, or for the ways clinicians usually use text and chat messaging to communicate with patients. Our evaluation instead mirrored the most common way by which people interact with LLMs today, leveraging a potentially scalable and familiar mechanism for AI systems to engage in remote diagnostic dialogue. In this setting, we observed that AMIE, an AI system optimised specifically for the task, outperformed PCPs on simulated diagnostic conversations when evaluated along multiple clinically-meaningful axes of consultation quality.

Diagnostic Performance.

The differential diagnoses provided by AMIE were more accurate and complete than those provided by board-certified PCPs, when both were evaluated by specialist physicians. Previous research has shown that AI systems may match or exceed human diagnostic performance in specific, narrow tasks kelly2019key; mcduff2023towards; kanjee2023accuracy in retrospective evaluation. However, these situations typically involved both AI and physicians interpreting the same fixed input (for example, identifying the presence of a specific finding in a medical image). Our study was significantly more challenging because it required the AI system to actively acquire relevant information through conversation rather than relying on clinical information collated by human efforts semigran2015evaluation. Therefore the system’s downstream differential diagnoses depended on not only its diagnostic inference capability, but also the quality of information gathered under uncertainty through natural conversation and building rapport.

Our results suggested that AMIE was as adept as PCPs in eliciting pertinent information during the simulated consultations and was more accurate than PCPs in formulating a complete differential diagnosis if given the same amount of acquired information. This finding corroborates other work that LLMs may be able to produce more complete differential diagnoses given the same clinical information as physicians in challenging cases (mcduff2023towards). Though not explored in this study, the assistive performance of AMIE therefore represents an interesting and important avenue for future research, particularly given the real-world importance of expert oversight for AI systems in safety-critical settings such as medicine.

Our study utilized a wide variety of simulated patients, comprising actors trained in both Canada and India and scenarios across a range of specialties. This allowed us to explore how performance varied along multiple axes: by specialty, and by the locations in which the scenario was derived and enacted. We observed that both PCPs and AMIE performed worse in obstetric/gynecology and internal medicine scenarios than those from other specialties (see Figure A.8). The study was not powered or designed to compare performance between different specialty topics, and we cannot exclude that the scenarios in some specialties might be harder than others. We observed that both AMIE and PCPs had higher diagnostic accuracy in consultations performed in the Canada OSCE lab compared to those enacted in the India OSCE lab (see Figure A.13). However, the differences were not statistically significant and in a subset of 40 scenarios enacted in both the Canada OSCE lab and the India OSCE lab, the performance of both AMIE and PCPs was equivalent (see Figure A.14).

Conversational Performance.

Patient actors and specialist raters both evaluated AMIE’s performance to be higher than PCPs on metrics related to empathy and communication skills. These axes comprised a majority of the dimensions that were evaluated. This general finding is consistent with a prior study where LLM responses were found to be more empathetic than the responses from clinicians to health questions posted on Reddit (ayers2023comparing). However, the findings in that study may not be generalised directly to our setting due to the differences in study design. Specifically, prior work has not involved a direct, randomised comparison of physicians and AI systems in a prospective simulation of multi-turn dialogue with the same patient. In both settings, the lack of voice-based and non-verbal visual communication may be an unfair disadvantage to clinicians.

The text-based chat interface used in this study introduces both advantages and disadvantages. People today most commonly engage with LLMs through synchronous text-chat interfaces openai2023website, and patients often use patient portals to send messages to their providers. We therefore chose this mode of interaction as a representative interface for LLMs to perform multi-turn conversation, adapting the virtual OSCE framework accordingly. While this allowed a fair comparison of diagnostic dialogue between LLMs and clinicians when both were restricted to a synchronous text-chat, it is important to acknowledge that our experiments do not emulate the expected quality of diagnostic dialogue in real clinical practice (including telemedicine). Physicians may be more used to history-taking and diagnostic dialogue by telephone or video consultation than synchronous text-chat communication carrillo2022effectiveness; wharton2019virtual. Instead, text is more commonly used by clinicians to communicate with patients for episodic or asynchronous needs such as prescription refills or communication about specific test results fuster2022asynchronous. Physicians may thus be more familiar with text/SMS or email rather than the synchronous text-chat medium we employed in this study. In both text/SMS and email, the conventions and expectations for communicating naturally and with empathic style might be different hammersley2019comparing. It is possible that the PCPs in our study had not yet become accustomed to the setting, and may have performed differently if subjected to a specific training program (similar in spirit to the training process for AMIE). Clinicians participating in the study undertook two preparatory pilot sessions of consultations with our synchronous text interface before the evaluation began, but this was not a formal training program, nor was it designed to optimize clinicians’ performance. Future research could explore this question more thoroughly including monitoring for the impact of a learning curve, or exploring whether performance varies according to the extent to which participating clinicians or simulated patients are familiar with telemedicine.

Additionally, our findings regarding empathic communication could also be partially attributed to the fact that AMIE responses were significantly longer than clinician responses (shown in Figure A.11), and presented with greater structure. This could potentially suggest to an observer that more time was spent preparing the response, analogous to known findings that patient satisfaction increases with time spend with their physicians (gross1998patient; tates2017effect; zyzanski1998trade).

Collectively, our findings suggest many avenues for further research that might leverage human-AI complementarity dvijotham2023enhancing, combining clinicians’ skills in the analysis of verbal and non-verbal cues with the potential strengths of LLMs to suggest more enriched conversational responses including empathic statements, structure, eloquence, or more complete differential diagnoses.

Simulated Dialogue.

The use of simulated data allowed us to quickly scale training to a broad set of conditions and patient contexts, while the injection of knowledge from search encouraged these dialogues to remain grounded and realistic. Though the simulated patients encompassed a wide range of conditions, they failed to capture the full range of potential patient backgrounds, personalities, and motivations. Through the inner self-play procedure, we were able to iteratively improve the simulated dialogue we generated and used in fine-tuning. However, these improvements were limited by our ability to articulate what makes a good dialogue in the critic instructions, the critic’s ability to produce effective feedback, and AMIE’s ability to adapt to such feedback. For example, in the simulated environment we impose that AMIE reaches a proposed differential and testing/treatment plan for the patient, but such an endpoint may be unrealistic for some conditions, especially in the virtual chat-based setting.

Evaluation Framework.

In contrast to prior works, we anchored our evaluation in criteria already established to be relevant for assessing physicians’ communication skills and history-taking quality. We performed more extensive and diverse human evaluation than prior studies of AI systems, with ratings from both clinicians and simulated patients perspective. Our raters and scenarios were sourced from multiple geographic locations, including North America, India and the UK. Our pilot evaluation rubric is, to our knowledge, the first to evaluate LLMs’ history-taking and communication skills using axes that are also measured in the real world for physicians themselves, increasing the clinical relevance of our research. Our evaluation framework is considerably more granular and specific than prior works on AI-generated clinical dialogue, which have not considered patient-centred communication best practice or clinically-relevant axes of consultation quality naseem2022incorporating; zeng2020meddialog; liu2022meddg; varshney2022cdialog; yan2022remedi; johri2023testing.

However, our pilot framework is not definitive and can be further improved in future research. History-taking itself is contextual and what determines a “good history” is dependent on the specific clinical situation, patient and physician attributes, cultural characteristics, and many other factors. Despite variation in models for clinical history-taking bird1990three; rezler1991missing; rosenberg1997lessons; smith2002patient, studies have shown that good clinical interviews are associated with not only problem detection and diagnostic accuracy, but also quadruple aims for care delivery berwick2008triple; bodenheimer2014triple ranging from patient and physician satisfaction, resilience to stress and illness, and health outcomes or cost. Future studies on the quality of LLM history-taking might therefore utilise prospective measures of these outcomes in real-world settings (for example reductions in patient complaints adamson1989physician, or improvements in cost and care effectiveness, patient and provider satisfaction), though evaluations as such may be challenging or impractical to compare to standard practice in the same individual patient, and randomisation of different approaches may also be challenging in real-world settings.

Breadth of Evaluation.

Our chosen axes of evaluation were not exhaustive and their interpretation was often subjective in nature. Although we conducted evaluations from both clinician and lay-perspectives, generating scenario-packs in three countries with assessors in both North America and India, the pool of clinicians and lay-people assessing the models could be expanded further to improve generalization of our insights. Our experiments could also undergo more extensive replication to explore other aspects such as inter-observer and inter-participant variability, including future work with an intentionally further diversified pool of human raters (clinicians and lay users). Participatory design in the development of model evaluation tools with a representative pool of patients, as well as clinical and health equity domain experts, could also be valuable.

Although our scenarios comprised many different clinical conditions and specialties, our experiments were not necessarily representative of the decades of clinical practice accumulated by even a single doctor (who on average may perform tens of thousands of consultations in a career silverman2010doctors). The range of conditions possible to examine in medicine is vast as is the variation in presentation of individual diseases. Our experiments were not designed to examine multi-morbidity and co-incident pathology, longitudinal case presentation or the consideration of sequential information from clinical investigations. We excluded entirely some clinical settings or specialties such as psychiatry, pediatrics, intensive care, and inpatient case management scenarios. Further research would be needed to understand the applicability of our findings in many settings such as these, where the requirements for high-quality history-taking might differ rahman2023inter; kantar2022history. The OSCE framework is commonly used in the assessment of clinicians’ skills. It encompasses a significant range of methodologies including real or simulated patients, interaction with physical artefacts or clinical materials, applications to a variety of medical specialties, tasks or settings; and both remote or in-person assessments. Although the OSCE approach is popular, there are significant limitations to its validity setyonugroho2015reliability. We utilised a remote text-based assessment, replicating known issues with the paradigm of “virtual OSCE” such as the inability to incorporate non-verbal symptoms, signs and communication features. Additionally, this format could introduce unfamiliar constraints to the communication of PCP participants chan2023implementation.

The tone, content, and nature of the OSCE dialogues in our study are likely not to be representative of real-world patient populations. For example, patient actors may have described their symptoms with greater structure, depth or clinical detail than could be routinely expected in many consultations, or had greater comprehension of clinical context than would be ordinarily expected. Furthermore, although evaluation was blinded, the style of responses from AMIE was notably different to that by PCPs which limits the practical extent of blinding in study design.

Therefore even within the distribution of diseases and specialties we addressed, our findings should be interpreted with humility and caution. There is a need for further research to examine varied presentations of the same diseases, alongside exploration of alternate approaches to evaluating history-taking and clinical dialogue in situations of different patient needs, preferences, behaviours and circumstances.

Fairness and Bias.

The evaluation protocol presented in this paper is limited in terms of its ability to capture potential issues related to fairness and bias, which remains an important open question that we will aim to address in subsequent system evaluations. Recent advances in the development of comprehensive frameworks for bias detection in large language models weidinger2022taxonomy; gallegos2023bias present a promising starting point for establishing such an approach. It should be noted that medical diagnostic dialogue is a particularly challenging use case, due to the complexity of the medical domain, the interactive information gathering nature of the dialogue, and the outcome-driven setting, with the potential of associated harms in case of incorrect diagnosis or incorrect medical advice. Nevertheless, disentangling these issues is an important further research area if LLMs in the domain are to overcome rather than propagate inequities in healthcare. For example, previous studies have found that physicians approach communication with their patients differently, on average, depending on patients’ race, resulting in Black patients receiving communication that was less patient-centered, and with a lower positive affect johnson2004patient. Other studies have found differences in physicians’ communication styles and conversation length based on gender roter2002physician. Effective intercultural communication skills are essential rahman2023inter. There is therefore a non-negligible risk that such historical conversational biases may be replicated or amplified in an AI dialogue system, but at the same time there is also an opportunity to work towards designing conversational systems that can be more inclusive, and more personalized to the individual patient’s needs.

To help inform the development of the necessary fairness, bias, and equity frameworks, it is important to employ a participatory approach to solicit representative views across a wide range of patient demographics, as well as clinical and health equity domain experts. Such evaluation frameworks should be complemented by extensive model red teaming and an adversarial approach to identifying any remaining gaps and failure modes. Recent advances in red teaming LLMs could be useful in this scenario perez2022red; ganguli2022red; yu2023gptfuzzer; ge2023mart. These practices should not only inform the evaluation of the final model, but also its development and iterative refinement. Model development should follow the established data and model reporting practices and provide transparency into the training data and the associated decision processes mitchell2019model; crisan2022interactive; pushkarna2022data. The dialogue research dataset contributing to AMIE training data in our study was de-identified, reducing the availability of socio-economic factors, patient demographics, and information about clinical settings and locations.

Further work is also needed to ensure the robustness of medical LLMs in multilingual settings choudhury2021linguistically; talat2022you; ahuja2023megaverse; ImaniGooghari_2023, and particularly their performance in low-resource languages nguyen2023democratizing. The great variety of cultures naous2023having, languages, localities, identities, and localized medical needs, makes the task of generating a priori static yet comprehensive fairness benchmarks practically infeasible. Measurement and mitigation of bias must move beyond the traditional narrow focus on specific axes that fails to scale globally ramesh2023fairness. LLM-based evaluators present a potential solution for preliminary assessments in languages where there are no systematic benchmarks, though prior studies have found these auto-evaluation frameworks to be biased, underscoring the need for calibrating them on native speaker evaluations, and using them with caution hada2023large.

Deployment.

This research demonstrates the potential of LLMs for future use in healthcare in the context of diagnostic dialogue. Transitioning from an LLM research prototype that has been evaluated in this study to a safe and robust tool that can be used by healthcare providers, administrators, and people will require significant additional research to ensure the safety, reliability, efficacy, and privacy of the technology. Careful consideration will need to be given to the ethical deployment of this technology including rigorous quality assessment across different clinical settings and research into reliable uncertainty estimation methods quach2023conformal; chen2023quantifying; huang2023look; yang2023uncertaintyaware that would allow for deferral to human clinical experts when needed. These and other guardrails are needed to mitigate potential overreliance on LLM technologies, with other specific measures for attention to ethical and regulatory requirements particular to future use-cases and the presence of qualified physicians in the loop to safeguard any model outputs. Additional research will also be needed to assess the extent to which biases and security vulnerabilities might arise either from base models or the circumstances of use in deployment, as we have highlighted in our prior work singhal2022large. Given the continuous evolution of clinical knowledge, it will also be important to develop ways for LLMs to utilize up-to-date clinical information lazaridou2021mind.

7 Conclusion

The utility of medical AI systems could be greatly improved if they are better able to interact conversationally, anchoring on large-scale medical knowledge while communicating with appropriate levels of empathy and trust. This research demonstrates the significant potential capabilities of LLM based AI systems for settings involving clinical history-taking and diagnostic dialogue. The performance of AMIE in simulated consultations represents a milestone for the field, as it was assessed along an evaluation framework that considered multiple clinically-relevant axes for conversational diagnostic medical AI. However, the results should be interpreted with appropriate caution. Translating from this limited scope of experimental simulated history-taking and diagnostic dialogue, towards real-world tools for people and those who provide care for them, requires significant additional research and development to ensure the safety, reliability, fairness, efficacy, and privacy of the technology. If successful, we believe AI systems such as AMIE can be at the core of next generation learning health systems that help scale world class healthcare to everyone.

Acknowledgments

This project was an extensive collaboration between many teams at Google Research and Google DeepMind. We thank Yun Liu, Daniel McDuff, Jake Sunshine, Ali Connell, Paul McGovern and Zoubin Ghahramani for their comprehensive review and detailed feedback on the manuscript. We also thank Sami Lachgar, Lauren Winer, John Guilyard and Maggie Shiels for contributions to the narratives and visuals. We are grateful to Julie Anne Seguin, Sally Goldman, Yuri Vasilevski, Xinying Song, Akshay Goel, Chu-ling Ko, Abhinav Das, Haiyang Yu, Chang Liu, Yuchen Liu, SiWai Man, Brett Hatfield, Sean Li, Ajay Joshi, Gordon Turner, Annisah Um’rani, Divya Pandya and Preeti Singh for their valuable insights, technical support and feedback during our research. We also thank our clinical provider partners in Canada and India for their partnership in conducting the OSCE study. Finally, we are grateful to Dale Webster, Ewa Dominowska, David Fleet, Philip Mansfield, Sushant Prakash, Renee Wong, Susan Thomas, Michael Howell, Karen DeSalvo, Jeff Dean, James Manyika, Zoubin Ghahramani and Demis Hassabis for their support during the course of this project.

Data Availability

Some of the real-world datasets used in the development of AMIE are open-source (MedQA). The scenario packs from UK used in the OSCE study are also available for download on the internet.

Code Availability

AMIE is an LLM based research AI system for diagnostic dialogue. We are not open-sourcing model code and weights due to the safety implications of unmonitored use of such a system in medical settings. In the interest of responsible innovation, we will be working with research partners, regulators, and providers to validate and explore safe onward uses of AMIE. For reproducibility, we have documented technical deep learning methods while keeping the paper accessible to a clinical and general scientific audience. Our work builds upon PaLM 2, for which technical details have been described extensively in the technical report google2023palm2.

Competing Interests

This study was funded by Alphabet Inc and/or a subsidiary thereof (‘Alphabet’). All authors are employees of Alphabet and may own stock as part of the standard compensation package.

\printbibliography

{refsection}

Appendix

In the following sections, we report additional data and detailed analyses to further illustrate the performance of AMIE.

We provide details on:

•

OSCE evaluation rubrics
•

Example of simulated dialogues after self-critique
•

AMIE user interfaces
•

Example consultation with OSCE agents
•

DDx top-k accuracy by degree of matching
•

DDx top-k accuracy by specialty
•
Auto-evaluation on DDx
- –
  
  Reproducing DDx accuracy via auto-evaluation
- –
  
  AMIE DDx accuracy on AMIE and PCP consultations
- –
  
  DDx accuracy as a function of the number of dialogue turns
•

DDx top-k accuracy by location
•
Model-based auto-evaluation of qualitative criteria:
- –
  
  Rank-order agreement of auto-evaluation to specialists
- –
  
  Auto-evaluation of simulated dialogues with self-play

A.1 OSCE Evaluation Rubrics

[Uncaptioned image] — Table A.1: General Medical Council Patient Questionnaire (GMCPQ) rubric details.

A.2 Example of Simulated Dialogue After Self-critique

A.3 AMIE User Interfaces

A.4 Example Consultation with OSCE Agents

The following are two example consultations for the same scenario pack and patient actor from AMIE and PCP, respectively.

A.4.1 Example AMIE Consultation

A.4.2 Example PCP Consultation

A.5 DDx Top-k Accuracy by Degree of Matching

In our OSCE study, the specialist was asked the following question:

Here we present an ablation analysis for varying degrees of matching to the ground truth where for each differential, we only considered a diagnosis a match if the specialist indicated in the answer to this question that the match was at least as close as the specified degree of matching. Note that all other specialist-rated DDx evaluations in this paper used the “Relevant” threshold when computing accuracy. The differences between AMIE and PCPs in DDx accuracy were statistically significant for all values of k at the matching levels “Relevant”, “Extremely Relevant”, and “Exact Match”.

A.6 DDx Top-k Accuracy by Specialty

Figure A.8 shows the DDx accuracy achieved by AMIE and PCPs for each specialty based on specialist ratings. Specifically, we observed that AMIE’s performance matched or surpassed PCPs performance for all specialties.

A.7 Auto-evaluation on DDx

Here we report the top-k DDx accuracy as computed by the auto-evaluation method. For each DDx in the DDx list generated by AMIE and PCPs, we used Med-PaLM 2 to determine whether the ground truth diagnosis appears within the top-k positions of the differential diagnosis list. Given a prediction and label, the auto-evaluator computes whether they match by prompting Med-PaLM 2 with the following question:

A.7.1 Reproducing DDx Accuracy via Auto-evaluation

The overall performance trends obtained through the auto-evaluator align well with specialist assessments in Figure 3 despite marginal differences in the computed accuracy values, as shown in Figure A.9. These results demonstrate that the auto-evaluator is a valid surrogate for the specialist raters.

A.7.2 AMIE DDx Accuracy on AMIE and PCP Consultations

We compared AMIE’s diagnosis accuracy based on its own consultations with its accuracy generated from corresponding PCP consultations, using the DDx auto-evaluator. Results in Figure A.10 showed that the diagnostic quality remained consistent regardless of whether AMIE processed information from its own dialogue or from the PCP’s conversation.

A.7.3 DDx Accuracy as a Function of Dialogue Turns

Distribution of words and turns.

Figure A.11 shows the distributions of words and turns for the OSCE conversations. Because the number of patient actor words and turns is consistent between groups, neither agent has an unfair advantage in terms of the amount of information used to make a diagnosis. However, it is important to note that AMIE is far more verbose in its responses which may have influenced the qualitative ratings from specialists.

Accuracy by number of turns.

Here we plotted the auto-evaluation of AMIE-generated differential diagnoses as a function of number of turns. We truncated conversations to the first $T$ turns, and then asked AMIE to produce a DDx with this truncated conversation. For both the AMIE and PCP conversations, we observed that AMIE’s average diagnostic accuracy began to plateau within 10 turns, with additional information gathering having diminishing returns on the diagnostic performance as shown in Figure A.12.

A.8 DDx Accuracy by Location

Accuracy by Location.

We compared the specialist ratings for the 67 scenarios conducted in Canada and the 82 scenarios conducted in India.

Shared Scenarios.

We repeated 40 of the scenarios at the other location, meaning if it was originally run in Canada, we then ran it in India and vice-versa. This included all of the UK scenarios and 26 of the India scenarios. Because these conversations did not have specialist ratings, we instead leveraged auto-evaluation to compare the produced differential diagnoses and ablate the effect of the OSCE location.

Results.

We observed a higher average diagnostic performance for AMIE in Canada than in India (see Figure A.13). However, when comparing scenarios performed at both study locations, we observed that AMIE and PCP performance remained consistent regardless of the study location (see Figure A.14), suggesting that the observed performance variations are likely not due to the patient actor or clinician rater differences, but instead might be attributed to inherent differences in the difficulty levels of scenarios in each location.

A.9 Model-based Auto-evaluation of Qualitative Criteria

In order to accurately emulate the ratings of specialists on the clinical criteria in our OSCE evaluation framework, we developed a model-based auto-evaluation procedure leveraging the AMIE model to score dialogues from 1 to 5 based on how well they exemplified those qualitative criteria. We initially focused on a subset of four clinical axes from the PACES criteria (see Table A.2), however, this procedure can be easily extended to other ratings.

Using the 298 dialogues produced by AMIE and PCPs in this study, we corroborated the results of the specialist ratings on these 4 criteria using the auto-evaluation procedure. We validated that the auto-evaluation rankings were well aligned with these specialist ratings (see Figures A.17 and A.18). Additionally, we applied it to the simulated dialogues generated via the inner-loop self-play procedure to test whether this iterative process resulted in measurable improvements in dialogue quality (see Figure A.19).

Self-CoT Procedure for Auto-Evaluation of Clinical Criteria.

The auto-evaluation procedure we employed was a two-step process in which we prompted AMIE itself to rate dialogues on the chosen subset of the PACES criteria (see Table A.2).

1.

First, we prompted AMIE to summarize good and bad aspects of several dialogues and provide an explanation of the provided human rating between 1 and 5 (see Figure A.15).
2.

Next, we used these self-generated explanations alongside their respective dialogues as examples in a 5-shot prompt to evaluate and rate a new dialogue. This few-shot prompt included one example for each point on the 5-point rating scale (see Figure A.16).

In both prompts, we included the rating scale and expert-derived examples of good or bad behaviour for a particular criterion, matching those shown in Table A.2. We referred to this prompting method as self-CoT (Chain-of-Thoughts) [wei2022chain] as the plausible reasoning for the human ratings are derived from the model itself.

Rank-order Agreement.

We evaluated our auto-evaluation method by quantifying its agreement with the specialist rankings of the OSCE dialogues. We limited our analysis to the 149 dialogue pairs in the study. Thus, each pair consisted of a AMIE conversation and a PCP conversation with the same patient actor, and rated by the same specialist. For a pair of two dialogues, the three possibilities were: the first one was rated better than the second one, they were equally rated, or the first one was rated worse than the second one. We defined the rank-order agreement as the proportion of dialogue pairs for which the specialist ranking was preserved by the auto-evaluation ratings. For example, we counted it as correct when our auto-evaluation rated AMIE’s dialogue as better than the PCP’s dialogue if specialists also rated AMIE’s dialogue as better, regardless of the exact scores each method assigned.

Auto-evaluation Prompting Strategies.

Using the rank-order agreement metric, we ablated the effect of the two-step prompting and compared it to other methods such as the five-shot prompting (i.e. dropping step 1), shuffled five-shot self-CoT prompting where the order of support examples was randomised each time, and 0-shot prompting using only the rating scale explanation itself (see Figure A.17). All methods outperformed the chance level, with the two-step process generally outperforming other methods, though this difference was marginal. Shuffling the examples in the self-CoT prompt made no difference on average.

Benchmarking Auto-evaluation.

While auto-evaluation was significantly better than random guessing at aligning with specialist preferences, it was unclear if the resulting performance was sufficient. To test this, we had 139 dialogue pairs each rated by a second specialist, and we computed the rank-order agreement of the two specialists on this subset (see figure A.18). We observed that auto-evaluation was about as accurate as the alternative specialists in predicting the first specialist’s rank-order, suggesting that it is useful to leverage auto-evaluation for these criteria.

Evaluating Self-play Dialogues.

We applied our auto-evaluation procedure to 1,142 dialogues (derived from common conditions) before and after being refined through the self-play critique. We demonstrated that, on average, the refined dialogues after an iteration of critique/revision were rated higher than the original baseline dialogues across all criteria A.19.

Figure A.15: Prompt for the explanation generation given an dialogue and the human rating. AMIE was prompted to summarize good and bad aspects of a particular dialogue and provide an explanation of the given human rating between 1 and 5.

Figure A.16: Prompt for the auto-evaluation of qualitative criteria. We constructed few-shot prompt with self-generated explanations alongside their respective dialogues as exemplars to evaluate and rate a new dialogue. This few-shot prompt included one example for each point on the 5-point rating scale.

A.9.1 Rank-order Agreement of Auto-evaluation to Specialist

A.9.2 Auto-evaluation of Simulated Dialogues with Self-play

Table A.5: Model card for AMIE.

	Model characteristics
Model initialization	The model was initialized from an LLM similar to PaLM-2 [google2023palm2]. Additional fine-tuning was performed as described in Section 2.3.
	Usage
Application	The primary use is research on LLMs for clinical history-taking and medical dialogue including advancing diagnostic accuracy, alignment methods, fairness, safety, and equity research, and understanding limitations of current LLMs for such applications.
	Data overview
Training dataset	The dataset was datasets of medical question answering/reasoning, summarization and medical dialogue datasets in Section 2.1.
Evaluation	Randomized OSCE was performed to understand AMIE’s capabilities and benchmark against expert primary care physicians.
	Evaluation results
Evaluation results	The results are described in Section 4.