Leveraging Open-Source Large Language Models to Identify Undiagnosed Patients with Rare Genetic Aortopathies

Abstract

Background: Rare genetic aortopathies are frequently undiagnosed due to phenotypic heterogeneity, and delayed diagnosis can lead to fatal cardiac outcomes. While genetic testing can enable early proactive interventions, it relies on primary care physicians to recognize a genetic basis for symptoms and then refer patients to clinical genetics. Broad-scale screening methods are needed to identify cases that do not fit an obvious diagnostic pattern. Clinical notes, rich in narrative details, may support the automated flagging of patients for genetic testing. This study investigates the use of large language models (LLMs) to identify undiagnosed cases by analyzing unstructured clinical text. Methods: Given the strength of LLMs in processing unstructured text, we developed an open-source LLM-enabled genetic testing recommendation pipeline for rare genetic aortopathies. The pipeline leverages retrieval augmented generation (RAG) on curated genetic aortopathy-related corpora to utilize relevant clinical knowledge for identifying patients likely to benefit from genetic testing. By combining base LLMs with domain-specific knowledge, the pipeline improves prediction accuracy in ambiguous cases. The pipeline was validated using 22,510 patient progress notes from 500 individuals (250 cases, 250 controls) in the Penn Medicine BioBank (PMBB). Findings: Our recommendation pipeline successfully categorized 425 out of 499 patients, with one case requiring further clinician evaluation due to incomplete information. The model achieved a patient-level recommendation accuracy of 0.852, precision of 0.889, recall of 0.803, F1-score of 0.844, and F3-score of 0.811. Interpretation: Our LLM-enabled workflow that integrates RAG showed strong performance in recommending genetic testing for patients with rare genetic aortopathies. This demonstrates its potential to support undiagnosed patient identification from free-text clinical notes, thereby automating early disease identification and improving patient outcomes. Funding: U.S. Department of Energy and U.S. National Institute of Health.

Competing Interest Statement

The authors have declared no competing interest.

Funding Statement

This research uses resources of the Argonne Leadership Computing Facility, a U.S. Department of Energy (DOE) Office of Science user facility at Argonne National Laboratory and is based on research supported by the U.S. DOE Office of Science-Advanced Scientific Computing Research Program, under Contract No. DE-AC02-06CH11357. This research also utilizes computing resources provided by the National Artificial Intelligence Research Resource (NAIRR) Pilot, supported by award NAIRR240008. This study was also supported by Genomic Medicine T32 Training Grant 5T32HG009495-08.

Author Declarations

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.

Yes

The details of the IRB/oversight body that provided approval or exemption for the research described are given below:

The IRB of University of Pennsylvania have given ethical approve for this work under protocol number 856828.

I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.

Yes

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).

Yes

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.

Yes

Data Availability

Patient notes data will not be made available to anyone due to HIPAA compliance and privacy restrictions. Implementation code will be provided on Github along with necessary documentation for access by all.

Comments (0)

No login
gif