To recruit volunteers more quickly for clinical trials, a team of National Institutes of Health (NIH) researchers has turned to artificial intelligence. The scientists built a large language model, TrialGPT, that can scan medical summaries of patients and match them to clinical trials they are eligible for.
The results were published in Nature Communications on Nov. 18.
“Our study shows that TrialGPT could help clinicians connect their patients to clinical trial opportunities more efficiently and save precious time that can be better spent on harder tasks that require human expertise,” study author Zhiyong Lu, Ph.D., senior investigator at the NIH’s National Library of Medicine, said in a Nov. 18 release.
Lu and colleagues evaluated TrialGPT by testing it against three publicly available synthetic patient cohorts, totaling 183 patients, that were previously matched with clinical trials. Synthetic patients are generated from real-world data to mimic characteristics seen in real patients. The AI model generates keywords for each patient and uses those to match them to studies in the ClinicalTrials.gov system; it then generates a summary of why the patient does or doesn't fit the trial population.
Overall, TrialGPT correctly matched patients to trials 87.3% of the time, which the study authors note is close to the range seen in human experts: 88.7% to 90%.
The researchers also asked two clinicians to review six patient summaries and evaluate their eligibility for six clinical trials. One clinician did this manually, while another used TrialGPT; using the model sped up screening time by 40% without losing accuracy.
The model made four main kinds of errors when matching patients to trials. The most common type, making up 30.7% of all errors, was TrialGPT using incorrect reasoning by concluding that there wasn’t enough information to say whether a patient fit a trial when in fact there was.
Other errors stemmed from the model’s lack of medical knowledge, for example incorrectly concluding that a cardiac catheterization is the same as a central venous puncture and excluding a patient from a trial because they’d received one before. The two procedures are different, and the patient should have been eligible for the trial.
As many as 85% of clinical trials fail to recruit enough patients to sustain the needed sample size for statistical analysis, according to a separate NIH study published in 2023, despite the fact that about $1.9 billion is spent on recruitment every year.
The researchers are now testing out TrialGPT in real-word clinical settings as part of a 2024 Director's Challenge Innovation Award, according to the release.