The Promise and Pitfalls of Artificial Intelligence (AI) Literature Search Tools in Medicine

Files

Authors

• Kimberly Forman, Michigan State University College of Human Medicine Medical Student
• Rachel Armstrong, Michigan State University College of Human Medicine Medical Student
• Allison Kowatch, Michigan State University College of Human Medicine Medical Student
• Megan Courtley, Emergency Medicine - Corewell Health West
• Michelle Padley, Allied Health - Corewell Health West
• Nawal Bennett, Pulmonary and Acute Health Research Operations - Corewell Health West
• Tiffany Fleeger, Pulmonary - Corewell Health West
• Lindsey Ouellette, Michigan State University
• Jeffrey Jones, Emergency Medicine - Corewell Health West

Description

Conducting comprehensive literature reviews often poses challenges for clinicians and medical researchers, requiring extensive reading and synthesis of existing works while advancing original insights. Numerous artificial intelligence (AI) research tools have emerged to assist medical researchers and clinicians; however, identifying the most effective tools can be overwhelming. The study assessed AI tools for their ability to accurately identify relevant research papers, extract key information, generate concise summaries, and facilitate trend analysis across studies.

This prospective descriptive analysis examined six semantic AI search tools that interpret the meaning of words and phrases in a query, and six citation-based search tools that match keywords. A panel of emergency medicine faculty developed evaluation criteria. These criteria included search functionality, data analysis and summarization, accuracy (compared to four human searches), relevance, limitations, unique features, user-friendliness, and cost. Descriptive statistics and frequency tables were used to describe the key quantitative and qualitative variables.

The six semantics-based tools (e.g., Consensus, Perplexity) were more suitable for exploring nuanced topics and emerging research. In contrast, the six citation-based tools (e.g., Connected Papers, Scite.ai) focused on the impact of research through citation metrics and network analysis. Using both approaches together can provide a more comprehensive and nuanced literature search. Most AI search tools offered a user-friendly interface (92%), quick summaries of research findings (75%), content analysis (67%), conversational search abilities (50%), and literature mapping (42%). Monthly fees ranged from free to $29 per month/month based on the selected features. Accuracy relative to human searches ranged from 100% to 87%. All of the tools flagged results as relevant when they were not (false positive rate of 3-16%). AI hallucinations and incorrect or mismatched citations (3-16%).

AI-driven literature search tools have become increasingly sophisticated and accessible, providing substantial efficiencies in the identification and synthesis of medical evidence. Both semantic and citation-based systems offer unique advantages, and their combined use offers the greatest potential benefits for clinicians and researchers. Nevertheless, inherent risks-such as false positives and inaccurate citations-require users to maintain a critical, evaluative approach and avoid overreliance on automated outputs.

Publication Date

5-8-2026

Disciplines

Emergency Medicine

Recommended Citation

Forman K, Armstrong R, Kowatch A, Courtley M, Padley M, Bennett N, Fleeger T, Ouellette L, Jones J. The promise and pitfalls of artificial intelligence (AI) literature search tools in medicine. Presented at: Research Day Corewell Health West; 2026 May 8; Grand Rapids, MI.

Comments

2026 Research Day Corewell Health West, Grand Rapids, MI, May 8, 2026. Abstract 1891