Publications

Selected publications for viewing. Some work were published during Yanjun’s time as postdoctoral researcher in ICU Data Science Lab, UW-Madison.

Shiyue Hu, Ruizhe Li, Yanjun Gao (2026). Race, Ethnicity and Their Implication on Bias in Large Language Models. medRxiv.

He Cheng, Yifu Wu, Saksham Khatwani, Maya Kruse, Dmitriy Dligach, Timothy a Miller, Majid Afshar, Yanjun Gao (2026). LogosKG: Hardware-Optimized Scalable and Interpretable Knowledge Graph Retrieval. arXiv preprint arXiv:2604.18913.

Shiyue Hu, He Cheng, Lucas Gillenwater, Keenan Manpearl, Aishwarya Mandava, Yifan Wang, Milton Pividori, Barbara Stranger, Arjun Krishnan, Casey Greene, Others (2026). Beyond Identifier Matching: An Empirical Characterization of Failure Modes in Biomedical Knowledge Graph Integration. medRxiv.

Maya Kruse, MS, Majid Afshar, Saksham Khatwani, MS, Anoop Mayampurath, Guanhua Chen, Yanjun Gao, PhD (2025). Simple Yet Effective: An Information-Theoretic Approach to Multi-LLM Uncertainty Quantification. In EMNLP 2025.

PDF Source Document

Maya Kruse, MS, Shiyue Hu, MS, Nicolas Derby, MS, Yifu Wu, PhD, Samantha Stonbraker, Bingsheng Yao, Dakuo Wang, Elizabeth Goldberg, Yanjun Gao, PhD (2025). Large Language Models with Temporal Reasoning for Longitudinal Clinical Summarization and Prediction.

PDF Source Document

Quintin Myers, Yanjun Gao (2025). Uncovering Hidden Violent Tendencies in LLMs: A Demographic Analysis via Behavioral Vignettes. arXiv preprint arXiv:2506.20822.

Yanjun Gao, Skatje Myers, Shan Chen, Dmitriy Dligach, Timothy Miller, Danielle S Bitterman, Guanhua Chen, Anoop Mayampurath, Matthew M Churpek, Majid Afshar (2025). Uncertainty estimation in diagnosis generation from large language models: next-word probability is not pre-test probability. JAMIA open.

Maya Kruse, Majid Afshar, Saksham Khatwani, Anoop Mayampurath, Guanhua Chen, Yanjun Gao (2025). Simple Yet Effective: An Information-Theoretic Approach to Multi-LLM Uncertainty Quantification. arXiv preprint arXiv:2507.07236.

Yanjun Gao, Ruizhe Li, Emma Croxford, John Caskey, Brian W Patterson, Matthew Churpek, Timothy Miller, Dmitriy Dligach, Majid Afshar (2025). Leveraging Medical Knowledge Graphs Into Large Language Models for Diagnosis Prediction: Design and Application Study. JMIR AI.

Maya Kruse, Shiyue Hu, Nicholas Derby, Yifu Wu, Samantha Stonbraker, Bingsheng Yao, Dakuo Wang, Elizabeth Goldberg, Yanjun Gao (2025). Large Language Models with Temporal Reasoning for Longitudinal Clinical Summarization and Prediction. Findings of the Association for Computational Linguistics: EMNLP 2025.

M Afshar, J Caskey, Y Gao, MM Churpek, A Mayampurath, D Dligach, R Sethi (2025). Large Language Models to Detect Stigmatizing Language in Critically Ill Patients With Substance Use Disorders. American Journal of Respiratory and Critical Care Medicine.

Kelli Henry, Steven Xu, Kaitlin Blotske, Moriah Cargile, Erin F Barreto, Brian Murray, Susan Smith, Seth R Bauer, Xingmeng Zhao, Adeleine Tilley, Others (2025). Large language models management of medications: three performance analyses. arXiv preprint arXiv:2509.22926.

Skatje Myers, Dmitriy Dligach, Timothy a Miller, Samantha Barr, Yanjun Gao, Matthew Churpek, Anoop Mayampurath, Majid Afshar (2025). Evaluating Retrieval-Augmented Generation vs. Long-Context Input for Clinical Reasoning over EHRs. arXiv preprint arXiv:2508.14817.

Emma Croxford, Yanjun Gao, Elliot First, Nicholas Pellegrino, Miranda Schnier, John Caskey, Madeline Oguss, Graham Wills, Guanhua Chen, Dmitriy Dligach, Others (2025). Evaluating clinical AI summaries with large language models as judges. npj Digital Medicine.

Emma Croxford, Yanjun Gao, Nicholas Pellegrino, Karen Wong, Graham Wills, Elliot First, Miranda Schnier, Kyle Burton, Cris Ebby, Jillian Gorski, Others (2025). Development and validation of the provider documentation summarization quality instrument for large language models. Journal of the American Medical Informatics Association.

Rohan Sethi, John Caskey, Yanjun Gao, Matthew M Churpek, Timothy a Miller, Anoop Mayampurath, Elizabeth S Afshar, Majid Afshar, Dmitriy Dligach (2025). Detecting Stigmatizing Language in Clinical Notes with Large Language Models for Addiction Care. medRxiv.

Emma Croxford, Yanjun Gao, Nicholas Pellegrino, Karen Wong, Graham Wills, Elliot First, Frank Liao, Cherodeep Goswami, Brian Patterson, Majid Afshar (2025). Current and future state of evaluation of large language models for medical summarization tasks. npj Health Systems.

Saksham Khatwani, He Cheng, Majid Afshar, Dmitriy Dligach, Yanjun Gao (2025). Brittleness and Promise: Knowledge Graph Based Reward Modeling for Diagnostic Reasoning. arXiv preprint arXiv:2509.18316.

Emma Croxford, Yanjun Gao, Elliot First, Nicholas Pellegrino, Miranda Schnier, John Caskey, Madeline Oguss, Graham Wills, Guanhua Chen, Dmitriy Dligach, Others (2025). Automating Evaluation of AI Text Generation in Healthcare with a Large Language Model (LLM)-as-a-Judge. medRxiv.

Ruizhe Li, Yanjun Gao (2025). Anchored Answers: Unravelling Positional Bias in GPT-2's Multiple-Choice Questions. In Findings of the Association for Computational Linguistics: ACL 2025.

Yanjun Gao, PhD, Skatje Myers, Shan Chen, Dmitriy Dligach, Tim Miller, Danielle Bitterman, Matthew MM Churpek, Majid Afshar (2024). When Raw Data Prevails: Are Large Language Model Embeddings Effective in Numerical Data Representation for Medical Machine Learning Applications?. In Findings of EMNLP 2024.

Ruizhe Li, Yanjun Gao, PhD (2024). Anchored Answers: Unravelling Positional Bias in GPT-2's Multiple-Choice Questions.

Xin Chen, Hanxian Huang, Yanjun Gao, PhD, Yi Wang, Jishen Zhao, Ke Ding (2024). Learning to Maximize Mutual Information for Chain-of-Thought Distillation.

PDF Code Dataset Poster Video Source Document

Yanjun Gao, Skatje Myers, Shan Chen, Dmitriy Dligach, Timothy a Miller, Danielle Bitterman, Matthew Churpek, Majid Afshar (2024). When Raw Data Prevails: Are Large Language Model Embeddings Effective in Numerical Data Representation for Medical Machine Learning Applications?. arXiv preprint arXiv:2408.11854.

Majid Afshar, Yanjun Gao, Graham Wills, Jason Wang, Matthew M Churpek, Christa J Westenberger, David T Kunstman, Joel E Gordon, Frank J Liao, Brian W Patterson (2024). Prompt Engineering GPT-4 to Answer Patient Inquiries: A Real-Time Implementation in the Electronic Health Record across Provider Clinics. medRxiv.

Yanjun Gao, Skatje Myers, Shan Chen, Dmitriy Dligach, Timothy Miller, Danielle S Bitterman, Guanhua Chen, Anoop Mayampurath, Matthew M Churpek, Majid Afshar (2024). Position Paper On Diagnostic Uncertainty Estimation from Large Language Models: Next-Word Probability Is Not Pre-test Probability. medRxiv.

Majid Afshar, Yanjun Gao, Deepak Gupta, Emma Croxford, Dina Demner-Fushman (2024). On the role of the UMLS in supporting diagnosis generation proposed by Large Language Models. Journal of Biomedical Informatics.

Skatje Myers, Timothy a Miller, Yanjun Gao, Matthew M Churpek, Anoop Mayampurath, Dmitriy Dligach, Majid Afshar (2024). Lessons learned on information retrieval in electronic health records: a comparison of embedding models and pooling strategies. Journal of the American Medical Informatics Association.

Xin Chen, Hanxian Huang, Yanjun Gao, Yi Wang, Jishen Zhao, Ke Ding (2024). Learning to Maximize Mutual Information for Chain-of-Thought Distillation. In Findings of the Association for Computational Linguistics: ACL 2024.

W Yoon, S Chen, Y Gao, D Dligach, DS Bitterman, M Afshar, T Miller (2024). LCD Benchmark: Long Clinical Document Benchmark on Mortality Prediction.. Medrxiv: the Preprint Server for Health Sciences.

Shan Chen, Jack Gallifant, Marco Guevara, Yanjun Gao, Majid Afshar, Timothy Miller, Dmitriy Dligach, Danielle S Bitterman (2024). Improving Clinical NLP Performance through Language Model-Generated Synthetic Clinical Data. arXiv preprint arXiv:2403.19511.

B Leding, Y Gao, D Dligach, E Croxford, A Mayampurath, MM Churpek, M Afshar (2024). Generation and Evaluation of Synthetic Critical Care Progress Notes With Large Language Models. C22. ARTIFICIAL INTELLIGENCE IN THE ICU: THE MACHINE WILL SEE YOU NOW.

Emma Croxford, Yanjun Gao, Nicholas Pellegrino, Karen K Wong, Graham Wills, Elliot First, Frank J Liao, Cherodeep Goswami, Brian Patterson, Majid Afshar (2024). Evaluation of Large Language Models for Summarization Tasks in the Medical Domain: A Narrative Review. arXiv preprint arXiv:2409.18170.

Emma Croxford, Yanjun Gao, Brian Patterson, Daniel To, Samuel Tesch, Dmitriy Dligach, Anoop Mayampurath, Matthew M Churpek, Majid Afshar (2024). Development of a Human Evaluation Framework and Correlation with Automated Metrics for Natural Language Generation of Medical Diagnoses. medRxiv.

Yanjun Gao, Diwakar Mahajan, \"Ozlem Uzuner, Meliha Yetisgen (2024). Clinical natural language processing for secondary uses. Journal of Biomedical Informatics.

Yanjun Gao, PhD, Dmitriy Dligach, Timothy Miller, John Caskey, Brihat Sharma, Matthew M Churpek, Majid Afshar (2023). DR. BENCH: Diagnostic Reasoning Benchmark for Clinical Natural Language Processing. JBI.

Yanjun Gao, PhD, Dmitriy Dligach, Timothy Miller, Dongfang Xu, Matthew MM Churpek, Majid Afshar (2022). Summarizing Patients’ Problems from Hospital Progress Notes Using Pre-trained Sequence-to-Sequence Models. In COLING.

Yanjun Gao, Timothy Miller, Dongfang Xu, Dmitriy Dligach, Matthew M Churpek, Majid Afshar (2022). Summarizing Patients’ Problems from Hospital Progress Notes Using Pre-trained Sequence-to-Sequence Models. Proceedings of COLING. International Conference on Computational Linguistics.

Meliha Yetisgen, Ozlem Uzuner, Yanjun Gao, Diwakar Mahajan (2022). Call for papers: Special issue on clinical natural language processing for secondary use applications.. Journal of Biomedical Informatics.