AMIA 2025 NLP Annual Symposium - Poster

We are excited to announce that our paper “Evaluating Large Language Models for Summarizing Long Clinical Texts and Longitudinal Patient Trajectories” has been accepted to the AMIA 2025 NLP Annual Symposium!

In this paper, we systematically evaluate several state-of-the-art open-source LLMs, along with their Retrieval-Augmented Generation (RAG) and chain-of-thought (CoT) variants, on long-context clinical summarization and prediction tasks. Our study re-engineers existing EHR-based tasks, such as discharge summarization and diagnosis prediction, to test how well LLMs integrate structured and unstructured patient data over time.

We find that while longer context windows improve input integration, they do not consistently enhance clinical reasoning, particularly in temporal progression and the prediction of rare diseases. This work establishes a foundation for evaluating LLMs on complex, multi-modal clinical data and highlights key challenges in achieving temporally coherent clinical reasoning.

Collaborating Authors:

• Dr. Samantha Stonbraker and Dr. Elizabeth Goldberg – University of Colorado Anschutz Medical Campus
• Dr. Bingsheng Yao and Dr. Dakuo Wang – Northeastern University

Please stop by our poster during the poster session on November 18, 2025, from 5:00 to 6:30 PM.