Synoptic reporting by summarizing cancer pathology reports using large language models

Abstract

Synoptic reporting, the documenting of clinical information in a structured manner, enhances patient care by improving accuracy, readability, and report completeness, but imposes significant administrative burdens on physicians. The potential of Large Language Models (LLMs) for automating synoptic reporting remains underexplored. In this study, we explore state-of-the-art LLMs for automatic synoptic reporting, using 7774 pathology reports from 8 cancer types, paired with physician annotated synoptic reports from the Mayo Clinic EHR. We developed a comprehensive automation framework, combining state-of-the-art LLMs, incorporating parameter-efficient optimization, scalable prompt templates, and robust evaluation strategies. We validate our results on both internal and external data, ensuring alignment with pathologist responses. Using our framework, fine-tuned LLAMA-2 achieved BERT F1 scores above 0.86 across all data elements and exceeding 0.94 over 50% (11 of 22) of the data elements, translating to manually assessed mean semantic accuracies of 77% and up to 81% for short clinical reports.