Abstract

Discharge notes contain complex clinical language that patients often struggle to understand. We investigate the impact of automatic highlighting and multi-step prompting on discharge note simplification using in-context learning with large language models. Highlighting is performed using a domain-specific Cardiology Interface Terminology. We used 28 MIMIC-III discharge notes paired with physician-authored summaries and evaluated four configurations (U1, U2, H1, H2): one-step and two-step simplification applied to either highlighted or unhighlighted notes. In one-step simplification (U1 or H1), the model generated a structured, patient-friendly summary directly from the note. In two-step simplification (U2 or H2), the model first generated a structured summary and then simplified it to a 6th-grade reading level. Manual evaluation showed that completeness improved from U1 to H2 (U1 < H1 < U2 < H2), with H2 achieving the highest completeness (92.45%) and the fewest errors (1) compared to U1 (79.05%, 5 errors). Improvements were statistically significant (p < 0.001), except between H1 and U1. Readability improves across all methods (e.g., FKGL reduced from 11 to 7.7 in U2). LLM-based evaluation using both ChatGPT and Gemini shows strong agreement (ρ = 0.88) and favors H2. This pilot study shows combining highlighting with two-step prompting yields more patient-comprehensible summaries.

IPC Classification

Keywords