Abstract

Background/Objectives: Secondary hypertension (SH) requires complex diagnostic reasoning and guideline-based management, posing substantial challenges for artificial intelligence–driven clinical decision-support systems. This study aimed to comparatively evaluate the performance of three large language models (LLMs) in diagnostic reasoning, clinical management, follow-up planning, and patient-oriented communication in SH. Methods: In this cross-sectional blinded study, three LLMs—GPT-5.2 (OpenAI), Claude Sonnet 4.6 (Anthropic), and Gemini 3.0 Pro (Google)—were evaluated using 10 expert-developed clinical case vignettes representing major etiologies of SH. Model outputs were anonymized and independently assessed by three senior clinicians (two endocrinologists and one cardiologist) using a 7-point Likert scale across five domains: (1) diagnostic accuracy and hallucination control, (2) quality and comprehensiveness, (3) reliability and clinical guidance, (4) efficiency of diagnostic workup, and (5) clinical usability. Group differences were analyzed using Kruskal–Wallis tests with Bonferroni-corrected pairwise comparisons. Inter-rater agreement was assessed using two-way mixed-effects intraclass correlation coefficients with absolute agreement. Results: A total of 90 blinded expert evaluations were analyzed. GPT-5.2 (6.0, Q1–Q3 5.40–6.05) and Gemini 3 Pro (5.2, Q1–Q3 4.55–6.20) (H = 40.055, p < 0.001). The results indicated a clear performance hierarchy, with Claude Sonnet 4.6 receiving the highest overall scores, followed by GPT-5.2 and Gemini 3 Pro. Pairwise analyses showed higher scores for Claude Sonnet 4.6 than the other models in most domains, while efficiency of diagnostic workup showed smaller between-model differences. GPT-5.2 generally showed intermediate performance, with higher ratings than Gemini 3 Pro in reliability and clinical usability. Performance differences were most pronounced in domains requiring complex clinical reasoning, whereas efficiency of diagnostic workup scores was relatively comparable among models. Claude Sonnet 4.6 ranked first in nine of the ten clinical vignettes. Inter-rater agreement analyses demonstrated consistent ranking patterns among evaluators. Conclusions: These exploratory findings suggest heterogeneous and model-dependent performance of LLMs in secondary hypertension–related clinical tasks. A clear clinician-rated performance hierarchy was observed, with differences most apparent in domains requiring complex clinical reasoning. However, given the pilot vignette-based design and limited sample size, these results should be interpreted as hypothesis-generating and require confirmation in larger, multicenter validation studies before routine clinical implementation can be considered.

IPC Classification

Keywords