LLM-Based Differential Diagnosis of Neurodegenerative Diseases from MRI-Derived Brain Atrophy Reports
Abstract
Differential diagnosis of neurodegenerative diseases is challenging due to the overlap and heterogeneity of brain atrophy patterns observed on structural MRI. Although supervised deep-learning classifiers have shown promising performance, they require large annotated datasets and remain sensitive to domain shift. Vision-Language Models (VLMs) offer a promising paradigm for zero-shot inference by leveraging representations learned from large-scale multimodal pretraining; however, current VLMs struggle with brain MRI interpretation, where diagnosis requires distinguishing subtle, distributed disease-related atrophy patterns from age-related changes. To overcome these challenges, we propose a modular framework that reframes differential diagnosis as reasoning over structured descriptions of brain atrophy rather than raw MRI scans. The framework combines automated brain parcellation and lifespan normative modeling to quantify deviations from expected aging trajectories and converts them into qualitative atrophy reports aligned with neurodegenerative terminology. These reports are then processed by Large Language Models (LLMs), which leverage their pretrained knowledge to perform zero-shot diagnostic reasoning, generating human-readable rationales and a ranked diagnostic hypothesis. Evaluated on a five-class neurodegenerative benchmark, the proposed zero-shot framework outperforms direct MRI-based VLM inference and achieves performance comparable to supervised MRI-based classifiers.