Multimodal Large Language Models for Medical Image Interpretation

PhD Qualifying Examination


Title: "Multimodal Large Language Models for Medical Image Interpretation"

by

Mr. Sunan HE


Abstract:

Multimodal large language models (MLLMs) are transforming medical image
interpretation by integrating visual perception with language, clinical context
and domain knowledge. Their emergence marks a shift from task-specific
prediction towards generalist, interactive systems that can address diverse
clinical questions and extend their capabilities through external knowledge and
computational tools. However, translating these capabilities into reliable
interpretation requires models to preserve clinically relevant visual
information, connect their responses to imaging evidence and generalize across
clinical settings. This survey critically examines the development of MLLMs
across medical imaging modalities and clinical tasks, organizing the literature
around four dimensions: visual representation and multimodal alignment,
adaptation to medical domains and tasks, inference-time evidence integration and
agentic workflows, and evaluation of clinical reliability and utility. It
compares how parameter updates, contextual learning, retrieval and tool use
contribute to model specialization, and examines the transition from individual
model responses to workflows involving planning, tool execution and iterative
verification. Particular attention is given to weak image-text correspondence,
the loss of spatial and temporal information, unreliable visual grounding and
error propagation in multistep systems. The discussion distinguishes gains in
benchmark performance and linguistic fluency from evidence of faithful
interpretation and clinical benefit. Finally, the survey identifies open
challenges and research directions for developing multimodal and agentic systems
that support verifiable, generalizable and clinically useful medical image
interpretation.


Date:                   Thursday, 3 December 2026

Time:                   10:00am - 11:00am

Venue:                  Room 4472
                        Lifts 25/26

Committee Members:      Dr. Hao Chen (Supervisor)
                        Dr. Long Chen (Chairperson)
                        Dr. Yinghao Xu