Supporting Pedagogical Decision-Making in AI-Mediated Learning: Instructor-Centered Human-AI Systems and Evaluation Methods

PhD Thesis Proposal Defence


Title: "Supporting Pedagogical Decision-Making in AI-Mediated Learning: 
Instructor-Centered Human-AI Systems and Evaluation Methods"

by

Mr. Zixin CHEN


Abstract:

Generative artificial intelligence is increasingly embedded in everyday
learning and teaching. Students increasingly use large language models (LLMs)
to ask questions, receive explanations, generate ideas, solve problems, and
obtain feedback, while instructors are beginning to use them to create
learning materials and support assessment. These capabilities promise more
responsive and scalable education, but they also complicate pedagogical
decision-making. When AI participates in learning, student work may no longer
fully reveal how students reasoned; model outputs may appear fluent while
failing in domain-specific tasks; instructional materials must be adapted for
diverse learners with limited prior feedback; and assessment must benefit
from automation without surrendering professional judgment. This thesis
investigates how instructor-centered human-AI systems and evaluation methods
can support pedagogical decision-making in AI-mediated learning.

The thesis argues that the key challenge is not to automate educational
tasks, but to preserve and extend instructors' ability to understand
learners, calibrate AI models, adapt instruction, and exercise agency in
consequential teaching workflows. Grounded in visual analytics, model
evaluation, learner simulation, and human-AI workflow design, the thesis
develops this argument through four connected studies. It begins with
StuGPTViz, which treats student-ChatGPT conversations as pedagogical traces
and helps instructors examine cognitive levels, prompting strategies,
model-response quality, and interaction patterns in a data visualization
course. It then turns from learners to models through Misleading ChartQA, a
taxonomy-driven benchmark that evaluates multimodal large language models
(MLLMs) on misleading chart question answering and reveals their capabilities
and limitations in visualization literacy. Building on this evidence-oriented
perspective, VisQStudio supports iterative design of visualization-literacy
multiple-choice questions by combining MLLM-assisted question generation,
configurable simulated student profiles, and visual analytics of simulated
reasoning, helping instructors anticipate misconceptions and refine materials
before classroom deployment. Finally, CoGrader extends instructor-centered AI
support to project-report assessment, integrating LLM assistance for metric
co-design, benchmark-driven regrading, report comparison, and feedback
generation while keeping instructors in control of assessment criteria,
revisions, and final feedback.

Together, these contributions advance an instructor-centered approach to
AI-mediated learning. They show that effective educational AI requires not
only capable models, but also systems and evaluation methods organized around
the evidence and judgments instructors need: understanding learner thinking,
calibrating model competence, adapting instructional materials, and assessing
complex student work fairly and accountably. By connecting learner-process
visualization, model capability benchmarking, simulation-informed design, and
instructor-controlled assessment, this thesis contributes practical systems
and design knowledge for preserving instructor agency in AI-mediated learning
environments.


Date:                   Thursday, 11 June 2026

Time:                   3:00pm - 5:00pm

Venue:                  Room 3494
                        Lift 25/26

Committee Members:      Prof. Huamin Qu (Supervisor)
                        Dr. Anyi Rao (Chairperson, AMC)
                        Dr. Arpit Narechania