Consistent Visual Editing in the Diffusion Era: Escalating from 2D Images to 3D Space and Temporal Dynamics

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Consistent Visual Editing in the Diffusion Era: Escalating from 2D
Images to 3D Space and Temporal Dynamics"

By

Mr. Qingyan BAI


Abstract:

Visual editing often requires one requested change to remain coherent across
related photographs, unseen viewpoints, or video frames. Diffusion models enable
flexible appearance changes, but independently edited outputs can disagree on
details that an instruction leaves unspecified. This dissertation investigates
how information about the relationships between observations can improve
consistency while retaining editing flexibility. First, we introduce Edicho, a
training-free method that separates correspondence estimation from appearance
synthesis. Matches estimated from source images guide self-attention and
classifier-free guidance, coordinating local and global edits across image sets.
Next, we present 3DPE, a real-time portrait editor that combines geometric
supervision from a portrait reconstructor with editing supervision from a
diffusion model. Its feedforward model predicts an edited 3D representation for
novel-view rendering, supports image and text prompts, and permits adaptation to
new styles. For video editing, we develop Ditto, a synthetic-data pipeline
combining image-edited references, in-context video generation, temporal
enhancement, and vision-language models for instruction generation and
filtering. Editto learns from the resulting paired videos through a modality
curriculum that gradually removes the edited reference frame while retaining the
source video and instruction. Experiments across the three settings show
improved consistency alongside instruction or reference alignment within the
evaluated tasks. Together, these studies connect correspondence-guided
inference, shared 3D representations, and learning from paired sequences as
complementary approaches to consistent editing across images, space, and time.


Date:                   Wednesday, 18 November 2026

Time:                   10:00am - 12:00am

Venue:                  Room 5501
                        Lifts 25/26

Chairman:               Prof. Liming ZHANG (CIVL)

Committee Members:      Dr. Qifeng CHEN (Supervisor)
                        Prof. Long QUAN
                        Dr. Yinghao XU
                        Dr. Wenhan LUO (AMC)
                        Dr. Kede MA (CityU)