Video Understanding and Generation: What's Next?

Speaker:  Dr. Mike Zheng Shou
          Assistant Professor
          National University of Singapore

Title:   "Video Understanding and Generation: What's Next?"

Date:    Tuesday, 30 April 2024

Time:    11:00am - 12 noon

Venue:   Room 4472 (via lift 25/26), HKUST


Abstract:

In this talk, I will introduce our pioneering research explorations in
video intelligence, respectively from understanding and generation, these
two key problems in video intelligence. (1) For understanding, I will
share our innovations that transform video models to be more efficient
(All-in-one, the first unified video-language pretrained model), become
online (VideoLLM-online, can process streaming video in real-time), to
handle egocentric video (EgoVLP, the first egocentric video-language
model) and complete complex tasks (AssistGUI, a multi-agent system that
can automate complicated tasks on computer screen). (2) For generation, I
will share our journey of developing a series of pioneering diffusion-based
video generation models that are efficient (Tune-A-Video, the first
diffusion-based algorithm for efficient video editing), safe (RingID,
diffusion-based watermark for identifying whether a video is generated or not,
and by who), and can generate robust long video (MagicAnimate, which animates
a single user's image according to a long reference video).


*******************
Biography:

Mike Zheng Shou joined National University of Singapore as a tenure-track
Assistant Professor in 2021. His research areas are computer vision and
multimedia, with a focus on video intelligence. He was a research
scientist at Facebook AI in the Bay Area. He obtained his Ph.D. degree at
Columbia University, working with Dean Prof. Shih-Fu Chang. He received
the best paper finalist at CVPR 2022, the best student paper nomination at
CVPR 2017, best paper award at PREMIA 2023. His team won 1st place in the
international challenges including ActivityNet 2017, EPIC-Kitchens 2022,
Ego4D 2022 & 2023. He regularly serves as Area Chair for top-tier
artificial intelligence conferences including CVPR, ECCV, ICCV, ACM MM. He
is a Fellow of National Research Foundation Singapore. He is on the Forbes
30 Under 30 Asia list.