CSE Undergraduate Student Xuanchi REN Awarded First Runner-up and Best Poster Award in Mr Armin and Mrs Lillian Kitchell Undergraduate Research Award 2021

COSC student Xuanchi REN was awarded First Runner-Up and Best Poster Award with his UROP project "Self-supervised Dance Video Synthesis Conditioned on Music". The project was supervised by Dr. Qifeng CHEN.

The project presented a self-supervised approach with pose perceptual loss for automatic dance video generation. This method can produce a realistic dance video that conforms to the beats and rhymes of given music. To achieve this, Xuanchi and Dr. Chen firstly generate a human skeleton sequence from music and then apply the learned pose-to-appearance mapping to generate the final video. In the stage of generating skeleton sequences, they utilize two discriminators to capture different aspects of the sequence and propose a novel pose perceptual loss to produce natural dances. Besides, they also provide a new cross-modal evaluation to evaluate the dance quality, which is able to estimate the similarity between two modalities (music and dance). Finally, experimental qualitative and quantitative results demonstrate that the dance video synthesis approach produces realistic and diverse results.

The Undergraduate Research Opportunities Program (UROP) is a HKUST signature program designed to provide undergraduate students with an exciting opportunity to engage in academic research, thereby helping them to develop insightful perspectives on their areas of interest and advance the frontiers of knowledge.

For more information, please refer to the announcement of UROP and the presentation video.

Congratulations to Xuanchi and Dr. Chen!

Project presentation by Xuanchi REN

Project presentation by Xuanchi REN

Project poster by Xuanchi REN

Project poster by Xuanchi REN

open icon
Navigation close icon