Lip Reading Project Supervised by Dr. Brian MAK Awarded Winner in IET PATW Global Final 2019

The lip reading project, Helen - An Accessibility Device for AI-based Lip Reading was created by CSE UG students Padmanabhan KRISHNAMURTHY and Amrutavarsh Sanganabasappa KINAGI under the supervision of Dr. Brian Mak. Presented by Padmanabhan, the project was awarded Winner at the Present Around The World (PATW) Global Final organized by The Institution of Engineering and Technology (IET) on 24 October 2019 at London, United Kingdom.

Their project, Helen, is an accessibility device that uses deep learning to perform automatic lip reading. Helen consists of a camera connected to a Raspberry Pi that streams video wirelessly to a remote computer running the LipNet: a deep learning model that achieves high accuracy on automatic lip reading. It is now designed to be worn around one's neck and the software workflow has been optimized to ensure that the entire process from recording video to obtaining annotations takes no more than 10 seconds.

Helen was previously also awarded winner of the HKUST President's Cup 2019.

Congratulations again to Padmanabhan, Amrutavarsh Sanganabasappa and Dr. Mak!

For more information, please refer to the announcement of IET, the presentation video, and the news release by HKUST.

Creators of the project, Amrutavarsh Sanganabasappa KINAGI and Padmanabhan KRISHNAMURTHY

Padmanabhan KRISHNAMURTHY's winning announcement presented by IET PATW Global Final 2019