Learning Perception and Control for Robot Intelligence

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Learning Perception and Control for Robot Intelligence"

By

Mr. Siyi LI


Abstract

Autonomous robots that can assist humans in the daily unstructured world 
have been a long standing vision of robotics and artificial intelligence 
(AI). Such autonomous intelligent robotic system requires two essential 
building blocks: perception and control. Meanwhile, the past few years 
have seen major advances in many perception and control tasks empowered by 
deep learning and reinforcement learning methods. Hence one natural 
question to ask is how AI techniques could help to accomplish those 
robotic tasks. In this thesis, we explore learning-based solutions to 
robotic tasks.

Our first attempt is constructing a unified benchmark for visual object 
tracking on the unmanned aerial vehicle (UAV) platform. We manually built 
a drone tracking dataset, consisting of a variety of videos with high 
diversity captured by drone cameras. We performed an extensive empirical 
study of the state-of-the-art methods on the dataset and identified their 
major weakness in the motion model. We also devised new motion models by 
explicitly estimating the camera motion in the tracking phase, which are 
especially suitable and effective for the drone tracking scenario.

Collecting real-world data with robotic systems is generally expensive due 
to the hardware cost and the manual labeling effort. However, deep 
learning and reinforcement learning methods require a data-hungry training 
paradigm. We proposed to address this issue by learning from synthetic 
data while minimizing the gap from simulation to reality at the same time. 
For robotic perception task, we investigated instance segmentation for 
robot manipulation. We developed an automated rendering pipeline to 
generate a variety of photorealistic synthetic images with pixel-level 
labels. The synthetic dataset is then used to train an objectness deep 
neural network model which can successfully generalize to real-world 
manipulation scenarios. For robotic control task, we focused on the 
challenging problem of learning UAV control for actively tracking a moving 
target. We proposed a hierarchical approach that combines model-free 
reinforcement learning methods with conventional feedback controllers to 
enable efficient and safe exploration in the training phase. We showed 
that this hierarchical control scheme can learn a target following policy 
in a simulator efficiently and the learned behavior can be successfully 
transferred to real-world quadrotor control.


Date:			Monday, 29 April 2019

Time:			3:00pm - 5:00pm

Venue:			Room 2408
 			Lifts 17/18

Chairman:		Prof. Yongsheng Gao (MAE)

Committee Members:	Prof. Dit-Yan Yeung (Supervisor)
 			Prof. Chi-Keung Tang
 			Prof. Nevin Zhang
 			Prof. Richard So (IEDA)
 			Prof. Anthoni Chan (CityU)


**** ALL are Welcome ****