Detection-Driven Reinforcement Learning to Act in Visual 3D Environment

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering

Final Year Thesis Oral Defense

Title: "Detection-Driven Reinforcement Learning to Act in Visual 3D 
        Environment"

by

GUO Shaopeng

Abstract:

We propose to empower artificial agents using deep reinforcement learning 
with visual object detection ability to learn how to act in a challenging 
3D environment. In 3D games, our Detection Driven Reinforcement Learning 
(DDRL) successfully makes the agents capable of recognizing enemies and 
hazards and thus excel in the required challenging task. This is in 
contrast to previous work in 2D and 3D games where mostly an end-to-end 
approach was taken. Although some methods considered concatenated video 
frames, no adequate 3D semantics (specically moving foreground objects) 
was utilized in training and testing. Our DDRL is particularly relevant in 
challenging 3D environment such as VizDoom (to simulate military combat) 
and Carla (autonomous driving), where life-or-death actions must be made 
in a split second based on what the agent has exactly seen in a previously 
unseen 3D environment during testing time. The DDRL consists of two deep 
networks, one for object detection and the other uses the deep recurrent 
asynchronous advantage actor-critic (A3C) or deep recurrent Q-network 
(DRQN) methods for reinforcement learning. We test DDRL in these 3D game 
platforms and in particular compare with the VizDoom 2017 winning entry.


Date            : 24 April 2018 (Tuesday)

Time            : 15:20 - 16:00

Venue           : Room 1505 (near lifts 25/26), HKUST

Advisor         : Prof. TANG Chi-Keung

2nd Reader      : Prof. CHUNG Albert Chi-Shing