Practical edge-assisted mobile computing: in the case of augmented reality and virtual reality

PhD Thesis Proposal Defence


Title: "Practical edge-assisted mobile computing: in the case of 
augmented reality and virtual reality"

by

Mr. Wenxiao ZHANG


Abstract:

Smartphone is now a necessity of people's daily life, and we are enjoying 
various services on it with numerous mobile applications. However, the 
resource and communication limitations of a single mobile device make it 
insufficient in satisfying the real-time and interactive constraints of 
some computation intensive applications, such as mobile Augmented Reality 
and mobile Virtual Reality. To bridge the gap, we utilize the processing 
power of edge servers via task offloading and build practical mobile 
systems which significantly outperforms state-of-the-art mobile systems in 
terms of scalability, battery life, latency, quality of experience (QoE), 
etc..

In the case of mobile Augmented Reality, large-scale object recognition is 
an essential but time consuming task. To offload the object recognition 
task and enhance the system performance, we explore how the GPU and the 
multi-core architecture on the edge servers would accelerate the 
large-scale object recognition process. With the carefully designed 
offloading pipeline and edge acceleration, we are able to finish the whole 
AR pipeline within one frame interval (33 ms for a 30 fps camera system) 
while maintaining high recognition accuracy with large-scale datasets.

Despite the performance concern on the mobile devices, edge servers are 
also faced with scalability issues, as too many concurrent offloading 
requests would exhaust the processing capacity of the edge and result in 
delayed execution with tasks queuing on the edge. To address this 
scalability issue, we propose a two-tier architecture for mobile AR: 1) 
when there is enough processing capacity remaining on the edge, the mobile 
client would offload the recognition task to achieve the best performance, 
and 2) when the edge is heavily occupied, the mobile client would derive 
its own result based on the previous recognition result from nearby 
device's cache, which is based on the fact that users in vicinity have a 
high chance of querying the same physical objects. This two-tier 
architecture not only guarantees the system performance when serving 
massive concurrent users, but also enables innovative features such as 
multi-player AR.

In the case of mobile Virtual Reality, existing 360 video streaming 
systems are suffering from insufficient pixel density, as the video 
resolution falling within the user's field of view (FoV) is relatively 
low. We utilize the edge server for ultra-high resolution video 
transcoding and implement a system which streams tile-based viewport 
adaptive 360 videos onto the mobile clients. With this edge proxy, we 
successfully achieve 16k 360 video streaming onto off-the-shelf 
smartphones, achieving high quality frames and fluent 30 fps playback 
without overwhelming the processing capacity of the smartphones.


Date:			Friday, 26 April 2019

Time:                  	2:00pm - 4:00pm

Venue:                  Room 5566
                         (lifts 27/28)

Committee Members:	Dr. Pan Hui (Supervisor)
 			Prof. James Kwok (Chairperson)
 			Prof. Gary Chan
 			Dr. Pedro Sander


**** ALL are Welcome ****