Pixels and Patches, Bigger, Faster and Better: PatchTable and Image Perforation

Speaker:        Prof. Connelly Barnes
                University of Virginia

Title:          "Pixels and Patches, Bigger, Faster and Better:
                 PatchTable and Image Perforation"

Date:           Wednesday, 15 June 2016

Time:           2:30pm - 4:00pm

Venue:          Room 5506 (via lifts 25/26), HKUST

Abstract:

I will present two projects related to making patch-based image synthesis
scalable and fast, and automatically optimizing image pipelines.

The first paper is called "PatchTable: Efficient Patch Queries for Large 
Datasets and Applications." It was presented at ACM SIGGRAPH 2015. This 
paper presents a data structure that reduces approximate nearest neighbor 
query times for image patches in large datasets. Our new algorithm, 
PatchTable, offloads as much of the computation as possible to a 
pre-computation stage that takes modest time, so patch queries can be as 
efficient as possible. The algorithm is based on a locality sensitive 
hashing scheme. We show experimentally that this accelerates the patch 
query operation by up to 9x over k-coherence, up to 12x over TreeCANN, and 
up to 200x over PatchMatch. Our fast algorithm allows us to explore 
efficient and practical imaging and computational photography applications. 
We show results for artistic video stylization, light field 
super-resolution, and multi-image inpainting.

The second paper is called "Image Perforation: Automatically Accelerating 
Image Pipelines by Intelligently Skipping Samples." It will be presented at 
SIGGRAPH 2016. It presents a new optimization technique that can be used to 
accelerate image pipelines by automatically trading off between performance 
and accuracy. Image perforation works by transforming loops over the image 
at each pipeline stage into coarser loops that effectively ``skip'' certain 
samples. These missing samples are reconstructed for later stages using a 
number of different interpolation strategies that are relatively 
inexpensive to perform compared to the original cost of computing the 
sample. For the applications we investigated, image perforation achieves 
speedups of 2x-10x with acceptable loss in visual quality.


*****************
Biography:

Connelly Barnes is an Assistant Professor of Computer Science at the 
University of Virginia. He received a Ph.D. in computer science from 
Princeton University in 2011. His group develops techniques for efficiently 
manipulating visual data in computer graphics by using semantic information 
from computer vision. Applications are in computational photography, image 
editing, art, and hiding visual information. Many computer graphics 
algorithms are more useful if they are interactive, therefore, his group 
also focuses on efficiency and optimization, including some compiler 
technologies.