CROSS-MATCHING BIG ASTRONOMIC CATALOGS ON HETEROGENEOUS CLUSTERS

PhD Thesis Proposal Defence


Title: "CROSS-MATCHING BIG ASTRONOMIC CATALOGS ON HETEROGENEOUS CLUSTERS"

by

Miss Xiaoying JIA


Abstract:

In astronomy, cross-match is a central operation to integrate multi-wavelength 
information by identifying celestial objects across multiple catalogs. With the 
rapid increase in data volume from space and ground-based surveys, it becomes 
crucial to process large astronomic catalogs efficiently. In this thesis 
proposal, we study how to accelerate the cross-match of billion-record catalogs 
on a cluster of computers with both CPUs and GPUs. Two critical factors are 
discussed in this proposal: (1) the choice of a suitable indexing method that 
supports efficient operations on GPU; (2) cross-match algorithms with design 
choices and optimizations targeting to the multi-node cluster environment. We 
present two cross-match algorithms, namely IB-CM and MASJ-CM, both of which 
work as follows: First, the positional cross-matching objects from astronomic 
catalogs is essentially a spatial distance join on two sets of points. Second, 
the query circle for each reference point overlaps a small set of cells under a 
partitioning scheme. Specifically, IB-CM follows a filter-and-refine approach 
to directly filter out most unlikely sample points, which fall out of the 
overlapping cells. MASJ-CM performs the cross-match by further replicating 
reference candidate objects for each sample object for matching. Our 
evaluations show that: (1) HEALPix was the best indexing method for cross-match 
tasks; (2) IB-CM outperformed MASJ-CM for cross-matching small scale catalogs 
on a single node, whereas MASJ-CM won on billion-record catalogs on a 
multi-node cluster; (3) self-match of a billion-record catalogs was completed 
under 4 minutes with MASJ-CM on a six-node cluster.


Date:			Thursday, 27 April 2017

Time:                  	10:00am - 12:00noon

Venue:                  Room 1505
                         (lifts 25/26)

Committee Members:	Dr. Qiong Luo (Supervisor)
  			Dr. Wei Wang (Chairperson)
  			Prof. Lei Chen
 			Dr. Ke Yi


**** ALL are Welcome ****