Automatically Extracting Bug-Reproducing Test Cases

MPhil Thesis Defence


Title: "Automatically Extracting Bug-Reproducing Test Cases"

By

Mr. Jaewoo SONG


Abstract

Bug-Reproducing Test cases (BRTs) are essential for developers to 
understand and fix bugs. BRTs are also necessary in various software 
engineering research areas such as fault localization, automated program 
repair and crash reproduction. However, it is extremely tedious and time 
consuming to collect them manually. Consequently, most of the research 
results in those areas are derived from small numbers of (8 to 119) bugs.

In this paper I propose an automatic approach of extracting BRTs from 
software repositories. My approach automatically finds, downloads and 
builds meaningful revisions. Then my algorithm automatically finds BRTs by 
using my definition of bugs and patches. Found BRTs are packaged with 
related data into readily usable forms. Using my approach, I collected 
1610 real-world BRTs from 891 bugs of 4 open source projects. I manually 
verified the collected bugs, and 72% of them were found to be real bugs. 
To further evaluate the usefulness of collected BRTs, I ran a program 
repair algorithm (GenProg) on them. A large number of BRTs allowed 
thorough analysis of the algorithm. I could identify a weakness of the 
algorithm, which had been presumed but not been able to be proven before 
because only a small number of BRTs had been available: many of its 
patches (71.4%) were nonsensical. Moreover, it was found that bugs were 
concentrated: top 19% of test classes revealed 50% of total bugs, 44% of 
runtime errors was solely due to null pointer exception, and 91% of 
runtime errors belonged to 8 error types. I believe my BRT collection and 
BRT collecting algorithm will benefit software engineering researchers in 
various areas, and allow them to run large-scale experiments.


Date:			Monday, 18 August 2014

Time:			2:00pm - 4:00pm

Venue:			Room 3501
 			Lifts 25/26

Committee Members:	Dr. Sunghun Kim (Supervisor)
 			Prof. Shing-Chi Cheung (Chairperson)
 			Dr. Qiong Luo


**** ALL are Welcome ****