Sanitizer Check Debloating with Reinforcement Learning

MPhil Thesis Defence


Title: "Sanitizer Check Debloating with Reinforcement Learning"

By

Mr. Kun Hung LUNG


Abstract

Sanitizers detect unsafe actions such as invalid memory accesses by 
inserting checks that are validated during a program’s execution. Despite 
their extensive use for vulnerability discovery, sanitizer checks often 
induce a high runtime cost, thus impeding its adoption in real-world 
scenarios. One important observation for the high cost is that many 
sanitizer checks are checking low security sensitivity code repeatedly — 
leading to unnecessarily wasted computing resources.

To help more profitably utilize sanitizer checks, we introduce DESAN, an 
effective and general approach to debloating sanitizer checks. Given a 
program with sanitizer checks fully enabled, DESAN progressively trains a 
reinforcement learning model to gradually identify an optimal sanitizer 
check debloating scheme, such that shaving each check notably reduces the 
program runtime cost, while retaining reasonably high vulnerability 
detectability. The contribution of each sanitizer check’s runtime cost can 
be identified via profiling. Nevertheless, to benchmark the vulnerability 
detectability of each sanitizer check, we conduct a hybrid analysis by 
first estimating a static security contribution score of each sanitizer 
check derived from existing metrics. We then fine-tune the score during 
the debloating process according to sanitizer check likely equality 
relations. Our evaluation on the SPEC benchmarks shows that DESAN can 
reduce the overhead of sanitizers significantly, from 76% to 26% for 
AddressSanitizer, and from 143% to 71% for Undefined- BehaviorSanitizer. 
Our further evaluation on 34 CVEs from 10 commonly-used programs shows 
that DESAN-reduced checks suffice to detect all 34 CVEs.


Date:  			Thursday, 29 July 2021

Time:			2:00pm - 4:00pm

Zoom meeting: 
https://hkust.zoom.us/j/99078043202?pwd=ZUc5SjgvSWdNNlQ4OWRuSzU5NUIrUT09

Committee Members:	Dr. Shuai Wang (Supervisor)
 			Dr. Dimitris Papadopoulos (Chairperson)
 			Dr. Lionel Parreaux


**** ALL are Welcome ****