Improving the usability of static security analysis

Speaker:        Dr. Omer Tripp
                IBM T.J. Watson Research Center

Title:          "Improving the usability of static security analysis"

Date:           Tuesday, 18 November 2014

Time:           3:00pm - 4:00pm

Venue:          Room 3598 (via lifts 27/28), HKUST

Abstract:

The scale and complexity of modern software systems complicate manual
security auditing. Automated analysis tools are gradually becoming a
necessity. Specifically, static security analyses carry the promise of
efficiently verifying large code bases. Yet, a critical usability barrier,
hindering the adoption of static security analysis by developers, is the
excess of false reports. Current tools do not offer the user any direct
means of customizing or cleansing the report. The user is thus left to
review hundreds, if not thousands, of potential warnings, and classify
them as either actionable or spurious. This is both burdensome and error
prone, leaving developers disenchanted by static security checkers. In
this talk, I will address this challenge by proposing a general technique
to refine the output of static security checkers. The key idea is to apply
statistical learning to the warnings output by the analysis based on user
feedback on a small set of warnings. This leads to an interactive
solution, whereby the user classifies a small fragment of the issues
reported by the analysis, and the learning algorithm then classifies the
remaining warnings automatically. An important aspect of this solution is
that it is user centric. The user can express different classification
policies, ranging from strong bias toward elimination of false warnings to
strong bias toward preservation of true warnings, which the filtering
system then executes.

I will present throughout the talk experimental data that validates this
user-centric approach. The experiments were done atop nearly 4,000
client-side JavaScript benchmarks, extracted from 675 popular Web sites,
that were manually annotated with vulnerabilities. The results show strong
promise. As an example, based only on 200 classified warnings, and with a
policy biased toward preservation of true warnings, a learning approach
can boost precision by a threefold factor (×2.868) while reducing recall
by a negligible factor (×1.006). Other policies are enforced with a
similarly high level of efficacy.

*****************
Biography:

Omer Tripp is a research staff member at the IBM T.J. Watson Research
Center in NY. Omer's research interests include language-based security,
program analysis and applications thereof (in particular, security and
concurrency), mobile technologies, as well as hybrid analysis methods that
feature statistical analysis, machine learning, rich user specifications
and other complementary techniques. Omer received his PhD from Tel Aviv
University under the supervision of Prof. Mooly Sagiv, and holds a BSc in
Computer Science from the Hebrew University.