COMP 6511B: Advanced Software Testing (Spring 2024) 

Lecture Details

Instructor: Dongdong She (dongdong@cse.ust.hk)
Office hours: Tuesday (1:30-2:30 pm) Room 3505
TA: Xunguang Wang (xwanghm@cse.ust.hk)
TA Office hours: TBD
Classroom: Room 2503
Class hours: Monday and Wednesday (12:00-1:20 pm)

Course Description

Software vulnerabilities profoundly impact our daily lives, from global ransomware attacks to various sensitive information leakage. Software testing is a program analysis technique to discover these vulnerabilities. This course will cover classic software techniques such as fuzzing, symbolic execution, and formal methods. The latest trend of leveraging machine learning (i.e., LLM, ChatGPT) to assist software testing and neural-symbolic software testing are also included.

Course Goal

The general goal of this course is to help you gain a solid understanding of software testing techniques such as fuzzing, symbolic execution, and formal methods. You can also learn the most recent works of software testing techniques and ML-based software testing.

Course Format and Student Workload

This course will center around paper readings, presentations, and discussions; and a final project. The course readings include a list of research papers selected from top-tier security, software engineering and systems conferences. We will discuss roughly one or two papers every class meeting. For the in-depth discussions to be possible, you will have to read the papers carefully before class.

You have three main responsibilities in the course:

Prerequisite

COMP 3633 Principles of Cybersecurity, COMP 4211 Machine Learning (optional), or equivalents of these two courses.  

Grading 

• Class participation - 40% 
• In-class discussion - 20%
• Paper presentation - 20%
• Final project - 60% 
• Project proposal - 15%
• Midterm demo - 15%
• Final report - 30%

Schedule

Advice

• Reading 

Read this article about how to read a research paper. The take-home message is that until you can answer a bunch of questions, you are not done reading a paper. William lists a number of important questions. I would add two: 1) What are the re-usable principles/tricks/algorithms presented in this paper? 2) What is the (authors') insight that drives the research? A system research paper often has a bunch of novel tricks. I believe the more such things you have in your toolbox, the more likely you can come up with an elegant/novel system design.

• Presentation 

Some advice on how to give a good presentation:

• Be very selective about the talk contents. It's almost always the case that you have more stuff than your audience can understand within a short amount of time. You should thus be very selective about what to include in your talk. What is the important thing about your proposal? What is neat, unusual, interesting to a listener? Figure it out, and say it in the talk, more than once. Do not try to include everything in your talk.
• Repeat the key points. Don't expect your listener to always follow your talk. It's a good idea to repeat and highlight the key points several times, for example, once at the beginning, once when you actually present them, and once at the end. Make sure your listener won't miss the most important stuff of your talk.
• Use an outline. A good way to keep your audience with you is to use an outline slide to describe the structure of your talk. I typically present an outline slide after the introduction of a talk. Then, as I go from one section to another, I may show the outline slide again, to let the audience know where we are.
• Get the timing right. Each content slide typically takes 1-3 minutes. Thus, for a 10 minutes talk, do not have more than 6-7 slides of real content! Note the title slide and outline slides do not count in this total because they take little time to present.
• Use visual aids, but do not abuse them. Pictures, animations are sometimes very handy at explaining complex ideas. However, use them only on the most important stuff; otherwise, they'll distract your listener.
• Above all else, practice, practice, practice. Practicing is the real key to give a good talk. I find it much more useful to practice aloud than to murmur to myself. If you can, try to give the talk in front of other people. Practicing is certainly the only way to get timing right.

Some online advice from others:

Read this paper about things to avoid when giving a talk
Read this paper about how to give a good conference talk. Many ideas apply to the mini-talk you'll give.

• Writing 
Some suggested readings (to make you a better writer):


Read this paper about how to write a technical paper. Many ideas apply to writing proposals as well.
Read this paper about how to write sentences, paragraphs, etc.
And, of course, read Strunk and White. Many times.