Towards Emulating Interactions for Effective Software Testing

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Towards Emulating Interactions for Effective Software Testing"

By

Mr. Hengcheng ZHU


Abstract:

With the rapid advancement of software technologies, the complexity of
software systems continues to grow. Modern software systems frequently
interact with numerous components or external systems, making the effective
emulation of such sophisticated interactions a fundamental challenge in
software testing. This thesis tackles this challenge by investigating the
emulation of interactions at two key testing levels: unit testing and system
testing.

At the unit testing level, interaction emulation is achieved through test
doubles, which isolate the component under test from its dependencies. The
most common role is the stub, which simulates the behavior of substituted
classes. Developers use stubs to simulate interactions that are complicated
to set up or difficult to reproduce in automated testing. The emulated
behavior is encoded in stub code, typically as a sequence of API calls to a
mocking framework. However, developing and maintaining stub code is
labor-intensive and error-prone. Therefore, we present StubCoder, a
technique that automatically generates and repairs stub code for regression
testing. StubCoder uses an evolutionary algorithm guided by the runtime
behavior of test cases. Our evaluation shows that StubCoder effectively
generates stub code for incomplete test cases and repairs obsolete test
cases.

In addition to stubs, mocks are another widely used form of test double,
configured with expectations about their interactions with the SUT. Mocks
record method calls and enable developers to verify anticipated interactions
through mock assertions. Mock assertions validate program behaviors that are
unobservable to test assertions, yet are rarely considered by automated test
generation techniques. Effective generation of mock assertions requires
understanding their practical usage. To bridge this gap, we conducted the
first empirical study of mock assertions, examining their adoption patterns,
characteristics of verified method invocations, and effectiveness in fault
detection. Our analysis of 4,652 test cases from 11 popular Java projects
reveals that mock assertions are primarily applied to validate method calls
involving external resources or reflecting code path traversal. We find that
mock assertions complement traditional test assertions by ensuring desired
side effects, validating control flow, and checking internal computation
results. Our findings provide a foundation for future research such as
automated test generation that supports mock assertions.

While unit testing focuses on isolating and emulating interactions at the
component level, system testing requires emulating interactions between the
system under test and external systems. A prominent example is code
intelligence tools, where the interaction between code editors and language
servers is central. At the core is the Language Server Protocol (LSP), which
standardizes communication to provide code intelligence features. The
reliability of LSP servers is a growing concern, as crashes can disrupt all
code intelligence features, and vulnerabilities can put developers at risk
when editing untrusted code. Despite widespread adoption, no existing
techniques specifically target LSP server testing. To bridge this gap, we
present LSPFuzz, a grey-box hybrid fuzzer that emulates code editor
behaviors for systematic LSP server testing. Effective LSP server testing
requires holistic mutation of source code and editor operations, as bugs
often manifest from their combinations. We employ a two-stage mutation
pipeline: syntax-aware mutations to source code, followed by context-aware
dispatching of editor operations. We evaluated LSPFuzz on four widely used
LSP servers. LSPFuzz demonstrated superior performance compared to baseline
fuzzers, and uncovered previously unknown bugs in real-world LSP servers.
Our work advances the quality assurance of LSP servers and provides
foundational insights for future research.

In summary, this thesis addresses the multifaceted challenges of emulating
interactions in software testing across both unit and system levels. These
contributions not only provide practical solutions for developers but also
lay the groundwork for future research in interaction emulation for
effective software testing. As software systems continue to grow in
complexity, the approaches and insights presented in this thesis will be
instrumental in ensuring their correctness and robustness.


Date:                   Wednesday, 12 November 2025

Time:                   2:30pm - 4:30pm

Venue:                  Room 5501
                        Lifts 25/26

Chairman:               Dr. Yiwen WANG (ECE)

Committee Members:      Prof. Shing-Chi CHEUNG (Supervisor)
                        Dr. Jiasi SHEN
                        Dr. Shuai WANG
                        Prof. Jianfeng CAI (MATH)
                        Prof. Mauro PEZZÈ (USI Università della Svizzera italiana)