Automated and Effective Metamorphic Testing: Relation Discovery, Deduction, and Generation.

The Hong Kong University of Science and Technology
Department of Computer Science and Engineering


PhD Thesis Defence


Title: "Automated and Effective Metamorphic Testing: Relation Discovery, 
Deduction, and Generation."

By

Mr. Congying XU


Abstract:

Software testing is a fundamental process for ensuring the reliability and 
correctness of software systems. Metamorphic Testing (MT), as a powerful 
technique, has been applied in diverse domains to address the oracle 
problem—a fundamental problem in software testing. However, its broader 
adoption remains limited due to the difficulty of constructing effective 
metamorphic relations (MRs), which requires domain-specific expertise. This 
thesis addresses this challenge by proposing automated approaches for deriving 
effective MRs.

This thesis makes three main contributions. First, it introduces a novel
direction of discovering and synthesizing MRs from existing tests. Building
on the observation that developer-written test cases often embed domain
knowledge that encodes MRs, we propose MR-Scout, an approach that
automatically discovers and synthesizes MRs encoded in existing test cases.
Using this approach, we discovered over 11,000 MR-encoded test cases from
701 open-source projects.

Second, we focus on deducing input relations from output relations and
example input-output pairs. While MR-Scout reveals that thousands of test
cases can encode MRs, over 70% of them lack explicit input relations, which
hinders their applicability. To address this, we propose MR-Adopt, which
leverages large language models to generate input transformation functions
that complement these incomplete MRs. Our evaluation shows that MR-Adopt
successfully generates valid input transformations for 72% of incomplete
MRs.

Finally, we explore generating MRs directly from a target program via
functional coupling analysis. Although MR-Scout and MR-Adopt effectively
derive MRs from existing tests, such MR-encoded tests are relatively few in
number, accounting for only 1% of test cases. To overcome this limitation,
we propose MR-Coupler, an automated and domain-agnostic approach that
generates metamorphic test cases via functional coupling analysis. The key
insight is that functional coupling between methods, which is readily
available in program code, can be formulated as MRs. This approach
successfully generated concrete metamorphic test cases for over 90% of
target programs, and the generated test cases successfully revealed 22
real-world bugs.

In summary, this thesis advances automated metamorphic testing through
relation discovery, deduction, and generation. These contributions advance
the practical and effective adoption of MT across diverse software systems.


Date:                   Friday, 27 February 2026

Time:                   4:30pm - 6:30pm

Venue:                  Room 2132C
                        Lift 22

Chairman:               Prof. Jianan Qu (ECE)

Committee Members:      Prof. Shing-Chi Cheung (Supervisor)
                        Dr. Lionel Parreaux
                        Prof. Raymond Wong
                        Dr. Shenghui Song (ECE)
                        Prof. David Lo (SMU)