A Survey on Explainable AI: From Interpreting Models to Driving Model Improvements

PhD Qualifying Examination


Title: "A Survey on Explainable AI: From Interpreting Models to Driving Model 
Improvements"

by

Miss Luyu QIU


Abstract:

The rapid advancement of Artificial Intelligence (AI) has had transformative 
effects across a wide array of fields. However, the lack of transparency and 
interpretability in AI models has created significant obstacles, particularly 
in applications where trust, accountability, and regulatory compliance are 
essential. This has led to the emergence of Explainable AI (XAI), a field 
dedicated to enhancing the transparency of AI systems and making their 
decision-making processes understandable to human users. This survey provides 
a comprehensive overview of XAI, detailing its background, importance, and 
categorizing key approaches to XAI methods. We analyze classic methods and 
discuss their strengths and limitations, highlighting the need for solutions 
that balance interpretability with predictive accuracy.

To demonstrate the depth of my research in the field of XAI, this survey will 
discuss one of my published papers, which addresses the challenge of 
Out-of-Distribution (OoD) data in perturbation-based XAI algorithms. 
Perturbation-based methods are promising for interpreting black-box models 
due to their simplicity and effectiveness, yet they often suffer from 
unreliable explanations when the perturbed data diverges from the original 
dataset distribution, an issue that has received limited attention.We address 
this OoD problem by introducing a novel module that measures the consistency 
between perturbed samples and the original dataset distribution. Our approach 
penalizes the influence of unreliable OoD data, enhancing the robustness of 
explanations by integrating inlier scores with target model predictions. This 
method significantly improves the reliability of popular perturbation-based 
XAI algorithms, such as RISE, OCCLUSION, and LIME, and notably mitigates the 
performance degradation issue observed in RISE. Extensive experiments confirm 
the effectiveness of our solution, yielding superior results on computational 
and cognitive metrics and introducing a novel faithfulness indicator that 
demonstrates robustness against the OoD problem.

Moreover, this survey highlights my ongoing research on explaining the 
behavior of large language models (LLMs). Specifically, we delve into 
understanding why LLMs underperform in multiplication tasks. By leveraging 
explainable AI (XAI) methods, we uncover the underlying causes of this issue 
and propose improvements to the Transformer model, achieving enhanced 
performance.

Finally, the survey concludes with a summary of key insights and future 
research directions, providing a comprehensive overview of the field and 
potential pathways for advancement.


Date:                   Wednesday, 18 December 2024

Time:                   4:00pm - 6:00pm

Venue:                  Room 2128A
                        Lift 19

Committee Members:      Prof. Lei Chen (Supervisor)
                        Prof. Qiong Luo (Chairperson)
                        Dr. Xiaomin Ouyang
                        Prof. Qian Zhang