Beyond Testing: Distribution-Aware Software Debugging
Date:
Abstract: In this talk, I will provide an overview of our work on employing input distribution to expose and diagnose software failures. The goal is to automatically explain or fix the root causes of failures, such as bugs/crashes, fairness violations and security backdoors.
Firstly, I will present a distribution-aware approach for exposing and reproducing software failures (i.e., crashes/bugs). This automated approach employs the distribution of input features to reveal faults in software systems (e.g., compilers, interpreters). Then, I will demonstrate how this technique is adapted to reveal and fix fairness violations and security backdoors in state-of-the-art AI-based software (i.e., CV and NLP systems). Next, I will present our ongoing (SE4LLM) work on exposing and diagnosing similar failures (unfair behaviours, inconsistencies, knowledge gaps and backdoors) in LLMs. Finally, I will conclude with our plans to address challenges in applying LLMs for software engineering tasks (LLM4SE).