Yukai Zhao

I am Yukai Zhao, a Ph.D. student at Zhejiang University (ZJU), advised by Prof. Xing Hu and Prof. Xin Xia.

My research interests are mainly in software engineering related fields such as Fuzzing, AI4SE and SE4AI.

News

  • [26/9/25] Our paper about LLM testing is accepted by ASE 2025.

  • [20/8/25] Our paper about SBOM Tool is accepted by TOSEM.

Publications

  • [ASE'25] HFuzzer: Testing Large Language Models for Package Hallucinations via Phrase-based Fuzzing
    Yukai ZHao, Menghan Wu, Xing Hu, Xin Xia
    Abstract Large Language Models (LLMs) are widely used for code generation, but they face critical security risks when applied to practical production due to package hallucinations, in which LLMs recommend non-existent packages. These hallucinations can be exploited in software supply chain attacks, where malicious attackers exploit them to register harmful packages. It is critical to test LLMs for package hallucinations to mitigate package hallucinations and defend against potential attacks. Although researchers have proposed testing frameworks for fact-conflicting hallucinations in natural language generation, there is a lack of research on package hallucinations. To fill this gap, we propose HFUZZER, a novel phrase-based fuzzing framework to test LLMs for package hallucinations. HFUZZER adopts fuzzing technology and guides the model to infer a wider range of reasonable information based on phrases, thereby generating enough and diverse coding tasks. Furthermore, HFUZZER extracts phrases from package information or coding tasks to ensure the relevance of phrases and code, thereby improving the relevance of generated tasks and code.
    💻 Code
  • [TOSEM'25] More Than Meets the Eye: On Evaluating SBOM Tools In Java
    Menghan Wu, Yukai ZHao, Xing Hu, Xian Zhan, Shanping Li, Xin Xia.
    Abstract Open-source software is widely used in current software development. Unfortunately, this integration introduces a spectrum of challenges and potential threats. Such challenges emerge due to the diversity of import scenarios, which in turn may introduce malicious or vulnerable code in the client software, thereby causing significant security risks. To improve the transparency of software supply chains, Software Bill of Materials (SBOM) tools are proposed to identify the components within software systems. However, there limit investigation of functionality (i.e., tool operational process and data fields) and their practical performance of SBOM tools across various import scenarios. In this paper, we perform a comprehensive empirical study to investigate the impact of different import scenarios on SBOM tools. Specifically, we focus on three distinct component import scenarios: Build Tool Import, Dynamic Loading, and Source Code Import across a new benchmark consisting of 152 projects. We find that (1) The detection capabilities of SBOM tools exhibit considerable variance, especially in identifying dependency relationships; (2) The effectiveness of SBOM tools within the import scenarios of Dynamic Loading and Source Code Import falls short of expectations. Based on our findings, we summarize the lessons learned from different perspectives, including practitioners, tool vendors, and researchers. Our study provides valuable insights into the intricate landscape of software component usage, contributing to enhancing SBOM tools in modern software development.
    📄 PDF | 💻 Code