Jing Chen

Yuqi Zhu, Xiaohan Wang, Jing Chen et al.

This paper presents an exhaustive quantitative and qualitative evaluation of Large Language Models (LLMs) for Knowledge Graph (KG) construction and reasoning. We engage in experiments across eight diverse datasets, focusing on four representative tasks encompassing entity and relation extraction, event extraction, link prediction, and question-answering, thereby thoroughly exploring LLMs' performance in the domain of construction and inference. Empirically, our findings suggest that LLMs, represented by GPT-4, are more suited as inference assistants rather than few-shot information extractors. Specifically, while GPT-4 exhibits good performance in tasks related to KG construction, it excels further in reasoning tasks, surpassing fine-tuned models in certain cases. Moreover, our investigation extends to the potential generalization ability of LLMs for information extraction, leading to the proposition of a Virtual Knowledge Extraction task and the development of the corresponding VINE dataset. Based on these empirical findings, we further propose AutoKG, a multi-agent-based approach employing LLMs and external sources for KG construction and reasoning. We anticipate that this research can provide invaluable insights for future undertakings in the field of knowledge graphs. The code and datasets are in https://github.com/zjunlp/AutoKG.

Chenyang Lyu, Shouling Ji, Yuwei Li et al.

Fuzzing is an automated application vulnerability detection method. For genetic algorithm-based fuzzing, it can mutate the seed files provided by users to obtain a number of inputs, which are then used to test the objective application in order to trigger potential crashes. As shown in existing literature, the seed file selection is crucial for the efficiency of fuzzing. However, current seed selection strategies do not seem to be better than randomly picking seed files. Therefore, in this paper, we propose a novel and generic system, named SmartSeed, to generate seed files towards efficient fuzzing. Specifically, SmartSeed is designed based on a machine learning model to learn and generate high-value binary seeds. We evaluate SmartSeed along with American Fuzzy Lop (AFL) on 12 open-source applications with the input formats of mp3, bmp or flv. We also combine SmartSeed with different fuzzing tools to examine its compatibility. From extensive experiments, we find that SmartSeed has the following advantages: First, it only requires tens of seconds to generate sufficient high-value seeds. Second, it can generate seeds with multiple kinds of input formats and significantly improves the fuzzing performance for most applications with the same input format. Third, SmartSeed is compatible to different fuzzing tools. In total, our system discovers more than twice unique crashes and 5,040 extra unique paths than the existing best seed selection strategy for the evaluated 12 applications. From the crashes found by SmartSeed, we discover 16 new vulnerabilities and have received their CVE IDs.