These labels were automatically added by AI and may be inaccurate. For details, see About Literature Database.
Abstract
Cyber threat intelligence is a critical tool that many organizations and
individuals use to protect themselves from sophisticated, organized,
persistent, and weaponized cyber attacks. However, few studies have focused on
the quality assessment of threat intelligence provided by intelligence
platforms, and this work still requires manual analysis by cybersecurity
experts. In this paper, we propose a knowledge graph-based verifier, a novel
Cyber Threat Intelligence (CTI) quality assessment framework that combines
knowledge graphs and Large Language Models (LLMs). Our approach introduces LLMs
to automatically extract OSCTI key claims to be verified and utilizes a
knowledge graph consisting of paragraphs for fact-checking. This method differs
from the traditional way of constructing complex knowledge graphs with entities
as nodes. By constructing knowledge graphs with paragraphs as nodes and
semantic similarity as edges, it effectively enhances the semantic
understanding ability of the model and simplifies labeling requirements.
Additionally, to fill the gap in the research field, we created and made public
the first dataset for threat intelligence assessment from heterogeneous
sources. To the best of our knowledge, this work is the first to create a
dataset on threat intelligence reliability verification, providing a reference
for future research. Experimental results show that KGV (Knowledge Graph
Verifier) significantly improves the performance of LLMs in intelligence
quality assessment. Compared with traditional methods, we reduce a large amount
of data annotation while the model still exhibits strong reasoning
capabilities. Finally, our method can achieve XXX accuracy in network threat
assessment.