What is SAC-KG
SAC-KG is a general framework for automatically constructing domain knowledge graphs based on large language models (LLMs). It consists of three main components: a generator, a verifier, and a pruner. It can automatically generate first-level knowledge graphs for a specific domain from a raw domain corpus, ensuring the accuracy of the generated triples. SAC-KG can construct domain knowledge graphs at a scale of over one million nodes, achieving 89.32% precision, which is more than 20% higher than existing state-of-the-art methods. This framework effectively uses LLMs as domain experts to generate professional and accurate multi-level knowledge graphs.
Main functions of SAC-KG
- Automated knowledge graph construction: Through its components, SAC-KG automatically constructs single-level knowledge graphs for a specific domain from a raw domain corpus and can iteratively construct multi-level knowledge graphs.
- Improved construction precision: The verifier and pruner of SAC-KG work together to ensure accuracy by correcting generation errors and determining whether newly generated tails require iteration for the next-level knowledge graph. In experiments, precision reached 89.32%, more than 20% higher than existing methods.
- Domain specialization: Based on large language models (LLMs) as domain experts, the knowledge graphs generated by SAC-KG are highly specialized and can generate triples related to a specific domain.
- Control of the generation process: By introducing an open knowledge retriever and a pruner, SAC-KG can effectively control the generation process, ensuring that the generated triples are correctly formatted and meet domain requirements.
- Large-scale construction capability: SAC-KG can automatically construct domain knowledge graphs at a scale of more than one million nodes, demonstrating its advantage in handling large-scale datasets.
- Unsupervised method: SAC-KG is an unsupervised method that can be applied to any domain with a large amount of unstructured text corpus without requiring labeled data.
- Consistency evaluation: Through consistency comparisons with GPT-4 and human evaluation, the high quality and reliability of the knowledge graphs generated by SAC-KG were verified.
Technical principles of SAC-KG
- Generator: The generator is responsible for retrieving relevant information from the raw domain corpus and open-source knowledge graphs, providing it as input to large language models (LLMs), and generating a first-level knowledge graph for a specific domain. It includes two submodules:
- Domain corpus retriever: Splits the domain corpus into sentences, sorts them according to the frequency of entity occurrence, and finally concatenates the sorted sentences into a fixed-length text input to the LLMs.
- Open knowledge retriever: For entities in the open-source knowledge graph, provides related triples as examples; for entities not in the open-source knowledge graph, tokenizes them and retrieves again; for entities that still do not match, randomly selects ten triples as prompts.
- Verifier: The role of the verifier is to detect and filter out erroneous triples generated by the generator. It includes two steps:
- Error detection: Uses more than 7,000 rule standards mined from RuleHub to perform quantity checks, format checks, and conflict checks on the generated triples.
- Error correction: Based on the detected error type, provide the corresponding prompt and re-prompt the LLMs to generate the correct triple.
- Pruner: The pruner works in coordination with the verifier to ensure accuracy by determining whether the newly generated tail needs to iterate to the next-level knowledge graph, enhancing the controllability of the constructed knowledge graph. Based on a T5 binary classifier model fine-tuned on the open-source knowledge graph DBpedia, the input consists of the tail entity of each correct triple, and the output is "growing" or "pruned," indicating whether the entity should continue generating the next-level knowledge graph or stop further generation.
SAC-KG project address
- arXiv technical paper：https://arxiv.org/html/2410.02811v1
SAC-KG application scenarios
- Professional domain knowledge graph construction: SAC-KG can be applied in multiple domains such as medicine, biology, and social networks, helping to construct knowledge graphs for specific domains.
- Improved automation and accuracy: Through an automated construction process, SAC-KG improves the degree of automation and precision of knowledge graph construction, achieving an accuracy of 89.32%, which is more than 20% higher than existing state-of-the-art methods.
- Domain specialization: SAC-KG uses large language models (LLMs) as domain experts to generate triples related to specific domains, giving the knowledge graph a high degree of specialization.
- Large-scale data processing: SAC-KG can automatically construct domain knowledge graphs at a scale of over one million nodes, demonstrating its advantage in processing large-scale datasets.