"Filter-then-Generate: Large Language Models with Structure-Text Adapter for Knowledge Graph Completion"

LLMs become knowledge graph experts when they only need to choose from pre-filtered candidates.

FtG introduces a filter-then-generate paradigm that helps LLMs perform better at Knowledge Graph Completion by first filtering unlikely entities and then using LLMs to predict from a smaller set.

-----

https://arxiv.org/abs/2412.09094

🤔 Original Problem:

LLMs struggle with Knowledge Graph Completion (KGC) tasks due to massive entity candidate sets and inability to understand graph structures. Even advanced LLMs like ChatGPT perform worse than conventional methods.

-----

🔧 Solution in this Paper:

→ FtG first uses conventional KGC methods to filter out unlikely entities and retain only top-k candidates

→ It transforms KGC into multiple-choice questions for LLMs to handle

→ An ego-graph serialization prompt captures local graph structure around query entities

→ A structure-text adapter maps graph features into text embedding space

→ The system uses LLaMA2-7B with LoRA fine-tuning and RotatE as filtering mechanism

-----

💡 Key Insights:

→ LLMs are better at discriminating between hard samples than handling entire entity sets

→ Structure-aware pruning significantly improves performance by filtering irrelevant neighbors

→ The filter-then-generate approach can enhance existing KGC methods in a plug-and-play manner

-----

📊 Results:

→ Achieved 33.2% Hits@1 improvement on FB15k-237 compared to RotatE

→ Outperformed state-of-the-art methods across three benchmarks

→ Showed 60.7% improvement on CoDEx-M dataset