"Can Graph Neural Networks Learn Language with Extremely Weak Text Supervision?"

Making graph networks bilingual: Teaching math structures to understand text.

This paper Proposes Morpher, a multi-modal prompt learning approach that adapts pre-trained Graph Neural Networks to downstream tasks using minimal text supervision.

-----

https://arxiv.org/abs/2412.08174

🤖 Original Problem:

→ Current graph-text alignment methods struggle with scarce graph data and weak text supervision, making it challenging to build transferable Graph Neural Networks (GNNs) that understand semantic meaning.

-----

🔧 Solution in this Paper:

→ Morpher introduces a novel multi-modal prompt learning framework that keeps both pre-trained GNN and LLM frozen while aligning their representations.

→ It improves existing graph prompt methods by balancing cross-connections between prompt and input graph to prevent feature overwhelming.

→ The framework simultaneously learns both graph prompts and text prompts to adapt GNN representations to LLM's semantic space.

→ Uses contrastive learning to align the graph embeddings with language embeddings without fine-tuning either model.

-----

💡 Key Insights:

→ Dense cross-connections between prompt and input graph can overwhelm original graph features

→ Multi-modal prompting provides better adaptation than single-modal approaches

→ Graph-text alignment can enable zero-shot generalization to novel classes

→ Balancing cross-connections is crucial for stable optimization

-----

📊 Results:

→ Achieves 4-7% accuracy improvement over baselines across datasets

→ Requires only 0.032-0.46% of parameters compared to fine-tuning

→ Successfully generalizes to unseen graph classes through semantic alignment

→ Outperforms existing methods on few-shot, multi-task and cross-domain settings