"LLMs for Cryptocurrency Transaction Analysis: A Bitcoin Case Study"
Below podcast on this paper is generated with Google's Illuminate.
https://arxiv.org/abs/2501.18158
The challenge in cryptocurrency analysis lies in the opacity and inflexibility of current black-box models, hindering effective understanding of transaction behaviors. This paper addresses this by investigating the potential of LLMs for cryptocurrency transaction graph analysis.
This paper proposes a three-tiered framework to evaluate LLMs' analytical capabilities on Bitcoin transaction graphs. The framework uses a new graph representation format and a sampling algorithm to enhance LLM input.
-----
📌 LLM4TG representation effectively transforms complex graph structures into a parseable text format. This allows direct utilization of LLMs for graph analytics without heavy preprocessing.
📌 CETraS algorithm intelligently samples graphs by prioritizing connectivity and node importance. It enables analysis of larger, more complex transaction graphs within LLM token limits.
📌 The three-level evaluation framework systematically benchmarks LLM capabilities in graph understanding. It reveals specific strengths and limitations across different analytical tasks on transaction graphs.
----------
Methods Explored in this Paper 🔧:
→ This paper introduces a three-tiered framework to evaluate LLMs in cryptocurrency transaction graph analysis. The levels are foundational metrics, characteristic overview, and contextual interpretation.
→ A new human-readable graph representation format called LLM4TG is proposed. LLM4TG minimizes redundancy and token usage while keeping data integrity. It organizes nodes in layers by type and integrates edge details within nodes.
→ To handle large graphs within LLM token limits, the Connectivity-Enhanced Transaction Graph Sampling (CETraS) algorithm is introduced. CETraS condenses graphs by removing less important nodes based on a node importance metric, while preserving key connections.
-----
Key Insights 💡:
→ LLMs demonstrate strong capabilities in understanding foundational graph metrics. They accurately extract node-level information like degrees and transaction values.
→ LLMs, especially GPT-4o, effectively provide characteristic overviews of transaction graphs. They identify key features and patterns relevant for analysis.
→ LLMs show promise in contextual interpretation for tasks like address classification, even with limited labeled examples.
-----
Results 📊:
→ In foundational metrics evaluation, LLMs achieved 98.50% to 100.00% accuracy in node-level metric identification. Global metric accuracy ranged from 24.00% to 58.00%.
→ For characteristic overview, GPT-4o achieved meaningful response rates of 95%, compared to 70% for GPT-4.
→ In raw graph-based classification, GPT-4o achieved 50.49% overall accuracy, significantly outperforming GPT-4's 18.81%.
→ GPT-4o in raw graph classification reached precision of 63.33%, recall of 53.95%, and F1 score of 50.14%.