"Does Representation Matter? Exploring Intermediate Layers in Large Language Models"

LLMs' hidden layers outshine the final layer in downstream tasks.

This paper explores the quality of intermediate representations in LLMs, revealing their superior performance in downstream tasks compared to final layers.

https://arxiv.org/abs/2412.09563

🤔 Original Problem:

Most studies focus on final-layer representations in LLMs, overlooking the potential of intermediate layers.

-----

🔍 Solution in this Paper:

→ The researchers investigate representation quality across different layers of LLMs, including Transformers and State Space Models (SSMs).

→ They adapt and apply metrics like prompt entropy, curvature, and augmentation-invariance to quantify representation quality.

→ The study examines how representations evolve throughout training and how factors like input randomness and prompt length affect each layer.

→ They analyze architectural differences between Transformers and SSMs in terms of representation quality.

-----

💡 Key Insights from this Paper:

→ Intermediate layers consistently provide better representations for downstream tasks than final layers

→ Transformers show more pronounced changes in representation quality across layers compared to SSMs

→ A negative correlation exists between prompt entropy and downstream performance

→ A bimodal distribution of entropy values appears in intermediate layers of Transformer models, but not in SSMs

-----

📊 Results:

→ Intermediate layers outperformed final layers across all tested architectures

→ LLM2Vec 8B: 66.8% best layer performance vs 64.7% last layer performance

→ Pythia 410M: 53.3% best layer performance vs 49.8% last layer performance

→ At least 2% improvement in average accuracy using intermediate layers