"HWPQ: Hessian-free Weight Pruning-Quantization For LLM Compression And Acceleration"

Below podcast on this paper is generated with Google's Illuminate.

https://arxiv.org/abs/2501.16376

The paper addresses the challenge of deploying LLMs on resource-limited devices due to the high computational cost of existing compression methods like pruning and quantization. Current methods using Hessian matrices are computationally expensive, hindering efficient LLM deployment.

This paper introduces Hessian-free Weight Pruning-Quantization (HWPQ). HWPQ avoids computationally intensive Hessian matrix calculations. It uses a contribution-based weight metric and Exponentially Weighted Moving Average (EWMA) to speed up compression while preserving accuracy.

-----

📌 HWPQ bypasses computationally expensive Hessian matrix. It uses EWMA and contribution metric for efficient LLM compression. This achieves near Hessian quality at O(n) complexity.

📌 EWMA adaptation in HWPQ dynamically assesses weight importance. This eliminates sorting and accelerates pruning and quantization. It offers a practical speedup for large language model compression.

📌 HWPQ's FP8 quantization and 2:4 sparsity are hardware-aware. This design optimizes for Tensor Cores. It enables efficient, dequantization-free inference on modern GPUs.

----------

Methods Explored in this Paper 🔧:

→ The paper introduces a Hessian-free Weight Pruning-Quantization (HWPQ) method. This method avoids the computationally expensive Hessian matrix.

→ HWPQ uses a contribution-oriented weight metric. This metric assesses weight importance without second-order derivatives.

→ The method employs Exponentially Weighted Moving Average (EWMA). EWMA replaces weight sorting to further reduce complexity.

→ HWPQ calculates a contribution metric L for each weight. L is based on the weight's value and the sum of squared input values.

→ Weights with lower L values are considered less important. These weights are pruned or quantized to FP8 precision.

→ The EWMA technique dynamically estimates the mean and deviation of L values. This helps in identifying weights for pruning or quantization without sorting.

→ The approach is extended to support 2:4 structured sparsity. This is beneficial for hardware accelerators.

-----

Key Insights 💡:

→ Identifying weights with minimal loss contribution is more about relative importance than absolute values.

→ Hessian matrix computation can be bypassed by using a contribution-oriented weight metric derived from loss values.

→ EWMA can effectively replace sorting for weight selection in pruning and quantization, reducing time complexity.

→ FP8 quantization and 2:4 structured sparsity can significantly improve inference speed without substantial accuracy loss.

-----

Results 📊:

→ Achieves average speedups of 5.97× in quantization time and 12.29× in pruning time compared to state-of-the-art methods. Quantization is 4.88× faster than AutoGPTQ, 2.82× faster than AutoAWQ, and 10.21× faster than SpQR. Pruning is 43.75× faster than SparseGPT and 12.29× faster than Wanda.

→ Achieves 1.50× inference speedup on Attention layers and 1.60× on MLP layers in LLaMA2-7B.

→ Maintains 99.4% of baseline model performance at 20% pruning ratio and 91.57% at 50% pruning ratio on LLaMA2-7B.

→ Reduces dequantization overhead by over 80%.