"Hardware and Software Platform Inference"

Hardware and Software Platform Inference (HSPI) detects if cloud providers secretly run your LLM on cheaper GPUs by analyzing output patterns

Like a GPU fingerprint detector, HSPI catches providers who swap premium hardware with cheaper alternatives

https://arxiv.org/abs/2411.05197

🎯 Original Problem:

→ When using third-party LLM inference services, clients cannot verify if providers are actually using the advertised expensive hardware (like NVIDIA H100) or secretly using cheaper alternatives to cut costs.

-----

🔧 Solution in this Paper:

→ The paper introduces Hardware and Software Platform Inference (HSPI), which identifies GPU architecture by analyzing subtle numerical patterns in model outputs.

→ HSPI uses two methods: Border Inputs (HSPI-BI) creates specially crafted inputs that produce different outputs across hardware, while Logits Distributions (HSPI-LD) analyzes probability patterns.

→ The technique exploits how different GPUs and software stacks perform calculations differently, creating unique computational fingerprints.

→ These differences arise from varying arithmetic units, register sizes, and optimization techniques across hardware platforms.

-----

🧠 Key Insights:

→ Different hardware/software configurations create distinct Equivalence Classes with unique computational behaviors

→ Hardware identification is possible through analyzing floating-point arithmetic variations

→ Batch size and model architecture significantly impact detection accuracy

→ The method works better with larger batch sizes but faces memory constraints

-----

📊 Results:

→ White-box setting: 83.9% to 100% accuracy in distinguishing between GPU platforms

→ Black-box setting: Up to 3x better than random guess accuracy

→ Perfect success rate in distinguishing between different quantization levels