MLPs naturally prevent data leakage in federated learning without sacrificing model performance.
This paper proves that using MLP-based architectures instead of CNNs can prevent data leakage in vertical federated learning without additional defense mechanisms.
-----
https://arxiv.org/abs/2412.11689
🔒 Original Problem:
→ In Vertical Federated Learning (VFL), feature reconstruction attacks can compromise private data by exploiting model architectures and prior data distribution knowledge.
→ Current defense mechanisms are computationally expensive and don't provide robust protection against sophisticated attacks.
-----
🛠️ Solution in this Paper:
→ The paper introduces a simple yet effective architectural change using MLP-based models instead of CNNs.
→ It proves mathematically that without prior distribution knowledge, feature reconstruction attacks cannot succeed on single-layer client models.
→ The solution leverages orthogonal transformations in client data and weights that preserve training protocol while making reconstruction impossible.
-----
💡 Key Insights:
→ CNN architectures are vulnerable due to their specific matrix structure that makes inverse transformations possible
→ Dense layers provide natural privacy protection by allowing infinite valid reconstructions
→ Prior knowledge of data distribution is crucial for attack success
-----
📊 Results:
→ MLP-based models achieved 98.42% accuracy on MNIST while being completely resistant to UnSplit attacks
→ Feature reconstruction attacks failed completely on MLP architectures while maintaining comparable model performance
→ FID scores show significantly better privacy protection compared to CNN models
Share this post