AI catches hyperrealistic fake news by spotting tiny glitches humans miss in images and text.
📚 https://arxiv.org/abs/2410.09045
Original Problem 🔍:
AI-generated fake news with hyperrealistic images poses a significant threat to information integrity. Existing datasets fail to accurately represent the challenges posed by advanced diffusion models in generating realistic fake news content.
-----
Solution in this Paper 🛠️:
• MiRAGeNews Dataset: 12,500 high-quality real and AI-generated image-caption pairs
🤖 How does the MiRAGe detector work and what makes it unique?
MiRAGe is a multimodal detector that fuses an image detector (MiRAGe-Img) and a text detector (MiRAGe-Txt). Both detectors are ensembles of a black-box linear model and an interpretable concept bottleneck model.
MiRAGe-Img uses an Object-Class Concept Bottleneck Model to detect regional anomalies in images, while MiRAGe-Txt uses a Text Bottleneck Model to extract distinguishing concept features from captions.
This combination allows MiRAGe to capture both global and local features in images and text, improving its detection capabilities and out-of-distribution robustness.
-----
Key Insights from this Paper 💡:
• Humans and state-of-the-art multimodal LLMs struggle to detect AI-generated news content
• Combining global and local features in images and text improves detection capabilities
• Out-of-domain robustness is crucial for real-world applicability of fake news detectors
• Interpretable concept-based approaches enhance detector performance
-----
Results 📊:
• MiRAGe outperforms humans (60% F-1) and state-of-the-art MLLMs (<24% F-1)
• Achieves over 98% F-1 score on in-domain data
• 85% F-1 score on out-of-domain data from unseen news publishers and image generators
• Improves by +5.1% F-1 over state-of-the-art baselines on out-of-domain image-caption pairs
Share this post