"What Features in Prompts Jailbreak LLMs? Investigating the Mechanisms Behind Attacks"

Study proves jailbreaks exploit unique nonlinear features, not universal patterns

Nonlinear probes reveal hidden mechanisms behind LLM jailbreak attacks.

https://arxiv.org/abs/2411.03343

🎯 Original Problem:

Understanding how jailbreak attacks work in LLMs remains a mystery. Previous works used only linear methods to analyze jailbreaks, leaving gaps in comprehending the underlying mechanisms that make these attacks successful.

-----

🔧 Solution in this Paper:

→ Created a dataset of 10,800 jailbreak attempts using 35 different attack methods on 300 harmful prompts

→ Used both linear and nonlinear (MLP) probes to analyze latent representations of prompt tokens in Gemma-7B-IT

→ Trained probes to predict whether a jailbreak attempt would succeed based only on model's internal representations

→ Developed a mechanistic jailbreaking method using nonlinear probe to produce latent-space adversarial attacks

-----

💡 Key Insights:

→ Different jailbreaking methods work through distinct nonlinear features in prompts, not universal or linear features alone

→ Probes can accurately identify successful jailbreaks within known attack methods but fail to transfer to new attack types

→ Nonlinear features in prompts are causally responsible for successful jailbreaks

→ Different attack methods exploit distinct mechanisms rather than using universal features

-----

📊 Results:

→ Linear probes achieved 93% accuracy on known attack types

→ MLP probes achieved 87% accuracy on known attack types

→ MLP probe-guided attacks achieved 74% success rate compared to 26% for linear probe-guided attacks

→ Both probe types showed poor transfer performance to unseen attacks, often performing barely better than random guessing