"An Engorgio Prompt Makes Large Language Model Babble on"

This paper introduces "Engorgio," a novel attack method that makes LLMs generate abnormally long outputs by suppressing their end-of-sequence tokens, potentially disrupting service availability.

-----

https://arxiv.org/abs/2412.19394v1

🤖 Original Problem:

LLM inference costs are becoming a major concern for service providers. While existing attacks target encoder-decoder models, modern decoder-only LLMs lack effective methods to exploit their inference costs.

-----

🔧 Solution in this Paper:

→ Engorgio uses a parameterized distribution to track LLM prediction trajectories during text generation

→ It employs two key losses: EOS escape loss to suppress end-of-sequence tokens, and self-mentor loss for stable generation

→ The method uses Gumbel-Softmax for effective gradient utilization and token selection

→ Implementation involves a two-stage process: generation stage for optimizing distribution matrix, and testing stage for final prompt selection

-----

🎯 Key Insights:

→ Modern LLMs are vulnerable to inference cost exploitation through carefully crafted prompts

→ End-of-sequence token suppression is key to extending output length

→ Even a small number of malicious users can significantly impact service availability

-----

📊 Results:

→ Achieved 90%+ of maximum length limit on base models vs 0-40% for normal queries

→ Generated outputs 2-13x longer than normal across 13 LLMs

→ Demonstrated effectiveness on both base and fine-tuned models

→ Real-world testing showed significant impact on service throughput

Hook Statement Options (Technical):

Engorgio exploits LLM token prediction to force endless text generation, challenging service providers.

A simple prompt trick makes LLMs talk forever, causing massive computational overhead.

By suppressing the stop signal, Engorgio makes LLMs babble until they hit length limits.

This attack turns chatty LLMs into unstoppable talking machines, draining server resources.

Hook Statement Options (Informal):

Ever wondered how to make an AI assistant talk until it runs out of breath?

It's like removing the period key from an AI's keyboard - it just keeps going and going.

When AI assistants forget how to stop talking, servers start sweating.

Think of it as giving an AI too much coffee - it won't stop chatting!