This paper proposes compact phrase representations for LLM rewriting, improving efficiency and accuracy in tasks like ASR post-editing.
-----
Paper - https://arxiv.org/abs/2501.13831
Original Problem 🤖:
→ LLMs excel at rewriting tasks but are computationally expensive.
→ Existing compact representations like edit spans sacrifice accuracy for efficiency.
-----
Solution in this Paper 💡:
→ This paper introduces two new compact phrase representations: phrase pair and target-phrase-only.
→ Phrase pair representation uses source-target phrase pairs, similar to phrase-based machine translation.
→ Target-phrase-only uses only the target phrase with surrounding context words.
→ These representations are less ambiguous and lead to more fluent outputs compared to span representations.
→ These are applied to ASR post-editing where an LLM corrects the output of a fast ASR model.
-----
Key Insights from this Paper 🤔:
→ Phrase representations offer a better efficiency-accuracy trade-off compared to span representations.
→ Target-phrase-only with a dilation span of 3 achieves the best balance.
→ Left context is more helpful than right context in decoding.
-----
Results ✨:
→ Target-phrase-only closes 54-57% of the WER gap between span and full rewrite models.
→ It loses only 12.5-22.2% of the length reduction rate compared to span representations.
→ Phrase pair and target only (k=3) representations achieve near 100% recovery rate.
