"Code-mixed LLM: Improve Large Language Models' Capability to Handle Code-Mixing through Reinforcement Learning from AI Feedback"

LLMs struggle with code-mixed languages, where multiple languages blend in conversations.

This research proposes using AI feedback to improve LLMs' ability to handle such mixed-language scenarios.

-----

https://arxiv.org/abs/2411.09073

Original Problem 🤔:

Code-mixing, prevalent in multilingual societies with 20% of online content, poses unique challenges like syntactic mismatches and semantic blending. Current LLMs, while good at individual languages, lack specific training for mixed-language scenarios.

-----

Solution in this Paper 🛠️:

→ The paper introduces Reinforcement Learning from AI Feedback (RLAIF) for code-mixing scenarios

→ First step involves supervised fine-tuning using parallel corpus with prompt templates

→ Next, they collect preference data from existing datasets and use advanced LLMs for preference annotation

→ Finally, they train a reward model and optimize the LLM through policy optimization

-----

Key Insights from this Paper 💡:

→ Fine-tuned models outperform prompt-based approaches for code-mixed tasks

→ AI feedback can replace costly human feedback in model alignment

→ Code-mixed machine translation serves as an effective base task for improvement

-----

Results 📊:

→ Higher win rates achieved by RLAIF-trained models (57.71%) compared to baseline (42.29%)

→ BLEU score improved from 7.86 to 9.50 after RLAIF implementation

→ Fine-tuned models showed superior performance over prompt-based LLMs in sentiment analysis