"On Creating A Brain-To-Text Decoder"

Real-time mind-reading: Brain waves to words, decoding inner speech with EEG.

This paper introduces a novel brain-to-text decoder using raw EEG signals, aiming for faster, real-time brain activity decoding. It explores the impact of vocabulary size, electrode density, and training data size on performance.

-----

https://arxiv.org/abs/2501.06326

Original Problem 🤔:

→ Current brain decoding methods, relying on fMRI, are slow, expensive, and not real-time.

→ Existing EEG-based methods often use processed features, not raw EEG signals, limiting their real-time applicability.

-----

Solution in this Paper 💡:

→ A novel brain-to-text decoder uses raw EEG signals as input.

→ It leverages a transformer encoder pre-trained with Data2Vec or Wav2Vec2 for feature extraction.

→ BENDR and EEG-Conformer are integrated for enhanced EEG encoding.

→ CTC loss is incorporated to handle variable-length input sequences.

→ A fully connected layer and a projection layer are added to predict character probabilities.

-----

Key Insights from this Paper 🔑:

→ Raw EEG signals can be directly used for brain-to-text decoding.

→ Pre-trained models like Data2Vec and Wav2Vec2 can effectively extract features from EEG data.

→ Specialized EEG encoders like BENDR and EEG-Conformer further improve performance.

→ CTC loss is crucial for handling the variable-length nature of EEG data.

-----

Results 💯:

→ Achieved competitive word error rates on Librispeech benchmark using limited labelled data.

→ Surpassed previous state-of-the-art techniques while using significantly fewer labels.

→ Analyzed error patterns in voice recognition and the influence of model size and unlabelled training data.