Time series becomes just another language in ChatTime's vocabulary, numbers become words, enabling zero-shot predictions.
ChatTime treats time series data as a foreign language, enabling seamless processing of both numerical and text data through LLM vocabulary expansion.
-----
https://arxiv.org/abs/2412.11376v1
🤔 Original Problem:
→ Current time series models either lack zero-shot capabilities or can't handle textual data effectively
→ Training foundation models from scratch is computationally expensive and inefficient
→ Existing methods struggle to process both time series and text simultaneously
-----
🔧 Solution in this Paper:
→ ChatTime converts continuous time series into discrete "foreign words" through normalization and binning
→ It expands LLaMA-2's vocabulary to include these time series tokens
→ The model uses mark characters "###" to efficiently encode time series values
→ Two-phase training combines continuous pre-training on time series and instruction fine-tuning
→ Supports zero-shot forecasting and bidirectional translation between time series and text
-----
💡 Key Insights:
→ Time series can be treated as a foreign language for LLMs
→ Vocabulary expansion is more efficient than training from scratch
→ Adding mark characters reduces token consumption significantly
→ Text-time series translation improves forecasting accuracy
-----
📊 Results:
→ Achieves 99.9% accuracy of SOTA using only 4% training data
→ Matches 90.9% accuracy of full-shot forecasting methods
→ Outperforms generic LLMs in time series tasks by 20%
→ Reduces training tokens from 25B to 1B
------
Are you into AI and LLMs❓ Join my daily AI newsletter. I will send you 7 emails a week analyzing the highest signal AI developments. ↓↓
🎉 https://rohanpaul.substack.com/
Share this post