"MarketGPT: Developing a Pre-trained transformer (GPT) for Modeling Financial Time Series"

GPT meets Wall Street: A transformer that speaks the language of stock markets

A transformer-based model that generates realistic financial market order flow, capturing complex market dynamics without explicitly training for them.

-----

https://arxiv.org/abs/2411.16585

🎯 Original Problem:

→ Traditional financial market simulations struggle to replicate real market behavior and microstructure features like limit order books.

-----

🔧 Solution in this Paper:

→ The paper introduces MarketGPT, a transformer model trained on NASDAQ market data to generate order messages.

→ It uses a 768-dimension embedding with 12 layers and 12 heads, totaling 100M parameters.

→ The model processes order messages through tokenization, converting 18 pre-processed elements into 24 tokens.

→ It employs attention sinks to handle long sequences beyond context windows.

→ The system integrates with a discrete event simulator to validate and execute generated orders.

-----

💡 Key Insights:

→ Pre-training on multiple stocks improves model performance

→ Attention sinks enable stable generation beyond initial context

→ Temperature scaling above 1.0 helps capture rare market events

→ Error correction prevents invalid order generation

-----

📊 Results:

→ Successfully reproduces key market properties like heavy-tailed returns and volatility clustering

→ Maintains realistic order flow generation with 93% valid messages

→ Accurately models inter-arrival times and order size distributions

→ Captures complex market dynamics without explicit training