Satoshi 7B is a Bitcoin-focused LLM I worked on when I was at Spirit of Satoshi.
It was based on Mistral 7B Instruct, fine-tuned with LoRA and later aligned with DPO, then released on Hugging Face.
My work on it included:
- curation and cleaning of training data
- evals
- continued pre-training
- LoRA fine-tuning
- DPO alignment
- data augmentation
The goal was simple: a model that answered Bitcoin questions from a Bitcoin-native frame instead of falling back to generic internet priors.