Researcher Hugo Vergnes successfully pretrained a 3.8 billion parameter language model from scratch for just $998. The model was trained on 65 billion tokens over 43 hours to achieve a score of 0.384 CORE, demonstrating a highly cost-effective approach to AI development.