The development of AI models capable of reasoning and coding has historically required significant financial resources and proprietary technology. However, researchers from the NovaSky team at UC Berkeley have introduced Sky-T1-32B-Preview, an open-source reasoning model trained for less than $450, demonstrating that advanced AI development can be both accessible and cost-effective.
Overview: Breaking Barriers in AI Reasoning
Sky-T1-32B-Preview competes with industry-leading models like o1-preview on reasoning and coding benchmarks. Unlike proprietary models that limit community engagement, Sky-T1 is fully open-source, providing transparency and tools for academic and open-source innovators.
Key Resources Provided:
- Infrastructure for data generation, training, and evaluation
- 17,000 curated training data points
- Model weights and a detailed technical report
- Open-source code and logs to replicate results
Data Curation for High-Quality Reasoning
To achieve strong performance, Sky-T1’s training process emphasized quality over quantity:
- Data Sources: 5,000 coding problems from datasets like APPs and TACO, 10,000 math problems from NuminaMATH subsets, and 1,000 science puzzles from STILL-2
- Rejection Sampling: Incorrect data points were filtered using exact solution matching and unit tests, enhancing accuracy rates
- Reformatting: Inspired by Still-2, GPT-4o-mini was used to rewrite reasoning traces to improve result parsing, boosting accuracy on complex datasets
Training Process
The model was fine-tuned using Qwen2.5-32B-Instruct, an open-source base without reasoning capabilities:
- Training Configuration:
- Epochs: 3
- Learning Rate: 1e-5
- Batch Size: 96
- Training Time: 19 hours on 8 H100 GPUs using DeepSpeed Zero-3 offload
- Cost: ~$450 using Lambda Cloud pricing
The training leveraged tools like Llama-Factory for optimal results, proving that efficient model training is achievable without high financial barriers.
Key Findings
- Model Size Matters: Smaller models (7B, 14B) showed limited reasoning improvements, often generating repetitive content. The 32B size proved optimal.
- Balanced Data Mixtures: Training exclusively on math data boosted accuracy, while combining math and coding data initially reduced performance. Enriching data with complex tasks restored accuracy and coding capabilities.
Driving Innovation with Open-Source Collaboration
Sky-T1 represents a commitment to democratizing AI research by fostering open-source innovation and transparency. Researchers can now build upon and refine these advancements to explore new frontiers in reasoning AI.
Future Directions
The NovaSky team plans to explore:
- Developing more efficient models with strong reasoning performance
- Exploring advanced techniques to enhance model efficiency and accuracy
Sky-T1 marks a significant step toward making high-level AI reasoning capabilities accessible to the broader research community. Stay tuned for updates as NovaSky continues pushing the boundaries of open-source AI innovation.
Acknowledgments
This project was made possible by the Berkeley Sky Computing Lab, Lambda Labs, Anyscale, and valuable input from the Still-2 and Qwen teams.
Reference: NovaSky