Pretraining data selection
3 methods in the atlas attack this one problem. They are rivals: each wins something the others do not.
Phrasings that mean this problem
Pretraining data selection
llm-training-alignment
- Curriculum data mixingPhase-scheduled corpus weightsspecialistllm-training-alignment
- Deduplication for pretrainingSuffix-array exact substring removalspecialistllm-training-alignment
- Data quality filteringClassifier-scored web textspecialistllm-training-alignment