The FineWeb Datasets
by Guilherme Penedo and coauthors, Hugging Face
Study how filtering and deduplication affect pretraining data.
The FineWeb paper reports controlled comparisons of dataset-construction choices and an educational-text subset. It connects data curation to downstream measurements, with released artifacts for readers who want to investigate the method.
- Level
- Expert
- Models
- any
- Category
- Model Training
- Tags
- Source date
- date not stated
- Added
- 2026-10-18
- Last verified
- 2026-10-18
- Status
- ok
- Prerequisites
- Acts on model-training, infrastructure or research tooling; requires training-stack expertise.
- Subcategory
- training data
- Region
- other
- Original language
- en
- Access
- free
- Resource reuse
- open: Source states by/4.0. Applies to the linked resource, not every referenced model or dataset. Facet definitions
Summary in our own words. Rights remain with the original creator. Credit and correction policy.