Data & datasets
Data & datasets
- data engineering & RAG (52)
- datasets (36)
88 resources · page 3 of 3
HANDBOOK.md Benchmark (Surge AI)
by Surge AI
Surge AI post on HANDBOOK.md, a benchmark testing whether agents follow long company policy handbooks.
HellaSwag or HellaBad? (Surge AI)
by Surge AI
Surge AI audit reporting errors in 36% of HellaSwag benchmark rows, with examples.
The Pile
by Leo Gao et al. (EleutherAI)
arXiv paper (2021) "The Pile: An 800GB Dataset of Diverse Text for Language Modeling", describing its 22 component sources and construction.
How Shopify and Toloka built a ground-truth flywheel
by Toloka
Toloka case study on building a human-labelled ground-truth loop with Shopify to keep skill agents accurate.
Writing tests with GitHub Copilot
by GitHub
GitHub tutorial on generating unit and integration tests with GitHub Copilot.
Ai2 model cards and datasets
by Ai2
Ai2's Hugging Face organisation page listing its model checkpoints, model cards and datasets, including OLMo.
Alibi Detect
by Seldon
Alibi Detect documentation: Python library for outlier, adversarial and data-drift detection in ML systems.
Cleanlab documentation
by Cleanlab
Cleanlab documentation: open-source library for finding label errors and data issues in ML datasets.
Datasheets for Datasets
by Timnit Gebru et al. (Microsoft Research)
Microsoft Research page for Datasheets for Datasets, proposing standard documentation for ML datasets.
LanceDB documentation
by LanceDB
LanceDB documentation: multimodal lakehouse and vector database for AI data, with versioned tables and retrieval.
LLM-jp corpora (GitLab)
by LLM-jp (National Institute of Informatics)
LLM-jp datasets on the NII GitLab: open Japanese pretraining corpora.
NeurIPS paper checklist
NeurIPS paper checklist covering assumptions, limitations, experimental details, reproducibility and broader impacts.
Qdrant documentation
by Qdrant
Qdrant documentation: vector search engine with quickstarts, filtering, hybrid search and deployment guides.
Retrieval-Augmented Generation
by Patrick Lewis et al. (Facebook AI Research)
arXiv paper (2020) "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", introducing retrieval-augmented generation (RAG).
RL environments for LLM agents: Design, rewards, and validation (Snorkel)
by Snorkel AI
Snorkel AI post on designing RL environments for LLM agents: state, actions, rewards and validation.
Partial Credit in RL: When It Opposes Completion (Snorkel)
by Snorkel AI
Snorkel AI post on how partial-credit rewards in reinforcement learning can work against task completion.
Training Compute-Optimal Large Language Models
by Jordan Hoffmann et al. (DeepMind)
arXiv paper (2022) "Training Compute-Optimal Large Language Models", describing Chinchilla and compute-optimal model size versus training tokens.
Weaviate documentation
by Weaviate
Weaviate documentation: open-source vector database with guides on indexing, search and RAG pipelines.
Synthetic Data in Investment Management (CFA Institute)
by CFA Institute Research and Policy Center
CFA Institute Research and Policy Center report (2025) on synthetic data in investment management.
DataComp / DCLM
by DataComp team (University of Washington and others)
DataComp site: data-centric benchmarks for building better training datasets for CLIP models and language models (DCLM).
DCLM paper
by Jeffrey Li et al. (University of Washington)
arXiv paper (2024) introducing DataComp-LM (DCLM), a testbed for curating language model training sets.
Microsoft GraphRAG
by Microsoft
Microsoft GraphRAG documentation: RAG approach that builds an LLM-generated knowledge graph for global summarisation queries.
Do people click on links in Google AI summaries? (Pew Research Center)
Pew Research short read on Google users clicking fewer links when an AI summary appears.
Yandex ШАД (School of Data Analysis)
by Yandex School of Data Analysis
Yandex School of Data Analysis: free two-year Russian programme in ML, data analysis and engineering.