Data Datasets
88 resources · page 3 of 3
HANDBOOK.md Benchmark (Surge AI)
Creator not supplied; see original source.
Surge AI post on HANDBOOK.md, a benchmark testing whether agents follow long company policy handbooks.
HellaSwag or HellaBad? (Surge AI)
Creator not supplied; see original source.
Surge AI audit reporting errors in 36% of HellaSwag benchmark rows, with examples.
The Pile
Creator not supplied; see original source.
arXiv paper (2021) "The Pile: An 800GB Dataset of Diverse Text for Language Modeling", describing its 22 component sources and construction.
How Shopify and Toloka built a ground-truth flywheel
Creator not supplied; see original source.
Toloka case study on building a human-labelled ground-truth loop with Shopify to keep skill agents accurate.
Writing tests with GitHub Copilot
Creator not supplied; see original source.
GitHub tutorial on generating unit and integration tests with GitHub Copilot.
Ai2 model cards and datasets
Creator not supplied; see original source.
Ai2's Hugging Face organisation page listing its model checkpoints, model cards and datasets, including OLMo.
Alibi Detect
Creator not supplied; see original source.
Alibi Detect documentation: Python library for outlier, adversarial and data-drift detection in ML systems.
Cleanlab documentation
Creator not supplied; see original source.
Cleanlab documentation: open-source library for finding label errors and data issues in ML datasets.
Datasheets for Datasets
Creator not supplied; see original source.
Microsoft Research page for Datasheets for Datasets, proposing standard documentation for ML datasets.
LanceDB documentation
Creator not supplied; see original source.
LanceDB documentation: multimodal lakehouse and vector database for AI data, with versioned tables and retrieval.
LLM-jp corpora (GitLab)
Creator not supplied; see original source.
LLM-jp datasets on the NII GitLab: open Japanese pretraining corpora.
NeurIPS paper checklist
Creator not supplied; see original source.
NeurIPS paper checklist covering assumptions, limitations, experimental details, reproducibility and broader impacts.
Qdrant documentation
Creator not supplied; see original source.
Qdrant documentation: vector search engine with quickstarts, filtering, hybrid search and deployment guides.
Retrieval-Augmented Generation
Creator not supplied; see original source.
arXiv paper (2020) "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", introducing retrieval-augmented generation (RAG).
RL environments for LLM agents: Design, rewards, and validation (Snorkel)
Creator not supplied; see original source.
Snorkel AI post on designing RL environments for LLM agents: state, actions, rewards and validation.
Partial Credit in RL: When It Opposes Completion (Snorkel)
Creator not supplied; see original source.
Snorkel AI post on how partial-credit rewards in reinforcement learning can work against task completion.
Training Compute-Optimal Large Language Models
Creator not supplied; see original source.
arXiv paper (2022) "Training Compute-Optimal Large Language Models", describing Chinchilla and compute-optimal model size versus training tokens.
Weaviate documentation
Creator not supplied; see original source.
Weaviate documentation: open-source vector database with guides on indexing, search and RAG pipelines.
Synthetic Data in Investment Management (CFA Institute)
Creator not supplied; see original source.
CFA Institute Research and Policy Center report (2025) on synthetic data in investment management.
DataComp / DCLM
Creator not supplied; see original source.
DataComp site: data-centric benchmarks for building better training datasets for CLIP models and language models (DCLM).
DCLM paper
Creator not supplied; see original source.
arXiv paper (2024) introducing DataComp-LM (DCLM), a testbed for curating language model training sets.
Microsoft GraphRAG
Creator not supplied; see original source.
Microsoft GraphRAG documentation: RAG approach that builds an LLM-generated knowledge graph for global summarisation queries.
Do people click on links in Google AI summaries? (Pew Research Center)
Creator not supplied; see original source.
Pew Research short read on Google users clicking fewer links when an AI summary appears.
Yandex ШАД (School of Data Analysis)
Creator not supplied; see original source.
Yandex School of Data Analysis: free two-year Russian programme in ML, data analysis and engineering.