Data & datasets for advanced learners

45 resources · page 1 of 2

• AI9Stars model and dataset registry

  by AI9Stars

  AI9Stars Hugging Face organisation: compact G9 language models and datasets.

  entity org International; domicile unverified en

• AMLSim

  by International Business Machines

  IBM AMLSim GitHub repo: multi-agent simulator generating synthetic bank transaction data for anti-money-laundering research.

  entity repo US/global en

• Argilla

  by Argilla

  Argilla GitHub repo: collaboration tool for building datasets with human feedback and annotation.

  entity repo Spain/EU en

• ColBERT

  by Future Data Systems

  Stanford ColBERT GitHub repo: late-interaction neural retrieval model and library.

  entity repo US en

• Common Crawl

  by Common Crawl

  Common Crawl: nonprofit maintaining an open repository of web crawl data, a common source of model training data.

  entity lab US en

• data engineering / RAG pipelines

  by InfiniFlow

  InfiniFlow RAGFlow GitHub repo: open-source retrieval-augmented generation engine with document parsing.

  entity repo US en

• datatrove

  by Hugging Face

  Hugging Face datatrove GitHub repo: large-scale text data processing library, used to build FineWeb.

  entity repo France en

• DatologyAI

  by DatologyAI

  DatologyAI: company offering automated curation of model training data.

  entity lab US en

• Docling (IBM)

  by Docling Project

  IBM Docling GitHub repo: converts PDFs and other documents into structured formats for generative AI.

  entity repo US en

• FathomNet

  by FathomNet

  FathomNet: labelled database of underwater images with ML tools for analysing ocean visual data.

  entity dataset US (MBARI) en

• FineWeb

  by FineData

  Hugging Face FineWeb dataset: about 15 trillion tokens of filtered English web text for LLM pretraining.

  entity dataset US en

• Global Fishing Watch

  by Global Fishing Watch

  Global Fishing Watch: uses machine learning on satellite and vessel data to map human activity at sea.

  entity org Global en|es

• HF datasets

  by Hugging Face

  Hugging Face Datasets hub: searchable catalogue of public machine-learning datasets.

  entity lab US en

• LightOn HF

  by LightOn AI

  LightOn Hugging Face organisation: retrieval, OCR and language models.

  entity lab EU-FR en

• OtterTune wiki: An End-To-End Tuning Example

  by CMU Database Group

  OtterTune wiki page on GitHub (CMU): end-to-end example of machine-learning database configuration tuning.

  entity repo US en

• Marker (Datalab)

  by Datalab

  Datalab Marker GitHub repo: converts PDF documents to Markdown and JSON.

  entity repo US en

• MinerU (OpenDataLab)

  by OpenDataLab

  OpenDataLab MinerU GitHub repo: converts PDFs and Office documents into Markdown and JSON.

  entity repo CN en

• Multimodal Universe

  by Multimodal Universe Collaboration

  Multimodal Universe GitHub repo: large dataset of astronomical images, spectra and light curves for training ML models.

  entity dataset Global en

• MVTec Anomaly Detection dataset

  by MVTec Software

  MVTec AD dataset page: over 5,000 high-resolution images for industrial anomaly detection and localisation.

  entity dataset EU (DE) en

• Qwen-Agent

  by Qwen

  QwenLM Qwen-Agent GitHub repo: agent framework with function calling, MCP and code interpreter for Qwen models.

  entity repo China zh

• RAG_Techniques

  by Nir Diamant

  RAG_Techniques GitHub repo (Nir Diamant): notebooks demonstrating many retrieval-augmented generation techniques.

  entity repo Israel en

• circuit_training: Ariane RISC-V example

  by Google Research

  Ariane documentation in Google's circuit_training repo: RL chip floorplanning on the Ariane RISC-V design.

  entity repo US en

• Vera C. Rubin Observatory

  by Vera C. Rubin Observatory

  Vera C. Rubin Observatory: NSF–DOE telescope running the LSST survey, whose alert stream requires ML triage.

  entity org US/CL en|es

• xView2 / xBD

  by Defense Innovation Unit (xView2)

  xView2 challenge site: xBD satellite imagery dataset for assessing building damage after natural disasters.

  entity benchmark US (DIU) en

• Yambda dataset

  by Yandex

  Yandex Yambda dataset on Hugging Face: large music-streaming interaction dataset for recommender-system research.

  entity dataset Russia en

• Databricks: Add AI-generated comments to Unity Catalog objects

  by Databricks

  Databricks documentation on AI-generated comments for Unity Catalog tables and columns.

  Intermediate–Advanced learning_resource docs en

• SDV: CTGANSynthesizer

  by DataCebo (Synthetic Data Vault)

  Synthetic Data Vault documentation for CTGANSynthesizer, a GAN-based generator of synthetic tabular data.

  Intermediate–Advanced learning_resource docs en

• Upgrading Java versions with Amazon Q Developer

  by Amazon Web Services

  AWS documentation on upgrading Java application versions with Amazon Q Developer code transformation.

  Intermediate–Advanced learning_resource docs US en

• AI-Network-Troubleshooting-PoC (Cisco Code Exchange)

  by Cisco

  Cisco Code Exchange entry for a proof-of-concept that uses AI agents and telemetry to troubleshoot network faults.

  Intermediate–Advanced learning_resource docs en

• aider docs: Usage

  by Aider

  Aider usage documentation: how to add files, chat and edit code in a git repo, including with local models.

  Intermediate–Advanced learning_resource docs West en

• Get started with Copilot in Excel (Excel support)

  by Microsoft

  Microsoft Support getting-started guide for Copilot in Excel: data analysis, formulas and charts.

  Intermediate–Advanced learning_resource docs US en

• Get started with Copilot in Excel

  by Microsoft

  Microsoft Support guide to Copilot in Excel for analysing data, generating formulas and creating charts.

  Intermediate–Advanced learning_resource docs US en

Plain-text Markdown copy