data engineering & RAG: 52 curated resources
data engineering & RAG
52 resources · page 2 of 2
AI-Network-Troubleshooting-PoC (Cisco Code Exchange)
by Cisco
Cisco Code Exchange entry for a proof-of-concept that uses AI agents and telemetry to troubleshoot network faults.
RAG survey (Gao et al.)
by Yunfan Gao et al. (Tongji University)
arXiv paper (2023) surveying retrieval-augmented generation for LLMs with a naive, advanced and modular RAG taxonomy.
Ragas
by Ragas (Exploding Gradients)
Ragas documentation: open-source framework with metrics for evaluating RAG and LLM applications.
aider docs: Usage
by Aider
Aider usage documentation: how to add files, chat and edit code in a git repo, including with local models.
Selectel tutorial: Deploying a private LLM in a Kubernetes cluster
by Selectel
Selectel tutorial in Russian on deploying a private LLM and RAG stack (vLLM, Qdrant, n8n) in Managed Kubernetes.
JetBrains AI Assistant: Generate documentation
by JetBrains
JetBrains AI Assistant documentation on generating code documentation with AI and customising the prompt.
Get started with Copilot in Excel (Excel support)
by Microsoft
Microsoft Support getting-started guide for Copilot in Excel: data analysis, formulas and charts.
The Pile
by Leo Gao et al. (EleutherAI)
arXiv paper (2021) "The Pile: An 800GB Dataset of Diverse Text for Language Modeling", describing its 22 component sources and construction.
Writing tests with GitHub Copilot
by GitHub
GitHub tutorial on generating unit and integration tests with GitHub Copilot.
Alibi Detect
by Seldon
Alibi Detect documentation: Python library for outlier, adversarial and data-drift detection in ML systems.
Cleanlab documentation
by Cleanlab
Cleanlab documentation: open-source library for finding label errors and data issues in ML datasets.
Datasheets for Datasets
by Timnit Gebru et al. (Microsoft Research)
Microsoft Research page for Datasheets for Datasets, proposing standard documentation for ML datasets.
LanceDB documentation
by LanceDB
LanceDB documentation: multimodal lakehouse and vector database for AI data, with versioned tables and retrieval.
NeurIPS paper checklist
NeurIPS paper checklist covering assumptions, limitations, experimental details, reproducibility and broader impacts.
Qdrant documentation
by Qdrant
Qdrant documentation: vector search engine with quickstarts, filtering, hybrid search and deployment guides.
Retrieval-Augmented Generation
by Patrick Lewis et al. (Facebook AI Research)
arXiv paper (2020) "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", introducing retrieval-augmented generation (RAG).
Weaviate documentation
by Weaviate
Weaviate documentation: open-source vector database with guides on indexing, search and RAG pipelines.
DataComp / DCLM
by DataComp team (University of Washington and others)
DataComp site: data-centric benchmarks for building better training datasets for CLIP models and language models (DCLM).
DCLM paper
by Jeffrey Li et al. (University of Washington)
arXiv paper (2024) introducing DataComp-LM (DCLM), a testbed for curating language model training sets.
Microsoft GraphRAG
by Microsoft
Microsoft GraphRAG documentation: RAG approach that builds an LLM-generated knowledge graph for global summarisation queries.