Data Engineering Rag
52 resources · page 1 of 2
AMLSim
Creator not supplied; see original source.
IBM AMLSim GitHub repo: multi-agent simulator generating synthetic bank transaction data for anti-money-laundering research.
ColBERT
Creator not supplied; see original source.
Stanford ColBERT GitHub repo: late-interaction neural retrieval model and library.
data engineering / RAG pipelines
Creator not supplied; see original source.
InfiniFlow RAGFlow GitHub repo: open-source retrieval-augmented generation engine with document parsing.
datatrove
Creator not supplied; see original source.
Hugging Face datatrove GitHub repo: large-scale text data processing library, used to build FineWeb.
OtterTune wiki: An End-To-End Tuning Example
Creator not supplied; see original source.
OtterTune wiki page on GitHub (CMU): end-to-end example of machine-learning database configuration tuning.
Qwen-Agent
Creator not supplied; see original source.
QwenLM Qwen-Agent GitHub repo: agent framework with function calling, MCP and code interpreter for Qwen models.
RAG_Techniques
Creator not supplied; see original source.
RAG_Techniques GitHub repo (Nir Diamant): notebooks demonstrating many retrieval-augmented generation techniques.
circuit_training: Ariane RISC-V example
Creator not supplied; see original source.
Ariane documentation in Google's circuit_training repo: RL chip floorplanning on the Ariane RISC-V design.
Postman: Write scripts to test API response data
Creator not supplied; see original source.
Postman documentation on writing test scripts that assert API response data.
DataTalksClub LLM Zoomcamp
Creator not supplied; see original source.
DataTalksClub LLM Zoomcamp GitHub repo: free course on building LLM and RAG applications.
Summarize incidents with Microsoft Copilot in Microsoft Defender
Creator not supplied; see original source.
Microsoft Learn page on generating incident summaries with Microsoft Copilot in Microsoft Defender.
Getting started with Pulumi Neo
Creator not supplied; see original source.
Pulumi documentation on setting up Pulumi Neo, an AI agent for infrastructure-as-code tasks.
Using GitHub Copilot code review
Creator not supplied; see original source.
GitHub documentation on requesting pull-request code reviews from GitHub Copilot.
Resolving code scanning alerts (GitHub Docs)
Creator not supplied; see original source.
GitHub documentation on viewing, fixing and dismissing code scanning alerts, including Copilot Autofix.
Introducing Contextual Retrieval (Anthropic)
Creator not supplied; see original source.
Anthropic announcement of Contextual Retrieval, combining contextual embeddings, BM25 and reranking, with measured retrieval gains.
Copilot in Power BI tutorial: Explore data and gain insights
Creator not supplied; see original source.
Microsoft tutorial on using Copilot in Power BI to explore report data and create summaries.
Diagnosing CI test failures (GitHub Copilot Chat Cookbook)
Creator not supplied; see original source.
GitHub Copilot cookbook guide on using Copilot CLI to pull CI logs, match failures to local code and fix them.
Grafana Assistant Investigations
Creator not supplied; see original source.
Grafana Cloud documentation on Grafana Assistant investigations for diagnosing production incidents across signals.
Databricks: Add AI-generated comments to Unity Catalog objects
Creator not supplied; see original source.
Databricks documentation on AI-generated comments for Unity Catalog tables and columns.
Datasette LLM documentation
Creator not supplied; see original source.
Documentation for Simon Willison's LLM, a CLI tool and Python library for prompting many models with plugins and logged conversations.
Cleanlab: Classification with Tabular Data using Scikit-Learn and Cleanlab
Creator not supplied; see original source.
Cleanlab tutorial on finding label errors in tabular classification data with scikit-learn models.
DVC documentation
Creator not supplied; see original source.
DVC documentation: open-source tool for versioning data, models and ML pipelines alongside Git.
LiteRT: Get started with microcontrollers
Creator not supplied; see original source.
Google AI Edge guide to running LiteRT neural-network inference on microcontrollers.
FastGPT docs: dataset engine
Creator not supplied; see original source.
FastGPT documentation in Chinese on its knowledge-base engine: how documents are indexed and retrieved for RAG.
SDV: CTGANSynthesizer
Creator not supplied; see original source.
Synthetic Data Vault documentation for CTGANSynthesizer, a GAN-based generator of synthetic tabular data.
Haystack by deepset
Creator not supplied; see original source.
Haystack documentation: deepset's open-source framework for building RAG pipelines and AI agents.
Jason Liu RAG writing
Creator not supplied; see original source.
Jason Liu's essays on RAG systems: evaluation, failure modes and lessons from coding-agent teams.
Upgrading Java versions with Amazon Q Developer
Creator not supplied; see original source.
AWS documentation on upgrading Java application versions with Amazon Q Developer code transformation.
Label Studio documentation
Creator not supplied; see original source.
Label Studio documentation: open-source tool for labelling text, image, audio and video data for ML.
LlamaIndex documentation
Creator not supplied; see original source.
LlamaIndex documentation: open-source framework for data ingestion, retrieval and document agents, plus LlamaParse.
Microsoft Presidio
Creator not supplied; see original source.
Microsoft Presidio documentation: open-source PII detection and anonymisation for text, images and structured data.
Snowflake Cortex Analyst tutorial: time-series revenue data
Creator not supplied; see original source.
Snowflake tutorial using Cortex Analyst to answer natural-language questions over time-series revenue data with SQL.