Infrastructure, chips & efficiency for intermediate learners

77 resources · page 2 of 3

• NVIDIA Dynamo

  by Dynamo

  ai-dynamo Dynamo GitHub repo: NVIDIA's datacenter-scale distributed, disaggregated inference serving framework.

  entity repo US en

• NVIDIA NeMo Speech

  by NVIDIA-NeMo

  NVIDIA NeMo Speech GitHub repo: framework for speech recognition and TTS model training and inference.

  entity repo US en

• NVIDIA/TensorRT-LLM

  by NVIDIA Corporation

  NVIDIA TensorRT-LLM GitHub repo: Python API and kernels for optimised LLM inference on NVIDIA GPUs.

  entity repo US en

• NVIDIA VerilogEval

  by NVIDIA Research Projects

  NVIDIA VerilogEval GitHub repo: benchmark for evaluating LLMs on generating Verilog hardware-description code.

  entity benchmark US en

• Ollama

  by Ollama

  Ollama GitHub repo: runtime for running open LLMs locally, with API and cloud options.

  entity repo US en

• OLMo training and inference

  by Ai2

  Ai2 OLMo GitHub repo: modelling, training, evaluation and inference code for OLMo open language models.

  entity repo US en

• OpenROAD

  by The OpenROAD Project

  OpenROAD GitHub repo: open RTL-to-GDS chip design flow, also used in ML-for-EDA research.

  entity repo US en

• SageAttention

  by TSAIL group

  Tsinghua SageAttention GitHub repo: quantised low-bit attention kernels for faster inference.

  entity repo China en

• onediff

  by SiliconFlow

  SiliconFlow onediff GitHub repo: acceleration library for diffusion model inference.

  entity repo China en|zh

• Tenstorrent

  by Tenstorrent

  Tenstorrent: company building RISC-V based AI accelerators and computers.

  entity lab US en

• ThunderKittens

  by HazyResearch

  HazyResearch ThunderKittens GitHub repo: tile primitives framework for writing fast GPU kernels.

  entity repo US en

• TileKernels

  by DeepSeek

  DeepSeek TileKernels GitHub repo: library of GPU kernels written in TileLang.

  entity repo China en

• Together cookbook

  by Together

  Together cookbook GitHub repo: notebooks with recipes for building with Together AI models.

  entity repo US en

• torchtitan

  by pytorch

  PyTorch torchtitan GitHub repo: PyTorch-native platform for large-scale generative model pretraining.

  entity repo US en

• fairseq: NLLB branch

  by Meta Research

  Meta fairseq nllb branch on GitHub: code for the No Language Left Behind translation models.

  entity repo US en

• Triton-distributed

  by ByteDance-Seed

  ByteDance Seed Triton-distributed GitHub repo: distributed compiler and parallel kernels overlapping compute and communication.

  entity repo China en

• TurboDiffusion (thu-ml)

  by TSAIL group

  Tsinghua TurboDiffusion GitHub repo: acceleration framework for video diffusion models.

  entity repo China en

• GPU Puzzles (Sasha Rush)

  by Sasha Rush

  Sasha Rush's GPU Puzzles GitHub repo: interactive notebook exercises for learning CUDA programming.

  Beginner–Intermediate learning_resource course US en

• GPU Glossary (Modal)

  by Modal

  Modal GPU Glossary: reference explaining GPU hardware and software terms from CUDA cores to memory hierarchy.

  Beginner–Expert learning_resource guide US en

• Alibaba Cloud Model Studio: Create a knowledge base

  by Alibaba Cloud

  Alibaba Cloud Model Studio documentation on creating knowledge bases for RAG, with parsing and chunking settings (in Chinese).

  Intermediate learning_resource docs China zh

• AMD ROCm

  by AMD

  AMD ROCm developer hub with documentation, training and tools for GPU-accelerated computing on AMD hardware.

  Intermediate–Advanced learning_resource docs US en

• Cerebras blog

  by Cerebras

  Cerebras blog on wafer-scale chips, fast inference, model performance and developer tooling.

  Intermediate learning_resource blog US en

• Cloudflare Workers AI

  by Cloudflare

  Cloudflare Workers AI documentation for running ML models on serverless GPUs across Cloudflare's network.

  Intermediate learning_resource docs US en

• Fireworks blog

  by Fireworks AI

  Fireworks AI blog on inference performance, fine-tuning and model releases on its serving platform.

  Intermediate learning_resource blog US en

• google-ai-edge/gallery: skills

  by google-ai-edge

  Google AI Edge Gallery skills folder on GitHub: skills for on-device ML and generative AI apps.

  Intermediate learning_resource skill-library US en

• Google TPU docs

  by Google Cloud

  Google Cloud TPU documentation for running machine-learning training and inference on Tensor Processing Units.

  Intermediate–Advanced learning_resource docs US en

• GPU MODE

  by GPU MODE community

  GPU MODE site: community for learning GPU programming through lectures, hackathons, kernel competitions and working groups.

  Intermediate–Advanced learning_resource course Global en

• GPU MODE lectures

  by GPU MODE

  GPU MODE lectures GitHub repo: slides and code from community lectures on GPU programming and CUDA kernels.

  Intermediate–Advanced learning_resource course Global en

• GPU MODE resource stream

  by GPU MODE

  GPU MODE resource-stream GitHub repo: curated links to CUDA and GPU programming books, papers, posts and videos.

  Intermediate learning_resource guide Global en

• Groq blog

  by Groq

  Groq blog with posts on its LPU inference hardware, cloud API and supported models.

  Intermediate learning_resource blog US en

• Groq documentation

  by Groq

  Groq documentation: OpenAI-compatible API, available models and rate limits for its LPU inference service.

  Intermediate learning_resource docs US en

• Making Deep Learning Go Brrrr From First Principles (Horace He)

  by Horace He

  Horace He's Making Deep Learning Go Brrrr: first-principles guide to whether models are compute-, memory- or overhead-bound.

  Intermediate learning_resource guide US en

Plain-text Markdown copy