Infrastructure, chips & efficiency
Infrastructure, chips & efficiency
- chips & hardware (37)
- efficiency & quantization (8)
- inference & serving (58)
- local/on-device (6)
- training systems (13)
122 resources · page 3 of 4
google-ai-edge/gallery: skills
Google AI Edge Gallery skills folder on GitHub: skills for on-device ML and generative AI apps.
Google TPU docs
by Google Cloud
Google Cloud TPU documentation for running machine-learning training and inference on Tensor Processing Units.
GPU MODE
GPU MODE site: community for learning GPU programming through lectures, hackathons, kernel competitions and working groups.
GPU MODE lectures
by GPU MODE
GPU MODE lectures GitHub repo: slides and code from community lectures on GPU programming and CUDA kernels.
GPU MODE resource stream
by GPU MODE
GPU MODE resource-stream GitHub repo: curated links to CUDA and GPU programming books, papers, posts and videos.
Groq blog
by Groq
Groq blog with posts on its LPU inference hardware, cloud API and supported models.
Groq documentation
by Groq
Groq documentation: OpenAI-compatible API, available models and rate limits for its LPU inference service.
Making Deep Learning Go Brrrr From First Principles (Horace He)
by Horace He
Horace He's Making Deep Learning Go Brrrr: first-principles guide to whether models are compute-, memory- or overhead-bound.
vLLM documentation
by vLLM project
vLLM documentation: open-source engine for high-throughput LLM inference and serving.
LM Studio blog
by LM Studio
LM Studio blog with release notes and technical posts on running LLMs locally.
Modal blog
by Modal
Modal blog on serverless GPU computing, infrastructure engineering and running AI workloads.
Ollama blog
by Ollama
Ollama blog with release notes, supported model announcements and guides for running models locally.
OpenRouter docs: Prompt Caching
by OpenRouter
OpenRouter guide to prompt caching across model providers, with cost implications.
PyTorch FSDP tutorial
PyTorch tutorial on Fully Sharded Data Parallel (FSDP2) for training large models across GPUs.
Qwen 中文文档
Qwen documentation in Chinese: usage, deployment and fine-tuning of Qwen language and multimodal models.
SGLang documentation
SGLang documentation: serving framework for large language and multimodal models.
Together AI blog
by Together AI
Together AI blog with product launches, research and technical guides on inference, fine-tuning and GPU clusters.
Transformers documentation
by Hugging Face
Hugging Face Transformers documentation: reference for model architectures, loading, training and generation.
Triton Puzzles
by GPU MODE
Triton Puzzles GitHub repo: interactive exercises for learning to write GPU kernels in Triton.
Triton tutorials
Triton tutorials: ordered examples from vector addition to fused softmax, matmul and fused attention GPU kernels.
Unsloth blog
by Unsloth AI
Unsloth blog with guides and release notes for fast, memory-efficient LLM fine-tuning.
A graph placement methodology for fast chip design
by Azalia Mirhoseini et al. (Google)
Nature paper (2021) by Mirhoseini et al. introducing the reinforcement learning chip floorplanning method later named AlphaChip.
Agentic inference optimization (Baseten)
by Baseten
Baseten post on optimising inference engines for agent workloads, reporting decode-speed and latency gains over vLLM.
Baseten documentation
by Baseten
Baseten documentation for running hosted models, deploying custom models and training on GPU infrastructure.
BentoML documentation
by BentoML
BentoML documentation: open-source inference platform for packaging, deploying and scaling AI models.
Code execution with MCP
by Anthropic
Anthropic post on code execution with MCP: agents write code to call tools, reducing token use and context load.
Defeating Nondeterminism in LLM Inference
Thinking Machines Lab post on the causes of nondeterminism in LLM inference and batch-invariant kernels to fix it.
DoWhy documentation
by PyWhy
DoWhy documentation: Python library for causal inference with modelling, identification, estimation and refutation steps.
EconML
by PyWhy
EconML documentation: Python methods for estimating heterogeneous treatment effects with machine learning.
Feast documentation
Feast documentation: open-source feature store for serving consistent features to ML training and inference.
Fireworks AI documentation
by Fireworks AI
Fireworks AI documentation: serverless and dedicated inference, fine-tuning and deployment of open models.
Three Tests to Run Before You Switch from LoRA to FullFT (Fireworks AI)
by Fireworks AI
Fireworks AI post with controlled experiments on Qwen3.5-9B showing what to test before switching from LoRA to full fine-tuning.