Model Training
Study the data, algorithms and infrastructure used to change model weights.
- Pretraining (3)
- Training Data (1)
- Post Training Rl (5)
- Fine Tuning (4)
- Training Infrastructure (123)
136 resources · page 4 of 5
Fireworks AI documentation
Creator not supplied; see original source.
Fireworks AI documentation: serverless and dedicated inference, fine-tuning and deployment of open models.
Three Tests to Run Before You Switch from LoRA to FullFT (Fireworks AI)
Creator not supplied; see original source.
Fireworks AI post with controlled experiments on Qwen3.5-9B showing what to test before switching from LoRA to full fine-tuning.
GigaChat Family: Efficient Russian Language Modeling Through Mixture of Experts Architecture
by GigaChat team / SaluteDevices
Study a Russian-origin model team’s account of mixture-of-experts architecture, pretraining and evaluation.
Building an efficient harness for advanced enterprise work (Glean)
Creator not supplied; see original source.
Glean post on its coding-agent harness using code execution, tool-output truncation and progressive disclosure to cut tokens.
Grouped-Query Attention
Creator not supplied; see original source.
arXiv paper (2023) introducing grouped-query attention (GQA) and uptraining multi-head models to it.
LlamaFactory
by hiyouga (Yaowei Zheng) et al.
Compare adaptation methods within a shared training interface.
TRL - Transformers Reinforcement Learning
by Hugging Face
Find training interfaces for supervised and preference-based adaptation.
MindSpore 中文 API 文档
Creator not supplied; see original source.
MindSpore API documentation in Chinese for Huawei's deep learning training and inference framework.
Modal guides
Creator not supplied; see original source.
Modal guides: deploying model serving, GPU jobs and scalable applications on Modal's serverless platform.
Scaling to 1 million concurrent sandboxes in seconds (Modal)
Creator not supplied; see original source.
Modal engineering post on building a scheduler that runs one million concurrent code sandboxes per workspace.
Moore Threads docs center
Creator not supplied; see original source.
Moore Threads documentation centre for its Chinese GPUs and the MUSA SDK (in Chinese).
Reinforcement Learning from Human Feedback (book)
A conceptual path through reward modeling and post-training methods.
PEFT documentation
Creator not supplied; see original source.
Hugging Face PEFT documentation: parameter-efficient fine-tuning methods such as LoRA, with recipes.
Qiskit Machine Learning
Creator not supplied; see original source.
Qiskit Machine Learning documentation: quantum kernels and quantum neural networks for ML experiments.
QLoRA
Creator not supplied; see original source.
arXiv paper (2023) introducing QLoRA, fine-tuning 4-bit quantized LLMs with low-rank adapters.
Ray Serve documentation
Creator not supplied; see original source.
Ray Serve documentation: scalable library for building online inference APIs and composing model deployments.
Controlling Reasoning Effort in LLMs (Ahead of AI)
A free Ahead of AI article on training controllable reasoning effort.
SGLang docs: HiCache best practices
Creator not supplied; see original source.
SGLang HiCache best practices: configuring hierarchical KV caching across GPU, CPU and storage.
Tinker documentation
Creator not supplied; see original source.
Tinker documentation from Thinking Machines Lab: training API for fine-tuning open-weight LLMs, with SDK reference and recipes.
AdapTive-LeArning Speculator System (ATLAS) (Together AI)
Creator not supplied; see original source.
Together AI post on ATLAS, a speculative decoding system whose speculators keep learning from the live workload.
Cache-aware prefill-decode disaggregation (Together AI)
Creator not supplied; see original source.
Together AI post on cache-aware prefill-decode disaggregation for faster long-context inference serving.
Together AI documentation
Creator not supplied; see original source.
Together AI documentation for running, fine-tuning and serving open-source models.
Unsloth
by Unsloth
Study local training and adaptation workflows for supported models.
vLLM docs: Automatic prefix caching
Creator not supplied; see original source.
vLLM design document on automatic prefix caching and how shared prompt prefixes reuse the KV cache.
vLLM docs
Creator not supplied; see original source.
vLLM latest documentation: installation, serving, supported models and quantisation for the LLM inference engine.
nanochat
A compact training harness spanning the model-development lifecycle.
CMU 11-868 LLM Systems
Creator not supplied; see original source.
CMU 11-868 LLM Systems course site: systems for training and serving large language models, with schedule and readings.
CUDA C++ Programming Guide
Creator not supplied; see original source.
NVIDIA CUDA C++ Programming Guide: reference for the CUDA programming model, memory hierarchy and interfaces.
DeepSeek-R1: Incentivizing Reasoning via RL (arXiv 2501.12948)
by DeepSeek
Study the reported reinforcement-learning recipe behind DeepSeek-R1.
EAGLE
Creator not supplied; see original source.
arXiv paper (2024) introducing EAGLE, speculative sampling with feature-level draft heads.
Fast Inference from Transformers via Speculative Decoding
Creator not supplied; see original source.
arXiv paper (2022) introducing speculative decoding to speed up transformer inference without changing outputs.
FlashAttention
Creator not supplied; see original source.
arXiv paper (2022) introducing FlashAttention, an IO-aware exact attention algorithm for GPUs.