--- id: "/tags/infrastructure-chips-efficiency/page/4/" title: "Infrastructure Chips Efficiency · page 4" url: "https://skillissue.dev/tags/infrastructure-chips-efficiency/page/4/" --- # Infrastructure Chips Efficiency [Beginner](/levels/beginner/index.md) [Intermediate](/levels/intermediate/index.md) [Advanced](/levels/advanced/index.md) [Expert](/levels/expert/index.md) 122 resources · page 4 of 4 - ### [Building an efficient harness for advanced enterprise work (Glean)](/model-training/training-infrastructure/glean-building-an-efficient-harness-for-advanced-enterprise-work/index.md) by [Glean](https://glean.com/) Glean post on its coding-agent harness using code execution, tool-output truncation and progressive disclosure to cut tokens. Advanced learning_resource blog US en - ### [Grouped-Query Attention](/model-training/training-infrastructure/grouped-query-attention/index.md) by [Joshua Ainslie et al. (Google)](https://ai.google/) arXiv paper (2023) introducing grouped-query attention (GQA) and uptraining multi-head models to it. Advanced learning_resource paper US en - ### [MindSpore 中文 API 文档](/model-training/training-infrastructure/mindspore-api/index.md) by [Huawei (MindSpore)](https://www.mindspore.cn/) MindSpore API documentation in Chinese for Huawei's deep learning training and inference framework. Advanced learning_resource docs China zh - ### [Modal guides](/model-training/training-infrastructure/modal-guides/index.md) by [Modal](https://modal.com/) Modal guides: deploying model serving, GPU jobs and scalable applications on Modal's serverless platform. Advanced learning_resource docs US en - ### [Scaling to 1 million concurrent sandboxes in seconds (Modal)](/model-training/training-infrastructure/modal-scaling-to-1m-concurrent-sandboxes/index.md) by [Modal](https://modal.com/) Modal engineering post on building a scheduler that runs one million concurrent code sandboxes per workspace. Advanced learning_resource blog US en - ### [Moore Threads docs center](/model-training/training-infrastructure/moore-threads-docs-center/index.md) by [Moore Threads (摩尔线程)](https://www.mthreads.com/) Moore Threads documentation centre for its Chinese GPUs and the MUSA SDK (in Chinese). Advanced learning_resource docs China zh - ### [PEFT documentation](/model-training/training-infrastructure/peft-documentation/index.md) by [Hugging Face](https://huggingface.co/) Hugging Face PEFT documentation: parameter-efficient fine-tuning methods such as LoRA, with recipes. Advanced learning_resource docs International en - ### [Qiskit Machine Learning](/model-training/training-infrastructure/qiskit-machine-learning/index.md) by [Qiskit Community](https://github.com/qiskit-community) Qiskit Machine Learning documentation: quantum kernels and quantum neural networks for ML experiments. Advanced learning_resource docs International en - ### [QLoRA](/model-training/training-infrastructure/qlora/index.md) by [Tim Dettmers et al. (University of Washington)](https://www.washington.edu/) arXiv paper (2023) introducing QLoRA, fine-tuning 4-bit quantized LLMs with low-rank adapters. Advanced learning_resource paper US en - ### [Ray Serve documentation](/model-training/training-infrastructure/ray-serve-documentation/index.md) by [Anyscale (Ray)](https://www.anyscale.com/) Ray Serve documentation: scalable library for building online inference APIs and composing model deployments. Advanced learning_resource docs US/global en - ### [SGLang docs: HiCache best practices](/model-training/training-infrastructure/sglang-hicache-best-practices/index.md) by [sgl-project](https://github.com/sgl-project) SGLang HiCache best practices: configuring hierarchical KV caching across GPU, CPU and storage. Advanced learning_resource guide US/China en - ### [Tinker documentation](/model-training/training-infrastructure/tinker-documentation/index.md) by [Thinking Machines Lab](https://thinkingmachines.ai/) Tinker documentation from Thinking Machines Lab: training API for fine-tuning open-weight LLMs, with SDK reference and recipes. Advanced learning_resource docs US en - ### [AdapTive-LeArning Speculator System (ATLAS) (Together AI)](/model-training/training-infrastructure/together-ai-atlas-runtime-learning-speculators/index.md) by [Together AI](https://together.ai/) Together AI post on ATLAS, a speculative decoding system whose speculators keep learning from the live workload. Advanced learning_resource blog US en - ### [Cache-aware prefill-decode disaggregation (Together AI)](/model-training/training-infrastructure/together-ai-cache-aware-prefill-decode-disaggregation/index.md) by [Together AI](https://together.ai/) Together AI post on cache-aware prefill-decode disaggregation for faster long-context inference serving. Advanced learning_resource blog US en - ### [Together AI documentation](/model-training/training-infrastructure/together-ai-documentation/index.md) by [Together AI](https://www.together.ai/) Together AI documentation for running, fine-tuning and serving open-source models. Advanced learning_resource docs US en - ### [vLLM docs: Automatic prefix caching](/model-training/training-infrastructure/vllm-automatic-prefix-caching-design/index.md) by [vLLM](https://github.com/vllm-project) vLLM design document on automatic prefix caching and how shared prompt prefixes reuse the KV cache. Advanced learning_resource docs Global en - ### [vLLM docs](/model-training/training-infrastructure/vllm-docs/index.md) by [vLLM project](https://vllm.ai/) vLLM latest documentation: installation, serving, supported models and quantisation for the LLM inference engine. Advanced learning_resource docs US/Global en - ### [CMU 11-868 LLM Systems](/model-training/training-infrastructure/cmu-11-868-llm-systems/index.md) by [llmsystem](https://github.com/llmsystem) CMU 11-868 LLM Systems course site: systems for training and serving large language models, with schedule and readings. Expert learning_resource course US en - ### [CUDA C++ Programming Guide](/model-training/training-infrastructure/cuda-c-programming-guide/index.md) by [NVIDIA](https://www.nvidia.com/) NVIDIA CUDA C++ Programming Guide: reference for the CUDA programming model, memory hierarchy and interfaces. Expert learning_resource docs US en - ### [EAGLE](/model-training/training-infrastructure/eagle/index.md) by [Yuhui Li et al.](https://orcid.org/0000-0003-3375-0273) arXiv paper (2024) introducing EAGLE, speculative sampling with feature-level draft heads. Expert learning_resource paper China/Canada en - ### [Fast Inference from Transformers via Speculative Decoding](/model-training/training-infrastructure/fast-inference-from-transformers-via-speculative-decoding/index.md) by [Yaniv Leviathan et al. (Google)](https://ai.google/) arXiv paper (2022) introducing speculative decoding to speed up transformer inference without changing outputs. Expert learning_resource paper Israel; US en - ### [FlashAttention](/model-training/training-infrastructure/flashattention/index.md) by [Tri Dao et al. (Stanford University)](https://www.stanford.edu/) arXiv paper (2022) introducing FlashAttention, an IO-aware exact attention algorithm for GPUs. Expert learning_resource paper US en - ### [How to Scale Your Model (JAX scaling book)](/model-training/training-infrastructure/how-to-scale-your-model-jax-scaling-book/index.md) by [jax-ml](https://docs.jax.dev) How to Scale Your Model: free online book on TPU/GPU rooflines, transformer math, parallelism and serving for LLMs. Expert learning_resource book US en - ### [PagedAttention / vLLM paper](/model-training/training-infrastructure/pagedattention-vllm-paper/index.md) by [Woosuk Kwon et al. (UC Berkeley)](https://www.berkeley.edu/) arXiv paper (2023) introducing PagedAttention and the vLLM serving system for efficient KV-cache memory management. Expert learning_resource paper US; international en - ### [SGLang docs](/model-training/training-infrastructure/sglang-docs/index.md) by [SGLang project (LMSYS)](https://lmsys.org/) SGLang documentation: LLM serving framework with RadixAttention prefix caching and fast structured outputs. Expert learning_resource docs US/China en - ### [How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance (Simon Boehm)](/model-training/training-infrastructure/simon-boehm-cuda-matmul-worklog/index.md) by [Simon Boehm](https://siboehm.com/) Simon Boehm's worklog optimising a CUDA matrix-multiplication kernel step by step towards cuBLAS performance. Expert learning_resource guide Germany en [Previous page](/tags/infrastructure-chips-efficiency/page/3/index.md) [Plain-text Markdown copy](/tags/infrastructure-chips-efficiency/page/4/index.md)