--- id: "/infrastructure-chips-efficiency/expert/page/2/" title: "Infrastructure, chips & efficiency for expert learners · page 2" url: "https://skillissue.dev/infrastructure-chips-efficiency/expert/page/2/" --- # Infrastructure, chips & efficiency for expert learners [Beginner](/infrastructure-chips-efficiency/beginner/index.md) [Intermediate](/infrastructure-chips-efficiency/intermediate/index.md) [Advanced](/infrastructure-chips-efficiency/advanced/index.md) [Expert](/infrastructure-chips-efficiency/expert/index.md) 59 resources · page 2 of 2 - ### [NVIDIA Dynamo](/model-training/training-infrastructure/nvidia-dynamo/index.md) by [Dynamo](https://developer.nvidia.com/dynamo) ai-dynamo Dynamo GitHub repo: NVIDIA's datacenter-scale distributed, disaggregated inference serving framework. entity repo US en - ### [NVIDIA NeMo Speech](/model-training/training-infrastructure/nvidia-nemo-speech/index.md) by [NVIDIA-NeMo](https://nvidia.com/) NVIDIA NeMo Speech GitHub repo: framework for speech recognition and TTS model training and inference. entity repo US en - ### [NVIDIA/TensorRT-LLM](/model-training/training-infrastructure/nvidia-tensorrt-llm/index.md) by [NVIDIA Corporation](https://nvidia.com) NVIDIA TensorRT-LLM GitHub repo: Python API and kernels for optimised LLM inference on NVIDIA GPUs. entity repo US en - ### [NVIDIA VerilogEval](/model-training/training-infrastructure/nvidia-verilogeval/index.md) by [NVIDIA Research Projects](https://research.nvidia.com/) NVIDIA VerilogEval GitHub repo: benchmark for evaluating LLMs on generating Verilog hardware-description code. entity benchmark US en - ### [Ollama](/model-training/training-infrastructure/ollama-repo/index.md) by [Ollama](https://ollama.com) Ollama GitHub repo: runtime for running open LLMs locally, with API and cloud options. entity repo US en - ### [OLMo training and inference](/model-training/training-infrastructure/olmo-training-and-inference/index.md) by [Ai2](https://allenai.org/) Ai2 OLMo GitHub repo: modelling, training, evaluation and inference code for OLMo open language models. entity repo US en - ### [OpenROAD](/model-training/training-infrastructure/openroad/index.md) by [The OpenROAD Project](https://theopenroadproject.org/) OpenROAD GitHub repo: open RTL-to-GDS chip design flow, also used in ML-for-EDA research. entity repo US en - ### [SageAttention](/model-training/training-infrastructure/sageattention/index.md) by [TSAIL group](https://ml.cs.tsinghua.edu.cn) Tsinghua SageAttention GitHub repo: quantised low-bit attention kernels for faster inference. entity repo China en - ### [onediff](/model-training/training-infrastructure/siliconflow-onediff/index.md) by [SiliconFlow](https://siliconflow.com) SiliconFlow onediff GitHub repo: acceleration library for diffusion model inference. entity repo China en|zh - ### [Tenstorrent](/model-training/training-infrastructure/tenstorrent/index.md) by [Tenstorrent](https://tenstorrent.com/) Tenstorrent: company building RISC-V based AI accelerators and computers. entity lab US en - ### [ThunderKittens](/model-training/training-infrastructure/thunderkittens/index.md) by [HazyResearch](https://cs.stanford.edu/people/chrismre/) HazyResearch ThunderKittens GitHub repo: tile primitives framework for writing fast GPU kernels. entity repo US en - ### [TileKernels](/model-training/training-infrastructure/tilekernels/index.md) by [DeepSeek](https://www.deepseek.com/) DeepSeek TileKernels GitHub repo: library of GPU kernels written in TileLang. entity repo China en - ### [Together cookbook](/model-training/training-infrastructure/together-cookbook/index.md) by [Together](https://github.com/togethercomputer) Together cookbook GitHub repo: notebooks with recipes for building with Together AI models. entity repo US en - ### [torchtitan](/model-training/training-infrastructure/torchtitan/index.md) by [pytorch](https://pytorch.org) PyTorch torchtitan GitHub repo: PyTorch-native platform for large-scale generative model pretraining. entity repo US en - ### [fairseq: NLLB branch](/model-training/training-infrastructure/translation-localization/index.md) by [Meta Research](https://opensource.fb.com) Meta fairseq nllb branch on GitHub: code for the No Language Left Behind translation models. entity repo US en - ### [Triton-distributed](/model-training/training-infrastructure/triton-distributed/index.md) by [ByteDance-Seed](https://seed.bytedance.com/) ByteDance Seed Triton-distributed GitHub repo: distributed compiler and parallel kernels overlapping compute and communication. entity repo China en - ### [TurboDiffusion (thu-ml)](/model-training/training-infrastructure/turbodiffusion-thu-ml/index.md) by [TSAIL group](https://ml.cs.tsinghua.edu.cn) Tsinghua TurboDiffusion GitHub repo: acceleration framework for video diffusion models. entity repo China en - ### [GPU Glossary (Modal)](/model-training/training-infrastructure/modal-gpu-glossary/index.md) by [Modal](https://modal.com/) Modal GPU Glossary: reference explaining GPU hardware and software terms from CUDA cores to memory hierarchy. Beginner–Expert learning_resource guide US en - ### [CMU 11-868 LLM Systems](/model-training/training-infrastructure/cmu-11-868-llm-systems/index.md) by [llmsystem](https://github.com/llmsystem) CMU 11-868 LLM Systems course site: systems for training and serving large language models, with schedule and readings. Expert learning_resource course US en - ### [CUDA C++ Programming Guide](/model-training/training-infrastructure/cuda-c-programming-guide/index.md) by [NVIDIA](https://www.nvidia.com/) NVIDIA CUDA C++ Programming Guide: reference for the CUDA programming model, memory hierarchy and interfaces. Expert learning_resource docs US en - ### [EAGLE](/model-training/training-infrastructure/eagle/index.md) by [Yuhui Li et al.](https://orcid.org/0000-0003-3375-0273) arXiv paper (2024) introducing EAGLE, speculative sampling with feature-level draft heads. Expert learning_resource paper China/Canada en - ### [Fast Inference from Transformers via Speculative Decoding](/model-training/training-infrastructure/fast-inference-from-transformers-via-speculative-decoding/index.md) by [Yaniv Leviathan et al. (Google)](https://ai.google/) arXiv paper (2022) introducing speculative decoding to speed up transformer inference without changing outputs. Expert learning_resource paper Israel; US en - ### [FlashAttention](/model-training/training-infrastructure/flashattention/index.md) by [Tri Dao et al. (Stanford University)](https://www.stanford.edu/) arXiv paper (2022) introducing FlashAttention, an IO-aware exact attention algorithm for GPUs. Expert learning_resource paper US en - ### [How to Scale Your Model (JAX scaling book)](/model-training/training-infrastructure/how-to-scale-your-model-jax-scaling-book/index.md) by [jax-ml](https://docs.jax.dev) How to Scale Your Model: free online book on TPU/GPU rooflines, transformer math, parallelism and serving for LLMs. Expert learning_resource book US en - ### [PagedAttention / vLLM paper](/model-training/training-infrastructure/pagedattention-vllm-paper/index.md) by [Woosuk Kwon et al. (UC Berkeley)](https://www.berkeley.edu/) arXiv paper (2023) introducing PagedAttention and the vLLM serving system for efficient KV-cache memory management. Expert learning_resource paper US; international en - ### [SGLang docs](/model-training/training-infrastructure/sglang-docs/index.md) by [SGLang project (LMSYS)](https://lmsys.org/) SGLang documentation: LLM serving framework with RadixAttention prefix caching and fast structured outputs. Expert learning_resource docs US/China en - ### [How to Optimize a CUDA Matmul Kernel for cuBLAS-like Performance (Simon Boehm)](/model-training/training-infrastructure/simon-boehm-cuda-matmul-worklog/index.md) by [Simon Boehm](https://siboehm.com/) Simon Boehm's worklog optimising a CUDA matrix-multiplication kernel step by step towards cuBLAS performance. Expert learning_resource guide Germany en [Previous page](/infrastructure-chips-efficiency/expert/index.md) [Plain-text Markdown copy](/infrastructure-chips-efficiency/expert/page/2/index.md)