--- id: "/infrastructure-chips-efficiency/page/2/" title: "Infrastructure, chips & efficiency · page 2" url: "https://skillissue.dev/infrastructure-chips-efficiency/page/2/" --- # Infrastructure, chips & efficiency Infrastructure, chips & efficiency - [chips & hardware](/infrastructure-chips-efficiency/chips-hardware/index.md) (37) - [efficiency & quantization](/infrastructure-chips-efficiency/efficiency-quantization/index.md) (8) - [inference & serving](/infrastructure-chips-efficiency/inference-serving/index.md) (58) - [local/on-device](/infrastructure-chips-efficiency/local-on-device/index.md) (6) - [training systems](/infrastructure-chips-efficiency/training-systems/index.md) (13) [Beginner](/infrastructure-chips-efficiency/beginner/index.md) [Intermediate](/infrastructure-chips-efficiency/intermediate/index.md) [Advanced](/infrastructure-chips-efficiency/advanced/index.md) [Expert](/infrastructure-chips-efficiency/expert/index.md) 122 resources · page 2 of 4 - ### [NVIDIA Dynamo](/model-training/training-infrastructure/nvidia-dynamo/index.md) by [Dynamo](https://developer.nvidia.com/dynamo) ai-dynamo Dynamo GitHub repo: NVIDIA's datacenter-scale distributed, disaggregated inference serving framework. entity repo US en - ### [NVIDIA NeMo Speech](/model-training/training-infrastructure/nvidia-nemo-speech/index.md) by [NVIDIA-NeMo](https://nvidia.com/) NVIDIA NeMo Speech GitHub repo: framework for speech recognition and TTS model training and inference. entity repo US en - ### [NVIDIA/TensorRT-LLM](/model-training/training-infrastructure/nvidia-tensorrt-llm/index.md) by [NVIDIA Corporation](https://nvidia.com) NVIDIA TensorRT-LLM GitHub repo: Python API and kernels for optimised LLM inference on NVIDIA GPUs. entity repo US en - ### [NVIDIA VerilogEval](/model-training/training-infrastructure/nvidia-verilogeval/index.md) by [NVIDIA Research Projects](https://research.nvidia.com/) NVIDIA VerilogEval GitHub repo: benchmark for evaluating LLMs on generating Verilog hardware-description code. entity benchmark US en - ### [Ollama](/model-training/training-infrastructure/ollama-repo/index.md) by [Ollama](https://ollama.com) Ollama GitHub repo: runtime for running open LLMs locally, with API and cloud options. entity repo US en - ### [OLMo training and inference](/model-training/training-infrastructure/olmo-training-and-inference/index.md) by [Ai2](https://allenai.org/) Ai2 OLMo GitHub repo: modelling, training, evaluation and inference code for OLMo open language models. entity repo US en - ### [OpenROAD](/model-training/training-infrastructure/openroad/index.md) by [The OpenROAD Project](https://theopenroadproject.org/) OpenROAD GitHub repo: open RTL-to-GDS chip design flow, also used in ML-for-EDA research. entity repo US en - ### [SageAttention](/model-training/training-infrastructure/sageattention/index.md) by [TSAIL group](https://ml.cs.tsinghua.edu.cn) Tsinghua SageAttention GitHub repo: quantised low-bit attention kernels for faster inference. entity repo China en - ### [onediff](/model-training/training-infrastructure/siliconflow-onediff/index.md) by [SiliconFlow](https://siliconflow.com) SiliconFlow onediff GitHub repo: acceleration library for diffusion model inference. entity repo China en|zh - ### [Tenstorrent](/model-training/training-infrastructure/tenstorrent/index.md) by [Tenstorrent](https://tenstorrent.com/) Tenstorrent: company building RISC-V based AI accelerators and computers. entity lab US en - ### [ThunderKittens](/model-training/training-infrastructure/thunderkittens/index.md) by [HazyResearch](https://cs.stanford.edu/people/chrismre/) HazyResearch ThunderKittens GitHub repo: tile primitives framework for writing fast GPU kernels. entity repo US en - ### [TileKernels](/model-training/training-infrastructure/tilekernels/index.md) by [DeepSeek](https://www.deepseek.com/) DeepSeek TileKernels GitHub repo: library of GPU kernels written in TileLang. entity repo China en - ### [Together cookbook](/model-training/training-infrastructure/together-cookbook/index.md) by [Together](https://github.com/togethercomputer) Together cookbook GitHub repo: notebooks with recipes for building with Together AI models. entity repo US en - ### [torchtitan](/model-training/training-infrastructure/torchtitan/index.md) by [pytorch](https://pytorch.org) PyTorch torchtitan GitHub repo: PyTorch-native platform for large-scale generative model pretraining. entity repo US en - ### [fairseq: NLLB branch](/model-training/training-infrastructure/translation-localization/index.md) by [Meta Research](https://opensource.fb.com) Meta fairseq nllb branch on GitHub: code for the No Language Left Behind translation models. entity repo US en - ### [Triton-distributed](/model-training/training-infrastructure/triton-distributed/index.md) by [ByteDance-Seed](https://seed.bytedance.com/) ByteDance Seed Triton-distributed GitHub repo: distributed compiler and parallel kernels overlapping compute and communication. entity repo China en - ### [TurboDiffusion (thu-ml)](/model-training/training-infrastructure/turbodiffusion-thu-ml/index.md) by [TSAIL group](https://ml.cs.tsinghua.edu.cn) Tsinghua TurboDiffusion GitHub repo: acceleration framework for video diffusion models. entity repo China en - ### [Alibaba Cloud Model Studio: Prompt engineering guide](/model-training/training-infrastructure/alibaba-cloud-bailian-prompt-engineering-guide/index.md) by [Alibaba Cloud](https://www.alibabacloud.com/) Alibaba Cloud Model Studio prompt engineering guide in Chinese with techniques for designing and optimising prompts. Beginner learning_resource guide China zh - ### [Cloud.ru Foundation Models documentation](/model-training/training-infrastructure/cloud-ru-sber-foundation-models/index.md) by [Cloud.ru](https://cloud.ru/) Cloud.ru Foundation Models documentation: Sber subsidiary's unified API for hosted language models (in Russian). Beginner learning_resource docs Russia ru - ### [fal Docs (fal.ai/docs)](/model-training/training-infrastructure/fal-docs-2/index.md) by [fal](https://fal.ai/) fal documentation for deploying and calling generative media models on fal's serving platform. Beginner learning_resource docs US en - ### [GPU Puzzles (Sasha Rush)](/model-training/training-infrastructure/gpu-puzzles-sasha-rush/index.md) by [Sasha Rush](https://rush-nlp.com/) Sasha Rush's GPU Puzzles GitHub repo: interactive notebook exercises for learning CUDA programming. Beginner–Intermediate learning_resource course US en - ### [GroqDocs: Prompt Basics](/model-training/training-infrastructure/groq-prompt-basics/index.md) by [Groq](https://groq.com/) Groq prompting guide covering roles, instructions, context, input and output formats, and sampling parameters. Beginner learning_resource docs US en - ### [LM Studio docs](/model-training/training-infrastructure/lm-studio-docs/index.md) by [LM Studio](https://lmstudio.ai/) LM Studio app documentation: running Llama, DeepSeek, Qwen and other LLMs locally on a desktop. Beginner learning_resource docs US en - ### [GPU Glossary (Modal)](/model-training/training-infrastructure/modal-gpu-glossary/index.md) by [Modal](https://modal.com/) Modal GPU Glossary: reference explaining GPU hardware and software terms from CUDA cores to memory hierarchy. Beginner–Expert learning_resource guide US en - ### [GigaChat docs: Prompt engineering basics](/model-training/training-infrastructure/sber-gigachat/index.md) by [Sber](https://www.sber.ru/) Sber GigaChat prompt engineering basics in Russian: how to write effective prompts for GigaChat. Beginner learning_resource guide Russia ru - ### [SiliconFlow docs: Platform introduction](/model-training/training-infrastructure/siliconflow/index.md) by [SiliconFlow](https://siliconflow.cn/) SiliconFlow platform introduction in Chinese: inference service offering API access to many open models. Beginner learning_resource docs China zh - ### [Volcano Engine Ark (方舟) docs](/model-training/training-infrastructure/volcano-engine-ark-docs/index.md) by [Volcano Engine (ByteDance)](https://www.volcengine.com/) Volcano Engine Ark documentation entry: ByteDance's model platform docs, redirecting to docs.volcengine.com. Beginner learning_resource docs China zh - ### [Alibaba Cloud Model Studio: Create a knowledge base](/model-training/training-infrastructure/alibaba-cloud-bailian-knowledge-base-rag/index.md) by [Alibaba Cloud](https://www.alibabacloud.com/) Alibaba Cloud Model Studio documentation on creating knowledge bases for RAG, with parsing and chunking settings (in Chinese). Intermediate learning_resource docs China zh - ### [AMD ROCm](/model-training/training-infrastructure/amd-rocm/index.md) by [AMD](https://amd.com/) AMD ROCm developer hub with documentation, training and tools for GPU-accelerated computing on AMD hardware. Intermediate–Advanced learning_resource docs US en - ### [Cerebras blog](/model-training/training-infrastructure/cerebras-blog/index.md) by [Cerebras](https://www.cerebras.ai/) Cerebras blog on wafer-scale chips, fast inference, model performance and developer tooling. Intermediate learning_resource blog US en - ### [Cloudflare Workers AI](/model-training/training-infrastructure/cloudflare-workers-ai/index.md) by [Cloudflare](https://developers.cloudflare.com/) Cloudflare Workers AI documentation for running ML models on serverless GPUs across Cloudflare's network. Intermediate learning_resource docs US en - ### [Fireworks blog](/model-training/training-infrastructure/fireworks-blog/index.md) by [Fireworks AI](https://fireworks.ai/) Fireworks AI blog on inference performance, fine-tuning and model releases on its serving platform. Intermediate learning_resource blog US en [Previous page](/infrastructure-chips-efficiency/index.md) [Next page](/infrastructure-chips-efficiency/page/3/index.md) [Plain-text Markdown copy](/infrastructure-chips-efficiency/page/2/index.md)