Infrastructure, chips & efficiency

Infrastructure, chips & efficiency

• chips & hardware (37)
• efficiency & quantization (8)
• inference & serving (58)
• local/on-device (6)
• training systems (13)

122 resources · page 2 of 4

• NVIDIA Dynamo

  by Dynamo

  ai-dynamo Dynamo GitHub repo: NVIDIA's datacenter-scale distributed, disaggregated inference serving framework.

  entity repo US en

• NVIDIA NeMo Speech

  by NVIDIA-NeMo

  NVIDIA NeMo Speech GitHub repo: framework for speech recognition and TTS model training and inference.

  entity repo US en

• NVIDIA/TensorRT-LLM

  by NVIDIA Corporation

  NVIDIA TensorRT-LLM GitHub repo: Python API and kernels for optimised LLM inference on NVIDIA GPUs.

  entity repo US en

• NVIDIA VerilogEval

  by NVIDIA Research Projects

  NVIDIA VerilogEval GitHub repo: benchmark for evaluating LLMs on generating Verilog hardware-description code.

  entity benchmark US en

• Ollama

  by Ollama

  Ollama GitHub repo: runtime for running open LLMs locally, with API and cloud options.

  entity repo US en

• OLMo training and inference

  by Ai2

  Ai2 OLMo GitHub repo: modelling, training, evaluation and inference code for OLMo open language models.

  entity repo US en

• OpenROAD

  by The OpenROAD Project

  OpenROAD GitHub repo: open RTL-to-GDS chip design flow, also used in ML-for-EDA research.

  entity repo US en

• SageAttention

  by TSAIL group

  Tsinghua SageAttention GitHub repo: quantised low-bit attention kernels for faster inference.

  entity repo China en

• onediff

  by SiliconFlow

  SiliconFlow onediff GitHub repo: acceleration library for diffusion model inference.

  entity repo China en|zh

• Tenstorrent

  by Tenstorrent

  Tenstorrent: company building RISC-V based AI accelerators and computers.

  entity lab US en

• ThunderKittens

  by HazyResearch

  HazyResearch ThunderKittens GitHub repo: tile primitives framework for writing fast GPU kernels.

  entity repo US en

• TileKernels

  by DeepSeek

  DeepSeek TileKernels GitHub repo: library of GPU kernels written in TileLang.

  entity repo China en

• Together cookbook

  by Together

  Together cookbook GitHub repo: notebooks with recipes for building with Together AI models.

  entity repo US en

• torchtitan

  by pytorch

  PyTorch torchtitan GitHub repo: PyTorch-native platform for large-scale generative model pretraining.

  entity repo US en

• fairseq: NLLB branch

  by Meta Research

  Meta fairseq nllb branch on GitHub: code for the No Language Left Behind translation models.

  entity repo US en

• Triton-distributed

  by ByteDance-Seed

  ByteDance Seed Triton-distributed GitHub repo: distributed compiler and parallel kernels overlapping compute and communication.

  entity repo China en

• TurboDiffusion (thu-ml)

  by TSAIL group

  Tsinghua TurboDiffusion GitHub repo: acceleration framework for video diffusion models.

  entity repo China en

• Alibaba Cloud Model Studio: Prompt engineering guide

  by Alibaba Cloud

  Alibaba Cloud Model Studio prompt engineering guide in Chinese with techniques for designing and optimising prompts.

  Beginner learning_resource guide China zh

• Cloud.ru Foundation Models documentation

  by Cloud.ru

  Cloud.ru Foundation Models documentation: Sber subsidiary's unified API for hosted language models (in Russian).

  Beginner learning_resource docs Russia ru

• fal Docs (fal.ai/docs)

  by fal

  fal documentation for deploying and calling generative media models on fal's serving platform.

  Beginner learning_resource docs US en

• GPU Puzzles (Sasha Rush)

  by Sasha Rush

  Sasha Rush's GPU Puzzles GitHub repo: interactive notebook exercises for learning CUDA programming.

  Beginner–Intermediate learning_resource course US en

• GroqDocs: Prompt Basics

  by Groq

  Groq prompting guide covering roles, instructions, context, input and output formats, and sampling parameters.

  Beginner learning_resource docs US en

• LM Studio docs

  by LM Studio

  LM Studio app documentation: running Llama, DeepSeek, Qwen and other LLMs locally on a desktop.

  Beginner learning_resource docs US en

• GPU Glossary (Modal)

  by Modal

  Modal GPU Glossary: reference explaining GPU hardware and software terms from CUDA cores to memory hierarchy.

  Beginner–Expert learning_resource guide US en

• GigaChat docs: Prompt engineering basics

  by Sber

  Sber GigaChat prompt engineering basics in Russian: how to write effective prompts for GigaChat.

  Beginner learning_resource guide Russia ru

• SiliconFlow docs: Platform introduction

  by SiliconFlow

  SiliconFlow platform introduction in Chinese: inference service offering API access to many open models.

  Beginner learning_resource docs China zh

• Volcano Engine Ark (方舟) docs

  by Volcano Engine (ByteDance)

  Volcano Engine Ark documentation entry: ByteDance's model platform docs, redirecting to docs.volcengine.com.

  Beginner learning_resource docs China zh

• Alibaba Cloud Model Studio: Create a knowledge base

  by Alibaba Cloud

  Alibaba Cloud Model Studio documentation on creating knowledge bases for RAG, with parsing and chunking settings (in Chinese).

  Intermediate learning_resource docs China zh

• AMD ROCm

  by AMD

  AMD ROCm developer hub with documentation, training and tools for GPU-accelerated computing on AMD hardware.

  Intermediate–Advanced learning_resource docs US en

• Cerebras blog

  by Cerebras

  Cerebras blog on wafer-scale chips, fast inference, model performance and developer tooling.

  Intermediate learning_resource blog US en

• Cloudflare Workers AI

  by Cloudflare

  Cloudflare Workers AI documentation for running ML models on serverless GPUs across Cloudflare's network.

  Intermediate learning_resource docs US en

• Fireworks blog

  by Fireworks AI

  Fireworks AI blog on inference performance, fine-tuning and model releases on its serving platform.

  Intermediate learning_resource blog US en

Plain-text Markdown copy