Infrastructure, chips & efficiency
Infrastructure, chips & efficiency
- chips & hardware (37)
- efficiency & quantization (8)
- inference & serving (58)
- local/on-device (6)
- training systems (13)
122 resources · page 1 of 4
Apple machine learning research
by Apple Machine Learning Research
Apple Machine Learning Research publications: papers on speech, language, vision, privacy and health.
Apple MLX
by ml-explore
Apple MLX GitHub repo: array framework for machine learning on Apple silicon.
OpenInference
by Arize AI
Arize OpenInference GitHub repo: OpenTelemetry instrumentation and semantic conventions for LLM tracing.
Truss
by Baseten
Baseten Truss GitHub repo: packaging and serving ML models in production.
checkpoint-engine
by Moonshot AI
Moonshot checkpoint-engine GitHub repo: middleware for updating model weights in running LLM inference engines.
Colossal-AI
by HPC-AI Tech
HPC-AI Tech Colossal-AI GitHub repo: parallel training system for large AI models.
DeepGEMM
by DeepSeek
DeepSeek DeepGEMM GitHub repo: GPU matrix multiplication kernel library including FP8 GEMMs.
DeepGEMM-Ascend
by DeepSeek
DeepSeek DeepGEMM-Ascend GitHub repo: matrix multiplication kernels for Huawei Ascend hardware.
DeepSeek Open Infra Index
by DeepSeek
DeepSeek Open Infra Index GitHub repo: index of released infrastructure tools such as FlashMLA, DeepEP, DeepGEMM and 3FS.
DeepSpeed
by deepspeedai
DeepSpeed GitHub repo: deep learning optimisation library for large-scale training and inference, origin of ZeRO.
py-why/dowhy
by PyWhy
PyWhy DoWhy GitHub repo: Python library for causal inference with explicit assumptions and refutation tests.
Etched
by Etched
Etched: company building transformer-specific AI inference chips, racks and software.
FastDeploy
by PaddlePaddle
PaddlePaddle FastDeploy GitHub repo: inference and deployment toolkit for LLMs and VLMs.
Fireworks cookbook
by Fireworks AI
Fireworks cookbook GitHub repo: recipes for fine-tuning, function calling and building with Fireworks AI.
FlashMLA
by DeepSeek
DeepSeek FlashMLA GitHub repo: multi-head latent attention decoding kernels for Hopper GPUs.
FlashQLA
by Qwen
QwenLM FlashQLA GitHub repo: linear attention kernel library built on TileLang.
Huawei Ascend
by Huawei
Huawei Ascend community site: Ascend AI processors, CANN software stack, frameworks and developer tools.
KTransformers
by kvcache.ai
KTransformers GitHub repo: framework for heterogeneous CPU-GPU inference of large LLMs.
Kunlunxin (昆仑芯)
Kunlunxin site: Baidu-affiliated maker of AI accelerator chips (in Chinese).
Liger Kernel
by LinkedIn
LinkedIn Liger Kernel GitHub repo: Triton kernels for LLM training.
Liquid AI research
by Liquid AI
Liquid AI research page: papers and technical reports on Liquid Foundation Models.
llm.c
by Andrej
llm.c GitHub repo (Andrej Karpathy): LLM training in plain C and CUDA.
LMDeploy
by InternLM
InternLM LMDeploy GitHub repo: toolkit for compressing, deploying and serving LLMs.
llama.cpp
by ggml
llama.cpp GitHub repo: LLM inference in C/C++ for local and on-device hardware.
Megatron-LM
NVIDIA Megatron-LM GitHub repo: research code for training transformer models at scale with tensor and pipeline parallelism.
MiniCPM
by OpenBMB
OpenBMB MiniCPM GitHub repo: on-device small language models with deployment resources.
MiniCPM-V
by OpenBMB
OpenBMB MiniCPM-V GitHub repo: on-device multimodal models MiniCPM-V and MiniCPM-o for vision and speech.
MLX / mlx-lm
by ml-explore
Apple mlx-lm GitHub repo: Python package for running and fine-tuning LLMs with MLX.
MobileCLIP research implementation
Apple ml-mobileclip GitHub repo: official implementation of MobileCLIP efficient image-text models.
Mooncake
by kvcache.ai
Mooncake GitHub repo: KV-cache-centric disaggregated serving platform used for Kimi.
Motif Technologies
Motif Technologies GitHub organisation: training and serving code for its Korean models.
Multiverse Computing model registry
Multiverse Computing Hugging Face organisation: compressed model cards such as HyperNova.