--- id: "/tags/inference-serving/page/2/" title: "Inference Serving · page 2" creator: null creator_url: null creator_x: null url: "https://skillissue.dev/tags/inference-serving/page/2/" level: null level_max: null category: null subcategory: null region: null license: null license_scope: null access: null models: null source_date: null date_added: null resource_kind: null verification_basis: null access_evidence: null domain_top: null domain_sub: null last_verified: null --- # Inference Serving [Beginner](/levels/beginner/index.md) [Intermediate](/levels/intermediate/index.md) [Advanced](/levels/advanced/index.md) [Expert](/levels/expert/index.md) 58 resources · page 2 of 2 - ### [Qwen 中文文档](/model-training/training-infrastructure/qwen/index.md) Creator not supplied; see original source. Qwen documentation in Chinese: usage, deployment and fine-tuning of Qwen language and multimodal models. Intermediate · unknown · unknown resource · zh - ### [Transformers documentation](/model-training/training-infrastructure/transformers-documentation/index.md) Creator not supplied; see original source. Hugging Face Transformers documentation: reference for model architectures, loading, training and generation. Intermediate · unknown · unknown resource · en - ### [Agentic inference optimization (Baseten)](/model-training/training-infrastructure/baseten-agentic-inference-optimization/index.md) Creator not supplied; see original source. Baseten post on optimising inference engines for agent workloads, reporting decode-speed and latency gains over vLLM. Advanced · unknown · unknown resource · en - ### [Baseten documentation](/model-training/training-infrastructure/baseten-documentation/index.md) Creator not supplied; see original source. Baseten documentation for running hosted models, deploying custom models and training on GPU infrastructure. Advanced · unknown · unknown resource · en - ### [BentoML documentation](/model-training/training-infrastructure/bentoml-documentation/index.md) Creator not supplied; see original source. BentoML documentation: open-source inference platform for packaging, deploying and scaling AI models. Advanced · unknown · unknown resource · en - ### [Defeating Nondeterminism in LLM Inference](/model-training/training-infrastructure/defeating-nondeterminism-in-llm-inference/index.md) Creator not supplied; see original source. Thinking Machines Lab post on the causes of nondeterminism in LLM inference and batch-invariant kernels to fix it. Advanced · unknown · unknown resource · en - ### [DoWhy documentation](/model-training/training-infrastructure/dowhy-2/index.md) Creator not supplied; see original source. DoWhy documentation: Python library for causal inference with modelling, identification, estimation and refutation steps. Advanced · unknown · unknown resource · en - ### [EconML](/model-training/training-infrastructure/econml/index.md) Creator not supplied; see original source. EconML documentation: Python methods for estimating heterogeneous treatment effects with machine learning. Advanced · unknown · unknown resource · en - ### [Feast documentation](/model-training/training-infrastructure/feast-documentation/index.md) Creator not supplied; see original source. Feast documentation: open-source feature store for serving consistent features to ML training and inference. Advanced · unknown · unknown resource · en - ### [Fireworks AI documentation](/model-training/training-infrastructure/fireworks-ai-documentation/index.md) Creator not supplied; see original source. Fireworks AI documentation: serverless and dedicated inference, fine-tuning and deployment of open models. Advanced · unknown · unknown resource · en - ### [Three Tests to Run Before You Switch from LoRA to FullFT (Fireworks AI)](/model-training/training-infrastructure/fireworks-three-tests-before-you-switch-from-lora-to-fullft/index.md) Creator not supplied; see original source. Fireworks AI post with controlled experiments on Qwen3.5-9B showing what to test before switching from LoRA to full fine-tuning. Advanced · unknown · unknown resource · en - ### [Grouped-Query Attention](/model-training/training-infrastructure/grouped-query-attention/index.md) Creator not supplied; see original source. arXiv paper (2023) introducing grouped-query attention (GQA) and uptraining multi-head models to it. Advanced · unknown · unknown resource · en - ### [Scaling to 1 million concurrent sandboxes in seconds (Modal)](/model-training/training-infrastructure/modal-scaling-to-1m-concurrent-sandboxes/index.md) Creator not supplied; see original source. Modal engineering post on building a scheduler that runs one million concurrent code sandboxes per workspace. Advanced · unknown · unknown resource · en - ### [Ray Serve documentation](/model-training/training-infrastructure/ray-serve-documentation/index.md) Creator not supplied; see original source. Ray Serve documentation: scalable library for building online inference APIs and composing model deployments. Advanced · unknown · unknown resource · en - ### [SGLang docs: HiCache best practices](/model-training/training-infrastructure/sglang-hicache-best-practices/index.md) Creator not supplied; see original source. SGLang HiCache best practices: configuring hierarchical KV caching across GPU, CPU and storage. Advanced · unknown · unknown resource · en - ### [AdapTive-LeArning Speculator System (ATLAS) (Together AI)](/model-training/training-infrastructure/together-ai-atlas-runtime-learning-speculators/index.md) Creator not supplied; see original source. Together AI post on ATLAS, a speculative decoding system whose speculators keep learning from the live workload. Advanced · unknown · unknown resource · en - ### [Cache-aware prefill-decode disaggregation (Together AI)](/model-training/training-infrastructure/together-ai-cache-aware-prefill-decode-disaggregation/index.md) Creator not supplied; see original source. Together AI post on cache-aware prefill-decode disaggregation for faster long-context inference serving. Advanced · unknown · unknown resource · en - ### [Together AI documentation](/model-training/training-infrastructure/together-ai-documentation/index.md) Creator not supplied; see original source. Together AI documentation for running, fine-tuning and serving open-source models. Advanced · unknown · unknown resource · en - ### [vLLM docs: Automatic prefix caching](/model-training/training-infrastructure/vllm-automatic-prefix-caching-design/index.md) Creator not supplied; see original source. vLLM design document on automatic prefix caching and how shared prompt prefixes reuse the KV cache. Advanced · unknown · unknown resource · en - ### [vLLM docs](/model-training/training-infrastructure/vllm-docs/index.md) Creator not supplied; see original source. vLLM latest documentation: installation, serving, supported models and quantisation for the LLM inference engine. Advanced · unknown · unknown resource · en - ### [CMU 11-868 LLM Systems](/model-training/training-infrastructure/cmu-11-868-llm-systems/index.md) Creator not supplied; see original source. CMU 11-868 LLM Systems course site: systems for training and serving large language models, with schedule and readings. Expert · unknown · unknown resource · en - ### [EAGLE](/model-training/training-infrastructure/eagle/index.md) Creator not supplied; see original source. arXiv paper (2024) introducing EAGLE, speculative sampling with feature-level draft heads. Expert · unknown · unknown resource · en - ### [Fast Inference from Transformers via Speculative Decoding](/model-training/training-infrastructure/fast-inference-from-transformers-via-speculative-decoding/index.md) Creator not supplied; see original source. arXiv paper (2022) introducing speculative decoding to speed up transformer inference without changing outputs. Expert · unknown · unknown resource · en - ### [How to Scale Your Model (JAX scaling book)](/model-training/training-infrastructure/how-to-scale-your-model-jax-scaling-book/index.md) Creator not supplied; see original source. How to Scale Your Model: free online book on TPU/GPU rooflines, transformer math, parallelism and serving for LLMs. Expert · unknown · unknown resource · en - ### [PagedAttention / vLLM paper](/model-training/training-infrastructure/pagedattention-vllm-paper/index.md) Creator not supplied; see original source. arXiv paper (2023) introducing PagedAttention and the vLLM serving system for efficient KV-cache memory management. Expert · unknown · unknown resource · en - ### [SGLang docs](/model-training/training-infrastructure/sglang-docs/index.md) Creator not supplied; see original source. SGLang documentation: LLM serving framework with RadixAttention prefix caching and fast structured outputs. Expert · unknown · unknown resource · en [Previous page](/tags/inference-serving/index.md) [Plain-text Markdown copy](/tags/inference-serving/page/2/index.md)