Introduction | LLM Inference in Production

Related

• LLM Inference

  Note · 2025-07-11

  LLM Inference in Production by BentoML; "core concepts and performance metrics, to optimization techniques and…

• truespar/paddock: Native Rust inference server for open models on NVIDIA and Apple Silicon.

  github.com · 2026-11-01 · Page

  OpenAI- and Anthropic-compatible APIs, GGUF + safetensors + MLX, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio

• A Simple Guide to Retrieval Augmented Generation

  manning.com · 2025-03-14 · Page

  Book outline: The components of a RAG system How to create a RAG knowledge base The indexing and generation pipeline…

• Reinforcement Learning: An Introduction : Andrew Barto and Richard S. Sutton : Internet Archive

  archive.org · 2025-03-05 · Page

  Textbook, second edition

• [2501.09223] Foundations of Large Language Models

  arxiv.org · 2025-01-18 · Page

• Grok

  Note · 2026-05-17

  Connect Grok to Hermes Agent