Local LLMs & Inference

plano

by katanemo · Rust

plano is an open-source LLM inference tool published on GitHub by katanemo. It has 7.1k GitHub stars and 488 forks, and was last updated in October 2026. Topics include ai-gateway, ai-gateway-support and envoy.

“Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.”

— katanemo, project description on GitHub

Details

CategoryLocal LLMs & Inference
GitHub stars7,092
Forks488
LanguageRust
LicenseApache-2.0
Last updated
Ownerkatanemo
Websiteplanoai.dev
Repositorykatanemo/plano
Also listed inLLM Frameworks & MLOps

Tags

ai-gatewayai-gateway-supportenvoyenvoyproxygatewaygenerative-aillm-gatewayllm-inference

Repository facts from the GitHub public API, refreshed October 18, 2026. The quoted tagline is written by the project owner; the summary above is generated from public facts.

All Local LLMs & Inference

turbo-fieldfare

Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook

Local LLMs & Inference6.9k stars

ai-infra-book

《深入理解 AI Infra:量化分析与系统设计》(李博杰 著)开源书稿:从硬件约束和模型架构出发,量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验

Local LLMs & Inference6.7k stars

flashinfer

FlashInfer: Kernel Library for LLM Serving

Local LLMs & Inference6.7k stars

kserve

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework…

Local LLMs & Inference6.1k stars

cactus

Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.

Local LLMs & Inference6.1k stars

shimmy

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No…

Local LLMs & Inference6k stars