tiny-vllm is an open-source LLM inference tool from jmaczan, licensed under Apache-2.0. The repository has collected 1.2k stars and 99 forks; the latest push was in September 2026. Topics include ai, attention and batching.
“Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM”
— jmaczan, project description on GitHub
Details
| Category | Local LLMs & Inference |
|---|---|
| GitHub stars | 1,156 |
| Forks | 99 |
| Language | C++ |
| License | Apache-2.0 |
| Last updated | |
| Owner | jmaczan |
| Repository | jmaczan/tiny-vllm |
Tags
aiattentionbatchingcoursecppcudahpcinference
Repository facts from the GitHub public API, refreshed October 18, 2026. The quoted tagline is written by the project owner; the summary above is generated from public facts.