Local LLMs & Inference

GenerativeAIExamples

by NVIDIA · Jupyter Notebook

Built in Jupyter Notebook, GenerativeAIExamples is an open-source LLM inference tool maintained by NVIDIA. The repository has collected 4.2k stars and 1.1k forks; the latest push was in September 2026. Topics include gpu-acceleration, large-language-models and llm.

“Generative AI reference workflows optimized for accelerated infrastructure and microservice architecture.”

— NVIDIA, project description on GitHub

Details

CategoryLocal LLMs & Inference
GitHub stars4,209
Forks1,100
LanguageJupyter Notebook
LicenseApache-2.0
Last updated
OwnerNVIDIA
RepositoryNVIDIA/GenerativeAIExamples

Tags

gpu-accelerationlarge-language-modelsllmllm-inferencemicroservicenemoragretrieval-augmented-generation

Repository facts from the GitHub public API, refreshed October 18, 2026. The quoted tagline is written by the project owner; the summary above is generated from public facts.

All Local LLMs & Inference

Rapid-MLX

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple…

Local LLMs & Inference4k stars

AI-Engineer-Headquarters

A collection of scientific methods, processes, algorithms, and systems to build stories & models.

Local LLMs & Inference3.8k stars

distributed-llama

Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More…

Local LLMs & Inference3.1k stars

spiceai

Add a real-time analytics node to your operational database. Spice is a portable, accelerated SQL query…

Local LLMs & Inference3.1k stars

Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

Local LLMs & Inference2.8k stars