Local LLMs & Inference

openvino

by openvinotoolkit · C++

openvino is an open-source LLM inference tool published on GitHub by openvinotoolkit. It has 11k GitHub stars and 3.5k forks, and was last updated in October 2026. Topics include ai, computer-vision and deep-learning.

“OpenVINO™ is an open source toolkit for optimizing and deploying AI inference”

— openvinotoolkit, project description on GitHub

Details

CategoryLocal LLMs & Inference
GitHub stars11,019
Forks3,469
LanguageC++
LicenseApache-2.0
Last updated
Owneropenvinotoolkit
Websitedocs.openvino.ai
Repositoryopenvinotoolkit/openvino
Also listed inImage & Video Generation

Tags

aicomputer-visiondeep-learningdeploy-aidiffusion-modelsgenerative-aigood-first-issueinference

Repository facts from the GitHub public API, refreshed October 18, 2026. The quoted tagline is written by the project owner; the summary above is generated from public facts.

All Local LLMs & Inference

PowerInfer

High-speed Large Language Model Serving for Local Deployment

Local LLMs & Inference9.8k stars

kimi-k3-in-c

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no…

Local LLMs & Inference9.1k stars

BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model…

Local LLMs & Inference8.9k stars

dynamo

A Datacenter Scale Distributed Inference Serving Framework

Local LLMs & Inference8.3k stars

lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Local LLMs & Inference8.1k stars

openevolve

Open-source implementation of AlphaEvolve

Local LLMs & Inference7.6k stars