Local LLMs & Inference

chatllm.cpp

by foldl · C++

View on GitHub

chatllm.cpp is an open-source LLM inference tool written in C++, released under the MIT license. The repository has collected 933 stars and 76 forks; the latest push was in October 2026. Topics include llm and llm-inference.

“Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)”

— foldl, project description on GitHub

Details

CategoryLocal LLMs & Inference
GitHub stars933
Forks76
LanguageC++
LicenseMIT
Last updated
Ownerfoldl
Repositoryfoldl/chatllm.cpp

Tags

llmllm-inference

Repository facts from the GitHub public API, refreshed October 18, 2026. The quoted tagline is written by the project owner; the summary above is generated from public facts.

All Local LLMs & Inference

ZhiLight

A highly optimized LLM inference acceleration engine for Llama and its variants.

Local LLMs & Inference907 stars

llm_note

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

Local LLMs & Inference893 stars

LLM.swift

LLM.swift is a simple and readable library that allows you to interact with large language models locally…

Local LLMs & Inference882 stars

lws

LeaderWorkerSet: An API for deploying a group of pods as a unit of replication

Local LLMs & Inference882 stars

gpt4all

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

Local LLMs & Inference77k stars

ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for…

Local LLMs & Inference44k stars