OpenAlpha_Evolve
OpenAlpha_Evolve is an open-source Python framework inspired by the groundbreaking research on autonomous…
Local LLMs & Inference1.1k stars
Local LLMs & Inference, page 2
71 open-source LLM inference tools, ranked by GitHub stars. Most popular: gpt4all, ray and gitleaks. Inference engines and local runners serve open models on your own hardware, from laptops to GPU clusters.
Showing 61–71 of 71 tools
backburner
Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable
Local LLMs & Inference1k stars
project-maya
Project Maya - run GLM-5.3-Flash (321B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or…
Local LLMs & Inference1k stars
time-to-first-token
A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization…
Local LLMs & Inference994 stars
AI-Compass
“AI-Compass”将为社区指引在 AI 技术海洋中航行的方向，无论你是初学者还是进阶开发者，都能在这里找到通往 AI 各大方向的路径。旨在帮助开发者系统性地了解 AI…
Local LLMs & Inference983 stars
awesome-hosting
List of awesome hosting sorted by minimal plan price
Local LLMs & Inference944 stars
chatllm.cpp
Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)
Local LLMs & Inference933 stars
ZhiLight
A highly optimized LLM inference acceleration engine for Llama and its variants.
Local LLMs & Inference907 stars
llm_note
LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.
Local LLMs & Inference893 stars
LLM.swift
LLM.swift is a simple and readable library that allows you to interact with large language models locally…
Local LLMs & Inference882 stars
lws
LeaderWorkerSet: An API for deploying a group of pods as a unit of replication
Local LLMs & Inference882 stars
More categories
AI Agents 73RAG & Search 59Coding Assistants 72Image & Video Generation 71Speech & Audio 68Chat UIs 53LLM Frameworks & MLOps 58