LLM Frameworks & MLOps: 58 open-source tools

litellm

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format…

LLM Frameworks & MLOps62k

Made-With-ML

Learn how to develop, deploy and iterate on production-grade ML applications.

LLM Frameworks & MLOps50k

langfuse

🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open…

LLM Frameworks & MLOps36k

mlflow

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes…

LLM Frameworks & MLOps28k

opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive…

LLM Frameworks & MLOps23k

serve

☁️ Build multimodal AI applications with cloud-native stack

LLM Frameworks & MLOps22k

SuperAGI

SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run…

LLM Frameworks & MLOps18k

RagaAI-Catalyst

Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent…

LLM Frameworks & MLOps16k

ragas

Supercharge Your LLM Application Evaluations 🚀

LLM Frameworks & MLOps16k

gateway

A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast…

LLM Frameworks & MLOps13k

phoenix

AI Observability & Evaluation

LLM Frameworks & MLOps12k

metaflow

Build, Manage and Deploy AI/ML Systems

LLM Frameworks & MLOps10k

Soup

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

LLM Frameworks & MLOps9.3k

rig

⚙️🦀 Build modular and scalable LLM Applications in Rust

LLM Frameworks & MLOps8.9k

evidently

Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered…

LLM Frameworks & MLOps8k

openllmetry

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

LLM Frameworks & MLOps7.5k

clearml

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management…

LLM Frameworks & MLOps6.9k

agent-starter-pack

Ship AI Agents to Google Cloud in minutes, not months. Production-ready templates with built-in CI/CD…

LLM Frameworks & MLOps6.6k

helicone

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

LLM Frameworks & MLOps6.2k

Awesome-LLMOps

An awesome & curated list of best LLMOps tools for developers

LLM Frameworks & MLOps6k

giskard-oss

🐢 Open-Source Evaluation & Testing library for LLM Agents

LLM Frameworks & MLOps5.9k

coze-loop

Next-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by…

LLM Frameworks & MLOps5.8k

zenml

ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.

LLM Frameworks & MLOps5.6k

LLM-Engineers-Handbook

The LLM's practical guide: From the fundamentals to deploying advanced LLM and RAG apps to AWS using LLMOps…

LLM Frameworks & MLOps5.4k

cube-studio

cube…

LLM Frameworks & MLOps5.1k

langwatch

Open-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent…

LLM Frameworks & MLOps5k

llm-twin-course

🤖 𝗟𝗲𝗮𝗿𝗻 for 𝗳𝗿𝗲𝗲 how to 𝗯𝘂𝗶𝗹𝗱 an end-to-end 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻-𝗿𝗲𝗮𝗱𝘆 𝗟𝗟𝗠 & 𝗥𝗔𝗚 𝘀𝘆𝘀𝘁𝗲𝗺 using 𝗟𝗟𝗠𝗢𝗽𝘀 best practices: ~…

LLM Frameworks & MLOps4.4k

Acontext

Agent Skills as a Memory Layer

LLM Frameworks & MLOps3.7k

trulens

Evaluation and Tracking for LLM Experiments and AI Agents

LLM Frameworks & MLOps3.6k

lmnr

Laminar - open-source observability platform purpose-built for AI agents. YC S24.

LLM Frameworks & MLOps3.4k

pezzo

🕹️ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management…

LLM Frameworks & MLOps3.3k

AGiXT

AGiXT is a dynamic AI Agent Automation Platform that seamlessly orchestrates instruction management and…

LLM Frameworks & MLOps3.2k

second-brain-ai-assistant-course

Learn to build your Second Brain AI assistant with LLMs, agents, RAG, fine-tuning, LLMOps and AI systems…

LLM Frameworks & MLOps3.1k

ChainForge

An open-source visual programming environment for battle-testing prompts to LLMs.

LLM Frameworks & MLOps3k

vllm-ascend

Community maintained hardware plugin for vLLM on Huawei Ascend

LLM Frameworks & MLOps3k

openlit

OpenLIT is the open-source agent harness engineering platform: trace, evaluate, guard, and improve…

LLM Frameworks & MLOps2.9k

OpenPipe

Turn expensive prompts into cheap fine-tuned models

LLM Frameworks & MLOps2.8k

hamilton

Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows…

LLM Frameworks & MLOps2.6k

burr

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and…

LLM Frameworks & MLOps2.6k

cube-studio

cubestudio开源云原生一站式机器学习/深度学习/大模型AI平台/MaaS/mlops/人工智能平台/训推平台，算法全链路流程，多租户，算力租赁平台，token中转，拖拉拽任务流pipeline编排，多机多卡分布…

LLM Frameworks & MLOps2.6k

bionic-gpt

Bionic is sovereign Agentic AI for the enterprise — Runs on-premise and can securely work with your…

LLM Frameworks & MLOps2.4k

uptrain

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide…

LLM Frameworks & MLOps2.4k

nextpy

🤖Self-Modifying Framework from the Future 🔮 World's First AMS

LLM Frameworks & MLOps2.3k

LLMStack

No-code multi-agent framework to build LLM Agents, workflows and applications with your data

LLM Frameworks & MLOps2.3k

envd

🏕️ Reproducible development environment for humans and agents

LLM Frameworks & MLOps2.2k

future-agi

Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications.…

LLM Frameworks & MLOps2.2k

waku-agent

Waku Waku! Waku Agent is a local-first AI agent harness you actually own, including loop, memory, eval, all…

LLM Frameworks & MLOps2k

VectorChord

Scalable, fast, and disk-friendly vector search in Postgres, the successor of pgvecto.rs.

LLM Frameworks & MLOps1.8k

paddler

Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙…

LLM Frameworks & MLOps1.7k

Agently

[GenAI Application Development Framework] 🚀 Build GenAI application quick and easy 💬 Easy to interact with…

LLM Frameworks & MLOps1.7k

better-agents

Standards for building agents, better

LLM Frameworks & MLOps1.6k

rtp-llm

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

LLM Frameworks & MLOps1.4k

kusion

Declarative Intent Driven Platform Orchestrator for Internal Developer Platform (IDP).

LLM Frameworks & MLOps1.3k

sre

The SmythOS Runtime Environment (SRE) is an open-source, cloud-native runtime for agentic AI. Secure…

LLM Frameworks & MLOps1.3k

openinference

OpenTelemetry Instrumentation for AI Observability

LLM Frameworks & MLOps1.3k

intellagent

A framework for comprehensive diagnosis and optimization of agents using simulated, realistic synthetic…

LLM Frameworks & MLOps1.3k

langtrace

Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability tool for LLM applications…

LLM Frameworks & MLOps1.2k

awesome-ai-sdks

A database of SDKs, frameworks, libraries, and tools for creating, monitoring, debugging and deploying…

LLM Frameworks & MLOps1.2k

More categories