A list of 109+ LLM services, tools, and infrastructure for running AI locally. Criteria for inclusion:
- Open Source
- Self-hostable
- Friendly to containerization (Docker, Podman, etc.)
- Relates to homelab or personal AI use cases
- Well-documented with setup instructions
Relevance score (
0–100%): a composite metric of Popularity (logarithm of stars) and Recency (exponential decay with a 90-day half-life). This highlights projects that are both widely recognized and actively maintained.
Frontends - Chat interfaces and web UIs (17)
Backends - Inference engines and model servers (17)
Satellites - Companion services and integrations (70)
Workflow & Automation - Visual programming platforms (12)
API & Proxies - LLM gateways and aggregators (9)
Audio & Speech - TTS and STT services (4)
CLI Tools - Terminal-based LLM tools (16)
Evaluation - Benchmarking and testing (2)
MCP Tools - Model Context Protocol (5)
Chat interfaces and web applications for interacting with language models.
90% 
53.5k 
issues 3.1k (249 open, 2.8k closed) 
2026-01-18
The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more.
79% 15.1k issues 3.4k (393 open, 3.0k closed) 2026-01-18
All-in-one agentic chatbot platform for multi-LLM conversations across messaging platforms with plugin system.
64% 
2.3k issues 405 (17 open, 388 closed) 2026-01-19
on-premise LLM web UI with support for OpenAI-compatible backends
43% 8.9k issues 309 (23 open, 286 closed) 2025-11-08
Comprehensive LLM web interface with built-in marketplace
96% 100.7k issues 7.6k (3.3k open, 4.3k closed) 2026-01-19
The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
27% 1.1k issues 203 (65 open, 138 closed) 2025-10-12
A minimal web-UI for talking to Ollama servers.
77% 10.4k issues 717 (325 open, 392 closed) 2026-01-19
A chat interface using open source models, eg OpenAssistant or Llama. It is a SvelteKit app and it powers the HuggingChat app on hf.co/chat.
75% 9.3k issues 1.3k (385 open, 873 closed) 2026-01-18
KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models.
86% 33.2k issues 4.0k (218 open, 3.8k closed) 2026-01-18
Open-source ChatGPT UI alternative supporting multiple AI providers (Anthropic, AWS, OpenAI, Azure, Groq, Mistral, Google) with features like model switching, message search, and multi-user support. Includes integration with DALL-E-3 and various APIs.
93% 70.3k issues 5.1k (1.0k open, 4.1k closed) 2026-01-19
An open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / Azure / DeepSeek), Knowledge Base (file upload / knowledge management / RAG ), Multi-Modals (Vision/TTS) and plugin system.
45% 
682 issues 66 (17 open, 49 closed) 2025-12-27
LLM Frontend in a single HMTL file
63% 2.3k issues 162 (47 open, 115 closed) 2026-01-17
A simple Gradio app implementing an o1-like chain of reasoning with Ollama.
7% 361 issues 11 (1 open, 10 closed) 2025-05-13
Visual programming for AI language models
81% 17.1k issues 1.0k (110 open, 890 closed) 2026-01-19
Open Source AI Platform with Chat UI, RAG, MCP support, and 40+ document connectors.
91% 121.2k issues 7.4k (172 open, 7.2k closed) 2026-01-10
widely adopted and feature rich web interface for interacting with LLMs. Supports OpenAI-compatible and Ollama backends, multi-users, multi-model chats, custom prompts, TTS, Web RAG, RAG, and much much more.
51% 2.3k issues 135 (9 open, 126 closed) 2025-12-19
The text-based terminal client for Ollama.
35% 408 issues 39 (8 open, 31 closed) 2025-12-03
TUI for Ollama
Inference engines and model serving platforms. These power the actual LLM responses.
26% 7.5k issues 198 (119 open, 79 closed) 2025-09-03
70B inference with single 4GB GPU (very slow, though)
61% 1.6k issues 244 (80 open, 164 closed) 2026-01-19
Large-scale LLM inference engine
55% 2.8k issues 293 (80 open, 213 closed) 2025-12-27
Legacy version of Speaches, use that instead.
75% 9.3k issues 1.3k (385 open, 873 closed) 2026-01-18
KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models.
79% 16.4k issues 1.1k (395 open, 737 closed) 2026-01-16
A Flexible Framework for Experiencing Cutting-edge LLM Inference Optimizations
95% 93.3k issues 6.7k (361 open, 6.4k closed) 2026-01-19
LLM inference in C/C++
73% 6.4k issues 562 (176 open, 386 closed) 2026-01-19
Blazingly fast LLM inference.
84% 25.5k issues 3.1k (694 open, 2.4k closed) 2026-01-19
MAX is a platform from Modular (creators of Mojo) for running LLMs.
72% 7.5k issues 260 (25 open, 235 closed) 2026-01-16
Nexa SDK is a comprehensive toolkit for supporting ONNX and GGML models.
99% 159.9k issues 9.0k (1.9k open, 7.1k closed) 2026-01-19
Get up and running with Llama 3.2, Mistral, Gemma 3, and other large language models.
4% 844 issues 85 (7 open, 78 closed) 2025-02-02
An OpenAI API compatible text to speech server using Coqui AI's xtts_v2 and/or piper tts as the backend.
3% 5.5k issues 154 (115 open, 39 closed) 2024-12-10
Inference and training library for high-quality TTS models.
83% 22.6k issues 4.5k (652 open, 3.8k closed) 2026-01-19
SGLang is a fast serving framework for large language models and vision language models.
55% 2.8k issues 293 (80 open, 213 closed) 2025-12-27
an OpenAI API-compatible speech server (formerly faster-whisper-server), both TTS and STT
46% 1.1k issues 228 (23 open, 205 closed) 2025-12-19
An OAI compatible exllamav2 API that's both lightweight and fast
71% 10.7k issues 1.6k (282 open, 1.3k closed) 2026-01-08
Inference engine from HuggingFace.
92% 67.9k issues 13.0k (1.7k open, 11.3k closed) 2026-01-19
A high-throughput and memory-efficient inference and serving engine for LLMs
Companion services, research tools, and integrations that enhance LLM workflows.
82% 20.4k issues 2.6k (265 open, 2.3k closed) 2026-01-19
Open-source workflow automation platform with AI capabilities and 200+ app connectors.
67% 13.7k issues 448 (124 open, 324 closed) 2025-12-29
General-purpose personal assistant with Web RAG, persistent memory, tools, browser use and more.
68% 9.1k issues 632 (21 open, 611 closed) 2026-01-06
All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI tools & agents.
88% 39.9k issues 3.9k (1.1k open, 2.8k closed) 2026-01-19
Aider is AI pair programming in your terminal.
72% 5.6k issues 88 (36 open, 52 closed) 2026-01-19
Airweave lets agents search any app by transforming its contents into agent-ready knowledge.
79% 15.1k issues 3.4k (393 open, 3.0k closed) 2026-01-18
All-in-one agentic chatbot platform for multi-LLM conversations across messaging platforms with plugin system.
97% 181.3k issues 3.7k (226 open, 3.4k closed) 2026-01-14
Create, deploy, and manage continuous AI agents that automate complex workflows.
42% 18.9k issues 943 (59 open, 884 closed) 2025-10-23
Prompt, run, edit, and deploy full-stack web applications.
27% 15.5k issues 433 (259 open, 174 closed) 2025-08-31
AI-powered browser automation with web UI
78% 12.7k issues 1.3k (451 open, 825 closed) 2026-01-19
A helper service allowing to expose Harbor services over the internet.
13% 121 issues 15 (8 open, 7 closed) 2025-08-23
Create Linux commands from natural language, in the shell.
81% 19.2k issues 433 (178 open, 255 closed) 2026-01-17
Community-driven deep research framework combining LLMs with web search, crawling, and multi-agent workflows for comprehensive research reports.
98% 126.5k issues 16.0k (438 open, 15.6k closed) 2026-01-19
An open-source LLM app development platform.
90% 50.5k issues 1.5k (774 open, 769 closed) 2026-01-19
Transform documents into format ready for LLMs.
81% 18.9k issues 210 (82 open, 128 closed) 2026-01-18
AI-powered diagram creation tool - generate draw.io diagrams from natural language.
88% 38.3k issues 812 (22 open, 790 closed) 2026-01-19
LLM-driven processing of the text data in the terminal.
88% 48.3k issues 2.5k (637 open, 1.8k closed) 2026-01-16
Drag & drop UI to build your customized LLM flow.
69% 4.1k issues 333 (28 open, 305 closed) 2026-01-19
A simple CLI tool to interact with LLMs.
63% 2.3k issues 162 (47 open, 115 closed) 2026-01-17
Harbor's own tool to evaluate LLMs and inference backends against custom tasks.
63% 2.3k issues 162 (47 open, 115 closed) 2026-01-17
Connects to downstream LLM API and serves a wrapper with custom workflow. For example, it can be used to add a CoT (Chain of Thought) to an existing LLM API, and much more. Scriptable with Python.
94% 84.3k issues 66.9k (2.7k open, 64.3k closed) 2026-01-19
Open source home automation platform for managing and controlling smart home devices.
80% 15.0k issues 9.0k (2.4k open, 6.6k closed) 2026-01-19
Helper service to author/run Jupyter notebooks in Python with access to Harbor services.
86% 29.7k issues 2.5k (736 open, 1.8k closed) 2026-01-19
A modern load testing tool, using Go and JavaScript - https://k6.io
83% 22.9k issues 1.5k (492 open, 968 closed) 2026-01-19
Self-hosted bookmark manager with AI-powered automatic tagging via OpenAI or Ollama.
78% 32.2k issues 559 (77 open, 482 closed) 2026-01-06
AI second brain for chat, search, and agents with your docs. Supports local and cloud LLMs.
75% 9.3k issues 1.3k (385 open, 873 closed) 2026-01-18
KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models.
18% 24.9k issues 455 (207 open, 248 closed) 2025-07-02
An open-source RAG-based tool for chatting with your documents.
99% 144.0k issues 3.1k (350 open, 2.7k closed) 2026-01-19
A low-code app builder for RAG and multi-agent AI applications.
83% 20.8k issues 2.2k (312 open, 1.9k closed) 2026-01-19
Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets.
34% 748 issues 74 (34 open, 40 closed) 2025-11-17
A new kind of workflow + tool for visualizing and exploring datasets through the lens of latent spaces.
78% 13.6k issues 548 (127 open, 421 closed) 2026-01-18
A free and open-source machine translation.
87% 34.1k issues 8.5k (825 open, 7.7k closed) 2026-01-19
LLM proxy that can aggregate multiple inference APIs together into a single endpoint.
2% 103 issues 16 (3 open, 13 closed) 2024-11-25
Simple analytics platform that leverages LLMs to automate data analysis.
64% 2.2k issues 243 (10 open, 233 closed) 2026-01-19
Runs multiple llama.cpp servers on demand for seamless switching between them.
77% 11.2k issues 1.6k (539 open, 1.1k closed) 2026-01-19
A de-facto standard framework for the few-shot evaluation of language models.
69% 3.8k issues 243 (36 open, 207 closed) 2026-01-19
Transforms complex questions into comprehensive, cited reports.
88% 42.2k issues 1.2k (144 open, 1.1k closed) 2026-01-19
Complete AI stack for running AI models locally. Allows downloading variety of LLMs, TTS/STT/Image models and running thme locally via Web UI.
67% 3.1k issues 1.0k (382 open, 647 closed) 2026-01-19
Gateway and admin UI for managing Model Context Protocol (MCP) servers, tools, and resources.
33% 3.9k issues 125 (37 open, 88 closed) 2025-10-14
Turn MCP servers into OpenAPI REST APIs - use them anywhere.
51% 1.9k issues 154 (59 open, 95 closed) 2025-12-23
Allows to manage MCPs via a WebUI, exposes multiple MCPs as a single server.
88% 38.3k issues 4.4k (67 open, 4.3k closed) 2026-01-19
AI platform for integrating ML models with data sources via HTTP and MySQL APIs.
57% 8.5k issues 253 (55 open, 198 closed) 2025-12-15
An AI-powered search engine with a generative UI, similar to Perplexity and Perplexica.
100% 170.0k issues 7.6k (490 open, 7.1k closed) 2026-01-19
Fair-code workflow automation platform with native AI capabilities.
93% 77.4k issues 8.2k (170 open, 8.0k closed) 2026-01-19
Real-time infrastructure monitoring with per-second metrics for systems, containers, and applications.
30% 24.2k issues 242 (173 open, 69 closed) 2025-09-09
A simple screen parsing tool towards pure vision based GUI agent.
65% 61.7k issues 1.0k (239 open, 803 closed) 2025-12-05
A natural language interface for computers.
81% 18.2k issues 244 (78 open, 166 closed) 2026-01-18
AI-powered research and note-taking platform with multi-provider LLM support, podcast generation, and content analysis.
20% 2.3k issues 272 (147 open, 125 closed) 2025-08-18
UI-Agnostic OpenAI API Plugin Framework.
94% 78.5k issues 5.8k (2.3k open, 3.5k closed) 2026-01-19
AI coding assistant with server API, TUI, and IDE extensions. Supports multiple LLM providers.
92% 66.8k issues 3.7k (141 open, 3.5k closed) 2026-01-19
A platform for software development agents powered by AI.
55% 3.3k issues 88 (12 open, 76 closed) 2025-12-25
Optimising LLM proxy that implements many advanced workflows to boost the performance of the LLMs.
79% 28.3k issues 600 (171 open, 429 closed) 2026-01-10
An AI-powered search engine. It is an Open source alternative to Perplexity AI.
86% 39.1k issues 2.5k (420 open, 2.0k closed) 2026-01-16
AI-powered photo management app with face recognition, image classification, and automatic organization.
35% 14.9k issues 195 (29 open, 166 closed) 2025-10-03
AI driven development in your terminal.
61% 3.7k issues 162 (45 open, 117 closed) 2026-01-04
Open-source AI presentation generator with custom layouts, multi-model support, and PDF/PPTX export.
76% 10.0k issues 1.1k (79 open, 992 closed) 2026-01-19
Test your prompts, agents, and RAGs. A developer-friendly local tool for testing LLM applications.
70% 28.3k issues 1.7k (377 open, 1.3k closed) 2025-12-24
Qdrant - High-performance, massive-scale Vector Database and Vector Search Engine.
57% 1.1k issues 41 (14 open, 27 closed) 2026-01-16
Python toolkit for Retrieval-Augmented Generation (RAG)
82% 21.3k issues 227 (121 open, 106 closed) 2026-01-18
A powerful tool that packs your entire repository into a single, AI-friendly file.
84% 25.7k issues 185 (12 open, 173 closed) 2026-01-19
AI-powered tool for comparing resumes against job descriptions using local LLMs via Ollama.
84% 24.2k issues 1.7k (181 open, 1.5k closed) 2026-01-19
A privacy-respecting, hackable metasearch engine. Highly configurable and can be used for Web RAG use-cases.
84% 25.8k issues 378 (98 open, 280 closed) 2026-01-18
Open-source platform to build and deploy AI agent workflows with visual canvas editor.
16% 5.7k issues 80 (20 open, 60 closed) 2025-07-12
Chat-based SQL client, which uses natural language to communicate with the database.
29% 2.4k issues 70 (28 open, 42 closed) 2025-10-09
A simple and powerful API gateway for LLMs.
17% 3.3k issues 120 (42 open, 78 closed) 2025-07-25
Automatic "Differentiation" via Text - using large language models to backpropagate textual gradients.
89% 61.1k issues 6.5k (664 open, 5.9k closed) 2026-01-16
A modern HTTP reverse proxy and load balancer that makes deploying microservices easy.
35% 435 issues 31 (2 open, 29 closed) 2025-12-01
RAG WebUI built with txtai.
88% 50.9k issues 2.9k (827 open, 2.0k closed) 2026-01-16
Jupyter Lab environment with Unsloth for fast LLM fine-tuning - 2x faster training with 70% less memory.
65% 3.8k issues 141 (9 open, 132 closed) 2026-01-13
Linux in a web browser supporting popular desktop environments.
80% 15.5k issues 1.4k (506 open, 860 closed) 2026-01-19
Open-source developer platform for internal tools, workflows, and UIs with multi-language script support.
Visual programming, workflow automation, and orchestration platforms for building LLM applications.
82% 20.4k issues 2.6k (265 open, 2.3k closed) 2026-01-19
Open-source workflow automation platform with AI capabilities and 200+ app connectors.
96% 100.7k issues 7.6k (3.3k open, 4.3k closed) 2026-01-19
The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface.
81% 19.2k issues 433 (178 open, 255 closed) 2026-01-17
Community-driven deep research framework combining LLMs with web search, crawling, and multi-agent workflows for comprehensive research reports.
98% 126.5k issues 16.0k (438 open, 15.6k closed) 2026-01-19
An open-source LLM app development platform.
88% 48.3k issues 2.5k (637 open, 1.8k closed) 2026-01-16
Drag & drop UI to build your customized LLM flow.
99% 144.0k issues 3.1k (350 open, 2.7k closed) 2026-01-19
A low-code app builder for RAG and multi-agent AI applications.
2% 103 issues 16 (3 open, 13 closed) 2024-11-25
Simple analytics platform that leverages LLMs to automate data analysis.
100% 170.0k issues 7.6k (490 open, 7.1k closed) 2026-01-19
Fair-code workflow automation platform with native AI capabilities.
81% 17.1k issues 1.0k (110 open, 890 closed) 2026-01-19
Open Source AI Platform with Chat UI, RAG, MCP support, and 40+ document connectors.
20% 2.3k issues 272 (147 open, 125 closed) 2025-08-18
UI-Agnostic OpenAI API Plugin Framework.
84% 25.8k issues 378 (98 open, 280 closed) 2026-01-18
Open-source platform to build and deploy AI agent workflows with visual canvas editor.
80% 15.5k issues 1.4k (506 open, 860 closed) 2026-01-19
Open-source developer platform for internal tools, workflows, and UIs with multi-language script support.
API gateways, proxies, and aggregation services for managing multiple LLM endpoints.
78% 12.7k issues 1.3k (451 open, 825 closed) 2026-01-19
A helper service allowing to expose Harbor services over the internet.
63% 2.3k issues 162 (47 open, 115 closed) 2026-01-17
Connects to downstream LLM API and serves a wrapper with custom workflow. For example, it can be used to add a CoT (Chain of Thought) to an existing LLM API, and much more. Scriptable with Python.
83% 20.8k issues 2.2k (312 open, 1.9k closed) 2026-01-19
Open source LLM engineering platform: LLM Observability, metrics, evals, prompt management, playground, datasets.
87% 34.1k issues 8.5k (825 open, 7.7k closed) 2026-01-19
LLM proxy that can aggregate multiple inference APIs together into a single endpoint.
64% 2.2k issues 243 (10 open, 233 closed) 2026-01-19
Runs multiple llama.cpp servers on demand for seamless switching between them.
88% 38.3k issues 4.4k (67 open, 4.3k closed) 2026-01-19
AI platform for integrating ML models with data sources via HTTP and MySQL APIs.
20% 2.3k issues 272 (147 open, 125 closed) 2025-08-18
UI-Agnostic OpenAI API Plugin Framework.
55% 3.3k issues 88 (12 open, 76 closed) 2025-12-25
Optimising LLM proxy that implements many advanced workflows to boost the performance of the LLMs.
89% 61.1k issues 6.5k (664 open, 5.9k closed) 2026-01-16
A modern HTTP reverse proxy and load balancer that makes deploying microservices easy.
Text-to-speech (TTS), speech-to-text (STT), and audio processing services.
55% 2.8k issues 293 (80 open, 213 closed) 2025-12-27
Legacy version of Speaches, use that instead.
4% 844 issues 85 (7 open, 78 closed) 2025-02-02
An OpenAI API compatible text to speech server using Coqui AI's xtts_v2 and/or piper tts as the backend.
3% 5.5k issues 154 (115 open, 39 closed) 2024-12-10
Inference and training library for high-quality TTS models.
55% 2.8k issues 293 (80 open, 213 closed) 2025-12-27
an OpenAI API-compatible speech server (formerly faster-whisper-server), both TTS and STT
Command-line interfaces and terminal-based tools for LLM interaction.
68% 9.1k issues 632 (21 open, 611 closed) 2026-01-06
All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI tools & agents.
88% 39.9k issues 3.9k (1.1k open, 2.8k closed) 2026-01-19
Aider is AI pair programming in your terminal.
78% 12.7k issues 1.3k (451 open, 825 closed) 2026-01-19
A helper service allowing to expose Harbor services over the internet.
13% 121 issues 15 (8 open, 7 closed) 2025-08-23
Create Linux commands from natural language, in the shell.
88% 38.3k issues 812 (22 open, 790 closed) 2026-01-19
LLM-driven processing of the text data in the terminal.
69% 4.1k issues 333 (28 open, 305 closed) 2026-01-19
A simple CLI tool to interact with LLMs.
63% 2.3k issues 162 (47 open, 115 closed) 2026-01-17
Harbor's own tool to evaluate LLMs and inference backends against custom tasks.
86% 29.7k issues 2.5k (736 open, 1.8k closed) 2026-01-19
A modern load testing tool, using Go and JavaScript - https://k6.io
77% 11.2k issues 1.6k (539 open, 1.1k closed) 2026-01-19
A de-facto standard framework for the few-shot evaluation of language models.
65% 61.7k issues 1.0k (239 open, 803 closed) 2025-12-05
A natural language interface for computers.
94% 78.5k issues 5.8k (2.3k open, 3.5k closed) 2026-01-19
AI coding assistant with server API, TUI, and IDE extensions. Supports multiple LLM providers.
51% 2.3k issues 135 (9 open, 126 closed) 2025-12-19
The text-based terminal client for Ollama.
35% 14.9k issues 195 (29 open, 166 closed) 2025-10-03
AI driven development in your terminal.
76% 10.0k issues 1.1k (79 open, 992 closed) 2026-01-19
Test your prompts, agents, and RAGs. A developer-friendly local tool for testing LLM applications.
82% 21.3k issues 227 (121 open, 106 closed) 2026-01-18
A powerful tool that packs your entire repository into a single, AI-friendly file.
29% 2.4k issues 70 (28 open, 42 closed) 2025-10-09
A simple and powerful API gateway for LLMs.
Benchmarking, evaluation, and testing tools for measuring LLM performance.
63% 2.3k issues 162 (47 open, 115 closed) 2026-01-17
Harbor's own tool to evaluate LLMs and inference backends against custom tasks.
77% 11.2k issues 1.6k (539 open, 1.1k closed) 2026-01-19
A de-facto standard framework for the few-shot evaluation of language models.
Model Context Protocol servers and tool integration services.
79% 15.1k issues 3.4k (393 open, 3.0k closed) 2026-01-18
All-in-one agentic chatbot platform for multi-LLM conversations across messaging platforms with plugin system.
67% 3.1k issues 1.0k (382 open, 647 closed) 2026-01-19
Gateway and admin UI for managing Model Context Protocol (MCP) servers, tools, and resources.
33% 3.9k issues 125 (37 open, 88 closed) 2025-10-14
Turn MCP servers into OpenAPI REST APIs - use them anywhere.
51% 1.9k issues 154 (59 open, 95 closed) 2025-12-23
Allows to manage MCPs via a WebUI, exposes multiple MCPs as a single server.
29% 2.4k issues 70 (28 open, 42 closed) 2025-10-09
A simple and powerful API gateway for LLMs.
This list is auto-generated from Harbor's service metadata.
Made with 
by the Harbor community