开源跨平台
llama.cpp
LLM inference in C/C++
ggerganov/llama.cpp: LLM inference in C/C++
ollama
Get up and running with large language models.
Run Llama 3.3 , Phi 4 , Mistral , Gemma 2 , and other models. Customize and create your own.
Ollama
ollama/ollama: Get up and running with Llama 3.3, Phi 4, Gemma 2, and other large language models.
lmstudio.ai
Discover, download, and run local LLMs
Jan
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer
Linux+MacOS+Windows
Jan: Open source ChatGPT-alternative that runs 100% offline - Jan
janhq/jan: Jan is an open source alternative to ChatGPT that runs 100% offline on your computer
LocalAI
🤖 The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed, P2P inference
Mac(Apple Silicion M+Intel)+Linux(arm64+x86_64)
[mudler/LocalAI: :robot: The free, Open Source alternative to OpenAI, Claude and others. Self-hosted and local-first. Drop-in replacement for OpenAI, running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. Features: Generate Text, Audio, Video, Images, Voice Cloning, Distributed, P2P inference](https://github.com/mudler/LocalAI
MLC LLM
Universal LLM Deployment Engine with ML Compilation
Python+iOS+Android+Web(对，就是直接在web端执行WebLLM Chat
)
mlc-ai/mlc-llm: Universal LLM Deployment Engine with ML Compilation
MLC LLM | Home
pocketpal-ai
pocketpal-ai
iOS+Android
a-ghorbani/pocketpal-ai: An app that brings language models directly to your phone.
## VLLM
>A high-throughput and memory-efficient inference and serving engine for LLMs
Python
Welcome to vLLM — vLLM
vllm-project/vllm: A high-throughput and memory-efficient inference and serving engine for LLMs
## node-llama-cpp
>Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level
node.js with example(Linux+macOS+Windows)
node-llama-cpp | Run AI models locally on your machine
withcatai/node-llama-cpp: Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level
Text Generation Inference
Text Generation Inference (TGI) is a toolkit for deploying and serving Large Language Models (LLMs). TGI enables high-performance text generation for the most popular open-source LLMs, including Llama, Falcon, StarCoder, BLOOM, GPT-NeoX, and T5.
Text Generation Inference
huggingface/text-generation-inference: Large Language Model Text Generation Inference
anythingllm
The all-in-one AI application
MacOS+Windows+Linux
AnythingLLM | The all-in-one AI application for everyone
Mintplex-Labs/anything-llm: The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more.
Cortex.cpp
Run and Customize Local LLMs
Windows+Linux+MacOS
janhq/cortex.cpp: Local AI API Platform
Homepage - Cortex
## catai
>Run AI ✨ assistant locally! with simple API for Node.js 🚀
node.js
withcatai/catai: Run AI ✨ assistant locally! with simple API for Node.js 🚀
edgen
⚡ Edgen: Local, private GenAI server alternative to OpenAI. No GPU required. Run AI models locally: LLMs (Llama2, Mistral, Mixtral…), Speech-to-text (whisper) and many others.
docs.edgen.co/
edgenai/edgen: ⚡ Edgen: Local, private GenAI server alternative to OpenAI. No GPU required. Run AI models locally: LLMs (Llama2, Mistral, Mixtral…), Speech-to-text (whisper) and many others.
## ariya
ariya/ask-llm: Interact with any LLM service
ariya/query-llm: Query LLM with Chain-of-Tought
开源单一平台
ollama-app
A modern and easy-to-use client for Ollama
Android
JHubi1/ollama-app: A modern and easy-to-use client for Ollama
LMPlayground
Language Model Playground
Android
andriydruk/LMPlayground: Language Model Playground
mLLM
Fast Multimodal LLM on Mobile Devices
Android
UbiquitousLearning/mllm: Fast Multimodal LLM on Mobile Devices
# 闭源免费跨平台
backyard.ai
Create Immersive AI-Powered Characters
Android+iOS+MacOS(Intel+M1/M2/M3)+Windows
Backyard AI | Home
jellybox
创造，生成，分享！
在本地运行AI模型，完全脱机！
MacOS(Apple Silicion M)+Windows
Jellybox - Create, Generate, Share!
msty
The easiest way to use local and online AI models
> notice:有增值服务
MacOS(Intel+M1/M2/M3/M4)+Windows+Linux
Msty - Using AI Models made Simple and Easy
Sanctum
Your Private Sanctum for AI
Run & interact with full-featured open-source LLMs locally on your device.
Your data is encrypted, secure, and never leaves your Sanctum.
MacOS(Intel+M1/M2/M3)+Windows
Sanctum — Your Private, Local AI Assistant
闭源免费——安卓
Private AI
Private AI - Google Play 上的应用
# 闭源收费
## Recurse Chat
Mac
RecurseChat
| 分类 (Category) | 名称 (Name) | 平台 (Platform) | 描述 (Description) | 链接 (Links) |
|---|---|---|---|---|
| 开源跨平台 (Open Source Cross-Platform) | llama.cpp | C/C++ | C/C++ 中的 LLM 推理 (LLM inference in C/C++) | GitHub |
| ollama | Linux, macOS, Windows | 轻松运行大型语言模型 (Get up and running with large language models.)，支持 Llama 3.3, Phi 4, Mistral, Gemma 2 等模型，可自定义和创建模型。 | Ollama / GitHub | |
| lmstudio.ai | Linux, macOS, Windows | 发现、下载和运行本地 LLM (Discover, download, and run local LLMs) | lmstudio.ai / GitHub | |
| Jan | Linux, macOS, Windows | 开源 ChatGPT 替代品，100% 离线运行 (Open source ChatGPT-alternative that runs 100% offline) | Jan / GitHub | |
| LocalAI | Mac (Apple Silicon M+, Intel), Linux (arm64, x86_64) | 免费开源的 OpenAI 等替代品 (The free, Open Source alternative to OpenAI, Claude and others.)，自托管和本地优先，无需 GPU，支持多种模型架构。 | GitHub | |
| MLC LLM | Python, iOS, Android, Web | 通用 LLM 部署引擎，使用 ML 编译 (Universal LLM Deployment Engine with ML Compilation)，可以直接在 Web 端执行！超酷的！😎 | MLC LLM / GitHub | |
| pocketpal-ai | iOS, Android | 将语言模型直接带到手机上的应用 (An app that brings language models directly to your phone.) | GitHub | |
| VLLM | Python | 高性能和内存高效的 LLM 推理和服务引擎 (A high-throughput and memory-efficient inference and serving engine for LLMs) | vLLM / GitHub | |
| node-llama-cpp | Node.js, Linux, macOS, Windows | 使用 node.js 绑定在本地运行 AI 模型 (Run AI models locally on your machine with node.js bindings for llama.cpp.)，在生成级别强制执行 JSON 模式。 | node-llama-cpp / GitHub | |
| Text Generation Inference (TGI) | - | 用于部署和 serving 大型语言模型 (LLMs) 的工具包 (Text Generation Inference (TGI) is a toolkit for deploying and serving Large Language Models (LLMs).)，支持多种流行的开源 LLM。 | Text Generation Inference / GitHub | |
| anythingllm | macOS, Windows, Linux | 一体化 AI 应用程序 (The all-in-one AI application) | AnythingLLM / GitHub | |
| Cortex.cpp | Windows, Linux, macOS | 运行和自定义本地 LLM (Run and Customize Local LLMs)，本地 AI API 平台。 | Cortex / GitHub | |
| catai | node.js | 本地运行 AI 助手，提供简单的 Node.js API (Run AI ✨ assistant locally! with simple API for Node.js 🚀) | GitHub | |
| edgen | Linux, macOS, Windows | 本地、私有的 GenAI 服务器，OpenAI 的替代品 (Local, private GenAI server alternative to OpenAI.)，无需 GPU，支持多种模型和语音转文本。 | edgen / GitHub | |
| ask-llm (ariya) | - | 与任何 LLM 服务交互 (Interact with any LLM service) | GitHub | |
| query-llm (ariya) | - | 使用 Chain-of-Thought 查询 LLM (Query LLM with Chain-of-Thought) | GitHub | |
| 开源单一平台 (Open Source Single-Platform) | ollama-app | Android | Ollama 的现代且易于使用的客户端 (A modern and easy-to-use client for Ollama) | GitHub |
| LMPlayground | Android | 语言模型游乐场 (Language Model Playground) | GitHub | |
| mLLM | Android | 移动设备上的快速多模态 LLM (Fast Multimodal LLM on Mobile Devices) | GitHub | |
| 闭源免费跨平台 (Closed Source Free Cross-Platform) | backyard.ai | Android, iOS, macOS (Intel, M1/M2/M3), Windows | 创建沉浸式 AI 驱动的角色 (Create Immersive AI-Powered Characters) | Backyard AI |
| jellybox | macOS (Apple Silicon M), Windows | 创造，生成，分享！本地运行 AI 模型，完全脱机！(Create, Generate, Share! Run AI models locally, completely offline!) | Jellybox | |
| msty | macOS (Intel, M1/M2/M3/M4), Windows, Linux | 使用本地和在线 AI 模型的最简单方法 (The easiest way to use local and online AI models)，注意有增值服务哦！ | Msty | |
| Sanctum | macOS (Intel, M1/M2/M3), Windows | 您的私人 AI 圣所 (Your Private Sanctum for AI)，在本地设备上运行全功能开源 LLM，数据加密安全。 | Sanctum | |
| 闭源免费 - 安卓 (Closed Source Free - Android) | Private AI | Android | 私有 AI 应用 (Private AI App) | Google Play |
| 闭源收费 (Closed Source Paid) | Recurse Chat | Mac | Mac 平台的聊天应用 (Mac Chat Application) | RecurseChat |