Official Docs
Beginner
Gradio Official Documentation
Gradio
Overview
A Python library for ML demo interfaces — wrap a model into a shareable web app in a few lines.
https://www.gradio.app/
Related resources
View all Deployment & Serving →
Open Source Project
Beginner
★ Featured
Ollama Local LLM Tool
Ollama · Deployment & Serving
Run open-source LLMs locally with a single command on Windows, macOS, or Linux — the go-to starting point for local deployment.
Open Source Project
Advanced
vLLM Official Documentation
vLLM 社区 · Deployment & Serving
A high-performance LLM inference and serving engine; PagedAttention dramatically boosts throughput, with support for mainstream open-source models.
Open Source Project
Beginner
Open WebUI
Open WebUI · Deployment & Serving
A self-hosted AI chat interface that supports Ollama and OpenAI-compatible APIs, and can be deployed offline on an intranet.
Tool & Platform
Beginner
LM Studio Local Model Tool
LM Studio · Deployment & Serving
A GUI tool for running LLMs locally, with one-click model downloads and local chat — beginner friendly.
Open Source Project
Advanced
llama.cpp Inference Engine
ggerganov · Deployment & Serving
A pure C/C++ inference engine that runs LLMs even on CPU — one of the de facto standards for local deployment.
Open Source Project
Advanced
Hugging Face Text Generation Inference
Hugging Face · Deployment & Serving
Hugging Face's official inference serving framework with continuous batching and tensor parallelism, built for API-style deployment.