Self-Hosted AI Stack

Overview

A Docker Compose stack for deploying a self-hosted AI environment with Ollama, LiteLLM, AnythingLLM, Whisper, and other components. Local-first and private by default, with optional HTTPS and NVIDIA CUDA acceleration. Supports amd64 and arm64 architectures.

Best for

Best for
Developers who want a private, self-hosted AI stack with Docker Compose.

Use cases

• Deploy a private AI stack locally with Docker Compose
• Run speech-to-text with Whisper and WhisperLive
• Expose models through LiteLLM and MCP Gateway

Notes

A Docker Compose stack for deploying a self-hosted AI environment with Ollama, LiteLLM, AnythingLLM, Whisper, and other components. Local-first and private by default, with optional HTTPS and NVIDIA CUDA acceleration. Supports amd64 and arm64 architectures.

161 stars on GitHub. Last updated 2026-09-30. Licensed MIT.

Use cases

• Deploy a private AI stack locally with Docker Compose
• Run speech-to-text with Whisper and WhisperLive
• Expose models through LiteLLM and MCP Gateway

Pros

• Easy deployment via Docker Compose
• Local-first and private by default
• Multi-architecture support

Cons

• Requires self-hosting and maintenance
• Small community with limited support
• No managed service or enterprise support

Indexed from awesome-llmops and enriched against its public facts.