LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required.
Deutsch | Español | français | 日本語 | 한국어 | Português | Русский | 中文 LocalAI is the open-source AI engine. Run any model - LLMs, vision, voice, image, video - on any hardware. No GPU required. A small core, not a bundle. Each backend wraps a best-in-class engine (llama.cpp, vLLM, whisper.cpp, stable-diffusion, MLX...) in its own image, pulled only when a model needs it. You install nothing you don't use. - Composable by design: backends are separate and pulled on demand, so you install only what your model needs - Open and extensible: load any model, or build your own backend in any language against an open interface - Drop-in API compatibility: OpenAI, Anthropic, and ElevenLabs APIs across every backend - Any model, any modality: LLMs, vision, voice, image, and video behind one API - Any hardware: NVIDIA, AMD, Intel, Apple Silicon, Vulkan, or CPU-only - Multi-user ready: API key auth, user quotas, role-based access - Built-in AI agents: autonomous agents with tool use, RAG, MCP, and skills - Privacy-first: your data never leaves your infrastructure  Created by [Ettore Di Giacinto](https://github.com/mudler) and maintained by the [LocalAI team](#team). > [:book: Documentation](https://localai.io/) | [:speech_balloon: Discord](https://discord.gg/uJAeKSAGDy) | [💻 Quickstart](https://localai.io/basics/getting_started/) | [🖼️ Models](https://models.localai.io/) | [❓FAQ](https://localai.io/faq/) ## Guided tour https://github.com/user-attachments/assets/08cbb692-57da-48f7-963d-2e7b43883c18 Click to see more! #### User and auth https://github.com/user-attachments/assets/228fa9ad-81a3-4d43-bfb9-31557e14a36c #### Agents https://github.com/user-attachments/assets/6270b331-e21d-4087-a540-6290006b381a #### Usage metrics per user https://github.com/user-attachments/assets/cbb03379-23b4-4e3d-bd26-d152f057007f #### Fine-tuning and Quantization https://github.com/user-attachments/assets/5ba4ace9-d3df-4795-b7d4-b0b404ea71ee #### WebRTC https://github.com/user-attachments/assets/ed88e34c-fed3-4b83-8a67-4716a9feeb7b ## Quickstart ### macOS > Note: The DMG is not signed by Apple. After installing, run: sudo xattr -d com.apple.quarantine /Applications/LocalAI.app. See [#6268](https://github.com/mudler/LocalAI/issues/6268) for details. ### Containers (Docker, podman, ...) > Already ran LocalAI before? Use docker start -i local-ai to restart an existing container. #### CPU only: ``bash docker run -ti --name local-ai -p 8080:8080 localai/localai:latest #### NVIDIA GPU: bash # CUDA 13 docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-13 # CUDA 12 docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12 # NVIDIA Jetson ARM64 (CUDA 12, for AGX Orin and similar) docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64 # NVIDIA Jetson ARM64 (CUDA 13, for DGX Spark) docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-nvidia-l4t-arm64-cuda-13 #### AMD GPU (ROCm): bash docker run -ti --name local-ai -p 8080:8080 --device=/dev/kfd --device=/dev/dri --group-add=video localai/localai:latest-gpu-hipblas #### Intel GPU (oneAPI): bash docker run -ti --name local-ai -p 8080:8080 --device=/dev/dri/card1 --device=/dev/dri/renderD128 localai/localai:latest-gpu-intel #### Vulkan GPU: bash docker run -ti --name local-ai -p 8080:8080 localai/localai:latest-gpu-vulkan ### Loading models bash # From the model gallery (see available models with local-ai models list or at https://models.localai.io) local-ai run llama-3.2-1b-instruct:q4_k_m # From Huggingface local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf # From the Ollama OCI registry local-ai run ollama://gemma:2b # From a YAML config local-ai run https://gist.githubusercontent.com/.../phi-2.yaml # From a standard OCI registry (e.g., Docker Hub) local-ai run oci://localai/phi-2:latest To test a running LocalAI server from the terminal, open an interactive chat session from another shell. Inside the prompt, /models lists installed models and /model switches between them. bash # Terminal 1 local-ai run llama-3.2-1b-instruct:q4_k_m # Terminal 2 local-ai chat --model llama-3.2-1b-instruct:q4_k_m `` > Automatic Backend Detection: LocalAI automatically detects your GPU capabilities and downloads the appropriate backend. For advanced options, see [GPU Acceleration](https://localai.io/features/gpu-acceleration/). For more details, see the [Getting Started guide](https://localai.io/basics/getting_started/). ## Latest News - June 2026: New [realtime voice assistant demo](https://github.com/localai-org/localai-realtime-demo) (a tiny Go client for