How to Self-Host Ollama
Ollama is a local runtime for large language models, with a command-line client, a model library, and an OpenAI-compatible HTTP API. The notes below were checked against the project site and source repository.
When Ollama is a good fit
Choose Ollama when you want to download and run models on hardware you control. It is a runtime and API, not a multi-user chat product. Pair it with a separate interface if you need accounts and conversation history. Read the current install notes before selecting a version, because GPU and container requirements change between releases.
Deployment notes
The project documents an install script for Linux and macOS, a Windows installer, and the official image ollama/ollama. The API listens on port 11434. In the official container, model files live under /root/.ollama; on a host install they live in the user's Ollama directory. Persist that directory. Model weights are large, so size the disk before you pull the first model.
Keep the API on a private network. It can pull and run models. Put any public access behind authentication you add yourself.
Data and recovery
Back up the model directory if you need the weights to survive a rebuild. The models can also be pulled again from the library, which is slower but does not require a backup of the blobs.
Write down the restore sequence, required secrets, exposed ports, and which hosts may call the API. A scheduled backup is not complete until a clean host can restore it and pass a functional check.
Important limitation
Ollama does not add user accounts in front of the API. An exposed port is an open model runner.
Production checklist
- Pin and record the deployed version.
- Persist every documented data and configuration path.
- Terminate TLS at a maintained reverse proxy or trusted tunnel.
- Restrict administrative interfaces and rotate default credentials.
- Monitor uptime, disk space, certificate expiry, and failed jobs.
- Test an off-host backup with a real restore.
