Open WebUI is a polished, self-hosted chat interface for large language models — conversations, model switching, document chat (RAG), user accounts and more. Paired with Ollama, which runs open models locally, it gives you a private ChatGPT-style assistant where prompts and documents never leave your server. It can also connect to OpenAI-compatible APIs, so one interface covers local and cloud models.

Be realistic about hardware

Typical VPS servers have no GPU, so models run on the CPU. That works — with the right model size:

Model size (quantized, ~4-bit) RAM needed CPU-only experience
1–3B parameters (e.g. Llama 3.2 3B, Qwen 2.5 3B, Gemma small) 3–4 GB Usable, a few words per second
7–8B (e.g. Llama 3.1 8B, Qwen 2.5 7B, Mistral 7B) 6–8 GB Slower; fine for short tasks with 6+ cores
13–14B 10–12 GB Slow on CPU
30B+ 20 GB+ Impractical without a GPU

More vCPU cores and AVX2/AVX-512 support help a lot (grep -o -w avx2 /proc/cpuinfo | head -1). For a quick start, an 8 GB RAM VPS with 4+ cores runs 3B–8B models. For deeper background on local models, see running a local LLM on a VPS.

1. Compose file

With Docker installed:

mkdir -p ~/openwebui && cd ~/openwebui

compose.yaml:

services:
  ollama:
    image: ollama/ollama:latest
    restart: unless-stopped
    volumes:
      - ollama:/root/.ollama
    # no published ports: only Open WebUI talks to Ollama

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    restart: unless-stopped
    depends_on:
      - ollama
    environment:
      OLLAMA_BASE_URL: http://ollama:11434
      WEBUI_SECRET_KEY: ${WEBUI_SECRET_KEY}
    volumes:
      - open-webui:/app/backend/data
    ports:
      - "127.0.0.1:3000:8080"

volumes:
  ollama:
  open-webui:
echo "WEBUI_SECRET_KEY=$(openssl rand -hex 32)" > .env
docker compose up -d

Ollama’s API (port 11434) has no authentication — never publish it to the internet. Here it’s only reachable inside the Docker network.

2. Download a model

docker compose exec ollama ollama pull llama3.2:3b
docker compose exec ollama ollama pull qwen2.5:7b
docker compose exec ollama ollama list

Models are stored in the ollama volume — several GB each, so watch disk space.

3. HTTPS and the first account

server {
    listen 80;
    server_name ai.example.com;

    client_max_body_size 50M;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 600s;
        proxy_buffering off;
    }
}
sudo nginx -t && sudo systemctl reload nginx
sudo certbot --nginx -d ai.example.com

Open https://ai.example.com and sign up immediately — the first account becomes the administrator. Then, in Admin Panel → Settings → General, disable new sign-ups or set new users to “pending” so you approve them.

Long responses stream over a single connection; disabling buffering and raising the read timeout prevents cut-offs.

4. Use it

Pick a model from the dropdown and chat. Useful features:

  • Documents/Knowledge — upload PDFs or text and ask questions about them (RAG). Embeddings run locally by default.
  • Model presets — save system prompts and parameters as custom “models”.
  • Multiple users — each with their own history; admins control which models users see.

5. Add cloud models (optional)

Admin Panel → Settings → Connections → add an OpenAI-compatible API with its base URL and key. Many providers offer OpenAI-compatible endpoints. You can then use local models for private or simple tasks and stronger hosted models when needed, all in one interface.

Note that some AI providers restrict API access by region. Our Tokyo VPS is located in a region supported by the major AI APIs.

6. Performance tips on CPU

  • Prefer smaller, well-quantized models (Q4_K_M is a good balance).
  • Limit context length for faster responses.
  • Set OLLAMA_NUM_PARALLEL=1 and OLLAMA_MAX_LOADED_MODELS=1 on small servers to avoid running out of RAM.
  • Keep an eye on memory with docker stats; if the OOM killer strikes, choose a smaller model or add RAM.

7. Backups and updates

Back up the open-webui volume (users, chats, settings, documents). Models can be re-downloaded.

docker compose pull && docker compose up -d

Open WebUI releases frequently; read release notes for breaking changes.

Our VPS plans go up to 14 vCPU and 40 GB RAM — enough to run mid-size models on CPU. Start with an 8 GB plan and a 3B–8B model, and scale when you know what you need.

  • AI
  • Ollama
  • Self-Hosting