AnythingLLM on a VPS: Self-Hosted Enterprise AI Knowledge Base with Multi-User Workspaces

AnythingLLM is a full-stack, enterprise-ready self-hosted AI application that turns your documents into a queryable knowledge base — upload PDFs, Word docs, URLs, YouTube videos, and more, then ask questions in natural language using any LLM backend. It provides multi-user workspaces (teams have separate document collections), fine-grained permissions, agent mode with internet search, and an API for integration. It goes further than Open WebUI by being document-management-first rather than chat-interface-first.

AnythingLLM vs Open WebUI vs Flowise

• AnythingLLM: Document-first, multi-user workspaces per team, enterprise permissions, built-in RAG management, no-code setup
• Open WebUI: Chat-interface-first, multi-model support, RAG via file upload per conversation, simpler
• Flowise: Developer-focused, visual pipeline builder, most flexible but requires flow design
• Choose AnythingLLM: Organization deploying AI knowledge base to non-technical teams across departments

Step 1: Docker Compose Setup

<code">mkdir -p /opt/anythingllm/storage && cd /opt/anythingllm
nano docker-compose.yml

<code">version: '3.8'

services:
  anythingllm:
    image: mintplexlabs/anythingllm:latest
    container_name: anythingllm
    restart: always
    ports:
      - "127.0.0.1:3001:3001"
    cap_add:
      - SYS_ADMIN
    environment:
      STORAGE_DIR: /app/server/storage
      JWT_SECRET: ${JWT_SECRET}
      LLM_PROVIDER: ollama
      OLLAMA_BASE_PATH: http://host.docker.internal:11434
      OLLAMA_MODEL_PREF: mistral
      OLLAMA_MODEL_TOKEN_LIMIT: 8192
      EMBEDDING_ENGINE: ollama
      EMBEDDING_BASE_PATH: http://host.docker.internal:11434
      EMBEDDING_MODEL_PREF: nomic-embed-text
      VECTOR_DB: lancedb   # Built-in vector store
      DISABLE_TELEMETRY: "true"
    volumes:
      - ./storage:/app/server/storage
    extra_hosts:
      - "host.docker.internal:host-gateway"

<code">echo "JWT_SECRET=$(openssl rand -hex 32)" > .env
chmod 600 .env
docker compose up -d

Step 2: Nginx Reverse Proxy

<code">sudo nano /etc/nginx/sites-available/anythingllm

<code">server {
    listen 443 ssl http2;
    server_name ai.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/ai.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/ai.yourdomain.com/privkey.pem;

    client_max_body_size 200M;   # Large document uploads

    location / {
        proxy_pass http://127.0.0.1:3001;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 300s;
    }
}

<code">sudo certbot --nginx -d ai.yourdomain.com
sudo ln -s /etc/nginx/sites-available/anythingllm /etc/nginx/sites-enabled/
sudo systemctl reload nginx

Step 3: Initial Setup and Admin

1. Visit https://ai.yourdomain.com
2. Create admin account
3. Settings → LLM Preference: verify Ollama connection
4. Settings → Embedding: verify nomic-embed-text
5. Settings → Multi-User Mode: Enable (for team use)

Step 4: Create Workspaces for Teams

<code"># Workspaces are isolated document collections for different teams:
# Admin Panel → Workspaces → New Workspace

# Create workspace: "Engineering Docs"
# - Upload: API documentation, architecture diagrams, runbooks
# - Grant access to: engineering team members

# Create workspace: "HR Policies"
# - Upload: employee handbook, benefits guide, PTO policy
# - Grant access to: all employees (read-only)

# Create workspace: "Sales Intelligence"
# - Upload: competitor analysis, pricing guides, case studies
# - Grant access to: sales team

# Each workspace has:
# - Its own document collection (vector embeddings)
# - Separate conversation history per user
# - Configurable AI behavior (temperature, context length)
# - Optional: restrict to only documents (no general AI knowledge)

Step 5: Upload and Process Documents

<code"># Supported document types:
# PDF, DOCX, XLSX, PPTX, TXT, MD, CSV
# Web URLs (scrapes content)
# YouTube videos (transcribes)
# GitHub repositories (clones and indexes)

# Upload via UI: Workspace → Upload document → drag and drop
# AnythingLLM automatically:
# 1. Parses document content
# 2. Splits into chunks
# 3. Generates embeddings with nomic-embed-text
# 4. Stores in vector database

# Upload via API:
curl -X POST https://ai.yourdomain.com/api/v1/document/upload \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -F "file=@company-handbook.pdf"

Step 6: Connect to OpenAI (Cloud Fallback)

<code"># To switch between Ollama and OpenAI:
# Settings → LLM Preference → select provider

# For hybrid: use Ollama for private docs, OpenAI for general questions
# Configure per-workspace in workspace settings:
# Workspace Settings → LLM Preference → override to OpenAI for this workspace only

Step 7: API Integration

<code">import httpx

API_KEY = "your_anythingllm_api_key"
WORKSPACE = "engineering-docs"

# Query the knowledge base
response = httpx.post(
    f"https://ai.yourdomain.com/api/v1/workspace/{WORKSPACE}/chat",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={
        "message": "What is our database migration process?",
        "mode": "query",   # "query" = only use documents; "chat" = general AI too
        "sessionId": "user-12345",
    },
    timeout=60,
)
result = response.json()
print(result["textResponse"])
# Print citations:
for source in result.get("sources", []):
    print(f"  Source: {source['title']} (similarity: {source['score']:.2f})")

Getting Started

AnythingLLM with Ollama needs 4–8 GB RAM depending on the model. A 8 GB Ubuntu VPS at VPS.DO runs AnythingLLM with a 7B Ollama model for small-medium teams. For larger models or higher concurrent users, 16 GB is recommended. With OpenAI as the LLM provider, a 2 GB VPS is sufficient since inference happens at OpenAI.

Conclusion

AnythingLLM transforms company documents into a searchable AI knowledge base accessible to all team members through a polished multi-user interface. The workspace system allows HR, Engineering, Sales, and other departments to maintain separate document collections while sharing the same infrastructure. Self-hosted on a VPS, sensitive business documents — contracts, HR policies, financial models — are never processed by cloud AI services. For organizations taking their first step into enterprise AI, AnythingLLM provides the fastest path from document collection to conversational AI knowledge base.