OpenObserve on a VPS: Unified Logs, Metrics, and Traces at 140x Lower Cost Than Datadog

OpenObserve on a VPS: Unified Logs, Metrics, and Traces at 140x Lower Cost Than Datadog

OpenObserve (O2) is an open-source observability platform that stores logs, metrics, and traces in a unified columnar format (Parquet) — enabling SQL queries across all telemetry data, 140× cheaper storage than Elasticsearch/Datadog, and a single platform instead of separate Loki + Prometheus + Jaeger stacks. A Rust-based single binary handles petabyte-scale data on modest VPS hardware.

OpenObserve vs Separate PLG/ELK Stacks

  • OpenObserve: Single binary, SQL queries across logs+metrics+traces, columnar storage (Parquet), 140× smaller storage vs Elasticsearch, built-in dashboards
  • PLG stack: Three separate services (Loki + Prometheus + Grafana), label-based log queries (LogQL), Prometheus query language (PromQL) — more setup, more RAM, but mature ecosystem
  • Choose OpenObserve: Unified telemetry with SQL queries, storage efficiency critical, single binary simplicity
  • Choose PLG: Prefer established tools, rich Grafana ecosystem, community already familiar with Prometheus/Loki

Step 1: Docker Compose Setup

<code">mkdir -p /opt/openobserve && cd /opt/openobserve
nano docker-compose.yml
<code">version: '3.8'

services:
  openobserve:
    image: public.ecr.aws/zinclabs/openobserve:latest
    container_name: openobserve
    restart: always
    ports:
      - "127.0.0.1:5080:5080"   # HTTP API and UI
      - "127.0.0.1:5081:5081"   # gRPC (OTLP traces)
    environment:
      ZO_ROOT_USER_EMAIL: admin@yourdomain.com
      ZO_ROOT_USER_PASSWORD: ${ADMIN_PASSWORD}
      ZO_DATA_DIR: /data
      ZO_LOCAL_MODE_STORAGE: disk     # Use S3 for production
      # For S3 storage (recommended for production):
      # ZO_LOCAL_MODE_STORAGE: s3
      # ZO_S3_BUCKET_NAME: openobserve-data
      # ZO_S3_REGION_NAME: us-east-1
      # AWS_ACCESS_KEY_ID: your_key
      # AWS_SECRET_ACCESS_KEY: your_secret
    volumes:
      - ./data:/data
<code">echo "ADMIN_PASSWORD=StrongO2AdminPassword!" > .env
chmod 600 .env
mkdir -p data
docker compose up -d
docker compose logs -f openobserve

Step 2: Nginx Reverse Proxy

<code">sudo nano /etc/nginx/sites-available/openobserve
<code">server {
    listen 443 ssl http2;
    server_name observe.yourdomain.com;

    ssl_certificate /etc/letsencrypt/live/observe.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/observe.yourdomain.com/privkey.pem;

    client_max_body_size 100M;

    location / {
        proxy_pass http://127.0.0.1:5080;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 300s;
    }
}
<code">sudo certbot --nginx -d observe.yourdomain.com
sudo ln -s /etc/nginx/sites-available/openobserve /etc/nginx/sites-enabled/
sudo systemctl reload nginx

Step 3: Ship Logs with Fluent Bit

<code"># Install Fluent Bit on your VPS
curl https://raw.githubusercontent.com/fluent/fluent-bit/master/install.sh | sh
sudo systemctl enable fluent-bit
<code">sudo nano /etc/fluent-bit/fluent-bit.conf
<code">[SERVICE]
    Flush     5
    Daemon    Off
    Log_Level warn

# Collect Nginx access logs
[INPUT]
    Name              tail
    Path              /var/log/nginx/access.log
    Tag               nginx.access
    Parser            nginx
    Mem_Buf_Limit     5MB

# Collect systemd/journal logs
[INPUT]
    Name              systemd
    Tag               systemd.*
    Strip_Underscores On

# Collect Docker container logs
[INPUT]
    Name              forward
    Port              24224

# Send to OpenObserve
[OUTPUT]
    Name              http
    Match             *
    Host              localhost
    Port              5080
    URI               /api/default/default/_json
    Format            json
    Http_User         admin@yourdomain.com
    Http_Passwd       StrongO2AdminPassword!
    compress          gzip
    TLS               Off
<code">sudo systemctl restart fluent-bit
sudo systemctl status fluent-bit

Step 4: Ingest Metrics (Prometheus Remote Write)

<code"># OpenObserve accepts Prometheus remote_write
# In Prometheus configuration:
sudo nano /etc/prometheus/prometheus.yml
<code">global:
  scrape_interval: 15s

remote_write:
  - url: http://localhost:5080/api/default/prometheus/api/v1/write
    basic_auth:
      username: admin@yourdomain.com
      password: StrongO2AdminPassword!

scrape_configs:
  - job_name: node
    static_configs:
      - targets: ['localhost:9100']

  - job_name: nginx
    static_configs:
      - targets: ['localhost:9113']

Step 5: SQL Queries on Log Data

<code"># OpenObserve supports SQL for log queries — far more powerful than LogQL

# All HTTP 5xx errors in last hour:
SELECT * FROM "nginx.access"
WHERE status >= 500
  AND _timestamp >= NOW() - INTERVAL '1 hour'
ORDER BY _timestamp DESC
LIMIT 100;

# Request rate by status code:
SELECT
    DATE_TRUNC('minute', _timestamp) AS time,
    status,
    COUNT(*) AS requests
FROM "nginx.access"
WHERE _timestamp >= NOW() - INTERVAL '6 hours'
GROUP BY time, status
ORDER BY time;

# Top 10 slowest endpoints:
SELECT
    uri,
    AVG(CAST(request_time AS FLOAT)) AS avg_duration,
    COUNT(*) AS hits
FROM "nginx.access"
WHERE _timestamp >= NOW() - INTERVAL '24 hours'
GROUP BY uri
HAVING COUNT(*) > 10
ORDER BY avg_duration DESC
LIMIT 10;

# Error messages from systemd services:
SELECT _timestamp, service, message
FROM "systemd.*"
WHERE level = 'err' OR level = 'crit'
  AND _timestamp >= NOW() - INTERVAL '1 day'
ORDER BY _timestamp DESC;

Step 6: OTLP Traces (OpenTelemetry)

<code">pip install opentelemetry-api opentelemetry-sdk \
    opentelemetry-exporter-otlp-proto-grpc
<code">from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import BatchSpanProcessor

# Configure OTLP exporter to send traces to OpenObserve
exporter = OTLPSpanExporter(
    endpoint="http://localhost:5081",
    headers={"Authorization": "Basic " + base64.b64encode(
        b"admin@yourdomain.com:StrongO2AdminPassword!"
    ).decode()},
)

provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(provider)

tracer = trace.get_tracer("myapp")

def process_request(user_id: int):
    with tracer.start_as_current_span("process_request") as span:
        span.set_attribute("user.id", user_id)
        # ... your code ...
        with tracer.start_as_current_span("database_query"):
            # ... DB query ...
            pass

Getting Started

OpenObserve is a single Rust binary — lightweight and fast. On a 4 GB Ubuntu VPS at VPS.DO, OpenObserve comfortably handles logs, metrics, and traces from a typical production stack. The columnar Parquet storage format means 1 GB of Elasticsearch log data compresses to ~7 MB in OpenObserve — making long retention periods practical on VPS disk sizes. For production, configure S3-compatible storage (MinIO or Cloudflare R2) to separate compute from storage.

Conclusion

OpenObserve unifies logs, metrics, and traces in a single platform with SQL query capability and 140× better storage efficiency than Elasticsearch. A Rust-based single binary eliminates the operational complexity of managing separate Loki, Prometheus, and Jaeger instances. For teams building observability from scratch or looking to consolidate tooling, OpenObserve on a VPS provides Datadog-grade visibility at infrastructure cost — with SQL as the universal query language across all telemetry types.

Fast • Reliable • Affordable VPS - DO It Now!

Get top VPS hosting with VPS.DO’s fast, low-cost plans. Try risk-free with our 7-day no-questions-asked refund and start today!