OpenObserve on a VPS: Unified Logs, Metrics, and Traces at 140x Lower Cost Than Datadog
OpenObserve (O2) is an open-source observability platform that stores logs, metrics, and traces in a unified columnar format (Parquet) — enabling SQL queries across all telemetry data, 140× cheaper storage than Elasticsearch/Datadog, and a single platform instead of separate Loki + Prometheus + Jaeger stacks. A Rust-based single binary handles petabyte-scale data on modest VPS hardware.
OpenObserve vs Separate PLG/ELK Stacks
- OpenObserve: Single binary, SQL queries across logs+metrics+traces, columnar storage (Parquet), 140× smaller storage vs Elasticsearch, built-in dashboards
- PLG stack: Three separate services (Loki + Prometheus + Grafana), label-based log queries (LogQL), Prometheus query language (PromQL) — more setup, more RAM, but mature ecosystem
- Choose OpenObserve: Unified telemetry with SQL queries, storage efficiency critical, single binary simplicity
- Choose PLG: Prefer established tools, rich Grafana ecosystem, community already familiar with Prometheus/Loki
Step 1: Docker Compose Setup
<code">mkdir -p /opt/openobserve && cd /opt/openobserve nano docker-compose.yml
<code">version: '3.8'
services:
openobserve:
image: public.ecr.aws/zinclabs/openobserve:latest
container_name: openobserve
restart: always
ports:
- "127.0.0.1:5080:5080" # HTTP API and UI
- "127.0.0.1:5081:5081" # gRPC (OTLP traces)
environment:
ZO_ROOT_USER_EMAIL: admin@yourdomain.com
ZO_ROOT_USER_PASSWORD: ${ADMIN_PASSWORD}
ZO_DATA_DIR: /data
ZO_LOCAL_MODE_STORAGE: disk # Use S3 for production
# For S3 storage (recommended for production):
# ZO_LOCAL_MODE_STORAGE: s3
# ZO_S3_BUCKET_NAME: openobserve-data
# ZO_S3_REGION_NAME: us-east-1
# AWS_ACCESS_KEY_ID: your_key
# AWS_SECRET_ACCESS_KEY: your_secret
volumes:
- ./data:/data
<code">echo "ADMIN_PASSWORD=StrongO2AdminPassword!" > .env chmod 600 .env mkdir -p data docker compose up -d docker compose logs -f openobserve
Step 2: Nginx Reverse Proxy
<code">sudo nano /etc/nginx/sites-available/openobserve
<code">server {
listen 443 ssl http2;
server_name observe.yourdomain.com;
ssl_certificate /etc/letsencrypt/live/observe.yourdomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/observe.yourdomain.com/privkey.pem;
client_max_body_size 100M;
location / {
proxy_pass http://127.0.0.1:5080;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_read_timeout 300s;
}
}
<code">sudo certbot --nginx -d observe.yourdomain.com sudo ln -s /etc/nginx/sites-available/openobserve /etc/nginx/sites-enabled/ sudo systemctl reload nginx
Step 3: Ship Logs with Fluent Bit
<code"># Install Fluent Bit on your VPS curl https://raw.githubusercontent.com/fluent/fluent-bit/master/install.sh | sh sudo systemctl enable fluent-bit
<code">sudo nano /etc/fluent-bit/fluent-bit.conf
<code">[SERVICE]
Flush 5
Daemon Off
Log_Level warn
# Collect Nginx access logs
[INPUT]
Name tail
Path /var/log/nginx/access.log
Tag nginx.access
Parser nginx
Mem_Buf_Limit 5MB
# Collect systemd/journal logs
[INPUT]
Name systemd
Tag systemd.*
Strip_Underscores On
# Collect Docker container logs
[INPUT]
Name forward
Port 24224
# Send to OpenObserve
[OUTPUT]
Name http
Match *
Host localhost
Port 5080
URI /api/default/default/_json
Format json
Http_User admin@yourdomain.com
Http_Passwd StrongO2AdminPassword!
compress gzip
TLS Off
<code">sudo systemctl restart fluent-bit sudo systemctl status fluent-bit
Step 4: Ingest Metrics (Prometheus Remote Write)
<code"># OpenObserve accepts Prometheus remote_write # In Prometheus configuration: sudo nano /etc/prometheus/prometheus.yml
<code">global:
scrape_interval: 15s
remote_write:
- url: http://localhost:5080/api/default/prometheus/api/v1/write
basic_auth:
username: admin@yourdomain.com
password: StrongO2AdminPassword!
scrape_configs:
- job_name: node
static_configs:
- targets: ['localhost:9100']
- job_name: nginx
static_configs:
- targets: ['localhost:9113']
Step 5: SQL Queries on Log Data
<code"># OpenObserve supports SQL for log queries — far more powerful than LogQL
# All HTTP 5xx errors in last hour:
SELECT * FROM "nginx.access"
WHERE status >= 500
AND _timestamp >= NOW() - INTERVAL '1 hour'
ORDER BY _timestamp DESC
LIMIT 100;
# Request rate by status code:
SELECT
DATE_TRUNC('minute', _timestamp) AS time,
status,
COUNT(*) AS requests
FROM "nginx.access"
WHERE _timestamp >= NOW() - INTERVAL '6 hours'
GROUP BY time, status
ORDER BY time;
# Top 10 slowest endpoints:
SELECT
uri,
AVG(CAST(request_time AS FLOAT)) AS avg_duration,
COUNT(*) AS hits
FROM "nginx.access"
WHERE _timestamp >= NOW() - INTERVAL '24 hours'
GROUP BY uri
HAVING COUNT(*) > 10
ORDER BY avg_duration DESC
LIMIT 10;
# Error messages from systemd services:
SELECT _timestamp, service, message
FROM "systemd.*"
WHERE level = 'err' OR level = 'crit'
AND _timestamp >= NOW() - INTERVAL '1 day'
ORDER BY _timestamp DESC;
Step 6: OTLP Traces (OpenTelemetry)
<code">pip install opentelemetry-api opentelemetry-sdk \
opentelemetry-exporter-otlp-proto-grpc
<code">from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import BatchSpanProcessor
# Configure OTLP exporter to send traces to OpenObserve
exporter = OTLPSpanExporter(
endpoint="http://localhost:5081",
headers={"Authorization": "Basic " + base64.b64encode(
b"admin@yourdomain.com:StrongO2AdminPassword!"
).decode()},
)
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(provider)
tracer = trace.get_tracer("myapp")
def process_request(user_id: int):
with tracer.start_as_current_span("process_request") as span:
span.set_attribute("user.id", user_id)
# ... your code ...
with tracer.start_as_current_span("database_query"):
# ... DB query ...
pass
Getting Started
OpenObserve is a single Rust binary — lightweight and fast. On a 4 GB Ubuntu VPS at VPS.DO, OpenObserve comfortably handles logs, metrics, and traces from a typical production stack. The columnar Parquet storage format means 1 GB of Elasticsearch log data compresses to ~7 MB in OpenObserve — making long retention periods practical on VPS disk sizes. For production, configure S3-compatible storage (MinIO or Cloudflare R2) to separate compute from storage.
Conclusion
OpenObserve unifies logs, metrics, and traces in a single platform with SQL query capability and 140× better storage efficiency than Elasticsearch. A Rust-based single binary eliminates the operational complexity of managing separate Loki, Prometheus, and Jaeger instances. For teams building observability from scratch or looking to consolidate tooling, OpenObserve on a VPS provides Datadog-grade visibility at infrastructure cost — with SQL as the universal query language across all telemetry types.