Langfuse on a VPS: Open-Source LLM Observability for AI Application Tracing and Evaluation
Langfuse is an open-source LLM engineering platform — it provides distributed tracing for AI applications (every prompt, completion, retrieval step, and tool call is captured), cost tracking across providers, prompt version management, and evaluation datasets for testing AI output quality. As AI applications move to production, understanding why they fail, how much they cost, and whether quality is improving requires observability tooling purpose-built for LLM workloads.
What Langfuse Provides
- Traces: Full execution traces of AI pipelines — prompts, completions, retrieved chunks, tool calls, latency per step
- Cost tracking: Token usage and cost per trace, aggregated by user/session/feature
- Prompt management: Version-controlled prompts with A/B testing
- Evaluation: Score traces manually or with LLM-as-judge evaluation
- Datasets: Build test datasets from production traces, run regressions
- User sessions: Track quality and cost per user
Step 1: Docker Compose Setup
<code">mkdir -p /opt/langfuse && cd /opt/langfuse nano docker-compose.yml
<code">version: '3.8'
services:
langfuse-server:
image: langfuse/langfuse:latest
container_name: langfuse
restart: always
ports:
- "127.0.0.1:3000:3000"
depends_on:
langfuse-db:
condition: service_healthy
environment:
DATABASE_URL: postgresql://langfuse:${POSTGRES_PASSWORD}@langfuse-db:5432/langfuse
NEXTAUTH_URL: https://langfuse.yourdomain.com
NEXTAUTH_SECRET: ${NEXTAUTH_SECRET}
SALT: ${SALT}
ENCRYPTION_KEY: ${ENCRYPTION_KEY}
TELEMETRY_ENABLED: "false"
LANGFUSE_ENABLE_EXPERIMENTAL_FEATURES: "true"
# Auth: password-based + optional SSO
AUTH_DISABLE_SIGNUP: "false"
langfuse-db:
image: postgres:16-alpine
restart: always
environment:
POSTGRES_DB: langfuse
POSTGRES_USER: langfuse
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
volumes:
- langfuse_db:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U langfuse"]
interval: 10s
timeout: 5s
retries: 5
volumes:
langfuse_db:
<code">cat > .env << 'EOF' POSTGRES_PASSWORD=StrongLangfuseDbPassword! NEXTAUTH_SECRET=$(openssl rand -hex 32) SALT=$(openssl rand -hex 16) ENCRYPTION_KEY=$(openssl rand -hex 32) EOF chmod 600 .env docker compose up -d
Step 2: Nginx Reverse Proxy
<code">sudo nano /etc/nginx/sites-available/langfuse
<code">server {
listen 443 ssl http2;
server_name langfuse.yourdomain.com;
ssl_certificate /etc/letsencrypt/live/langfuse.yourdomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/langfuse.yourdomain.com/privkey.pem;
location / {
proxy_pass http://127.0.0.1:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
<code">sudo certbot --nginx -d langfuse.yourdomain.com sudo systemctl reload nginx
Step 3: Instrument Python AI Applications
<code">pip install langfuse openai
<code">from langfuse import Langfuse
from langfuse.decorators import observe, langfuse_context
from openai import OpenAI
# Initialize Langfuse (reads LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_HOST from env)
import os
os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-your-public-key"
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-your-secret-key"
os.environ["LANGFUSE_HOST"] = "https://langfuse.yourdomain.com"
langfuse = Langfuse()
openai_client = OpenAI()
@observe() # Decorator automatically creates a trace for this function
def answer_question(question: str, user_id: str) -> str:
"""Full RAG pipeline with automatic Langfuse tracing."""
# Update trace metadata
langfuse_context.update_current_trace(
user_id=user_id,
metadata={"question_type": "support"},
)
# Step 1: Retrieve context (traced as a span)
with langfuse_context.current_observation() as obs:
obs.update(name="retrieve-context", input=question)
context = retrieve_relevant_docs(question)
obs.update(output=context)
# Step 2: Generate answer (LLM call automatically traced)
response = openai_client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": f"Context: {context}"},
{"role": "user", "content": question},
],
)
answer = response.choices[0].message.content
# Score the output quality
langfuse_context.score_current_observation(
name="relevance",
value=0.9, # 0-1 scale
comment="Highly relevant answer"
)
return answer
# Result: In Langfuse dashboard, see:
# - Full trace: retrieve-context span + OpenAI API call
# - Token usage: input/output tokens, estimated cost
# - Latency: per-step timing
# - User: which user made the request
Step 4: Track with LangChain Integration
<code">pip install langfuse langchain-openai
<code">from langchain_openai import ChatOpenAI
from langchain.callbacks import get_openai_callback
from langfuse.callback import CallbackHandler
# Langfuse callback handler — instruments all LangChain calls
langfuse_handler = CallbackHandler(
public_key="pk-lf-your-key",
secret_key="sk-lf-your-key",
host="https://langfuse.yourdomain.com",
user_id="user-123",
session_id="session-abc",
trace_name="support-chat",
)
llm = ChatOpenAI(model="gpt-4o-mini")
# All LangChain calls are automatically traced via the callback
response = llm.invoke(
"What is the refund policy?",
config={"callbacks": [langfuse_handler]},
)
Step 5: Prompt Management
<code"># Create and version prompts in Langfuse UI:
# Prompts → Create Prompt → "support-system-prompt"
# Version 1: "You are a helpful support agent. Be concise."
# Version 2: "You are a helpful support agent. Be concise. Always end with 'Is there anything else I can help you with?'"
# Use prompt from Langfuse in code:
from langfuse import Langfuse
langfuse = Langfuse()
# Get the production-tagged version of a prompt:
prompt = langfuse.get_prompt("support-system-prompt")
compiled = prompt.compile(customer_name="Alice") # Inject variables
# Using the fetched prompt keeps your AI behavior version-controlled
# and changeable without code deployments
Step 6: Evaluation and Testing
<code"># Create an evaluation dataset from production traces:
# Langfuse UI → Datasets → Create → add trace items as test cases
# Run automated evaluation with LLM-as-judge:
from langfuse import Langfuse
langfuse = Langfuse()
# Evaluate a trace
langfuse.score(
trace_id="trace-uuid",
name="answer_quality",
value=0.85,
comment="Good answer, slightly verbose",
source="llm-judge",
)
# Get dashboard showing quality scores over time
# Track: are new prompt versions better than old ones?
Getting Started
Langfuse uses 200–500 MB RAM. A 2 GB Ubuntu VPS at VPS.DO runs Langfuse for small-medium AI applications. For teams deploying AI features to production, Langfuse provides the observability layer that generic APM tools (Datadog, New Relic) don’t cover: prompt-level debugging, per-user cost tracking, and AI output quality evaluation. Self-hosted keeps all LLM interactions (often containing sensitive user data) within your infrastructure.
Conclusion
Langfuse brings production-grade observability to LLM applications — tracing every prompt and completion, tracking costs per user and feature, managing prompt versions with A/B testing capability, and evaluating output quality over time. As AI applications move from prototypes to production, understanding why AI pipelines fail and measuring whether quality improves requires tooling purpose-built for LLM workloads. Self-hosted Langfuse provides this at infrastructure cost without sending AI interactions to a third-party analytics service.
