Redis Deep Dive: Caching, Pub/Sub, Rate Limiting, and Session Storage on a VPS

Most developers know Redis as a cache. But Redis is more accurately described as an in-memory data structure server — it supports strings, hashes, lists, sets, sorted sets, streams, and more, each with different performance characteristics and use cases. This guide goes beyond basic caching to cover four high-value Redis patterns on a VPS: application caching with proper eviction, Pub/Sub for real-time messaging, API rate limiting, and session storage. Each pattern is production-ready and immediately applicable.

Redis Installation and Production Configuration

Install Redis

sudo apt update
sudo apt install redis-server -y
sudo systemctl enable redis-server

Production Configuration

sudo nano /etc/redis/redis.conf

Key production settings:

# Bind to localhost only (never expose Redis publicly without authentication)
bind 127.0.0.1 ::1

# Require authentication
requirepass YourVeryStrongRedisPassword!

# Memory management
maxmemory 512mb
maxmemory-policy allkeys-lru

# Disable persistence for pure cache use cases
save ""
appendonly no

# Connection settings
tcp-backlog 511
timeout 300
tcp-keepalive 60

# Performance
hz 20
dynamic-hz yes
lazyfree-lazy-eviction yes
lazyfree-lazy-expire yes

The lazyfree-lazy-eviction yes and lazyfree-lazy-expire yes settings perform memory reclamation in background threads, preventing the main Redis thread from blocking during large key deletions — critical for consistent low latency.

sudo systemctl restart redis-server

# Test connectivity with authentication
redis-cli -a YourVeryStrongRedisPassword! ping

Pattern 1: Application Caching with Proper Eviction Policies

Understanding Eviction Policies

When Redis reaches maxmemory, it must decide which keys to evict. The policy choice dramatically affects cache behavior:

Caching Pattern in Python

import redis
import json
import hashlib
from functools import wraps

r = redis.Redis(host='localhost', port=6379, password='YourPassword', decode_responses=True)

def cache(ttl=300, key_prefix='cache'):
    """Decorator that caches function results in Redis."""
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            # Generate cache key from function name and arguments
            cache_key = f"{key_prefix}:{func.__name__}:{hashlib.md5(str(args).encode()).hexdigest()}"

            # Try to get from cache
            cached = r.get(cache_key)
            if cached is not None:
                return json.loads(cached)

            # Cache miss: call the function
            result = func(*args, **kwargs)

            # Store in cache with TTL
            r.setex(cache_key, ttl, json.dumps(result))
            return result
        return wrapper
    return decorator

@cache(ttl=600, key_prefix='products')
def get_product_catalog(category_id):
    # Expensive database query
    return db.query("SELECT * FROM products WHERE category_id = ?", category_id)

# Cache invalidation: delete specific keys or use patterns
def invalidate_product_cache(category_id):
    pattern = f"products:get_product_catalog:*"
    for key in r.scan_iter(match=pattern):
        r.delete(key)

Caching Pattern in Node.js

const Redis = require('ioredis');
const redis = new Redis({ password: 'YourPassword' });

async function withCache(key, ttl, fetchFn) {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const data = await fetchFn();
  await redis.setex(key, ttl, JSON.stringify(data));
  return data;
}

// Usage
const user = await withCache(
  `user:${userId}`,
  3600,
  () => db.query('SELECT * FROM users WHERE id = ?', [userId])
);

Pattern 2: Pub/Sub for Real-Time Messaging

Redis Pub/Sub enables real-time messaging between different parts of your application — perfect for chat systems, live notifications, and event broadcasting.

Publisher (sends messages)

import redis
import json

publisher = redis.Redis(host='localhost', port=6379, password='YourPassword')

def notify_order_update(order_id, status, user_id):
    message = json.dumps({
        'order_id': order_id,
        'status': status,
        'user_id': user_id,
        'timestamp': time.time()
    })
    # Publish to a channel - all subscribers receive this instantly
    subscriber_count = publisher.publish(f'order_updates:{user_id}', message)
    return subscriber_count

Subscriber (receives messages)

import redis
import json

subscriber_client = redis.Redis(host='localhost', port=6379, password='YourPassword')
pubsub = subscriber_client.pubsub()

# Subscribe to a channel
pubsub.subscribe('order_updates:*')  # Pattern subscription

# Listen for messages
for message in pubsub.listen():
    if message['type'] == 'message':
        data = json.loads(message['data'])
        print(f"Order {data['order_id']} updated to {data['status']}")
        # Push notification, update WebSocket clients, etc.

Pub/Sub with Redis Streams (More Durable)

For event streaming that requires message persistence and consumer groups (multiple workers processing the same stream), Redis Streams are more appropriate than basic Pub/Sub:

import redis

r = redis.Redis(host='localhost', password='YourPassword')

# Producer: append events to stream
r.xadd('orders', {
    'order_id': '12345',
    'action': 'created',
    'user_id': '789',
    'amount': '99.99'
})

# Consumer group (allows multiple workers to share stream processing)
r.xgroup_create('orders', 'order-processors', id='0', mkstream=True)

# Worker: read from consumer group
messages = r.xreadgroup(
    'order-processors', 'worker-1',
    {'orders': '>'},  # '>' means undelivered messages
    count=10,
    block=2000  # Block for 2 seconds if no messages
)

for stream, msgs in (messages or []):
    for msg_id, fields in msgs:
        process_order(fields)
        r.xack('orders', 'order-processors', msg_id)  # Acknowledge processing

Pattern 3: API Rate Limiting

Rate limiting protects your API from abuse and ensures fair usage. Redis’s atomic operations make it ideal for distributed rate limiting across multiple application servers.

Sliding Window Rate Limiter

import redis
import time

r = redis.Redis(host='localhost', password='YourPassword')

def is_rate_limited(identifier, limit=100, window=60):
    """
    Sliding window rate limiter.
    identifier: unique key (user ID, IP address, API key)
    limit: max requests allowed in window
    window: time window in seconds
    Returns True if rate limited, False if allowed
    """
    key = f"rate_limit:{identifier}"
    now = time.time()
    window_start = now - window

    pipe = r.pipeline()
    # Remove old entries outside the current window
    pipe.zremrangebyscore(key, 0, window_start)
    # Add current request timestamp
    pipe.zadd(key, {str(now): now})
    # Count requests in current window
    pipe.zcard(key)
    # Set key expiry (cleanup)
    pipe.expire(key, window)
    results = pipe.execute()

    request_count = results[2]
    return request_count > limit

# Flask middleware example
from flask import request, jsonify
from functools import wraps

def rate_limit(limit=100, window=60, key_func=None):
    def decorator(f):
        @wraps(f)
        def wrapped(*args, **kwargs):
            identifier = key_func() if key_func else request.remote_addr
            if is_rate_limited(identifier, limit, window):
                return jsonify({'error': 'Rate limit exceeded. Try again later.'}), 429
            return f(*args, **kwargs)
        return wrapped
    return decorator

@app.route('/api/data')
@rate_limit(limit=60, window=60)  # 60 requests per minute per IP
def get_data():
    return jsonify({'data': 'response'})

Token Bucket Rate Limiter (Allows Bursts)

import redis
import time

r = redis.Redis(host='localhost', password='YourPassword')

def token_bucket_allowed(key, rate=10, capacity=50):
    """
    Token bucket: allows bursts up to capacity,
    refills at rate tokens per second.
    """
    now = time.time()
    bucket_key = f"token_bucket:{key}"

    pipe = r.pipeline(True)  # Optimistic locking
    try:
        pipe.watch(bucket_key)
        bucket = pipe.hgetall(bucket_key)

        tokens = float(bucket.get(b'tokens', capacity))
        last_update = float(bucket.get(b'last_update', now))

        # Refill tokens based on elapsed time
        elapsed = now - last_update
        tokens = min(capacity, tokens + elapsed * rate)

        if tokens >= 1:
            tokens -= 1
            pipe.multi()
            pipe.hset(bucket_key, mapping={
                'tokens': tokens,
                'last_update': now
            })
            pipe.expire(bucket_key, 3600)
            pipe.execute()
            return True
        return False
    except redis.WatchError:
        return token_bucket_allowed(key, rate, capacity)  # Retry

Pattern 4: Session Storage

Storing sessions in Redis rather than files or the database provides several advantages: sessions persist across server restarts, work seamlessly with multiple application servers, and can be expired automatically.

PHP Session Storage in Redis

sudo nano /etc/php/8.2/fpm/pool.d/www.conf

Add PHP session configuration:

php_value[session.save_handler] = redis
php_value[session.save_path] = "tcp://127.0.0.1:6379?auth=YourPassword&database=1"
php_value[session.gc_maxlifetime] = 3600

Express.js Session Storage

const session = require('express-session');
const RedisStore = require('connect-redis')(session);
const { createClient } = require('redis');

const redisClient = createClient({
  socket: { host: 'localhost', port: 6379 },
  password: 'YourPassword',
  database: 1  // Use a different database number than your cache
});

redisClient.connect().catch(console.error);

app.use(session({
  store: new RedisStore({ client: redisClient }),
  secret: process.env.SESSION_SECRET,
  resave: false,
  saveUninitialized: false,
  cookie: {
    secure: true,   // HTTPS only
    httpOnly: true,
    maxAge: 3600000 // 1 hour
  }
}));

Monitoring Redis in Production

Key Metrics to Watch

redis-cli -a YourPassword INFO all | grep -E \
  "used_memory_human|maxmemory_human|hit_rate|connected_clients|blocked_clients|evicted_keys|keyspace_hits|keyspace_misses"

Calculate Cache Hit Rate

redis-cli -a YourPassword INFO stats | grep -E "keyspace_hits|keyspace_misses"
# hit_rate = hits / (hits + misses) * 100
# Aim for >90% hit rate; below 80% suggests cache configuration issues

Monitor Memory Usage

redis-cli -a YourPassword MEMORY DOCTOR
redis-cli -a YourPassword MEMORY USAGE keyname  # Memory cost of specific key

Real-Time Monitoring

redis-cli -a YourPassword MONITOR  # Stream all commands (development only — impacts performance)
redis-cli -a YourPassword --stat   # Live stats every second

Redis Data Separation Strategy

Use Redis database numbers to separate different types of data:

• DB 0: Application cache (evictable, use allkeys-lru)
• DB 1: Session storage (use noeviction, configure separately if possible)
• DB 2: Queue jobs (never evict — use dedicated Redis instance for production queues)
• DB 3: Rate limiting counters (short-lived, volatile-ttl)

For production systems where queue reliability is critical, run a second Redis instance with persistence enabled (appendonly yes) specifically for queues, separate from the evictable cache instance.

Getting Started

Redis runs efficiently on modest hardware — a 1 GB RAM VPS is sufficient for most caching and session workloads. For large caches or high-throughput Pub/Sub, 2–4 GB RAM provides comfortable headroom. KVM VPS plans at VPS.DO provide the low-latency local network access that makes Redis’s microsecond-level response times practical, with NVMe storage for fast Redis persistence operations when appendonly yes is required.

Conclusion

Redis is a Swiss Army knife for application infrastructure. Beyond simple key-value caching, its atomic operations, Pub/Sub messaging, sorted sets, and streams solve a wide range of real-time and performance problems elegantly. The four patterns in this guide — application caching with proper eviction, Pub/Sub for event broadcasting, sliding window rate limiting, and session storage — address the most common Redis use cases and provide a foundation for more advanced patterns like distributed locks, leaderboards, and real-time analytics.