AI applications, such as chatbots that answer from your documents, semantic search and recommendations, need to find content that’s similar in meaning, not just text that matches keywords. Vector databases store embeddings (lists of numbers that represent meaning) and quickly find the nearest ones to a query. Here’s how they work, the main options, and how to self-host one.

Embeddings in one paragraph

An embedding model (from OpenAI, Cohere, Voyage, or open models like bge and e5) turns text, an image or other data into a vector, for example 768 or 1536 floating-point numbers. Content with similar meaning ends up with vectors that are close together. “How do I reset my password?” and “I forgot my login credentials” are close, even though they share few words.

What a vector database does

  1. Store vectors together with an ID and metadata (source document, date, user, tags).
  2. Search by similarity: given a query vector, return the k nearest vectors, by cosine similarity, dot product or Euclidean distance.
  3. Filter by metadata at the same time (“similar chunks, but only from this customer’s documents”).

Exact nearest-neighbour search over millions of vectors is slow, so vector databases use approximate nearest neighbour (ANN) indexes, most commonly HNSW (a navigable graph) or IVF (clustering). These trade a tiny bit of accuracy for huge speedups.

The typical use: RAG

Retrieval-augmented generation: an LLM answers questions using your content.

  1. Split documents into chunks, embed each chunk, and store them in the vector database.
  2. At question time, embed the question and retrieve the most similar chunks.
  3. Send those chunks plus the question to the LLM, which answers using them.

Other uses: semantic site search, duplicate detection, recommendations (“similar products”), image search and classification.

Options

Option Type Notes
pgvector PostgreSQL extension Vectors next to your relational data, with SQL filtering. HNSW and IVFFlat. The easiest start if you already use PostgreSQL
Qdrant Dedicated (Rust) Fast, strong filtering, simple Docker deploy, quantisation for lower RAM
Milvus Dedicated, distributed For very large scale. More components
Weaviate Dedicated Built-in modules for vectorisation, hybrid search
Chroma Lightweight Popular for prototypes and local development
Elasticsearch / OpenSearch Search engines with vector support Combine keyword and vector (“hybrid”) search. See Elasticsearch
Redis / Valkey (search modules) In-memory Low latency for smaller sets

Start with pgvector if your app uses PostgreSQL and you have up to a few million vectors. Move to a dedicated engine when scale, filtering performance or features demand it.

pgvector quick start

sudo apt install postgresql-16-pgvector      # package name varies by PostgreSQL version
CREATE EXTENSION vector;

CREATE TABLE chunks (
  id bigserial PRIMARY KEY,
  doc_id bigint,
  content text,
  embedding vector(1536)
);

CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);

-- nearest 5 chunks to a query embedding
SELECT id, content
FROM chunks
ORDER BY embedding <=> '[0.012, -0.034, …]'::vector
LIMIT 5;

<=> is cosine distance, <-> Euclidean and <#> negative inner product. See PostgreSQL users and databases.

Qdrant with Docker

docker run -d --name qdrant --restart unless-stopped \
  -p 127.0.0.1:6333:6333 -v qdrant-data:/qdrant/storage qdrant/qdrant

Create a collection and search through its REST API or client libraries (Python, JS, Go, Rust). Keep it bound to localhost or a private network, and enable an API key. See Docker security.

Sizing a VPS for vectors

HNSW indexes perform best when they fit in RAM:

memory ≈ vectors × dimensions × 4 bytes × ~1.5 (index overhead)
Vectors Dimensions Raw float32 With index (approx.)
100,000 768 ~0.3 GB ~0.5 GB
1,000,000 768 ~3 GB ~4.5 GB
1,000,000 1536 ~6 GB ~9 GB

Quantisation (storing vectors as int8 or binary) cuts memory 4–32× with a modest accuracy loss. Qdrant, Milvus and pgvector (halfvec, binary) support variants.

Embedding generation is a separate cost: either API calls, or running an embedding model, which is fine on CPU for moderate volumes, and much faster on a GPU. See how much RAM a VPS needs.

Practical tips

  • Chunking matters more than the database. Split documents into sensible pieces (a few hundred tokens, with overlap), and keep titles and context.
  • Store metadata for filtering (tenant, language, date), and always filter by tenant in multi-user apps.
  • Hybrid search (keyword + vector) often beats pure vector search for names, codes and rare terms.
  • Re-embed on model changes. Vectors from different models aren’t comparable.
  • Back up like any database. See database backup automation.

Quick reference

Need Suggestion
Already on PostgreSQL, < a few million vectors pgvector
Dedicated, fast, easy to self-host Qdrant
Massive scale, distributed Milvus
Keyword + vector hybrid Elasticsearch / OpenSearch, Weaviate
Prototype locally Chroma

Related: SQL vs NoSQL and Open WebUI.

  • Databases
  • AI
  • Self-Hosting