System Architecture

Vector databases build production-ready systems on top of index structures — adding sharding, replication, filtering, and real-time updates.

---

1. Library vs Database

	Faiss (library)	Milvus/Qdrant (database)
What it is	In-process index	Distributed service
Persistence	Manual (save/load files)	Built-in (WAL, snapshots)
Scaling	Single machine	Horizontal sharding
Updates	Rebuild index	Real-time insert/delete
Filtering	Post-filter only	Pre-filter + vector search
Best for	Research, offline batch	Production serving

---

2. System Comparison

System	Architecture	Index types	GPU support	Best for
Faiss	Library (in-process)	IVF, HNSW, PQ	✅ GPU build + search	Batch processing, research
Milvus	Distributed (segment-based)	IVF, HNSW, DiskANN	✅ GPU indexing	Large-scale production
Qdrant	Distributed (Rust)	HNSW + quantization	❌	Low-latency serving
Weaviate	Distributed (Go)	HNSW	❌	Multi-modal, ease of use
Pinecone	Managed service	Proprietary	N/A	Zero-ops, serverless
pgvector	PostgreSQL extension	IVF, HNSW	❌	Existing Postgres infra

---

3. Sharding and Replication

For billion-scale vector databases:

Collection: 1B vectors
  → Shard 0 (250M vectors) → Node 0 (primary) + Node 3 (replica)
  → Shard 1 (250M vectors) → Node 1 (primary) + Node 4 (replica)
  → Shard 2 (250M vectors) → Node 2 (primary) + Node 5 (replica)
  → Shard 3 (250M vectors) → Node 3 (primary) + Node 0 (replica)

Query: scatter to all shards → each returns local top-K → merge → global top-K

---

4. Hybrid Search (Vector + Metadata Filter)

Real RAG queries combine vector similarity with metadata filters:

-- "Find similar documents from 2024 in the 'medical' category"
SELECT * FROM documents
WHERE category = 'medical' AND year = 2024
ORDER BY vector_distance(embedding, query_vector)
LIMIT 10

Pre-filter vs post-filter:

• Pre-filter: Filter metadata first, then vector search on filtered subset → fast but may miss results
• Post-filter: Vector search first, then filter results → accurate but wastes search on irrelevant vectors

For performance tuning, see Performance Tuning.

---

Self-Check

• Compare Faiss (library) vs Milvus (database) for production use
• Design sharding for a 1B vector collection across 4 nodes
• Explain pre-filter vs post-filter for hybrid search
• Choose a vector database for a given scale and requirements

---

References

• Milvus Documentation
• Qdrant Documentation