Database Deep Dives

Redis and Valkey are separate projects that diverged in 2024 when Redis Ltd. changed the Redis license from BSD to RSALv2/SSPL. Valkey is a Linux Foundation fork of Redis 7.2 that maintains the original BSD license and is governed by an open community including AWS, Google Cloud, Oracle, and Ericsson. Redis 8.0 is the continued development from Redis Ltd. under RSALv2 licensing, with additional features including native vector search and probabilistic data structures. Major cloud providers (AWS ElastiCache, Google Cloud Memorystore, Azure Cache for Redis) have moved their managed offerings to Valkey. For self-hosted deployments, both are technically compatible for most use cases.

Redis 8.0 includes a native vector search engine (previously requiring the RediSearch module) that allows storing high-dimensional vector embeddings as fields in Redis Hashes or JSON documents and performing approximate nearest neighbour (ANN) searches to find semantically similar vectors. You create a vector index using FT.CREATE specifying the vector field, dimensionality, data type, and similarity metric (cosine, L2, inner product). Searches use FT.SEARCH with a KNN query to find the K nearest vectors to a query vector. Indexing uses either HNSW (fast approximate search for large datasets) or FLAT (exact brute-force for smaller datasets). This enables RAG pipelines, semantic search, and recommendation systems using Redis as the vector store.

Redis 8.0 includes several probabilistic data structures natively (previously requiring the RedisBloom module): t-digest for approximate quantile computation over streaming data (e.g., p99 latency tracking); Count-Min Sketch for frequency estimation in data streams; Top-K for identifying the most frequent items in a stream; and Cuckoo Filters as a more capable alternative to Bloom filters (supporting deletion). These structures use fixed or sub-linear memory regardless of data volume, making them ideal for high-velocity streaming analytics where exact computation would require storing all raw data.

For new projects using managed cloud services (AWS, GCP, Azure), Valkey is the practical default since major cloud providers have moved to Valkey for their managed Redis-compatible offerings. For self-hosted deployments: if you need Redis Stack features (vector search, JSON, full-text search, time series) and are comfortable with RSALv2 licensing, Redis 8.0 is compelling as these are now native. If you require an OSI-approved open source license or prefer Linux Foundation governance, Valkey is the choice. Valkey is currently working on adding similar features, so the capability gap will narrow over time.

t-digest is a probabilistic data structure for computing accurate quantile estimates (median, p90, p95, p99, etc.) over streaming data with O(1) memory per percentile, regardless of how many data points have been observed. It is particularly accurate at the tails of the distribution (p99, p99.9) where latency SLAs are typically defined. Use t-digest when you need real-time percentile tracking for latency monitoring, response time SLA dashboards, or any streaming metric where storing all raw values is impractical. In Redis 8.0, the commands are TDIGEST.CREATE, TDIGEST.ADD, and TDIGEST.QUANTILE.

For a RAG (Retrieval-Augmented Generation) pipeline with Redis: create a vector index on your document collection (FT.CREATE with VECTOR field); generate embeddings for your documents using an embedding model (OpenAI text-embedding-3-small, Cohere embed-v3, or a local model); store each document with its embedding in a Redis Hash or JSON document (HSET or JSON.SET); at query time, generate an embedding for the user's question; search for the K nearest document embeddings to the query vector (FT.SEARCH with KNN); retrieve the matching documents and include them in the LLM prompt as context. Redis's in-memory speed makes it particularly suited for real-time RAG applications requiring sub-millisecond vector search latency.

In Redis 8.0, the previously separate Redis Stack modules — RediSearch (full-text search and vector search), RedisJSON (JSON data type), RedisBloom (probabilistic data structures), and RedisTimeSeries (time series data type) — have been integrated into the Redis core engine. You no longer need to load them as external modules with the loadmodule configuration directive. The commands remain the same (FT.CREATE, JSON.GET, BF.ADD, TS.CREATE), but the functionality is available in the base Redis 8.0 installation without any additional configuration. This simplifies deployment and eliminates the need to manage separate module versioning.

The Redis Source Available License v2 (RSALv2) allows free use of Redis for almost all purposes including production commercial use, SaaS applications, and internal enterprise deployments. The restriction is narrow: you cannot use Redis to offer a commercial managed service that competes directly with Redis Ltd.'s own cloud offerings (Redis Cloud). In other words, if you are AWS, Google Cloud, or Azure wanting to sell Redis as a managed service commercially, you cannot use the RSALv2 version without a commercial agreement with Redis Ltd. — which is why they forked Valkey. For virtually all enterprise users who run Redis themselves or use it in their own applications, RSALv2 imposes no practical restrictions versus the previous BSD license.