9. Search and AI Workloads
Core Redis finds data only by key. Any other question, such as "which products cost less than 50 euros?" or "which documents talk about refunds?", needs an index, and chapter 8.4 showed how much work it takes to build and maintain those indexes by hand. This chapter covers the parts of Redis that answer those questions for you: the Query Engine, which indexes hashes and JSON documents for full-text, numeric, tag, geo and aggregation queries, and vector search, which finds items by meaning rather than by exact value. The last subchapter puts these together into the patterns behind most AI applications built on Redis: retrieval-augmented generation, semantic caching, conversation memory and feature stores.
All examples in this chapter were run against Redis Open Source 8.10. The Query Engine, JSON and vector sets are part of Redis Open Source since 8.0 (chapter 1). On Redis 7.x they need Redis Stack or the separately loaded modules, and on Valkey the equivalent features come from different modules with different commands.
<a id="91-query-engine"></a>
9.1 Query Engine
This subchapter introduces the Redis Query Engine (formerly the RediSearch module): how to declare an index over existing keys, how to search text, how to filter by numbers, tags and locations, how to compute grouped statistics on the server, and how the same index works over hashes and JSON documents.
Creating Indexes with FT.CREATE
Chapter 8.4 built secondary indexes by hand: a set category:shoes with product IDs, a sorted set idx:product:price with prices as scores, all updated in the same transaction as the product hash. That works, but every new query needs a new index structure, every write path must remember to update all of them, and a missed update leaves the index silently wrong.
The Query Engine replaces that manual work with a declared index. You tell Redis once which keys to watch (for example all hashes whose name starts with product:) and which fields to index, with which type. From then on, every write to a matching key updates the index as part of the same command. The application writes plain HSET or JSON.SET commands and never touches the index directly.
An analogy: the keyspace is a library's shelves, where every book is found only by its shelf number (the key). An index is the library catalog. The librarian (the Query Engine) updates the catalog card every time a book is added, changed or removed, so you can look up books by author, title words or year without walking the shelves.
How an index works
HSET product:1 name "Wireless Headphones" price 199.99 brand Acme
│
▼
┌────────────────────────┐ key matches PREFIX "product:"? ┌──────────────────────────────┐
│ keyspace (the data) │ ────────────── yes ──────────────▶ │ idx:product (the index) │
│ product:1 (hash) │ │ name → inverted index │
│ product:2 (hash) │ │ brand → tag index │
│ order:9 (ignored) │ │ price → numeric range tree │
└────────────────────────┘ └──────────────────────────────┘
▲
FT.SEARCH idx:product "@brand:{acme} @price:[50 150]"
Important properties:
- The index is a separate structure in memory, not a Redis key. It does not appear in
KEYSorSCAN, and it uses memory in addition to the data. - Indexing is synchronous for ordinary fields: when
HSETreturns, the change is visible to queries. (Vector fields may be inserted into their graph by background threads, but they are searchable immediately, see chapter 9.2.) - When you create an index over keys that already exist, Redis scans the keyspace in the background and indexes them. Queries work during the scan but see only part of the data until
percent_indexedreaches 1. - The index definition is saved in RDB and AOF, but the index contents are not: after a restart Redis rebuilds them by scanning the keys again. Large datasets therefore take extra time to be fully searchable after a restart.
- Deleting or expiring a key removes it from the index. TTLs work normally.
The FT.CREATE syntax
FT.CREATE <index>
[ON HASH | JSON] # what kind of keys to index (default HASH)
[PREFIX <count> <prefix> ...] # which keys; without PREFIX, every key of that type
[FILTER <expression>] # optional: index only keys matching an expression
[LANGUAGE <lang>] # default stemming language (english)
[STOPWORDS <count> <word> ...] # custom stop-word list (0 = none)
[SKIPINITIALSCAN] # do not index keys that already exist
SCHEMA <field> [AS <alias>] <TYPE> [options] ...
Field types
| Type | Indexes | Typical fields | Query syntax |
|---|---|---|---|
TEXT | Words, after tokenizing, stemming and stop-word removal | Titles, descriptions, comments | wireless headphones, @name:(wire*) |
TAG | Exact values, optionally split on a separator | Category, status, brand, user ID | @brand:{acme | globex} |
NUMERIC | Numbers in a range tree | Price, rating, timestamp | @price:[10 100] |
GEO | Longitude/latitude points | Store location | @loc:[9.19 45.46 5 km] |
GEOSHAPE | Points and polygons (WKT) | Delivery zones, regions | @zone:[CONTAINS $point] |
VECTOR | Embeddings for similarity search | Text or image embeddings | *=>[KNN 5 @vec $q] (chapter 9.2) |
Common field options
| Option | Applies to | Effect |
|---|---|---|
SORTABLE | TEXT, TAG, NUMERIC, GEO | Keeps a copy of the value in the index so SORTBY and aggregations do not have to read the key. Costs memory. |
UNF | SORTABLE TEXT/TAG | Un-normalized form: sort on the original value instead of the lowercased one |
NOINDEX | Most types | Field is stored for sorting or returning but not searchable (use with SORTABLE) |
WEIGHT <n> | TEXT | Matches in this field count n times more in relevance scoring (default 1) |
NOSTEM | TEXT | Disable stemming (useful for names and codes) |
SEPARATOR <c> | TAG | Character that splits one value into several tags (default ,) |
CASESENSITIVE | TAG | Keep tag case; by default tags are lowercased |
INDEXMISSING | Most types | Allow ismissing(@field) queries for documents without the field |
INDEXEMPTY | TEXT, TAG | Allow searching for empty strings |
Example: a product catalog
127.0.0.1:6379> FT.CREATE idx:product ON HASH PREFIX 1 product:
SCHEMA name TEXT WEIGHT 2.0
description TEXT
brand TAG
category TAG SEPARATOR ,
price NUMERIC SORTABLE
rating NUMERIC
location GEO
OK
127.0.0.1:6379> HSET product:1 name "Wireless Headphones"
description "Noise cancelling over-ear headphones with long battery life"
brand "Acme" category "audio,electronics" price 199.99 rating 4.6 location "9.19,45.46"
(integer) 7
127.0.0.1:6379> HSET product:2 name "Wired Earbuds" description "Lightweight earbuds, running and gym friendly"
brand "Globex" category "audio" price 19.90 rating 4.1 location "12.49,41.90"
(integer) 7
127.0.0.1:6379> HSET product:3 name "Bluetooth Speaker" description "Portable waterproof speaker with deep bass"
brand "Acme" category "audio,outdoor" price 89 rating 4.4 location "9.20,45.48"
(integer) 7
127.0.0.1:6379> HSET product:4 name "Running Shoes" description "Lightweight running shoes for long distance"
brand "Initech" category "sport" price 120 rating 4.8 location "9.18,45.47"
(integer) 7
127.0.0.1:6379> FT.SEARCH idx:product "headphones" RETURN 2 name price
1) (integer) 1 # total number of matches
2) "product:1" # key of the first match
3) 1) "name"
2) "Wireless Headphones"
3) "price"
4) "199.99"
(The multi-line commands are shown wrapped for readability. In redis-cli they are typed on one line.)
Note on GEO values: the hash field holds "longitude,latitude", the same order as GEOADD (chapter 3.3).
Managing indexes
| Command | Purpose |
|---|---|
FT._LIST | List all indexes |
FT.INFO idx | Definition, document count, memory usage, percent_indexed, hash_indexing_failures |
FT.ALTER idx SCHEMA ADD field TYPE | Add a field. Existing keys are rescanned in the background so they become searchable on it too. |
FT.DROPINDEX idx | Drop the index, keep the data |
FT.DROPINDEX idx DD | Drop the index and delete every indexed key ("delete documents") |
FT.ALIASADD alias idx / FT.ALIASUPDATE alias idx | Point a stable name at an index |
FT.EXPLAIN idx "query" | Show how a query is parsed |
FT.PROFILE idx SEARCH QUERY "query" | Run a query and show where time is spent |
FT.ALTER cannot change an existing field's type or options. To change the schema safely without downtime, use an alias:
FT.CREATE idx:product:v2 ON HASH PREFIX 1 product: SCHEMA ... # new schema, scans in the background
FT.INFO idx:product:v2 # wait until percent_indexed is 1
FT.ALIASUPDATE products idx:product:v2 # queries on "products" switch atomically
FT.DROPINDEX idx:product:v1 # without DD: the data stays
Selecting keys with FILTER
FILTER indexes only keys whose fields match an expression, which keeps the index small when most keys with the prefix are irrelevant:
127.0.0.1:6379> FT.CREATE idx:user ON HASH PREFIX 1 user: FILTER "@status == 'active'"
SCHEMA name TEXT email TAG INDEXMISSING status TAG
A user whose status changes to banned leaves the index on that write.
Query dialects
The query language has several dialects (versions of the parser). The server default is still dialect 1 (search-default-dialect), but dialects 1, 3 and 4 are deprecated and the Redis documentation recommends dialect 2, which is required for parameters ($name), vector queries and many newer syntax features. Client libraries such as redis-py and RedisVL send DIALECT 2 for you. When typing queries in redis-cli, add DIALECT 2 yourself, or set it once with CONFIG SET search-default-dialect 2.
Query engine in a cluster
In Redis Cluster each shard indexes only its own keys. A query is sent to all shards, each returns its best results, and a coordinator merges them. Keys do not need hash tags to be searchable together, but every query costs work on every shard, and a query with LIMIT 0 10 asks each shard for its own top 10 before merging. Deep pagination (LIMIT 100000 10) is therefore expensive in a cluster, as it is in most search engines.
Key Takeaways
FT.CREATEdeclares an index over keys selected by type, prefix and optional filter. Writes to those keys update the index automatically and synchronously.- Choose field types by how you query:
TEXTfor words,TAGfor exact values,NUMERICfor ranges,GEO/GEOSHAPEfor location,VECTORfor similarity. - The index is held in memory and rebuilt from the keys after a restart. Check
percent_indexedbefore relying on complete results. FT.ALTERonly adds fields. Change schemas by building a new index and switching an alias.FT.DROPINDEX ... DDdeletes your data. Leave outDDunless that is what you want.- Use
DIALECT 2explicitly when typing queries by hand.
Practice
- (Beginner) Create
idx:productas above, load the four products, and runFT.INFO idx:product. Findnum_docs,total_index_memory_sz_mbandpercent_indexed. ThenDEL product:2and check thatnum_docsdrops to 3.- (Intermediate) Load 500,000 hashes with a script, then create an index over them and poll
FT.INFOevery 100 ms while running aFT.SEARCH "*"query. Plot how the result count grows. Repeat withSKIPINITIALSCANand explain the difference.- (Interview) "You need to change the
brandfield of a live index fromTEXTtoTAGwithout any search downtime. Describe the steps." Hint:FT.ALTERcannot do it; think about two indexes over the same keys, how you know the new one is complete, and what the application should query in the meantime.
Full-Text Search
A LIKE '%headphone%' query in SQL reads every row and compares strings, and it still misses "Headphones" if case or plural forms differ. Full-text search solves both problems by preparing the text once, at write time, so that queries become fast lookups.
From text to an inverted index
When a TEXT field is indexed, the Query Engine:
- Tokenizes it: splits on spaces and punctuation and lowercases each token.
- Removes stop words: very common words such as "the", "and", "with" that carry little meaning. A search for
thealone returns nothing. - Stems each word: reduces it to a root, so that "running", "runs" and "run" are the same term. Stemming is language-specific (English by default, set with
LANGUAGE). - Adds the document ID to the posting list of each term, together with the word positions (used for phrase queries and highlighting).
product:1 name "Wireless Headphones" product:4 name "Running Shoes"
product:2 name "Wired Earbuds"
inverted index for "name"
term posting list (doc, positions)
─────────── ────────────────────────────
wireless → product:1 [1]
headphon → product:1 [2] ("headphones", "headphone" share this stem)
wire → product:2 [1]
earbud → product:2 [2]
run → product:4 [1] ("running", "run", "runs")
shoe → product:4 [2]
The query wireless headphones is tokenized and stemmed the same way, and the engine intersects the posting lists of wireless and headphon. That is the analogy of the index at the back of a book: to find pages about "Redis cluster", you do not read the book, you intersect the page numbers listed under "Redis" and under "cluster".
Relevance scoring
When many documents match, the engine ranks them by a relevance score. The default scorer since Redis 8.0 is BM25STD, a standard form of BM25. Its intuition:
- A term that appears more often in a document raises the score, with diminishing returns.
- A rare term across the whole index counts more than a common one ("waterproof" says more than "speaker" in a catalog of speakers).
- Matches in shorter fields count more than the same match in a long description.
- Field
WEIGHTmultiplies the contribution (above, a match innamecounts twice as much as indescription).
Other scorers can be chosen per query with SCORER (for example TFIDF, BM25, DISMAX, DOCSCORE). Add WITHSCORES to see the score, and EXPLAINSCORE to see how it was computed.
Query syntax
| Syntax | Meaning | Example |
|---|---|---|
a b | Both terms (AND) | wireless headphones |
a | b | Either term (OR) | speaker | earbuds |
-a | Exclude documents with the term | lightweight -shoes |
"a b" | Exact phrase, words adjacent and in order | "long battery" |
a* | Prefix match (at least 2 characters by default) | wire* matches "wired", "wireless" |
%a% | Fuzzy match, Levenshtein distance 1 (%%a%% = 2, %%%a%%% = 3) | %%headfones%% |
@field:(...) | Restrict to one field | @name:(wire*) |
~a | Optional term: not required, boosts the score if present | speaker ~waterproof |
(...) | Grouping | (speaker | earbuds) -wired |
Operator precedence matters: AND binds tighter than OR. @brand:{acme} wireless | speaker means (@brand:{acme} wireless) | speaker, so a speaker from any brand matches. Use parentheses whenever you mix the two, and check with FT.EXPLAIN.
Examples
127.0.0.1:6379> FT.SEARCH idx:product "running" WITHSCORES RETURN 1 name
1) (integer) 2
2) "product:4"
3) "2.0816393700398494" # "Running" in the name (weight 2) and "running" in the description
4) 1) "name"
2) "Running Shoes"
5) "product:2"
6) "1.4313364170143106" # only "running" in the description
7) 1) "name"
2) "Wired Earbuds"
127.0.0.1:6379> FT.SEARCH idx:product "%%headfones%%" RETURN 1 name
1) (integer) 1
2) "product:1"
3) 1) "name"
2) "Wireless Headphones"
127.0.0.1:6379> FT.SEARCH idx:product "headphones" HIGHLIGHT FIELDS 1 name RETURN 1 name
1) (integer) 1
2) "product:1"
3) 1) "name"
2) "Wireless <b>Headphones</b>"
Other useful FT.SEARCH options:
| Option | Effect |
|---|---|
LIMIT offset count | Pagination. The default is LIMIT 0 10. |
RETURN n field ... | Return only these fields (RETURN 0 returns IDs only) |
NOCONTENT | Return only keys |
SORTBY field [ASC|DESC] | Sort by a field instead of relevance |
INFIELDS n field ... | Search only these fields |
SLOP n / INORDER | Allow up to n words between query terms / require query order |
SUMMARIZE | Return text fragments around the matches |
VERBATIM | Do not stem the query |
TIMEOUT ms | Per-query time limit (server default search-timeout is 500 ms; by default partial results are returned on timeout) |
Python
import redis
from redis.commands.search.query import Query
r = redis.Redis(decode_responses=True)
def search_products(text: str, page: int = 0, size: int = 10):
q = (Query(text)
.return_fields("name", "price")
.with_scores()
.paging(page * size, size) # translates to LIMIT offset count
.dialect(2))
res = r.ft("idx:product").search(q)
return res.total, [(d.id, d.name, float(d.price), d.score) for d in res.docs]
print(search_products("lightweight -shoes")) # (1, [('product:2', 'Wired Earbuds', 19.9, ...)])
Escaping user input
Characters such as , . < > { } [ ] " ' : ; ! @ # $ % ^ & * ( ) - + = ~ | have meaning in the query language. Never paste raw user input into a query string: escape punctuation with a backslash, or pass values as parameters (PARAMS, dialect 2). Otherwise a search for C++ or t-shirt either fails with a syntax error or means something else (t -shirt excludes "shirt").
Key Takeaways
- Full-text search tokenizes, removes stop words and stems at write time, then answers queries from an inverted index.
- Results are ranked by BM25STD by default. Field weights and
WITHSCORES/EXPLAINSCOREhelp tune relevance. - The syntax supports AND (space), OR (
|), NOT (-), phrases, prefixes, fuzzy matching and field restrictions. AND binds tighter than OR. FT.SEARCHreturns 10 results unless you setLIMIT.- Always escape or parameterize user input.
Practice
- (Beginner) Using
idx:product, write queries for: products mentioning "battery"; products whose name starts with "wir"; products mentioning "lightweight" but not "shoes"; the exact phrase "deep bass". Verify each withFT.EXPLAIN.- (Intermediate) Index 10,000 article titles and bodies with
titleweighted 3. Compare the top 5 results for a query with weight 1 and weight 3, and useEXPLAINSCOREon one document to explain the difference.- (Interview) "Users complain that searching for
iphone 15 proreturns phones that are not Pro, ranked above the Pro model. How do you investigate and improve this?" Hint: look at how the query is tokenized, phrase matching andSLOP, field weights, and whether model names should also be stored as aTAG.
Numeric, Tag and Geo Filters
Full-text search answers "which documents are about X?". Most application queries are different: "products from Acme between 50 and 150 euros", "orders with status shipped", "stores within 5 km". These are filters: a document either matches or not, with no notion of relevance. The Query Engine has a dedicated index structure for each kind of filter, and filters combine freely with each other and with text.
Numeric ranges
NUMERIC fields are stored in a range tree, so range queries do not scan all documents.
| Syntax | Meaning |
|---|---|
@price:[50 150] | 50 ≤ price ≤ 150 |
@price:[(50 150] | 50 < price ≤ 150 (( makes a bound exclusive) |
@price:[-inf 100] | price ≤ 100 |
@price:[100 +inf] | price ≥ 100 |
@price>=100, @price<50, @price==120 | Comparison operators (dialect 2, recent Redis 8 versions) |
Tags
A TAG field stores values exactly: no stemming, no stop words, no tokenizing on spaces. That makes tags the right type for anything you would put in an = or IN clause in SQL: categories, statuses, user IDs, SKUs, email addresses.
@category:{audio} # documents tagged "audio"
@category:{outdoor | sport} # either tag (IN)
-@category:{sport} # not tagged "sport"
@category:{audio} @category:{outdoor} # both tags
@category:{elec*} # tag prefix
The field category "audio,electronics" above holds two tags because of SEPARATOR ,. Tags are lowercased unless the field is CASESENSITIVE. Punctuation inside a tag value must be escaped:
127.0.0.1:6379> FT.SEARCH idx:user "@email:{ada@example.com}" DIALECT 2
(error) Syntax error at offset 11 near example
127.0.0.1:6379> FT.SEARCH idx:user "@email:{ada\\@example\\.com}" RETURN 1 name DIALECT 2
1) (integer) 1
2) "user:1"
3) 1) "name"
2) "Ada"
(In redis-cli the backslash itself must be escaped inside double quotes, hence \\. In client code, one backslash in the final string is what counts.)
TEXT or TAG?
| Question | TEXT | TAG |
|---|---|---|
| Matches words inside the value? | Yes | No, whole value only (or prefix) |
| Stemming and stop words? | Yes | No |
| Relevance score? | Yes | No |
| Memory | Higher (positions, frequencies) | Lower |
| Use for | Free text written by people | Identifiers, enumerations, labels |
A common mistake is indexing status or country as TEXT. The value pending review is then split into two words, so @status:review matches it; stemming makes shipped and shipping the same term; every match gets a relevance score nobody needs; and the index uses more memory. Use TAG.
Geographic filters
A GEO field holds one point per document. The radius filter is @field:[lon lat radius unit] with units m, km, mi or ft:
127.0.0.1:6379> FT.SEARCH idx:product "@location:[9.19 45.46 5 km] @category:{audio}" RETURN 1 name DIALECT 2
1) (integer) 2
2) "product:3"
3) 1) "name"
2) "Bluetooth Speaker"
4) "product:1"
5) 1) "name"
2) "Wireless Headphones"
GEOSHAPE fields go further: they store points and polygons written in WKT (Well-Known Text) and support the relations WITHIN, CONTAINS, INTERSECTS and DISJOINT. This answers "which delivery zone contains this address?", which plain GEO cannot (chapter 3.3 noted that geo commands support only circles and boxes). Use SPHERICAL for longitude/latitude and FLAT for planar coordinates such as a floor plan. Shape queries need dialect 2 or later.
127.0.0.1:6379> FT.CREATE idx:zone ON HASH PREFIX 1 zone: SCHEMA name TEXT area GEOSHAPE SPHERICAL
OK
127.0.0.1:6379> HSET zone:center name "City center"
area "POLYGON((9.17 45.45, 9.21 45.45, 9.21 45.48, 9.17 45.48, 9.17 45.45))"
(integer) 2
127.0.0.1:6379> FT.SEARCH idx:zone "@area:[CONTAINS $pt]" PARAMS 2 pt "POINT(9.19 45.46)" RETURN 1 name DIALECT 2
1) (integer) 1
2) "zone:center"
3) 1) "name"
2) "City center"
A polygon's first and last point must be the same (the ring is closed).
Combining everything
Filters and text combine with the same operators as terms. A query may even consist only of filters, in which case the score is meaningless and you should sort explicitly:
from redis.commands.search.query import Query
q = (Query("(@brand:{acme} | @rating:[4.5 +inf]) @price:[$min $max] @location:[$lon $lat 10 km] -@category:{sport}")
.sort_by("price", asc=True) # price is SORTABLE
.return_fields("name", "price")
.paging(0, 20)
.dialect(2))
res = r.ft("idx:product").search(q, query_params={"min": 50, "max": 250, "lon": 9.19, "lat": 45.46})
Parameters ($min, $max, ...) keep values out of the query string, which removes escaping problems for numbers and coordinates.
Key Takeaways
NUMERICfields answer range queries with[min max], using(for exclusive bounds and-inf/+inffor open ranges.TAGfields match whole values exactly and are the right type for IDs, statuses and categories. Escape punctuation in tag values.GEOanswers radius queries on points.GEOSHAPEanswers containment and intersection queries on polygons.- Filters combine with each other and with text. Sort filter-only queries explicitly.
- Prefer
PARAMSover building query strings from values.
Practice
- (Beginner) Write one
FT.SEARCHthat returns audio products from Acme cheaper than 150, sorted by price descending.- (Intermediate) Index 1,000 restaurants with
cuisine(TAG),rating(NUMERIC SORTABLE),open_hoursand aGEOlocation. Write the query "Italian or Japanese, rating at least 4, within 2 km, best rated first", and measure its latency withFT.PROFILE.- (Interview) "A product has a
colorfield stored asTEXT. A filter for@color:redsometimes returns products whose color isdark red, and sometimes missesRed/Black. Explain both and fix the schema." Hint: tokenizing on spaces and on punctuation, and whatSEPARATORdoes on aTAG.
Aggregations with FT.AGGREGATE
FT.SEARCH returns documents. Many questions are about groups of documents instead: "how many products per brand?", "average price per category?", "top 10 cities by number of orders this week?". Without aggregation, the application would fetch every matching document and compute these numbers itself, moving megabytes over the network to produce a few lines. FT.AGGREGATE runs the computation on the server, the equivalent of SQL's GROUP BY with aggregate functions.
The pipeline model
An aggregation is a pipeline of steps applied in the order written. Each step receives rows from the previous one, like a Unix shell pipeline:
query LOAD APPLY GROUPBY + REDUCE SORTBY LIMIT
"*" ──▶ add fields ──▶ compute new ──▶ one row per group ──▶ order rows ──▶ keep N
from the key fields with aggregates
(FILTER can appear between any steps to drop rows)
| Step | Purpose | SQL equivalent |
|---|---|---|
| query | Select documents (same syntax as FT.SEARCH) | WHERE |
LOAD n @field ... | Read fields that are not SORTABLE from the key | Selecting columns |
APPLY "expr" AS name | Compute a new field per row | Expression in SELECT |
GROUPBY n @field ... | Group rows (GROUPBY 0 = one global group) | GROUP BY |
REDUCE fn n args AS name | Aggregate within each group | COUNT, SUM, ... |
FILTER "expr" | Drop rows | WHERE / HAVING |
SORTBY n @field ASC|DESC | Order rows | ORDER BY |
LIMIT offset count | Pagination | LIMIT |
Reducers
| Reducer | Result |
|---|---|
COUNT 0 | Number of rows in the group |
COUNT_DISTINCT 1 @f / COUNT_DISTINCTISH 1 @f | Exact / approximate (HyperLogLog) distinct count |
SUM, MIN, MAX, AVG (1 @f) | Usual aggregates |
STDDEV 1 @f | Standard deviation |
QUANTILE 2 @f 0.5 | Quantile (0.5 = median) |
TOLIST 1 @f | All distinct values as a list |
FIRST_VALUE n @f [BY @g ASC|DESC] | Value of @f from the first row ordered by @g |
RANDOM_SAMPLE 2 @f n | Sample of n values |
Examples
Products and average price per brand:
127.0.0.1:6379> FT.AGGREGATE idx:product "*"
GROUPBY 1 @brand
REDUCE COUNT 0 AS products
REDUCE AVG 1 @price AS avg_price
SORTBY 2 @products DESC
1) (integer) 3
2) 1) "brand"
2) "Acme"
3) "products"
4) "2"
5) "avg_price"
6) "144.495"
3) 1) "brand"
2) "Initech"
...
A computed field and a filter on it (a HAVING-style step):
127.0.0.1:6379> FT.AGGREGATE idx:product "@category:{audio}"
LOAD 2 @price @rating
APPLY "@price * 0.9" AS discounted
FILTER "@discounted < 100"
SORTBY 2 @discounted ASC
1) (integer) 2
2) 1) "price"
2) "19.9"
3) "rating"
4) "4.1"
5) "discounted"
6) "17.91"
3) 1) "price"
2) "89"
...
Products by distance from a point, using the geodistance function (result in meters):
127.0.0.1:6379> FT.AGGREGATE idx:product "*" LOAD 1 @location
APPLY "geodistance(@location, 9.19, 45.46)" AS meters
SORTBY 2 @meters ASC LIMIT 0 2
APPLY and FILTER expressions support arithmetic, comparisons, and functions for strings (upper, lower, substr, format, split), numbers (floor, ceil, abs, log, sqrt), dates (timefmt, day, hour, month, year on Unix timestamps) and exists(@f).
LOAD and SORTABLE
A field used in APPLY, GROUPBY or SORTBY must be in the pipeline. SORTABLE fields are available directly from the index. Other fields must be loaded with LOAD, which reads each matching key and is much slower on large result sets. If you forget, the error is explicit:
127.0.0.1:6379> FT.AGGREGATE idx:product "*" APPLY "floor(@rating)" AS stars GROUPBY 1 @stars REDUCE COUNT 0 AS n
(error) Property not loaded nor in pipeline: `rating`
Fix it with LOAD 1 @rating, or better, declare rating as SORTABLE if you aggregate on it often.
Large result sets with cursors
An aggregation over millions of documents can produce more rows than you want in one reply. WITHCURSOR COUNT n returns the first n rows plus a cursor ID, and FT.CURSOR READ idx <cursor> fetches the next batch. Cursors expire after an idle timeout (search-cursor-max-idle, 300 seconds by default) and should be deleted with FT.CURSOR DEL when you stop early.
import redis
from redis.commands.search.aggregation import AggregateRequest
import redis.commands.search.reducers as reducers
r = redis.Redis(decode_responses=True)
ft = r.ft("idx:product")
# GROUPBY brand with two reducers, the same as the first redis-cli example
req = (AggregateRequest("*")
.group_by("@brand",
reducers.count().alias("products"),
reducers.avg("@price").alias("avg_price"))
.sort_by("@products", max=10))
for row in ft.aggregate(req).rows: # each row is a flat [name, value, name, value, ...] list
print(dict(zip(row[::2], row[1::2])))
# Streaming a large result in batches of 1,000 rows with a cursor
res = ft.aggregate(AggregateRequest("*").load("@name").cursor(count=1000))
while True:
for row in res.rows:
... # process the batch
if not res.cursor or res.cursor.cid == 0:
break # cursor id 0 means the result is exhausted
res = ft.aggregate(res.cursor) # sends FT.CURSOR READ
Key Takeaways
FT.AGGREGATEis a server-side pipeline: query,LOAD,APPLY,GROUPBY/REDUCE,FILTER,SORTBY,LIMIT, executed in the written order.- Reducers cover counts, sums, averages, quantiles, distinct counts and lists.
- Fields must be
SORTABLEor explicitlyLOADed before they are used.LOADreads keys and is slow at scale. - Use
WITHCURSORto stream large results in batches.
Practice
- (Beginner) For
idx:product, compute the number of products and the minimum, maximum and median price per category. (Remember that a product can have several categories.)- (Intermediate) Store 100,000 orders as hashes with
customer(TAG),total(NUMERIC SORTABLE) andcreated_at(NUMERIC SORTABLE, Unix seconds). Write one aggregation that returns revenue per day for the last 7 days, usingAPPLY "timefmt(@created_at, \"%Y-%m-%d\")". Compare its latency with the same query withoutSORTABLEontotal.- (Interview) "Product managers want a live dashboard of sales by region, updated every second, over 50 million orders. Would you run
FT.AGGREGATEfor every refresh?" Hint: compare the cost of re-aggregating everything with maintaining counters on write (chapter 8.3) or with time series, and think about what happens to the server's other clients.
Indexing Hashes and JSON
The Query Engine indexes two kinds of values: hashes and JSON documents (chapter 3.5). The query language is the same for both. What changes is how a field in the schema points into the value and what shapes of data can be indexed.
Hashes: flat fields
In a hash index, each schema field names a hash field directly. Values are strings, so numbers are parsed when indexed, and a vector is stored as a binary string (chapter 9.2). Hashes are compact and fast, and are the best choice for flat records.
JSON: paths into a document
In a JSON index (ON JSON), each schema field is a JSONPath expression, usually given a short alias with AS. This lets you index nested values and arrays:
127.0.0.1:6379> FT.CREATE idx:book ON JSON PREFIX 1 book:
SCHEMA $.title AS title TEXT
$.author.name AS author TAG
$.tags[*] AS tags TAG
$.year AS year NUMERIC SORTABLE
$.ratings[*] AS ratings NUMERIC
OK
127.0.0.1:6379> JSON.SET book:1 $ '{"title":"Dune","author":{"name":"Frank Herbert"},
"tags":["scifi","classic"],"year":1965,"ratings":[5,4,5]}'
OK
127.0.0.1:6379> JSON.SET book:2 $ '{"title":"Neuromancer","author":{"name":"William Gibson"},
"tags":["scifi","cyberpunk"],"year":1984,"ratings":[4,3]}'
OK
127.0.0.1:6379> FT.SEARCH idx:book "@tags:{classic}" RETURN 2 title year DIALECT 2
1) (integer) 1
2) "book:1"
3) 1) "title"
2) "Dune"
3) "year"
4) "1965"
127.0.0.1:6379> FT.SEARCH idx:book "@ratings:[5 5]" RETURN 1 title DIALECT 2 # any rating equal to 5
1) (integer) 1
2) "book:1"
3) 1) "title"
2) "Dune"
127.0.0.1:6379> FT.SEARCH idx:book "@tags:{scifi}" RETURN 1 $.author.name DIALECT 2
1) (integer) 2
2) "book:1"
3) 1) "$.author.name"
2) "Frank Herbert"
4) "book:2"
5) 1) "$.author.name"
2) "William Gibson"
Notes:
$.tags[*]indexes every element of the array as a separate tag. The same works forTEXT,NUMERICandGEOarrays: a document matches if any element matches.RETURNaccepts a JSONPath, so you can project part of a document. WithoutRETURN,FT.SEARCHreturns the whole document under the field$.- A JSON vector is an array of numbers (
"embedding": [0.12, -0.03, ...]), not a binary string.
When a document does not fit the schema
If a value has the wrong type for its field, the whole document is skipped, not just that field. The write itself succeeds, so the problem is easy to miss:
127.0.0.1:6379> JSON.SET book:3 $ '{"title":"The Hobbit","tags":["fantasy"],"year":"unknown"}'
OK
127.0.0.1:6379> FT.SEARCH idx:book "hobbit" DIALECT 2
1) (integer) 0 # not found, even by title
127.0.0.1:6379> FT.INFO idx:book
...
hash_indexing_failures 1
...
last indexing error "Invalid JSON type: String type can represent only TEXT, TAG, GEO or GEOMETRY field"
last indexing error key "book:3"
Monitor hash_indexing_failures (it counts JSON failures too) and validate documents before writing them. For hashes the same happens when, for example, a NUMERIC field holds "n/a".
Hash or JSON?
| Aspect | Hash | JSON |
|---|---|---|
| Structure | Flat field-value pairs | Nested objects and arrays |
| Multi-value fields | Only via TAG SEPARATOR | Arrays with [*], for all field types |
| Vectors | Binary string (compact, fast to write) | Array of numbers (readable, larger to transfer) |
| Memory | Lower | Higher (tree structure per document) |
| Partial updates | HSET of single fields | JSON.SET on any path |
| Expiration | Whole key, or per field (HEXPIRE, Redis 7.4+) | Whole key |
| Best for | Flat records, high write rates, large vector collections | Documents with nesting or arrays, data already in JSON |
Key Takeaways
- Hash indexes refer to field names. JSON indexes use JSONPath expressions with an
ASalias. [*]indexes every array element. A document matches when any element matches.- A document with a value of the wrong type is skipped entirely. Watch
hash_indexing_failuresinFT.INFO. - Prefer hashes for flat, write-heavy data and large vector sets. Use JSON when the data is naturally nested.
Practice
- (Beginner) Store three orders as JSON with nested
customer.countryand an array ofitems[*].sku. Create an index so you can find orders by country and by any SKU they contain.- (Intermediate) Write the same 100,000 products once as hashes and once as JSON documents, each with its own index. Compare
INFO memory,FT.INFO ... total_index_memory_sz_mband the time of a typical filtered query.- (Interview) "After a deploy, some new products no longer appear in search results, but they exist in Redis and no error was logged. What do you check?" Hint: the write succeeds even when indexing fails; look at
FT.INFOand at what changed in the data format.
<a id="92-vector-search"></a>
9.2 Vector Search
This subchapter covers searching by meaning instead of by exact words or values. It explains what embeddings are, how Redis indexes them (FLAT, HNSW and SVS-VAMANA indexes in the Query Engine), how distance is measured, how to run nearest-neighbor and range queries, how to combine them with ordinary filters and full-text search, and finally the vector set, a native Redis data type for similarity search.
Vector Embeddings
Full-text search matches words. A user who types "cheap flights to the seaside" will not find a document titled "Budget airfare for beach holidays", because the two share no terms, even after stemming. Search by meaning needs a representation of text in which similar meanings are close to each other. That representation is an embedding.
An embedding model is a neural network that turns an input (a sentence, a paragraph, an image, a product, a user) into a fixed-length list of numbers, a vector, for example 384 or 1,536 floating-point values. The model is trained so that inputs with similar meaning get vectors that point in similar directions. You never interpret individual numbers. Only the distances between vectors matter.
An analogy: imagine placing every sentence on a giant map, where nearby points are sentences about the same thing. A real embedding space has hundreds of dimensions instead of two, but the idea is the same: similarity search becomes a geometry problem, "find the points nearest to this one".
dimension 2
▲
│ "budget airfare for beach holidays" ●
│ ● "cheap flights to the seaside" (query)
│ "low-cost trips to the coast" ●
│
│ ● "how to reset a router password"
│ ● "change Wi-Fi admin credentials"
└──────────────────────────────────────────────────────────▶ dimension 1
Where embeddings come from
| Source | Examples | Notes |
|---|---|---|
| Hosted embedding APIs | Embedding endpoints from AI providers | No infrastructure, cost per token, data leaves your network |
| Open-source models run locally | Sentence Transformers models such as all-MiniLM-L6-v2 (384 dimensions) | Free to run, need CPU/GPU capacity |
| Multimodal models | CLIP-style models | Text and images in the same space |
| Your own models | Recommendation or two-tower models | Users and items as vectors |
Rules that hold regardless of the model:
- Use the same model for documents and queries. Vectors from different models (or different versions of the same model) live in different spaces and cannot be compared.
- The dimension is fixed by the model. The index must be created with exactly that
DIM. - Changing the model means re-embedding everything. Store the model name and version with your data so you know what to rebuild.
- Many models produce normalized vectors (length 1), which matters for the choice of distance metric (see "Distance Metrics").
Storing an embedding in Redis
In a hash, a vector is stored as a binary string: the raw bytes of the numbers in little-endian order. With NumPy this is array.astype(np.float32).tobytes(). In a JSON document it is an ordinary array of numbers.
import numpy as np
import redis
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2") # 384-dimensional embeddings
r = redis.Redis() # binary-safe: no decode_responses
texts = {
"doc:1": "Budget airfare for beach holidays",
"doc:2": "How to reset a router password",
}
vectors = model.encode(list(texts.values()), normalize_embeddings=True) # shape (2, 384)
for (key, text), vec in zip(texts.items(), vectors):
r.hset(key, mapping={
"text": text,
"embedding": vec.astype(np.float32).tobytes(), # 384 * 4 = 1,536 bytes
"model": "all-MiniLM-L6-v2", # remember which model produced it
})
Memory: the number that drives every design decision
A vector's size is dimensions × bytes per number:
| Dimensions | FLOAT32 per vector | 1 million vectors | 10 million vectors |
|---|---|---|---|
| 384 | 1.5 KB | 1.5 GB | 15 GB |
| 768 | 3 KB | 3 GB | 30 GB |
| 1,536 | 6 KB | 6 GB | 60 GB |
| 3,072 | 12 KB | 12 GB | 120 GB |
The index adds its own overhead on top (graph links for HNSW), and Redis keeps everything in RAM. This is why smaller models, smaller number types (FLOAT16, INT8) and quantization appear throughout this subchapter.
Key Takeaways
- An embedding is a fixed-length vector produced by a model so that similar inputs are close together.
- Similarity search turns "find related items" into "find nearest vectors".
- Documents and queries must be embedded with the same model and dimension. Changing the model means re-embedding all data.
- In hashes, store vectors as raw little-endian bytes (
np.float32(...).tobytes()). In JSON, store arrays of numbers. - Memory grows linearly with dimension and count. Estimate it before choosing a model.
Practice
- (Beginner) Embed ten short sentences with any embedding model, five about cooking and five about networking. Compute the cosine similarity of every pair with NumPy and check that sentences on the same topic score higher.
- (Intermediate) Estimate the RAM needed for 20 million product descriptions embedded with a 1,024-dimensional model, as FLOAT32 and as FLOAT16, including a rough 20% index overhead. Would it fit on one 64 GB node?
- (Interview) "Your team wants to switch to a better embedding model. The index holds 50 million vectors and must stay online. How do you migrate?" Hint: the old and new vectors cannot be compared; think about a second field or index, backfilling, dual writes, and switching with an alias (chapter 9.1).
Vector Fields (FLAT, HNSW)
Finding the exact nearest neighbors of a query means computing its distance to every stored vector. With 10 million vectors of 768 dimensions, that is about 7.7 billion multiplications per query. Approximate nearest neighbor (ANN) indexes trade a little accuracy for a huge speed-up: they find almost all of the true nearest neighbors while looking at a small fraction of the data. In the Query Engine, a VECTOR field declares which algorithm to use.
Declaring a vector field
FT.CREATE idx:docs ON HASH PREFIX 1 doc:
SCHEMA text TEXT
category TAG
embedding VECTOR HNSW 6 # algorithm, then the NUMBER of attribute arguments that follow
TYPE FLOAT32 # number type of each element
DIM 384 # must equal the model's dimension
DISTANCE_METRIC COSINE
The count after the algorithm name (6 above) is the number of arguments that follow, not the number of attributes. TYPE FLOAT32 DIM 384 DISTANCE_METRIC COSINE is 6 arguments. Adding M 16 EF_CONSTRUCTION 200 makes it 10.
Element types
TYPE | Bytes per element | Notes |
|---|---|---|
FLOAT32 | 4 | The default choice. What most models output. |
FLOAT64 | 8 | Rarely needed |
FLOAT16 | 2 | Half the memory, small precision loss |
BFLOAT16 | 2 | Wider range than FLOAT16, lower precision |
INT8, UINT8 | 1 | For models that output quantized integer embeddings |
The bytes you store must match TYPE and DIM exactly. A 384-dimension FLOAT32 field expects exactly 1,536 bytes. Anything else makes the document fail to index.
FLAT: brute force
A FLAT index stores vectors in contiguous blocks and compares the query with every one of them. Results are exact (100% recall), and inserts are cheap. Query time grows linearly with the number of vectors.
query ──▶ [v1][v2][v3][v4] ... [vN] compute N distances, keep the best K
Use FLAT for small collections (up to the low hundreds of thousands, depending on dimension and latency target), when exact results are required, or when a filter already narrows the candidates to a few thousand. Optional attribute: BLOCK_SIZE (vectors per memory block).
HNSW: a navigable graph
HNSW (Hierarchical Navigable Small World) connects each vector to its nearest neighbors, forming a graph, and stacks several layers of that graph. Upper layers contain few nodes and long links, for fast travel across the space. The bottom layer contains every node, linked to close neighbors, for precise local search.
The analogy is travel: to reach a street in another country, you take a flight to the nearest big airport (top layer), then a train to the right town (middle layer), then walk to the address (bottom layer). At every level you move greedily toward the destination.
layer 2 A ─────────────────────────── F few nodes, long links
│ │
layer 1 A ──────── C ──────── E ───── F ──── H more nodes
│ │ │ │ │
layer 0 A ── B ── C ── D ── E ── F ── G ── H ── I ── J every node, short links
search: enter at the top, walk greedily toward the query,
drop one layer, repeat; on layer 0 explore EF_RUNTIME candidates and return the best K
HNSW attributes:
| Attribute | Default | Effect of increasing it |
|---|---|---|
M | 16 | More links per node: better recall, more memory, slower inserts |
EF_CONSTRUCTION | 200 | More candidates considered while building: better graph, slower inserts |
EF_RUNTIME | 10 | More candidates explored per query: better recall, slower queries (can be overridden per query) |
EPSILON | 0.01 | Wider search boundary for range queries |
HNSW gives search times that grow roughly logarithmically with collection size, at the cost of memory for the graph and slower inserts and deletes than FLAT. It is the usual choice for collections from hundreds of thousands to many millions of vectors.
With worker threads enabled (search-workers), new vectors are first placed in a small flat buffer and inserted into the HNSW graph by background threads, so writes do not wait for graph insertion. The buffer is searched together with the graph, so a vector is findable as soon as HSET returns.
SVS-VAMANA: a compressed graph (Redis 8.2+)
Redis 8.2 added a third algorithm, SVS-VAMANA, based on Intel's Scalable Vector Search library. It builds a single-layer graph (the Vamana algorithm) and can store vectors compressed, which reduces memory significantly compared with HNSW at similar recall. It supports FLOAT32 and FLOAT16 only. The advanced compression modes (COMPRESSION LVQ... and LeanVec...) are optimized for Intel CPUs with AVX-512; on other platforms Redis falls back to a basic 8-bit scalar quantization. Building an SVS-VAMANA index is slower than building HNSW, so it suits large, mostly-read collections where memory is the main cost.
embedding VECTOR SVS-VAMANA 8 TYPE FLOAT32 DIM 768 DISTANCE_METRIC COSINE COMPRESSION LVQ8
Choosing an algorithm
| FLAT | HNSW | SVS-VAMANA | |
|---|---|---|---|
| Results | Exact | Approximate (tunable) | Approximate (tunable) |
| Query cost | Linear in N | ~Logarithmic in N | ~Logarithmic in N |
| Memory | Vectors only | Vectors + graph | Compressed vectors + graph |
| Insert cost | Lowest | Higher | Highest |
| Good for | Small sets, exact needs, heavily pre-filtered queries | General purpose, millions of vectors | Very large, read-mostly sets where RAM is the limit |
Example
import redis
from redis.commands.search.field import TextField, TagField, NumericField, VectorField
from redis.commands.search.index_definition import IndexDefinition, IndexType
r = redis.Redis()
schema = (
TextField("text"),
TagField("category"),
NumericField("price"),
VectorField("embedding", "HNSW", {
"TYPE": "FLOAT32",
"DIM": 384,
"DISTANCE_METRIC": "COSINE",
"M": 16, # links per node
"EF_CONSTRUCTION": 200, # build-time candidate list
}),
)
r.ft("idx:docs").create_index(
schema,
definition=IndexDefinition(prefix=["doc:"], index_type=IndexType.HASH),
)
Key Takeaways
- A
VECTORfield needs an algorithm,TYPE,DIMandDISTANCE_METRIC. The number after the algorithm counts the arguments that follow. - FLAT is exact and linear. HNSW is approximate and fast at scale. SVS-VAMANA (Redis 8.2+) trades build time for lower memory.
- HNSW is tuned with
MandEF_CONSTRUCTIONat build time andEF_RUNTIMEat query time. - Stored bytes must match
TYPE × DIMexactly, or the document is not indexed. - Smaller types (FLOAT16, INT8) and compression are the main levers for memory.
Practice
- (Beginner) Create two indexes over the same 50,000 random 128-dimensional vectors, one FLAT and one HNSW. Compare
vector_index_sz_mbinFT.INFOand the latency of a 10-NN query.- (Intermediate) Measure HNSW recall@10: for 200 query vectors, compute the true top 10 with the FLAT index and count how many of them HNSW returns. Repeat with
EF_RUNTIME10, 50 and 200 and plot recall against latency.- (Interview) "Your HNSW index has 30 million vectors, recall is fine, but p99 query latency is too high and memory is at 90%. What are your options?" Hint: consider
EF_RUNTIME, smaller element types, a smaller model or dimension reduction, SVS-VAMANA compression, and sharding across a cluster.
Distance Metrics
"Nearest" needs a definition of distance. The Query Engine supports three metrics, chosen per field with DISTANCE_METRIC. The right choice depends on how the embedding model was trained, and model documentation usually says which one to use.
The three metrics
▲ L2 (Euclidean): length of the segment between the tips
│ b COSINE: depends only on the angle θ between a and b
│ ╱ IP (inner product): |a| · |b| · cos θ
│ ╱ ⟍ L2 distance
│ ╱ θ ⟍
│ ╱───────────── a
└──────────────────────▶
| Metric | What it measures | Value returned by Redis | Best match | Use when |
|---|---|---|---|---|
L2 | Straight-line distance between the points | Squared Euclidean distance, Σ(aᵢ − bᵢ)² | 0 | Magnitude matters (some image and recommendation models) |
COSINE | Angle between the vectors, ignoring length | 1 − cos θ, from 0 to 2 | 0 | Text embeddings; the safe default when unsure |
IP | Inner (dot) product | 1 − a·b | Lowest (can be negative) | Normalized vectors, or models trained for dot product |
For all three, lower is more similar, so results are sorted ascending.
These values were checked against Redis 8.10 with tiny 2-dimensional vectors:
query q = (1, 0) a = (3, 4) b = (1, 0)
COSINE: cos(q, a) = 3 / 5 = 0.6 → score 1 − 0.6 = 0.4 b → 0
IP: q · a = 3 → score 1 − 3 = −2 b → 1 − 1 = 0 (a ranks first!)
L2 with q = (0, 0.0001): a → 9 + 15.9992 = 24.9992 (squared, not 4.99992)
The IP line shows why inner product needs care: with unnormalized vectors, a longer vector can rank above an identical-direction match. That is the intended behavior for models trained with dot product (length carries meaning, such as item popularity), and a bug for everything else.
Normalization connects the metrics
When every vector has length 1 (it is normalized):
a · b = cos θ, so IP and COSINE produce the same ranking.‖a − b‖² = 2 − 2 cos θ, so L2 produces the same ranking too.
Many text embedding models already return normalized vectors, or have an option for it (normalize_embeddings=True in Sentence Transformers). Normalize once at write time and the metric choice becomes a matter of convenience.
Turning scores into similarities
Applications often want "similarity from 0 to 1" for thresholds:
def cosine_similarity_from_score(score: float) -> float:
return 1.0 - score # COSINE: score = 1 - cos θ, so cos θ = 1 - score
def l2_distance_from_score(score: float) -> float:
return score ** 0.5 # L2: Redis returns the squared distance
Thresholds are model-specific. A cosine similarity of 0.8 may mean "nearly identical" for one model and "loosely related" for another. Calibrate them on your own data.
Key Takeaways
L2returns squared Euclidean distance,COSINEreturns1 − cos θ,IPreturns1 − a·b. Lower is always better.- Use the metric the embedding model was trained for. For text, COSINE is the safe default.
- For normalized vectors, all three metrics give the same ranking.
- Inner product on unnormalized vectors favors long vectors.
- Similarity thresholds depend on the model. Measure them, do not guess them.
Practice
- (Beginner) Store the vectors (1, 0), (0.7, 0.7), (0, 1) and (3, 3) in three indexes with L2, COSINE and IP. Query each with (1, 0) and explain every score you get.
- (Intermediate) Take 1,000 normalized embeddings and show experimentally that the top 10 for 20 queries is identical under all three metrics. Then multiply half of the stored vectors by 5 and repeat.
- (Interview) "A colleague set a duplicate-detection threshold of
score < 0.1on an L2 index and gets almost no duplicates, although there clearly are many. What could be wrong?" Hint: what L2 returns, the scale of the vectors, and which metric the threshold was designed for.
K-Nearest Neighbor Queries
With vectors indexed, the basic question is: "which K stored items are most similar to this query?" That is a K-nearest neighbor (KNN) query. The flow is always the same: embed the user's input with the same model, send the vector as a query parameter, and get back keys sorted by distance.
Syntax
FT.SEARCH idx:docs "*=>[KNN 5 @embedding $vec AS score]"
PARAMS 2 vec <384*4 bytes>
SORTBY score
RETURN 2 text score
DIALECT 2
| Part | Meaning |
|---|---|
* | Pre-filter: which documents may be candidates (* = all) |
=>[KNN 5 @embedding $vec ...] | Return the 5 nearest by field embedding to the parameter vec |
AS score | Name of the field that receives the distance. Without AS, it is __embedding_score. |
PARAMS 2 vec <bytes> | Parameter values (count of name/value arguments, then the pairs) |
SORTBY score | Order by distance. Include it: without it, results are not guaranteed to come back in distance order. |
DIALECT 2 | Required for parameters and vector syntax |
Query-time attributes go inside the brackets, for example [KNN 10 @embedding $vec EF_RUNTIME 100 AS score] to raise recall for one query.
K and LIMIT are independent: KNN 50 with the default LIMIT 0 10 finds 50 neighbors but returns only 10. Set both.
Python example
import numpy as np
import redis
from redis.commands.search.query import Query
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("all-MiniLM-L6-v2")
r = redis.Redis(decode_responses=True) # safe here: RETURN excludes the binary embedding field
def semantic_search(text: str, k: int = 5) -> list[tuple[str, str, float]]:
qvec = model.encode(text, normalize_embeddings=True).astype(np.float32).tobytes()
q = (Query(f"*=>[KNN {k} @embedding $vec AS score]")
.sort_by("score") # ascending: most similar first
.return_fields("text", "score") # never return the raw vector with decode_responses=True
.paging(0, k)
.dialect(2))
res = r.ft("idx:docs").search(q, query_params={"vec": qvec})
return [(d.id, d.text, 1 - float(d.score)) for d in res.docs] # cosine similarity
for key, text, sim in semantic_search("cheap flights to the seaside"):
print(f"{sim:.3f} {key} {text}")
If you return the embedding field with decode_responses=True, redis-py tries to decode the binary vector as UTF-8 and fails. Exclude it with return_fields, or use a separate connection without decoding when you need the raw bytes.
Range queries: "everything within a distance"
KNN always returns K results, even if the nearest one is unrelated. A range query returns all documents within a distance radius, however many there are:
FT.SEARCH idx:docs "@embedding:[VECTOR_RANGE $radius $vec]=>{$YIELD_DISTANCE_AS: score}"
PARAMS 4 radius 0.2 vec <bytes>
SORTBY score LIMIT 0 20
DIALECT 2
With COSINE, radius 0.2 means "cosine similarity at least 0.8". Range queries are the natural fit for thresholds: duplicate detection, semantic caching (chapter 9.3) and "related items, if any".
What makes queries slow
- Large K with HNSW: the search explores at least K candidates.
- High
EF_RUNTIME: more recall, more work. - Returning many fields: each returned field is read from the key. Return only what you show.
- FLAT on large collections: every query is a full scan.
Use FT.PROFILE idx:docs SEARCH QUERY "..." PARAMS ... DIALECT 2 to see where time goes.
Key Takeaways
- KNN syntax:
"<filter>=>[KNN k @field $param AS alias]"withPARAMSandDIALECT 2. - Always
SORTBYthe distance alias, and setLIMITto at least K. EF_RUNTIMEcan be raised per query for better recall.- Range queries (
VECTOR_RANGE) return everything within a radius and suit threshold-based features. - Do not return raw vectors through a client that decodes responses as text.
Practice
- (Beginner) Index 1,000 movie plot summaries with their embeddings. Write a function that takes a sentence and returns the five most similar titles with their cosine similarity.
- (Intermediate) For the same data, find near-duplicate summaries: for each movie, run a range query with radius 0.05 and report pairs. Measure how the runtime changes when you increase the radius.
- (Interview) "Your semantic search always shows 10 results, even for nonsense queries like 'asdfgh', and users complain about irrelevant results. How would you fix it?" Hint: KNN always fills K; think about thresholds, range queries, and combining a minimum similarity with a maximum count.
Hybrid Queries
Pure vector search ignores constraints that users take for granted. "Running shoes like this one" should not return a pair that costs 900 euros, is out of stock, or belongs to another tenant. And pure semantics misses exact matches that keyword search finds easily: a product code like XR-2200 has no meaningful embedding. Hybrid queries combine vector similarity with the filters and full-text search of chapter 9.1. There are two different kinds:
| Kind | Question | Mechanism |
|---|---|---|
| Filtered vector search | "Nearest neighbors among the documents that satisfy these conditions" | A filter expression before =>[KNN ...] in FT.SEARCH |
| Hybrid search (fusion) | "Best documents according to both keyword relevance and semantic similarity" | FT.HYBRID (Redis 8.4+), which merges two ranked lists |
Filtered vector search
Any query expression from chapter 9.1 can replace the * before =>:
FT.SEARCH idx:docs "(@category:{shoes} @price:[0 150] -@status:{discontinued})=>[KNN 10 @embedding $vec AS score]"
PARAMS 2 vec <bytes> SORTBY score RETURN 3 title price score DIALECT 2
Only documents that match the filter are candidates, so the results are the 10 nearest among matching documents, not the 10 nearest overall filtered afterwards (which could leave fewer than 10).
Internally, the engine chooses between two strategies (the "hybrid policy"):
ADHOC_BF (ad-hoc brute force) BATCHES
────────────────────────────── ───────────────────────────────────────
1. evaluate the filter → small set 1. ask the vector index for the next batch
2. compute the distance to each one of nearest neighbors
3. keep the best K 2. keep the ones that pass the filter
3. repeat until K results are found
good when the filter is selective good when the filter matches most documents
The engine picks a policy from the estimated filter selectivity, and you can force one with HYBRID_POLICY ADHOC_BF or HYBRID_POLICY BATCHES BATCH_SIZE n inside the brackets. FT.PROFILE shows the choice as Vector search mode. A very selective filter makes even HNSW queries fast (only a few distances to compute), while a medium-selectivity filter with BATCHES is the slowest case, because many batches may be fetched before K matches are found.
Hybrid search with FT.HYBRID
Keyword and semantic search fail in different ways: keywords miss paraphrases, vectors miss exact identifiers and rare names. Running both and fusing the two ranked lists gives better results than either alone, which is why most production retrieval systems do it. Before Redis 8.4 the application ran two queries and merged them itself. FT.HYBRID does it in one command:
FT.HYBRID idx:docs
SEARCH "running shoes waterproof" # full-text part (BM25STD by default)
YIELD_SCORE_AS text_score
VSIM @embedding $vec # vector part
KNN 2 K 20 # 20 nearest neighbors
FILTER "@category:{shoes}" # pre-filter for the vector part
YIELD_SCORE_AS vector_score
COMBINE RRF 4 WINDOW 20 CONSTANT 60 # fusion method and its parameters
YIELD_SCORE_AS hybrid_score
LOAD 2 @title @price
SORTBY 2 @hybrid_score DESC
LIMIT 0 10
PARAMS 2 vec <bytes>
Two fusion methods are available:
| Method | How it works | Tuning |
|---|---|---|
RRF (Reciprocal Rank Fusion, default) | Each list contributes 1 / (CONSTANT + rank) for every document it contains; contributions are summed | CONSTANT (default 60): higher values flatten the difference between ranks. WINDOW (default 20): how many results from each list take part. |
LINEAR | ALPHA × text_score + BETA × vector_score | Weights must be tuned, because text and vector scores are on different scales |
RRF needs no calibration because it uses ranks, not raw scores. A document ranked first in one list and absent from the other gets 1/61 ≈ 0.0164. A document ranked third in both gets 2 × 1/63 ≈ 0.0317 and wins. Agreement between the two methods is rewarded.
Points to remember about FT.HYBRID:
FILTERinside theVSIMclause pre-filters the vector candidates and must come before that clause'sYIELD_SCORE_AS. AFILTERafterCOMBINEis a post-processing step with aggregation-style expressions such as"@price < 100".- The output format is the one of
FT.AGGREGATE(rows of fields), andLOAD,APPLY,GROUPBYandSORTBYwork after the fusion. - It is newer than the rest of the engine: some tools (
FT.EXPLAIN, parts ofFT.PROFILE) and options such asWITHCURSORdo not support it. Check the release notes of your version.
Key Takeaways
- Filtered vector search puts a normal query expression before
=>[KNN ...]and returns the K nearest among matching documents. - The engine chooses ADHOC_BF (selective filters) or BATCHES (broad filters).
FT.PROFILEshows which one was used. - Hybrid search fuses keyword and vector rankings.
FT.HYBRID(8.4+) does it server-side with RRF or LINEAR. - RRF combines ranks and needs no score calibration. LINEAR combines scores and needs tuned weights.
Practice
- (Beginner) On a product index with embeddings, write a filtered KNN query for "comfortable running shoes" restricted to one brand and a price range. Check with
FT.PROFILEwhich hybrid policy was used.- (Intermediate) Build an evaluation set of 30 queries with known relevant documents, including some with exact product codes. Compare recall@10 for full-text only, vector only and
FT.HYBRIDwith RRF.- (Interview) "In a multi-tenant SaaS, every document belongs to a tenant, and vector search must never return another tenant's data. How do you design the index, and what are the performance trade-offs?" Hint: compare a TAG pre-filter on
tenant_idin one shared index with one index (or prefix) per tenant; consider filter selectivity, the number of tenants and index management cost.
Vector Sets
The Query Engine treats vectors as one field type inside an index over hashes or JSON. Redis 8.0 also introduced a vector set, a native data type created by Redis's original author, Salvatore Sanfilippo. It follows the design of the sorted set: a key holds a collection of string elements, but instead of a score each element has a vector, and the main query is "which elements are most similar to this vector (or to this element)?". There is no schema, no FT.CREATE and no separate index: the data structure is the index (an HNSW graph).
Core commands
| Command | Purpose |
|---|---|
VADD key [REDUCE dim] (VALUES n v1 ... | FP32 blob) element [CAS] [NOQUANT | Q8 | BIN] [EF n] [SETATTR json] [M n] | Add or update an element |
VSIM key (ELE element | VALUES n ... | FP32 blob) [WITHSCORES] [WITHATTRIBS] [COUNT n] [EF n] [FILTER expr] [FILTER-EF n] [EPSILON d] [TRUTH] [NOTHREAD] | Similarity search |
VREM key element | Remove an element |
VCARD key / VDIM key | Number of elements / vector dimension |
VEMB key element [RAW] | Get an element's (approximate) vector |
VSETATTR / VGETATTR key element | Set / get the JSON attributes of an element |
VISMEMBER key element | Membership test |
VRANDMEMBER key [count] | Random elements |
VRANGE key start end [count] | Iterate elements in lexicographic order |
VINFO key / VLINKS key element | Internals: quantization, graph neighbors |
Example
127.0.0.1:6379> VADD movies VALUES 3 0.9 0.1 0.0 alien SETATTR '{"year":1979,"genre":"scifi"}'
(integer) 1
127.0.0.1:6379> VADD movies VALUES 3 0.8 0.2 0.1 blade-runner SETATTR '{"year":1982,"genre":"scifi"}'
(integer) 1
127.0.0.1:6379> VADD movies VALUES 3 0.1 0.9 0.2 notting-hill SETATTR '{"year":1999,"genre":"romance"}'
(integer) 1
127.0.0.1:6379> VADD movies VALUES 3 0.7 0.1 0.6 the-thing SETATTR '{"year":1982,"genre":"horror"}'
(integer) 1
127.0.0.1:6379> VSIM movies ELE alien WITHSCORES COUNT 3 # "more like alien"
1) "alien"
2) "0.9999521672725677"
3) "blade-runner"
4) "0.9919387996196747"
5) "the-thing"
6) "0.8810034692287445"
127.0.0.1:6379> VSIM movies ELE alien FILTER '.year >= 1980 and .genre == "scifi"' WITHSCORES
1) "blade-runner"
2) "0.9919387996196747"
Scores. VSIM returns a similarity from 0 to 1, where 1 means identical direction. It is the cosine similarity rescaled: (1 + cos θ) / 2. Unlike the Query Engine, higher is better. The query element itself is returned first, with a score slightly below 1 because of quantization.
Attributes and filters
Each element can carry a JSON object of attributes. FILTER evaluates an expression against it during the graph search:
.year >= 1980 and .genre == "scifi"
.genre in ["scifi", "horror"] and .year < 1990
"classic" in .tags
!(.price > 50)
Expressions support comparisons, and/or/not (also &&, ||, !), in for arrays, and arithmetic. An element without the referenced attribute does not match. By default a filtered search explores up to COUNT × 100 candidates looking for matches. Raise it with FILTER-EF for very selective filters, or set FILTER-EF 0 to search until COUNT matches are found (possibly scanning everything).
Quantization
By default every vector is stored as 8-bit integers (Q8), not as 32-bit floats. The alternatives are NOQUANT (full FLOAT32) and BIN (1 bit per dimension). Measured on 2,000 random 384-dimensional vectors in Redis 8.10:
| Option | Memory | Bytes per element | Recall |
|---|---|---|---|
NOQUANT | 4.46 MB | ~2,230 | Highest |
Q8 (default) | 2.15 MB | ~1,075 | Very close to NOQUANT for most embeddings |
BIN | 1.48 MB | ~740 | Noticeably lower, fastest |
(The fixed part per element is the HNSW graph, which quantization does not shrink.) The quantization is chosen by the first VADD and must be repeated on every later VADD to the same key. Leaving it out means "Q8", and adding to a NOQUANT set without NOQUANT fails with asked quantization mismatch with existing vector set.
REDUCE dim applies a random projection to a lower dimension when the set is created, trading accuracy for memory. Like quantization, it is fixed at creation.
Vector set or Query Engine vector index?
| Aspect | Vector set | Query Engine (VECTOR field) |
|---|---|---|
| Setup | None: VADD creates the key | FT.CREATE with a schema |
| Data model | Element name + vector + JSON attributes in one key | Hash or JSON documents with many fields |
| Filters | Expression on JSON attributes, during the graph search | Full query language: text, tags, numeric, geo |
| Full-text and hybrid fusion | No | Yes (FT.HYBRID) |
| Aggregations | No | Yes |
| Scale-out | One key lives on one shard; split across several keys yourself | Index spans all shards of a cluster |
| Default storage | Q8 quantized | Full TYPE precision (unless compressed) |
| Feels like | A sorted set for similarity | A search engine |
Choose a vector set for straightforward "more like this" features, recommendations and deduplication where the data is just names and vectors with a few attributes. Choose the Query Engine when queries combine similarity with text, rich filters, aggregations, or when one collection is larger than a single shard.
Python
import redis
r = redis.Redis(decode_responses=True)
vs = r.vset()
vs.vadd("movies", [0.9, 0.1, 0.0], "alien", attributes={"year": 1979, "genre": "scifi"})
vs.vadd("movies", [0.8, 0.2, 0.1], "blade-runner", attributes={"year": 1982, "genre": "scifi"})
similar = vs.vsim("movies", "alien", with_scores=True, count=5, # a string input means ELE
filter='.genre == "scifi"')
print(similar) # {'alien': 0.99..., 'blade-runner': 0.99...}
Key Takeaways
- A vector set is a native type: string elements with vectors and optional JSON attributes, searched with
VSIM, with no schema or separate index. VSIMscores are similarities from 0 to 1, higher is better,(1 + cos θ) / 2.FILTERexpressions on attributes are evaluated during the search. TuneFILTER-EFfor selective filters.- Vectors are Q8-quantized by default. Quantization and
REDUCEare fixed by the firstVADD, and the quantization option must be repeated on laterVADDs. - Use vector sets for simple similarity features, and the Query Engine when you need text, rich filters or cluster-wide collections.
Practice
- (Beginner) Create a vector set of 20 words with 3-dimensional hand-made vectors (for example "animal-ness", "size", "danger"). Query
VSIM ... ELEfor a few words and check whether the results make sense.- (Intermediate) Load 100,000 real embeddings into three vector sets (
NOQUANT,Q8,BIN). CompareMEMORY USAGE, query latency and recall@10 against exact results computed with NumPy.- (Interview) "You use a single vector set for 200 million product embeddings and it no longer fits on one node. How do you scale it?" Hint: a key cannot span shards; think about splitting into N keys by a hash of the element, querying all N in parallel, and merging the results client-side by score.
<a id="93-ai-application-patterns"></a>
9.3 AI Application Patterns
This subchapter applies the Query Engine and vector search to the problems that come up when building applications around large language models (LLMs) and machine-learning models. Redis is a common choice here because it already sits in most stacks as a cache and session store, and its latency suits interactive AI features. The patterns are retrieval-augmented generation, semantic caching of LLM responses, conversation memory for chatbots and agents, and online feature stores for ML inference.
The code uses two placeholder functions so that it works with any provider: embed(text) -> np.ndarray returns a normalized FLOAT32 embedding (as in chapter 9.2), and call_llm(prompt) -> str sends a prompt to whatever LLM you use. All examples were run against Redis 8.10 with stub implementations of both.
Retrieval-Augmented Generation
An LLM knows only what was in its training data. It does not know your company's refund policy, last week's release notes or a customer's contract, and when asked about them it may invent a plausible answer (a hallucination). Retraining the model every time a document changes is impractical. Retrieval-augmented generation (RAG) solves this at query time: retrieve the passages most relevant to the question from your own data, put them in the prompt, and ask the model to answer from that context.
The analogy is an open-book exam. The model is a capable student who has not memorized your documents. RAG hands it the right pages before it answers, and asks it to cite them.
The two pipelines
INGESTION (when documents change)
┌──────────┐ ┌──────────┐ ┌──────────────┐ ┌────────────────────────────────┐
│ document │──▶│ chunk │──▶│ embed chunks │──▶│ Redis: HSET chunk:<doc>:<n> │
│ (PDF, │ │ ~500-1000│ │ (same model │ │ text, doc_id, tenant, vector │
│ page) │ │ chars │ │ as queries) │ │ indexed by idx:chunks │
└──────────┘ └──────────┘ └──────────────┘ └────────────────────────────────┘
QUERY (every question)
┌──────────┐ ┌──────────┐ ┌──────────────────────┐ ┌───────────────┐ ┌────────┐
│ question │──▶│ embed │──▶│ KNN / hybrid search │──▶│ build prompt │──▶│ LLM │──▶ answer
└──────────┘ └──────────┘ │ + tenant/ACL filter │ │ question + │ └────────┘ with
│ + distance threshold │ │ top chunks │ citations
└──────────────────────┘ └───────────────┘
Chunking
Documents are split into chunks before embedding, for two reasons. An embedding of a 50-page document averages everything into one vector and matches nothing well. And the LLM's context window is limited and paid per token, so you send only the relevant passages. Typical choices:
| Decision | Common choice | Trade-off |
|---|---|---|
| Chunk size | 500–1,000 characters (or 200–500 tokens) | Smaller: more precise matches, less context per chunk. Larger: more context, blurrier embeddings. |
| Overlap | 10–20% of the chunk size | Keeps sentences that cross a boundary retrievable from either side |
| Boundaries | Paragraphs, headings, sentences | Splitting mid-sentence hurts both embedding and answer quality |
| Metadata | Document ID, title, section, tenant, permissions, timestamp | Needed for filtering, citations and updates |
Ingestion and retrieval with redis-py
import time
import numpy as np
import redis
from redis.commands.search.field import TextField, TagField, NumericField, VectorField
from redis.commands.search.index_definition import IndexDefinition, IndexType
from redis.commands.search.query import Query
r = redis.Redis(decode_responses=True)
DIM = 384
r.ft("idx:chunks").create_index(
(
TagField("doc_id"),
TagField("tenant"), # every query filters on it
TextField("text"), # enables keyword and hybrid search too
NumericField("updated_at"),
VectorField("embedding", "HNSW",
{"TYPE": "FLOAT32", "DIM": DIM, "DISTANCE_METRIC": "COSINE"}),
),
definition=IndexDefinition(prefix=["chunk:"], index_type=IndexType.HASH),
)
def chunk_text(text: str, size: int = 800, overlap: int = 150) -> list[str]:
chunks, start = [], 0
while start < len(text):
end = min(start + size, len(text))
chunks.append(text[start:end]) # production code should split on paragraphs/sentences
if end == len(text):
break
start = end - overlap
return chunks
def ingest_document(tenant: str, doc_id: str, text: str) -> None:
"""(Re)index a document: old chunks are removed in the same transaction."""
old_chunks = r.smembers(f"doc:{doc_id}:chunks") # set of chunk keys from the previous version
pipe = r.pipeline(transaction=True)
if old_chunks:
pipe.delete(*old_chunks) # a shorter new version must not leave stale chunks
pipe.delete(f"doc:{doc_id}:chunks")
now = int(time.time())
for i, chunk in enumerate(chunk_text(text)):
key = f"chunk:{doc_id}:{i}"
pipe.hset(key, mapping={
"doc_id": doc_id, "tenant": tenant, "text": chunk, "updated_at": now,
"embedding": embed(chunk).astype(np.float32).tobytes(),
})
pipe.sadd(f"doc:{doc_id}:chunks", key)
pipe.execute()
def retrieve(tenant: str, question: str, k: int = 4, max_distance: float = 0.6):
q = (Query("(@tenant:{$tenant})=>[KNN $k @embedding $vec AS distance]")
.sort_by("distance")
.return_fields("doc_id", "text", "distance")
.paging(0, k)
.dialect(2))
res = r.ft("idx:chunks").search(q, query_params={
"tenant": tenant, "k": k, "vec": embed(question).astype(np.float32).tobytes()})
return [d for d in res.docs if float(d.distance) <= max_distance] # drop weak matches
def answer(tenant: str, question: str) -> str:
chunks = retrieve(tenant, question)
if not chunks:
return "I could not find this in the documentation." # do not let the LLM guess
context = "\n\n".join(f"[{c.doc_id}] {c.text}" for c in chunks)
prompt = (
"Answer the question using only the context below. "
"Cite sources as [doc_id]. If the context does not contain the answer, say so.\n\n"
f"Context:\n{context}\n\nQuestion: {question}"
)
return call_llm(prompt)
Design points in this code:
- The tenant filter is part of the vector query, not applied afterwards. Retrieval must never place another customer's data in the prompt, and a pre-filter guarantees K results from the right tenant (chapter 9.2, "Hybrid Queries"). Document-level permissions work the same way, with a
TAGof allowed groups. - A distance threshold prevents the model from answering from irrelevant chunks. With nothing relevant, the honest answer is "not found".
- Re-ingestion deletes old chunks. The per-document set of chunk keys makes updates and deletions exact. Without it, a document that got shorter keeps stale chunks that are still retrieved.
- The
TEXTfield allows switching retrieval toFT.HYBRID(keyword + vector fusion), which usually improves results for questions containing names, codes and acronyms.
Improving retrieval quality
| Problem | Technique |
|---|---|
| Exact terms (error codes, product names) not found | Hybrid search with FT.HYBRID |
| Relevant chunk is ranked 8th, only top 4 are used | Retrieve 20–50 candidates, then re-rank them with a cross-encoder model and keep the best few |
| Chunks lack context ("it supports this since v2") | Prepend the document title and section heading to each chunk before embedding |
| Old and new versions both retrieved | Re-ingest with chunk replacement as above; filter on updated_at or a version tag |
| Hard to know if changes help | Build a small evaluation set of questions with known source chunks and measure recall@k |
Key Takeaways
- RAG retrieves relevant passages from your data at query time and asks the LLM to answer from them, which keeps answers current and reduces hallucinations.
- Split documents into overlapping chunks with metadata, and embed chunks and questions with the same model.
- Apply tenant and permission filters inside the vector query, and drop matches beyond a distance threshold.
- Track the chunks of each document so updates replace them completely.
- Hybrid search, re-ranking and an evaluation set are the usual next steps for quality.
Practice
- (Beginner) Ingest three Markdown files of your own notes with the code above (implement
embedwith any local model). Ask five questions and print the retrieved chunks and their distances before calling any LLM.- (Intermediate) Change the chunk size to 300, 800 and 2,000 characters. For 15 questions with known answers, measure how often the correct chunk is in the top 4. Then add the document title to each chunk and repeat.
- (Interview) "Your RAG assistant answers questions about internal documents. A user from the sales team got an answer quoting an HR document they are not allowed to read. What went wrong, and how do you design retrieval so it cannot happen?" Hint: where permissions are enforced (before retrieval, after retrieval, or in the prompt), pre-filtering by allowed groups, and keeping permissions in sync when they change.
Semantic Caching
LLM calls are slow (often seconds) and expensive (paid per token). In many applications a large share of questions repeat: support bots, FAQ assistants and internal tools get the same questions all day, worded differently. An exact-match cache (chapter 8.1) keyed by the prompt string misses almost all of them, because "What's your refund policy?" and "how do refunds work" are different strings.
A semantic cache stores previous prompts with their embeddings and responses. A new prompt is embedded and compared with cached prompts. If one is close enough, its response is returned without calling the LLM.
prompt ──▶ embed ──▶ vector search in cache (scope filter, K=1)
│
distance ≤ threshold?
┌─────────────┴─────────────┐
yes no
│ │
return cached response call the LLM (seconds, costs tokens)
(milliseconds, free) store prompt + vector + response with a TTL
│
return response
Implementation with the Query Engine
import uuid
import numpy as np
import redis
from redis.commands.search.field import TagField, TextField, VectorField
from redis.commands.search.index_definition import IndexDefinition, IndexType
from redis.commands.search.query import Query
r = redis.Redis(decode_responses=True)
r.ft("idx:llmcache").create_index(
(
TagField("scope"), # model + prompt template + language (+ tenant)
TextField("prompt"),
VectorField("embedding", "HNSW",
{"TYPE": "FLOAT32", "DIM": 384, "DISTANCE_METRIC": "COSINE"}),
),
definition=IndexDefinition(prefix=["llmcache:"], index_type=IndexType.HASH),
)
def cache_lookup(scope: str, prompt: str, threshold: float = 0.1) -> str | None:
q = (Query("(@scope:{$scope})=>[KNN 1 @embedding $vec AS distance]")
.sort_by("distance")
.return_fields("response", "distance")
.paging(0, 1)
.dialect(2))
res = r.ft("idx:llmcache").search(q, query_params={
"scope": scope, "vec": embed(prompt).astype(np.float32).tobytes()})
if res.docs and float(res.docs[0].distance) <= threshold: # cosine distance: 0.1 = similarity 0.9
return res.docs[0].response
return None
def cache_store(scope: str, prompt: str, response: str, ttl: int = 3600) -> None:
key = f"llmcache:{uuid.uuid4().hex}"
r.hset(key, mapping={"scope": scope, "prompt": prompt, "response": response,
"embedding": embed(prompt).astype(np.float32).tobytes()})
r.expire(key, ttl) # stale answers age out; expired keys leave the index
def cached_completion(scope: str, prompt: str) -> tuple[str, bool]:
hit = cache_lookup(scope, prompt)
if hit is not None:
return hit, True
response = call_llm(prompt)
cache_store(scope, prompt, response)
return response, False
A scope tag such as "model-x:faq-v3:en" is part of every lookup. A response generated by a different model, prompt template or language, or for a different tenant, must never be served.
The threshold is the whole game
The threshold decides the balance between two kinds of error:
| Threshold (cosine distance) | Hit rate | Risk |
|---|---|---|
| Very strict (0.02–0.05) | Low: only near-identical wording | Little benefit |
| Moderate (0.1–0.15) | Good for FAQ-style traffic | Occasional wrong answers to similar-looking questions |
| Loose (0.3+) | High | Frequent false hits: "How do I cancel my order?" answered with the response to "How do I cancel my subscription?" |
Short prompts that differ in one important word ("enable" vs "disable", "2023" vs "2024") are the classic false-hit trap, because their embeddings are very close. Calibrate on real traffic: collect pairs of prompts labeled "same answer" or "different answer", and pick the threshold that keeps false hits at an acceptable rate.
When not to use a semantic cache
- Responses that depend on user-specific or real-time data (account balance, order status). Scope the cache by user, or do not cache.
- Conversations where the answer depends on previous turns. The last message alone does not identify the question. Cache only standalone questions, or embed a summary of the context.
- Creative tasks where users expect varied output.
Libraries and services
The RedisVL Python library (redisvl) provides a ready-made SemanticCache with the same design: an index, a distance threshold, TTLs and filterable fields. Redis also offers LangCache, a managed semantic caching service. The core logic is the same as the code above:
from redisvl.extensions.cache.llm import SemanticCache
from redisvl.utils.vectorize import HFTextVectorizer
cache = SemanticCache(
name="llmcache",
redis_url="redis://localhost:6379",
vectorizer=HFTextVectorizer("sentence-transformers/all-MiniLM-L6-v2"),
distance_threshold=0.1, # cosine distance
ttl=3600,
)
hits = cache.check(prompt="how do refunds work?") # list of matches, [] on a miss
if hits:
response = hits[0]["response"]
else:
response = call_llm("how do refunds work?")
cache.store(prompt="how do refunds work?", response=response)
Key Takeaways
- A semantic cache returns a stored LLM response when a new prompt is close enough in embedding space, saving latency and cost.
- Scope every lookup by model, prompt template, language and, where needed, tenant or user.
- The distance threshold trades hit rate against wrong answers. Calibrate it on labeled real traffic.
- Use TTLs so cached answers age out when the underlying information changes.
- Do not cache answers that depend on personal, real-time or conversational context.
Practice
- (Beginner) Implement the cache above and send 20 prompts, 10 of which are paraphrases of the others. Log hits, misses and the distance of each lookup.
- (Intermediate) Create 50 prompt pairs labeled "same answer" or "different answer" (include tricky ones like enable/disable). Compute the false-hit rate and hit rate for thresholds from 0.02 to 0.3 and choose one.
- (Interview) "After deploying a semantic cache, costs dropped 40%, but the support team reports customers receiving answers to questions they did not ask. How do you diagnose and fix this without losing all the savings?" Hint: log distances of hits, look at the false-hit patterns, consider a stricter threshold, narrower scopes, or a cheap verification step on borderline hits.
LLM Conversation Memory
LLM APIs are stateless: each request contains everything the model sees. A chatbot that "remembers" what you said three messages ago does so because the application stores the conversation and sends it back with every request. Agents need even more: notes about the user, results of earlier tool calls, and checkpoints of multi-step work. Redis is a natural store for this state: it is fast, shared by all application instances, and expires data on its own.
There are two kinds of memory, with different structures:
| Short-term memory | Long-term memory | |
|---|---|---|
| Content | The recent messages of the current conversation | Facts and past exchanges worth remembering across sessions ("prefers vegetarian recipes") |
| Access | "The last N messages", in order | "Memories relevant to the current message", by meaning |
| Redis structure | List (or stream) per session | Hashes or JSON with embeddings, in a vector index |
| Lifetime | Hours to days (TTL) | Months, or until deleted |
Short-term memory: a capped list per session
import json
import time
import redis
r = redis.Redis(decode_responses=True)
MAX_MESSAGES = 20
def add_message(session_id: str, role: str, content: str, ttl: int = 86400) -> None:
key = f"chat:{session_id}:messages"
pipe = r.pipeline(transaction=True)
pipe.rpush(key, json.dumps({"role": role, "content": content, "ts": time.time()}))
pipe.ltrim(key, -MAX_MESSAGES, -1) # keep only the newest 20 messages
pipe.expire(key, ttl) # sliding expiration: idle sessions disappear after a day
pipe.execute()
def recent_messages(session_id: str, n: int = 10) -> list[dict]:
return [json.loads(m) for m in r.lrange(f"chat:{session_id}:messages", -n, -1)]
def chat(session_id: str, user_text: str) -> str:
add_message(session_id, "user", user_text)
history = recent_messages(session_id)
prompt = "\n".join(f"{m['role']}: {m['content']}" for m in history) + "\nassistant:"
reply = call_llm(prompt)
add_message(session_id, "assistant", reply)
return reply
A message count is a rough limit. Models limit tokens, so production code usually walks the list from newest to oldest, adding messages until a token budget is reached.
Summarizing old turns
Trimming loses information. A common compromise keeps a running summary: when the list grows past a limit, the oldest messages are summarized by the LLM, the summary is stored in a separate key, and those messages are removed.
chat:s1:summary "User is planning a 5-day trip to Lisbon in May, budget 1,200 EUR, vegetarian."
chat:s1:messages [ last 10 messages verbatim ]
prompt = system instructions + summary + last messages + new message
Long-term memory: semantic recall
Long-term memories are stored as documents with embeddings, filtered by user, and retrieved by similarity to the current message, exactly like RAG chunks:
HSET memory:u42:7f3a user u42 text "Prefers vegetarian food" created_at 1762780000 embedding <bytes>
FT.SEARCH idx:memory "(@user:{u42})=>[KNN 3 @embedding $vec AS distance]"
PARAMS 2 vec <embedding of the current message> SORTBY distance DIALECT 2
Deciding what to store is the hard part. Usual approaches are asking the LLM to extract durable facts after each conversation, and deduplicating with a range query (do not store "likes vegetarian food" if "prefers vegetarian food" is already within a small distance).
Libraries
RedisVL provides MessageHistory (ordered recent messages per session) and SemanticMessageHistory (adds get_relevant(), a vector search over past messages). Agent frameworks have Redis integrations too, for example the langgraph-checkpoint-redis package, which stores LangGraph agent checkpoints and long-term memory in Redis.
from redisvl.extensions.message_history import MessageHistory
history = MessageHistory(name="support-bot", redis_url="redis://localhost:6379")
history.add_message({"role": "user", "content": "My order 1234 has not arrived."}, session_tag="s1")
history.add_message({"role": "assistant", "content": "Let me check order 1234."}, session_tag="s1")
recent = history.get_recent(top_k=10, session_tag="s1")
Privacy
Conversation memory is personal data. Use TTLs for short-term memory, provide a way to delete everything stored for a user (keep a set of a user's memory keys or a user tag you can query), and avoid storing secrets the user pastes into a chat.
Key Takeaways
- LLMs are stateless; the application stores the conversation and sends the relevant part with each request.
- Short-term memory: a list per session with
RPUSH,LTRIMand a slidingEXPIRE, trimmed to a token budget. - Summaries keep the gist of long conversations within the context window.
- Long-term memory: facts stored with embeddings, filtered by user, recalled by vector search.
- Treat memory as personal data: expiration and complete deletion per user.
Practice
- (Beginner) Implement
add_messageandrecent_messages. Add 30 messages to one session and verify withLLENandTTLthat only 20 remain and the expiration is refreshed on each message.- (Intermediate) Replace the message-count limit with a token budget of 2,000 tokens (approximate tokens as words × 1.3). Then add a running summary: when the budget is exceeded, summarize the oldest half with
call_llmand store it underchat:<id>:summary.- (Interview) "Your chatbot runs on 20 stateless application servers behind a load balancer. Users complain the bot sometimes forgets the previous message. What is happening, and how does Redis fix it? What do you do when Redis fails over?" Hint: in-process memory versus a shared store, replication lag after failover (chapter 7), and what a lost last message costs here compared with a lost payment.
Feature Stores
Machine-learning models used in production rarely take raw input only. A fraud model scoring a card payment also needs features such as "number of transactions by this card in the last hour", "average amount over 30 days" and "distance from the last transaction". These values are computed from historical data by pipelines, and must be available at prediction time, within a few milliseconds, for whichever card is paying right now.
A feature store is the system that manages these features. It has two parts:
| Offline store | Online store | |
|---|---|---|
| Purpose | Training data: full history of feature values | Serving: latest feature values per entity |
| Access pattern | Large scans, point-in-time joins | Key lookups for one or a few entities |
| Latency | Seconds to minutes | Single-digit milliseconds |
| Typical technology | Data warehouse, data lake (Parquet) | Redis, other key-value stores |
batch jobs / stream processors
│ compute features
┌─────────────┴──────────────┐
▼ ▼
┌───────────────┐ ┌───────────────┐ HMGET / pipeline ┌────────────────┐
│ offline store │ │ online store │◀─────────────────────│ model service │◀── request
│ (history, │ │ Redis: latest │ features for the │ (fraud scoring)│
│ training) │ │ value / entity│ entities involved └────────────────┘
└───────────────┘ └───────────────┘
│ "materialize": copy latest values ──▲
└─────────────────────────────────────┘
Why Redis fits the online store
- Lookups by entity key are exactly what Redis does fastest, and a pipeline fetches features for many entities in one round trip.
- Hashes map naturally to "entity → feature name → value".
- TTLs express freshness: a feature older than its validity window disappears instead of silently feeding stale values to the model.
- Atomic counters and sorted sets let a stream processor update real-time features (counts in the last hour, chapter 8.3) directly in the online store.
A hand-rolled online store
import redis
r = redis.Redis(decode_responses=True)
def write_features(entity: str, entity_id: str, view: str, features: dict, ttl: int) -> None:
key = f"fs:{view}:{entity}:{entity_id}" # one hash per feature view and entity
pipe = r.pipeline(transaction=False)
pipe.hset(key, mapping=features)
pipe.expire(key, ttl) # freshness guarantee
pipe.execute()
def read_features(entity: str, ids: list[str], view: str, names: list[str]) -> list[dict]:
pipe = r.pipeline(transaction=False)
for entity_id in ids:
pipe.hmget(f"fs:{view}:{entity}:{entity_id}", names)
rows = pipe.execute() # one round trip for all entities
return [dict(zip(names, values)) for values in rows] # None marks a missing or expired feature
write_features("card", "4111", "card_stats_1h",
{"txn_count_1h": 3, "amount_sum_1h": 182.40, "distinct_merchants_1h": 2}, ttl=3600)
print(read_features("card", ["4111", "9999"], "card_stats_1h", ["txn_count_1h", "amount_sum_1h"]))
# [{'txn_count_1h': '3', 'amount_sum_1h': '182.4'}, {'txn_count_1h': None, 'amount_sum_1h': None}]
The model service must decide what to do with missing values (None): use a default, use a fallback model, or reject the request. That decision belongs to the model's design, not to Redis.
Using a feature store framework
Frameworks such as Feast manage feature definitions, materialization from the offline store, and point-in-time correct training data, and support Redis as the online store. With Feast, Redis is configured in feature_store.yaml:
project: fraud_detection
registry: data/registry.db
provider: local
online_store:
type: redis
connection_string: "localhost:6379" # for a cluster: redis_type: redis_cluster
offline_store:
type: file
from feast import FeatureStore
store = FeatureStore(repo_path=".")
features = store.get_online_features(
features=["card_stats_1h:txn_count_1h", "card_stats_1h:amount_sum_1h"],
entity_rows=[{"card_id": 4111}],
).to_dict()
Training/serving skew
The most dangerous feature-store bug is training/serving skew: the model was trained on features computed one way (in the offline pipeline) and receives values computed differently online (another time window, another rounding, values that are hours stale). The model then performs worse in production than in testing, with no error anywhere. A feature store reduces skew by defining each feature once and producing both stores from the same definition. TTLs and monitoring of feature age catch the staleness part.
Vectors as features
Embeddings are features too. Recommendation systems store user and item embeddings in Redis and combine both sides of the pattern: a key lookup for the user's embedding, then a vector search (chapter 9.2) for the nearest items, filtered by availability.
Key Takeaways
- A feature store serves precomputed model inputs: an offline store for training history, an online store for the latest values at prediction time.
- Redis is a common online store: entity lookups in milliseconds, hashes per entity, pipelines for batches, TTLs for freshness.
- Missing or expired features must have an explicit handling strategy in the model service.
- Training/serving skew silently degrades models. Define features once and monitor their freshness.
- Embeddings stored as features connect the feature store to vector search for recommendations.
Practice
- (Beginner) Implement
write_featuresandread_features. Write features for 1,000 cards and read 100 random cards in one pipeline. Measure the latency with and without the pipeline.- (Intermediate) Build a real-time feature
txn_count_1hupdated on every transaction with a sorted set per card (timestamp scores, removing entries older than an hour, like the sliding-window log in chapter 8.2). Compare its value with a batch job that recomputes it every 10 minutes, and describe the skew between them.- (Interview) "Your fraud model reads 40 features for each payment from Redis with a 10 ms budget. At peak, p99 latency is 25 ms. How do you investigate and fix it?" Hint: count round trips per request, pipeline or group features into fewer keys, check for big or hot keys (chapter 8.4), cluster topology and client connection pooling, and whether some features can be cached in the model service.
<a id="10-client-development"></a>