I share search tips, blog articles, and free events I'm hosting about the search+retreval industry, vector databases, information retrieval and more.
Why do single vector representations fail? (daily search tip)
|
This week we’ll talk a bit about late interaction. But to get there, we need to think about why single vector representations fail. Let’s think about restaurants. Here’s an article reviewing local restaurants. I have three Italian restaurants and two Chinese ones. What’s the average of these? Russian or something!? Maybe Middle Eastern food? If my document lists these restaurants, then that’s exactly what I’ll get in a single vector encoding. A confusing muddle somewhere in the middle of every cuisine. The user comes along looking for “best Italian restaurant in my town” and they don’t get my document listing 3 Italian and 2 Chinese. Because the cosine similarity between the query, “italian restaurant,” and this document, lost somewhere in the Middle East, has become so low. As documents grow in complexity, the problem only worsens This sort of failure mode happens all the time with embeddings, where forever reason the whole washes out the parts. In any information-heavy search, the tension between retrieving the whole and narrowing in on the particular, sometimes diverse, facts in a document becomes stronger. And that’s why this week we’ll learn one approach: late interaction! -Doug PS today, 12:30PM ET is the last day to sign up for Cheat at Search with Agents: http://maven.com/softwaredoug/cheat-at-search Events · Consulting · Training (use code search-tips) You're subscribed to Doug Turnbull's daily search tips where I share tips, blog articles, events, and more. You can always manage your profile: |
Doug Turnbull
I share search tips, blog articles, and free events I'm hosting about the search+retreval industry, vector databases, information retrieval and more.
My work is split between mature search teams and new AI teams. Search teams are often farther along and manage a mature, traditional search product. AI teams, however, often don't know what they don't know yet. They've just been cobbled together, have built a few agent demos, and are often in the process of discovering the three big mistakes I blog here: Evals+measurement need to dominate a lot of your product thinking Retrieval isn't "one thing" (ie classic RAG) - its extremely custom to...
I'm writing about the weak spots in vector databases. Where you should prod and poke when selecting a vendor. Today: Updating Vector Databases If you think about the old vector search regime, it involved Indexing everything up front Never updating the index Search-only We overindexed on this paradigm, creating data structures focused on good search performance that couldn't tolerate updates. I wrote about how sensitive graph-based vector DBs in particular are to updates, picking on Lucene...
Hey all, I wrote a new article about a technique that has come up over and over in my work, especially in my Cheat at Search training, for doing effective query understanding into a large vocabulary. Instead of asking an LLM to classify into a vocabulary. Ask it to hallucinate fake entities, then resolve those to real ones client side. Save yourself a lot of tokens and use cheaper models. https://softwaredoug.com/blog/2026/08/10/hypothetical-classifications -Doug PS - A reminder that Vectors...