I share search tips, blog articles, and free events I'm hosting about the search+retreval industry, vector databases, information retrieval and more.
Don't classify, hallucinate!
|
Hey all, I wrote a new article about a technique that has come up over and over in my work, especially in my Cheat at Search training, for doing effective query understanding into a large vocabulary. Instead of asking an LLM to classify into a vocabulary. Ask it to hallucinate fake entities, then resolve those to real ones client side. Save yourself a lot of tokens and use cheaper models. https://softwaredoug.com/blog/2026/08/10/hypothetical-classifications -Doug PS - A reminder that Vectors Week is happening now if you want to chat about vector search, embeddings, and learn about the data structures that power your favorite vector database. Slack Community * Events · Consulting · Training (use code search-tips) You're subscribed to Doug Turnbull's daily search tips where I share tips, blog articles, events, and more. You can always manage your profile: |
Doug Turnbull
I share search tips, blog articles, and free events I'm hosting about the search+retreval industry, vector databases, information retrieval and more.
My work is split between mature search teams and new AI teams. Search teams are often farther along and manage a mature, traditional search product. AI teams, however, often don't know what they don't know yet. They've just been cobbled together, have built a few agent demos, and are often in the process of discovering the three big mistakes I blog here: Evals+measurement need to dominate a lot of your product thinking Retrieval isn't "one thing" (ie classic RAG) - its extremely custom to...
I'm writing about the weak spots in vector databases. Where you should prod and poke when selecting a vendor. Today: Updating Vector Databases If you think about the old vector search regime, it involved Indexing everything up front Never updating the index Search-only We overindexed on this paradigm, creating data structures focused on good search performance that couldn't tolerate updates. I wrote about how sensitive graph-based vector DBs in particular are to updates, picking on Lucene...
There's a class of search problems where: The total number of records is small ~1 million The index is built up front and doesn't change frequently Don't get hoodwinked into buying a vector database. Just brute-force search your embeddings. A naive Numpy search can do exhaustive vector search at 170QPS. You can replicate and copy that data to scale even further. More in my blog article: https://softwaredoug.com/blog/2026/07/29/just-brute-force-embeddings Wectors Week + Context Camp A reminder...