I share search tips, blog articles, and free events I'm hosting about the search+retreval industry, vector databases, information retrieval and more.
Evaluating search: free class Tuesday
Final free class Tuesday:Tuesday will be the last Cheat at Search Essentials before we begin Cheat at Search with Agents next week Have you ever had to evaluate a search application? To figure out if its satisfying users? To distinguish your team's opinion from what users actually click on? You may hear terms like "NDCG" or "judgment list". WTF is that? If you want to backfill some basics, understand search evaluation, come to the final Cheat at Search Essentials class on Tuesday. I'll define these core concepts while also giving opinions of my own of what you should do. Feb's paid course:Please consider signing up for my paid training in Feb. I'm grateful for any support! It lets me continue this newsletter, my blog, and free speaking. To that end, coming to Tuesday's class is the final discount for Feb's Cheat at Search with Agents. Prices will go up after Feb 2nd. What is Cheat at Search with Agents? I'll be teaching what has become significant themes in 2026 using Agents + LLMs in search:
Best, -Doug Events · Consulting · Training (use code search-tips) You're subscribed to Doug Turnbull's daily search tips where I share tips, blog articles, events, and more. You can always manage your profile: |
Doug Turnbull
I share search tips, blog articles, and free events I'm hosting about the search+retreval industry, vector databases, information retrieval and more.
Hey all, I wrote a new article about a technique that has come up over and over in my work, especially in my Cheat at Search training, for doing effective query understanding into a large vocabulary. Instead of asking an LLM to classify into a vocabulary. Ask it to hallucinate fake entities, then resolve those to real ones client side. Save yourself a lot of tokens and use cheaper models. https://softwaredoug.com/blog/2026/08/10/hypothetical-classifications -Doug PS - A reminder that Vectors...
There's a class of search problems where: The total number of records is small ~1 million The index is built up front and doesn't change frequently Don't get hoodwinked into buying a vector database. Just brute-force search your embeddings. A naive Numpy search can do exhaustive vector search at 170QPS. You can replicate and copy that data to scale even further. More in my blog article: https://softwaredoug.com/blog/2026/07/29/just-brute-force-embeddings Wectors Week + Context Camp A reminder...
Almost all my clients face this issue. They want to build a RAG system. They have: Millions of PDFs That turn into Billions of chunks + embeddings Turns into become petabytes of floats That's the recipe of nearly every speculative RAG project I've worked on. A mature search team stands up a vector database and can't tame the sheer scale of data they're facing. That drags project timelines out months while costs skyrocket. It's important to have tools to manage the scale. One often overlooked...