Today: AI-Search Fundamentals


This week+next there's some upcoming, free search events I'm cohosting:

AI-Search Fundamentals (free 'intro to search' training)

Today 12PM ET (w/ Me + Trey Grainger)

Sign up -- https://maven.com/p/b9e696/ai-search-fundamentals-vector-spaces-matching-ranking​

This session covers foundational concepts in information retrieval around matching and ranking. We'll cover sparse vs. dense vector spaces, lexical (BM25) vs. semantic (embedding) search, and matching and relevance ranking fundamentals. This session is free (open to all) and also provides the prerequisite background needed for anyone to join Trey & Doug's month-long "AI-Powered Search" course.

Train your own decision model

Monday Nov 9 (w/ Nish Tahir)

Sign up -- https://maven.com/p/9e1ae7/train-your-own-decision-model​

Instead of training ranking models, should search teams train decision models? Agentic systems increasingly need to make fast, reliable choices—not generate long-form text. Decision models offer a way to turn small language models into typed classifiers with confidence scores, giving agents a cheaper and more controllable path for routing, policy decisions, tool selection, and other high-frequency judgments.

----

AI-Powered Search Training (signup still available)

Nov 3 - Dec 3 (w/ me + Trey Grainger)

You can still join with late-signup, my subscribers get promo code doug-newsletter for $300 off

Sign up -- http://maven.com/search-school/ai-powered-search​

-----

Hope to see you at some of these events!

Cheers,

-Doug

​Slack Community · Events · Search consulting · AI-Powered Search training (use code doug-newsletter)

You're subscribed to Doug Turnbull's newsletter where I share tips, blog articles, events, and more.

To unsubscribe: https://preview.kit-mail3.com/unsubscribe

Doug Turnbull

I share search tips, blog articles, and free events I'm hosting about the search+retreval industry, vector databases, information retrieval and more.

Read more from Doug Turnbull

In a world where agents easily drive browsers, why spend time on classic ranking metrics like NDCG? Instead, task an agent with a search use case. See how it fairs in the UI and ranking. In this article, I discuss an alternative eval philosophy. Instead of LLM as a Judge, let agents simulate search users. Observe how they interact with the search app to solve real use-cases. We try to answer the question: can we make search lower effort and easier to accomplish common tasks? Build the...

I am a loyal customer of Alamo Drafthouse and Cinema - a boutique movie theater chain in the US. One nice perk is they allow you to order with a QR code ordering system from your table. Sadly the search doesn't work that well, when you search for "burger" you get "nachos" as the first result. Well that's an annoyance purpose-built for me. So I took it upon myself to recreate the search, then fix it. Did I build vector search? No. Did I try to improve query understanding? No. What I did...

I wrote a detailed blog article critiquing the traditional judgment list search evaluation regime. It's impossible to NOT have some kind of sampling bias. IE your labels all come from BM25 or some legacy search algorithm. So solutions can, at best, measure their effectiveness reranking within that sample. Indeed, its hard to think of an alternative. Labeling random results will just end up wasting time (99% of the corpus will be irrelevant). In the end, you need a sampling algorithm to mine...