Spreadsheet implementation of word2vec (daily search tip)

Published about 1 month ago • 1 min read

Here’s a fun spreadsheet that implements word2vec. Use it for jumping off point.

It has:

A single small vocabulary of 9 words
A single example mary had a little lamb
We move positive vectors closer together (mary is IN the context window of had); We move negative vectors farther apart (toenail is NOT IN the context window of mary)

In word2vec we maintain two embeddings per vocabulary entry (input and output) vectors.

They mean subtly different things:

Let’s say two inputs are similar, mary and jane.This says: when mary and jane act as centers, they co-occur with similar in-context words. Maybe poppins?
Let’s say two outputs are similar: little and lamb . This says: they often appear near the same center words, ie mary

In the end, they’ll be very correlated. But most people take the input vectors after training

The spreadsheet models two word2vec variants

Softmax: A model to predict probability of an adjacent word directly, given a center word. IE poppins near mary would have a reasonably high probability compared to monkeys near mary. Then backprop to the embeddings so it predicts these probabilities more accurately. Sadly, when you get to actual vocabularies of 100k-1m+, doing this during training becomes infeasible
Skipgram, negative sampling: Learn on samples: tweak dot products of one positive to the center word (ie poppins closer to mary) - but push farther from out-of-context words, like mary to toenail . This method scales better and is more common

Vector search means query understanding!? (daily search tip)

Good vector search means more than embeddings. Embeddings don’t know when a result matches / doesn’t match. Similarity floors don’t work consistently - a cutoff that works for one query might be disastrous for another. Even worse: your embedding usually can’t capture every little bit of meaning from your corpus. You need to efficiently pick the best top N candidates from your vector database. What do you need? Query Understanding - translating the query to domain language (categories, colors,...

about 19 hours ago • 1 min read

Use RRF sparingly in hybrid search

Reciprocal Rank Fusion merges one system’s search ranking with another’s (ie lexical + embedding search). RRF scores a document with ∑1/rank of each underlying system. I’ve found RRF is not enough. Here’s the typical pattern I see on teams: A mature lexical solution exists. It’s pretty good, The team wants to add untuned, embedding based retrieval, They deploy a vector DB, and RRF embedding results with the mature system, Disaster ensues! The poor embedding results drag down the lexical...

3 days ago • 1 min read

Blog post on Bayesian BM25

Just sharing my post on Bayesian BM25 and other ways of normalizing BM25 scores. Enjoy! https://softwaredoug.com/blog/2026/03/06/probabilistic-bm25-utopia Do you have any thoughts on normalizing BM25 scores? -Doug Events · Consulting · Training (use code search-tips) You're subscribed to Doug Turnbull's daily search tips where I share tips, blog articles, events, and more. You can always manage your profile:

4 days ago • 1 min read