Project: Find Similar Sentences with NumPy

What you'll build

A support team gets the same questions again and again. When a customer types "how do I get a refund", the help widget should show the closest existing answers. You'll build the search behind it: give it a sentence, get back the most similar sentences from a list, ranked by a score.

You'll use only NumPy and the ideas from this part: sentences become vectors, similarity is a cosine, and scoring every sentence at once is one matrix multiply. By the end you'll also see exactly where this approach breaks, which is the reason embedding models (Part 5) exist.

Prerequisites

• Vectors and Matrices: dot product, cosine similarity, shapes.
• Probability and Statistics: enough to read the IDF formula as "rarer means more informative".
• Python with NumPy installed (pip install numpy).

Architecture

Play

Two phases, like every search engine:

Phase	When it runs	Cost
Index	once, when the sentences change	tokenize, count, weight and normalize every sentence
Query	on every search	turn one query into a vector, one matrix multiply, one sort

The trick that keeps queries cheap: normalize every row to length 1 when you build the index. Then cosine similarity is just a dot product, and all of them at once is matrix @ query.

Build it step by step

Step 1: tokenize. Lowercase the text and keep runs of letters. Punctuation and case stop mattering: "Password?" and "password" are the same word.

Step 2: vocabulary. Collect every distinct word across all sentences and give each one a column number. Sorting them makes the columns stable between runs.

Step 3: count matrix. One row per sentence, one column per word, each cell is how many times that word appears. This is a bag of words: order is thrown away, only counts stay.

Step 4: IDF weights. Words like "how", "do", "i", "my" appear everywhere, so matching them says little. Inverse document frequency gives each word a weight that's small when many sentences contain it and large when few do: log((1 + n) / (1 + df)) + 1, where df is the number of sentences containing the word. Multiply each column by its weight. Counts × IDF is called TF-IDF.

Step 5: normalize. Divide each row by its length. Guard the empty row: a sentence with no known words would divide by zero and produce NaN.

Step 6: query. Push the query through the same steps 1, 3, 4, 5 (with the same vocabulary and weights), multiply, sort, return the top k.

The whole module:

"""Find similar sentences with nothing but NumPy: bag of words, TF-IDF, cosine, top-k."""
import re
import numpy as np

DOCS = [
    "How do I reset my password",
    "I forgot my password and cannot log in",
    "Change the email on my account",
    "The app crashes when I open the camera",
    "Camera does not work after the update",
    "How do I delete my account",
    "Refund for a cancelled order",
    "My order arrived late and the box was damaged",
    "Where is my order",
    "Cancel my order and give me a refund",
    "How do I change my phone number",
    "How do I track my parcel",
    "How do I contact support",
]

def tokenize(text):
    return re.findall(r"[a-z]+", text.lower())

def build_vocab(docs):
    words = sorted({w for d in docs for w in tokenize(d)})
    return {w: i for i, w in enumerate(words)}

def count_matrix(docs, vocab):
    m = np.zeros((len(docs), len(vocab)))
    for row, d in enumerate(docs):
        for w in tokenize(d):
            if w in vocab:  # words never seen while building the vocabulary are dropped
                m[row, vocab[w]] += 1
    return m

def idf_weights(counts):
    df = (counts > 0).sum(axis=0)              # how many docs contain each word
    return np.log((1 + len(counts)) / (1 + df)) + 1

def normalize(m):
    norms = np.linalg.norm(m, axis=1, keepdims=True)
    norms[norms == 0] = 1                      # an empty row stays all zeros, no NaN
    return m / norms

class Index:
    def __init__(self, docs, use_idf=True):
        self.docs = docs
        self.vocab = build_vocab(docs)
        counts = count_matrix(docs, self.vocab)
        self.idf = idf_weights(counts) if use_idf else np.ones(len(self.vocab))
        self.matrix = normalize(counts * self.idf)  # (n_docs, n_words), each row length 1

    def search(self, query, k=3):
        q = normalize(count_matrix([query], self.vocab) * self.idf)[0]
        scores = self.matrix @ q                    # one matrix multiply = every cosine at once
        top = np.argsort(-scores, kind="stable")[:k]
        return [(float(scores[i]), self.docs[i]) for i in top]

A script that runs each step and compares plain counts against TF-IDF:

from search import DOCS, Index, build_vocab, count_matrix, tokenize

print("step 1  tokens:", tokenize("How do I reset my Password?"))
vocab = build_vocab(DOCS)
print("step 2  vocabulary size:", len(vocab))
counts = count_matrix(DOCS, vocab)
print("step 3  count matrix shape:", counts.shape)

for use_idf in (False, True):
    index = Index(DOCS, use_idf=use_idf)
    print(f"\nstep 4  {'TF-IDF' if use_idf else 'plain counts'}")
    for query in ["how do i get a refund", "i can't log in", "automobile insurance"]:
        print(f"  query: {query!r}")
        for score, doc in index.search(query, k=2):
            print(f"    {score:.3f}  {doc}")

Output:

step 1  tokens: ['how', 'do', 'i', 'reset', 'my', 'password']
step 2  vocabulary size: 48
step 3  count matrix shape: (13, 48)

step 4  plain counts
  query: 'how do i get a refund'
    0.600  How do I contact support
    0.548  How do I reset my password
  query: "i can't log in"
    0.612  I forgot my password and cannot log in
    0.258  How do I contact support
  query: 'automobile insurance'
    0.000  How do I reset my password
    0.000  I forgot my password and cannot log in

step 4  TF-IDF
  query: 'how do i get a refund'
    0.468  Refund for a cancelled order
    0.388  Cancel my order and give me a refund
  query: "i can't log in"
    0.626  I forgot my password and cannot log in
    0.107  How do I reset my password
  query: 'automobile insurance'
    0.000  How do I reset my password
    0.000  I forgot my password and cannot log in

Reading it:

• 13 sentences produced 48 distinct words, so the index is a (13, 48) matrix. Every sentence is a vector in 48 dimensions, mostly zeros.
• Plain counts get "how do i get a refund" wrong. The top hits are "How do I contact support" and "How do I reset my password", because "how", "do", "i" match. The refund sentences lose.
• TF-IDF fixes it. Four FAQ lines start with "How do I", so those words get low weights, and "refund" (in only 2 sentences) gets a high one. Now both refund sentences are on top.
• "automobile insurance" scores 0.000 everywhere. None of its words are in the vocabulary, so its vector is all zeros. The results it "returns" are just the first rows. A bag of words knows nothing about meaning: "car" and "automobile" would also score 0 against each other.

Test it

Three properties every version of this search must keep:

from search import DOCS, Index

index = Index(DOCS)

def test_exact_sentence_ranks_first_with_score_one():
    for doc in DOCS:
        score, top = index.search(doc, k=1)[0]
        assert top == doc and abs(score - 1) < 1e-9

def test_unknown_words_give_zero_not_nan():
    for score, _ in index.search("zzz qqq", k=3):
        assert score == 0

def test_scores_are_sorted():
    scores = [s for s, _ in index.search("refund my order", k=5)]
    assert scores == sorted(scores, reverse=True)

if __name__ == "__main__":
    for name, fn in list(globals().items()):
        if name.startswith("test_"):
            fn()
            print("pass", name)

pass test_exact_sentence_ranks_first_with_score_one
pass test_unknown_words_give_zero_not_nan
pass test_scores_are_sorted

• A sentence finds itself with score 1. If this fails, normalization or the vocabulary is broken.
• Unknown words give 0, never NaN. This is the empty-row guard from step 5. Without it, one empty query poisons the sort.
• Scores come back sorted. Cheap to check, and it catches an inverted sort, a common slip with argsort.

Beyond unit tests, keep a short list of real queries with the answer you expect (a tiny golden set, Part 8) and check the top result after every change.

Trade-offs

Choice	Good	Costs
Bag of words	simple, fast, explainable: you can see which words matched	no meaning: synonyms score 0, word order is lost ("dog bites man" = "man bites dog")
TF-IDF over counts	common words stop drowning out the important ones	weights depend on the corpus; adding sentences changes every score
Dense matrix	one NumPy line does the scoring	13 × 48 is nothing, but 1 million sentences × 100,000 words is 400 GB of mostly zeros; real systems use sparse matrices or inverted indexes
Brute-force scoring	exact, no index to tune	cost grows with every sentence; at millions you need the approximate search of Part 5

This method is still alive in production as BM25, a refined TF-IDF used by Elasticsearch and others. Part 5 combines it with embeddings in hybrid search, because each one catches what the other misses.

Take it further

1. Add bigrams (pairs of neighbouring words like "log in") as extra columns, and see whether "i can't log in" ranks better.
2. Swap the dense matrix for scipy.sparse.csr_matrix and time both with 50,000 generated sentences.
3. Add a minimum score: return "no good match" below 0.2 instead of showing unrelated rows like the "automobile insurance" results.
4. Write down three queries where meaning matters ("my parcel is late", "money back") and keep them. When you reach embeddings in Part 5, run them again and compare.