L2 · 05 · Embeddings & Semantic Search

Keyword search finds words. Semantic search finds meaning: "something to charge my phone in the car" matches "USB-C car adapter" even though they share no words. This page explains embeddings properly and builds a product search for the Store Assistant in Scala and Python.

Embeddings, properly

An embedding model turns a piece of text (a word, a sentence, a product description, a code function) into a fixed-length vector, for example 768 or 1,536 numbers. It is trained so that texts with similar meaning get vectors that point in similar directions. Embedding models are usually much smaller and cheaper than chat models.

Cosine similarity

The standard measure is cosine similarity, the cosine of the angle between two vectors:

cos(a, b) = (a · b) / (|a| × |b|)       range -1 .. 1   (1 = same direction)

If vectors are normalised to length 1 (many embedding APIs already do this), cosine similarity is just the dot product, which is very fast to compute.

Semantic vs keyword search

Production systems usually combine both (hybrid search, below).

The embeddings API

{ "model": "nomic-embed-text", "input": ["USB-C car adapter, 30W", "leather watch strap"] }

{ "data": [
    { "index": 0, "embedding": [0.0123, -0.0456, 0.0789, "..."] },
    { "index": 1, "embedding": [-0.0311, 0.0042, 0.0510, "..."] }
  ],
  "usage": { "prompt_tokens": 12 } }

• Batch inputs: send many texts per call. It is much faster and cheaper than one call per text.
• Use the same model for documents and queries. Vectors from different models are not comparable.
• Changing the model means re-embedding everything. Store the model name and version next to each vector.
• Local option: Ollama serves embedding models (for example ollama pull nomic-embed-text) on the same endpoint.

Scala 3: product search

To run offline, this example includes a tiny toy embedder (hashed character trigrams). It only captures spelling similarity, not meaning, but the search code is identical to the real thing. Pass --live to use a real embedding model and see true semantic matches.

//> using scala 3
//> using dep io.circe::circe-core:0.14.15
//> using dep io.circe::circe-parser:0.14.15
import io.circe.Json
import io.circe.parser.parse
import io.circe.syntax.*
import java.net.URI
import java.net.http.{HttpClient, HttpRequest, HttpResponse}

final case class Product(sku: String, text: String)

val catalogue = Vector(
  Product("CAR-ADAPTER-30W", "USB-C car adapter, 30W fast charging for phones and tablets"),
  Product("CABLE-USBC-2M",   "Braided USB-C to USB-C charging cable, 2 metres"),
  Product("POWERBANK-10K",   "Portable battery pack 10,000 mAh for travel"),
  Product("EARBUDS-SPORT",   "Wireless sport earbuds, sweat resistant, secure fit for running"),
  Product("HEADPHONES-ANC",  "Over-ear headphones with active noise cancelling for flights"),
  Product("STRAP-LEATHER",   "Leather watch strap, brown, 41 mm"),
  Product("CASE-RUGGED",     "Rugged phone case with drop protection"),
  Product("STAND-DESK",      "Aluminium desk stand for tablets and laptops")
)

type Vec = Array[Double]
type Embedder = Vector[String] => Either[String, Vector[Vec]]

def normalise(v: Vec): Vec =
  val n = math.sqrt(v.map(x => x * x).sum)
  if n == 0 then v else v.map(_ / n)

def cosine(a: Vec, b: Vec): Double = a.indices.map(i => a(i) * b(i)).sum // vectors are normalised

/** Offline stand-in: hashed character trigrams. Spelling similarity only. */
val toyEmbedder: Embedder = texts => Right(texts.map { t =>
  val v = new Array[Double](512)
  s"  ${t.toLowerCase}  ".sliding(3).foreach(g => v(math.floorMod(g.hashCode, 512)) += 1)
  normalise(v)
})

/** Real embeddings from any OpenAI-compatible endpoint (local Ollama by default). */
def liveEmbedder(base: String, model: String, key: Option[String]): Embedder = texts =>
  val body = Json.obj("model" -> model.asJson, "input" -> texts.asJson).noSpaces
  val b = HttpRequest.newBuilder(URI.create(s"$base/embeddings"))
    .header("Content-Type", "application/json").POST(HttpRequest.BodyPublishers.ofString(body))
  key.foreach(k => b.header("Authorization", s"Bearer $k"))
  val resp = HttpClient.newHttpClient().send(b.build(), HttpResponse.BodyHandlers.ofString())
  if resp.statusCode() != 200 then Left(s"HTTP ${resp.statusCode()}: ${resp.body().take(200)}")
  else parse(resp.body()).flatMap(_.hcursor.downField("data").as[Vector[Json]]).left.map(_.getMessage).flatMap { items =>
    val vecs = items.flatMap(_.hcursor.get[Vector[Double]]("embedding").toOption).map(v => normalise(v.toArray))
    Either.cond(vecs.size == texts.size, vecs, "unexpected embeddings response")
  }

final class VectorIndex(products: Vector[Product], vectors: Vector[Vec], embed: Embedder):
  def search(query: String, k: Int = 3): Either[String, Vector[(Product, Double)]] =
    embed(Vector(query)).map { q =>
      products.zip(vectors).map((p, v) => p -> cosine(q.head, v)).sortBy(-_._2).take(k)
    }

object VectorIndex:
  def build(products: Vector[Product], embed: Embedder): Either[String, VectorIndex] =
    embed(products.map(_.text)).map(vs => VectorIndex(products, vs, embed))

@main def productSearch(args: String*): Unit =
  val embed =
    if args.contains("--live") then liveEmbedder(sys.env.getOrElse("LLM_BASE_URL", "http://localhost:11434/v1"),
      sys.env.getOrElse("EMBED_MODEL", "nomic-embed-text"), sys.env.get("LLM_API_KEY"))
    else toyEmbedder
  val queries = List("charger for the car", "earphones for jogging", "protect my phone if I drop it")
  VectorIndex.build(catalogue, embed) match
    case Left(err) => println(s"index failed: $err")
    case Right(index) => queries.foreach { q =>
      println(s"\n> $q")
      index.search(q).fold(e => println(s"  error: $e"),
        _.foreach((p, s) => println(f"  $s%.3f  ${p.sku}%-16s ${p.text}")))
    }

Python: the same, plus hybrid search

import math
import os
import re
from collections import Counter

CATALOGUE = {
    "CAR-ADAPTER-30W": "USB-C car adapter, 30W fast charging for phones and tablets",
    "CABLE-USBC-2M": "Braided USB-C to USB-C charging cable, 2 metres",
    "EARBUDS-SPORT": "Wireless sport earbuds, sweat resistant, secure fit for running",
    "HEADPHONES-ANC": "Over-ear headphones with active noise cancelling for flights",
    "CASE-RUGGED": "Rugged phone case with drop protection",
    "STRAP-LEATHER": "Leather watch strap, brown, 41 mm",
}


def normalise(v: list[float]) -> list[float]:
    n = math.sqrt(sum(x * x for x in v)) or 1.0
    return [x / n for x in v]


def toy_embed(texts: list[str], dims: int = 512) -> list[list[float]]:
    out = []
    for t in texts:
        v = [0.0] * dims
        s = f"  {t.lower()}  "
        for i in range(len(s) - 2):
            v[hash(s[i:i + 3]) % dims] += 1
        out.append(normalise(v))
    return out


def live_embed(texts: list[str]) -> list[list[float]]:
    import httpx
    base = os.environ.get("LLM_BASE_URL", "http://localhost:11434/v1")
    r = httpx.post(f"{base}/embeddings", timeout=60,
                   json={"model": os.environ.get("EMBED_MODEL", "nomic-embed-text"), "input": texts})
    r.raise_for_status()
    return [normalise(d["embedding"]) for d in sorted(r.json()["data"], key=lambda d: d["index"])]


embed = live_embed if os.environ.get("LIVE") else toy_embed
SKUS = list(CATALOGUE)
VECTORS = embed([CATALOGUE[s] for s in SKUS])


def semantic(query: str) -> list[str]:
    q = embed([query])[0]
    scores = {s: sum(a * b for a, b in zip(q, v)) for s, v in zip(SKUS, VECTORS)}
    return sorted(scores, key=scores.get, reverse=True)


def keyword(query: str) -> list[str]:
    words = set(re.findall(r"\w+", query.lower()))
    scores = Counter({s: len(words & set(re.findall(r"\w+", t.lower()))) for s, t in CATALOGUE.items()})
    return [s for s, c in scores.most_common() if c > 0]


def rrf(*rankings: list[str], k: int = 60) -> list[str]:
    """Reciprocal Rank Fusion: combine rankings without comparing their raw scores."""
    fused: Counter = Counter()
    for ranking in rankings:
        for rank, sku in enumerate(ranking, start=1):
            fused[sku] += 1 / (k + rank)
    return [s for s, _ in fused.most_common()]


for q in ["usb-c cable", "earphones for jogging", "CASE-RUGGED"]:
    print(f"\n> {q}")
    print("  semantic:", semantic(q)[:3])
    print("  keyword: ", keyword(q)[:3])
    print("  hybrid:  ", rrf(semantic(q), keyword(q))[:3])

Hybrid search and reciprocal rank fusion

Keyword and vector scores are on different scales, so you can't simply add them. Reciprocal Rank Fusion (RRF) uses only positions: each result gets 1 / (k + rank) from each list (k is about 60), summed. Items ranked well by both methods rise to the top. It is simple, robust, and used by many search engines and vector databases.

Where to store vectors

Practical guidance

• Embed the right text. For products: name + category + key attributes. For docs: chunks with their heading (page 06).
• Use metadata filters. "In stock at Riverside" is a filter, not a similarity. Filter, then rank.
• Evaluate retrieval. Make 30 queries with the expected top result and measure hit@3. Change one thing at a time.
• Watch for language and domain gaps. Pick a multilingual or domain-tuned embedding model if needed.
• Privacy. Embeddings can leak information about the source text. Protect them like the data itself.