ActiveRetrieval systems · 2026

Local retrieval engine

gloss

gloss turns a source text into a portable SQLite corpus, then searches it through independent lexical and local-vector channels. The design keeps the source verbatim, makes every hit inspectable, and treats retrieval quality as something to measure rather than vibe-check.

My role

Independent project · design and implementation

System map

Source text
Deterministic units
SQLite / FTS5
Hybrid retrieval
Cited passage
Retrieval systems
Artifact
One portable SQLite database
Retrieval
BM25 + local vectors + RRF
Evaluation
266 rank-sensitive cases
01

The constraint

Most AI search products optimize for a fluent answer. That is the wrong output when the job is to recover what a source actually said. gloss is built around a stricter contract: return the original passage, preserve its location, and expose why it ranked.

The first corpus is A Philosophy of Software Design, but the engine is deliberately corpus-agnostic. Book-specific taxonomy, evaluation cases, and enrichment prompts live outside the retrieval core.

02

Build once, retrieve locally

An offline build parses the PDF with font-aware structure, segments it deterministically, and asks an LLM only for retrieval metadata such as context lines, symptom-shaped questions, and key terms. The LLM never rewrites the source text.

The build is checkpointed and resumable. Enrichment, lexical indexes, vectors, and the verbatim units all land in the same SQLite artifact, so the finished corpus is easy to move and inspect.

03

Two channels, visible evidence

At query time, FTS5/BM25 and max-similarity vector search rank independently. Reciprocal-rank fusion combines the lists without pretending their raw scores are comparable, and every result reports which channels supported it.

Lexical mode has no runtime dependencies beyond Python's standard library. Hybrid mode uses a local Ollama embedding model and degrades cleanly back to lexical search when that service is unavailable.

04

Evaluation is part of the product

The repository carries a rank-sensitive evaluation set, hit-at-k and mean-reciprocal-rank scoring, and paired sign-flip tests for comparing retrieval changes. The expanded suite includes curated, developer-voice, and adversarially phrased slices so one flattering aggregate cannot hide a weak usage mode.

Engineering choices

The decisions that shaped it.

The implementation details matter because each one closes a specific failure mode or keeps an important boundary visible.

01

Keep source text immutable

LLMs enrich retrieval fields offline; they never author the passage returned to the user.

02

Fuse ranks, not scores

Lexical and semantic channels remain independently legible, then combine through reciprocal-rank fusion.

03

Ship the evaluation harness

Every ranking change can be compared on the same cases instead of being accepted from a demo query.

Where it landed

A working open-source prototype that builds a cited corpus end to end, runs locally, and has a measurable path for deciding whether a retrieval change actually helped.

Next case study

Celery Fork-Safety Investigation

Keep reading