Keep source text immutable
LLMs enrich retrieval fields offline; they never author the passage returned to the user.
Local retrieval engine
gloss turns a source text into a portable SQLite corpus, then searches it through independent lexical and local-vector channels. The design keeps the source verbatim, makes every hit inspectable, and treats retrieval quality as something to measure rather than vibe-check.
System map
Most AI search products optimize for a fluent answer. That is the wrong output when the job is to recover what a source actually said. gloss is built around a stricter contract: return the original passage, preserve its location, and expose why it ranked.
The first corpus is A Philosophy of Software Design, but the engine is deliberately corpus-agnostic. Book-specific taxonomy, evaluation cases, and enrichment prompts live outside the retrieval core.
An offline build parses the PDF with font-aware structure, segments it deterministically, and asks an LLM only for retrieval metadata such as context lines, symptom-shaped questions, and key terms. The LLM never rewrites the source text.
The build is checkpointed and resumable. Enrichment, lexical indexes, vectors, and the verbatim units all land in the same SQLite artifact, so the finished corpus is easy to move and inspect.
At query time, FTS5/BM25 and max-similarity vector search rank independently. Reciprocal-rank fusion combines the lists without pretending their raw scores are comparable, and every result reports which channels supported it.
Lexical mode has no runtime dependencies beyond Python's standard library. Hybrid mode uses a local Ollama embedding model and degrades cleanly back to lexical search when that service is unavailable.
The repository carries a rank-sensitive evaluation set, hit-at-k and mean-reciprocal-rank scoring, and paired sign-flip tests for comparing retrieval changes. The expanded suite includes curated, developer-voice, and adversarially phrased slices so one flattering aggregate cannot hide a weak usage mode.
Engineering choices
The implementation details matter because each one closes a specific failure mode or keeps an important boundary visible.
LLMs enrich retrieval fields offline; they never author the passage returned to the user.
Lexical and semantic channels remain independently legible, then combine through reciprocal-rank fusion.
Every ranking change can be compared on the same cases instead of being accepted from a demo query.
Where it landed
A working open-source prototype that builds a cited corpus end to end, runs locally, and has a measurable path for deciding whether a retrieval change actually helped.
Next case study
Celery Fork-Safety Investigation