Total Recall: How QMD Works

Every session starts with amnesia

A new session does not start with your notes in it. Whatever the agent needs from them, it has to find, every session, from zero.

On my machine the notes are about four thousand markdown files across several projects: decisions, runbooks, postmortems, research, meeting notes. Pasting all of that into every request is not practical, and it would bury the one paragraph that matters anyway: "Lost in the Middle" (Liu et al.) measured accuracy drops of more than 20% when the relevant passage sat mid-context.

So agents search, and their default search tool is grep. Grep matches characters, not meaning. Ask "how do we undo a bad release?" and the runbook that answers it says "## Rollback" and "flip the load balancer back to blue". Not one word in common. Grep for that phrase returns nothing; grep needs the right word before it can find anything.

The worse failure is quieter: the agent does not search at all. It answers from general knowledge, fluently and confidently, while the sourced answer sits in a file it never opened. My agents' rules file now says it in one line: "Answering 'from principle' when the file already has a sourced answer is a failure, and it produces confidently wrong output."

Code is different, and that is worth saying. Anthropic describes Claude Code's own approach as runtime exploration with tools like glob and grep, "effectively bypassing the issues of stale indexing". In a code repository, identifiers are exact and the files change every minute, so grep is the right tool. A notes corpus is prose: synonyms, paraphrases, years of decisions. That is where an index earns its keep. QMD finds the file; the harness around it makes checking the file part of the workflow.

What QMD is

QMD (Query Markup Documents) describes itself as "an on-device search engine for everything you need to remember". Tobi Lütke, Shopify's CEO, started it in December 2025; the first changelog entry says it was built in a single day for his own notes, journals and meeting transcripts. It is MIT-licensed, installs from npm as @tobilu/qmd, and had about 30,000 GitHub stars when I checked.

All three models run locally through node-llama-cpp (on Metal, on a Mac). They download once from Hugging Face on first use; after that, in the research sandbox for this article, I blocked outbound network at the operating-system level and ran the whole pipeline anyway: index, embed, search, query. No cloud, no API key, no bill per query.

How it remembers

You point QMD at a folder and it becomes a collection. qmd update (or the first collection add) reads every markdown file, hashes it, and writes it into SQLite's FTS5 keyword index. The first six hex characters of the file's SHA-256 become its docid, the short #c181d9-style handle you see in results.

Keyword ranking is BM25, the classic formula: rare words count more, and each repeat of a word counts a little less than the last. It is fast, exact and blind to synonyms.

Then you can add context: a sentence that describes a collection or a sub-folder, returned with every result from it. QMD's README calls context "the key feature" and adds "Don't sleep on it!". It is cheap and it helps agents decide which hit matters.

qmd embed builds the meaning side. Long documents are cut into chunks of about 900 tokens with 15% overlap, and the cut lands on the best markdown break point within the last 200 tokens: a # heading scores 100, ## 90, a code fence 80, a blank line 20, a plain line break 1, discounted by distance. Ideas stay together, and code blocks stay whole when possible. A short note (under about 2,700 characters, where the first cut is made) stays a single chunk. Each chunk is embedded with its document title attached, and the vectors land in the same SQLite file as the words.

One real query, stage by stage

The demo for the film is a small, fictional set of seven engineering notes in an isolated index. The question is "how do we undo a bad release?". The answer is a blue/green rollout runbook whose ## Rollback section says to flip the load balancer back to blue. No note contains "undo", "bad" or "release".

qmd search (keywords only) answered in 0.21 seconds: "No results found." Here is what qmd query did instead.

1. Shortcut check. If plain BM25 already has a clear winner (a score of at least 0.85, at least 0.15 ahead of second place), QMD skips the expensive steps. Here keyword search had nothing, so it continued.

2. Expansion. The 1.7B model wrote typed lines: two lex keyword lines ("strategies for reversing", "approaches to rectifying"), two vec meaning lines, and one hyde line, a hypothetical answer. HyDE comes from a 2023 paper (Gao et al.): embed what the answer might look like, and let the encoder's dense bottleneck filter out the invented details. The small model was not perfect: it read "release" as a policy release, and its hypothetical answer ended with a stray </think> tag. The pipeline still worked, which is the point of fusing several signals.

3. Routing. "Typed expansions are routed exclusively: lex → BM25/FTS, vec and hyde → vector search. The original query is sent to both backends." The keyword lists came back empty. The four vector lists ranked the runbook first, first, fourth and first.

4. Fusion. Reciprocal Rank Fusion (Cormack, Clarke and Büttcher, SIGIR 2009) scores each document by summing weight / (60 + rank) over every list it appears in. Lists from the original question count double, and a document that ranks first anywhere gets a +0.05 bonus (+0.02 for second or third). The runbook: 2/61 + 1/61 + 1/64 + 1/61 = 0.0812, plus the bonus, 0.1312. The runner-up, a postmortem, scored 0.1309. A photo finish.

5. Reranking. Up to 40 candidates go to Qwen3-Reranker. For each document QMD picks the chunk with the most overlap with the question and asks a yes-or-no relevance judgement, scored as a probability. For this paraphrase both finalists received very low relevance scores: 0.003 and 0.002. Asked "roll back the checkout deployment" instead, it scored the runbook 0.994. Rerankers are sensitive to phrasing.

6. Blend. The final score is w × (1 / fusion rank) + (1 − w) × reranker score, with w = 0.75 for ranks 1–3, 0.60 for 4–10 and 0.40 below. The README's reason: "Position-aware blending prevents the reranker from destroying high-confidence retrieval results." A side effect: the fusion winner always scores at least 0.75. The runbook's "75%" is rank position, not confidence. Read it that way, and open the source before trusting the answer.

Result: the rollback runbook first, in 2.14 seconds with the expansion already cached (a fresh question took 3.16 seconds), with #c181d9, the path, the line of the best match and the score. qmd get "#c181d9:16:5" then returns exactly lines 16 to 20.

Does hybrid matter beyond one demo? QMD ships a tiny benchmark: 10 questions over 6 documents. On this machine the right file ranked first for 50% of questions with keywords only, 80% with vectors only, and 100% with hybrid, with or without reranking. It is a toy, and the shape is the point. The demo shows semantic retrieval working; it does not prove the reranker was needed. Anthropic's larger contextual-retrieval study (their numbers, their setup) points the same way: contextual embeddings plus contextual BM25, then reranking, cut the top-20 retrieval failure rate from 5.7% to 1.9%, and their summary reads "Embeddings+BM25 is better than embeddings on their own".

How agents use it

Agents can run qmd from the shell, and QMD's README says that works fine. MCP makes it a native tool: QMD speaks the Model Context Protocol and exposes four read-only tools:

• query: up to 10 typed sub-queries (lex, vec, hyde), an optional intent and collection filter; the first sub-query gets double weight.
• get: one document by path or docid, with :from:count line ranges.
• multi_get: several documents by glob, list or docids.
• status: index health and collections.

When an agent connects, the server sends its own instructions: how many documents it holds and how to write lex, vec and hyde sub-queries. QMD's bundled agent skill goes further and tells agents they are "a better query expander than the built-in model". In practice a capable agent writes its own sub-queries, and QMD then skips its own expansion step.

qmd mcp runs over stdio, one process per client, each loading its own models. For several agents, use a shared server instead of the stdio registration in step 6: start one long-lived server (qmd mcp --http --daemon, localhost port 8181 by default) and register it by URL: claude mcp add --scope user --transport http qmd http://localhost:8181/mcp and codex mcp add qmd --url http://localhost:8181/mcp. On my machine a launchd job keeps that server alive, so every session of both agents shares one index and one set of models. After a few idle minutes the first query took 11 seconds here while the models reloaded; warm repeats took 0.6 to 1.5 seconds.

The harness around it

Installing QMD did not change my agents' behaviour. They still reached for grep. A written preference did not hold, so the harness enforces it in three layers.

Rule one: search before you answer. If a question touches anything the notes document (the owner's own world rather than general knowledge), the agent queries QMD first, answers from what it reads, keeps the notes' own labels (an analysis stays an analysis), and cites the file. This rule exists because an agent once answered a question from general principle while a sourced analysis of exactly that question sat in the notes. Retrieval shows what a note says; it does not prove the note is right.

Rule two: a hook that blocks the shortcut. A PreToolUse hook runs before every shell and grep tool call. When a grep, ripgrep or git grep could sweep the knowledge folders (a search aimed at them, a recursive search from the repository root, or any git grep), it denies the call and answers: "Knowledge-base search blocked — use mcp_qmd_query (lex/vec/hyde) instead." Reading one known file is allowed, as are searches confined to code folders that do not use git grep. The hook fails open: if the guard crashes, the call goes through, because a broken guard must never brick an agent's tools. Codex runs the same guard script through a small adapter that only executes a guard whose hash was explicitly registered. Every block is logged: 3,518 since 29 May, retries counted separately. The log cannot tell a justified block from a false positive, and there is a known one: a command that searches code but also mentions a notes path gets caught, so the agents split such commands. It is a guardrail, not a security boundary. While this article was being researched, it blocked one of the agents doing the research.

Rule three: freshness is a queue. An index is only as good as its last update. Every git commit runs a post-commit hook that files a durable request and returns immediately. One shared updater, behind one global lock, drains the requests from all six enrolled repositories: qmd update, then qmd embed. If a run fails, it writes a failure record and raises a health flag, a file the agents are instructed to check at session start, and only a successful run clears it. Since 5 September the updater has run 1,583 times and embedded 227,102 chunks. Two runs failed on 15 October; the next one recovered.

That design came from a silent failure. In May a Node.js upgrade broke the native SQLite module. The background reindex had been failing silently for weeks, its output going to a temp log nobody read, and the search server survived only on the old module still loaded in memory: one reboot away from a total outage. Now a doctor script checks the parts that break quietly: does the native module load, does the server answer a real MCP handshake, is the server running the current binary, and is the queue free of pending or failed work. It repairs what it can and otherwise fails loudly. The working rule: silence is not freshness.

Overview first. Each major knowledge area has a short overview file, around 200 lines, that agents read before anything else. It says what exists and which rules apply. Depth goes through QMD instead of through reading multi-thousand-line files whole.

I also piloted a hand-curated relation graph on top. It did not beat plain QMD discovery (the needed sources were reached on 29 of 31 test questions without the graph, 28 of 31 with it), so QMD stayed the baseline.

The receipts

Two caveats. These are single-machine measurements, and my install carries a reviewed patch for a memory problem in 2.8.3's vector lookup, so they are not a clean benchmark of the stock package. The server log also holds 683 cancellation notices of unknown cause, so the median describes completed queries only. And the latency deserves a straight reading. Of the completed agent queries, a third finished in under a second; 38% took 15 to 60 seconds. When an agent knows the exact term, a lex-only query with reranking turned off came back from the running server in under 10 milliseconds (on a small collection).

Limits worth knowing

• Freshness is your job. Grep is always current; an index is only as current as its last update. Queue reindexing on change and make failure loud.
• Small models have quirks. The expansion model misread a question in the demo; the fusion of several signals absorbed it. Let capable agents write their own lex/vec/hyde sub-queries.
• A score is not a confidence. The reranker can assign very low scores while the blend still prints 75% for rank one. Read the passage before trusting the answer.
• Indexes rot. Three quarters of the vectors in my index belong to old or deleted versions of files. qmd cleanup exists; run it after a verified backup.
• Collections are not access control. A collection filter scopes a query. Material with different permissions needs an enforced access boundary; a separate index or process alone is not enough.
• Small corpora may not need it. Anthropic's guidance: below about 200,000 tokens (roughly 500 pages), you can put the whole knowledge base in the prompt.
• Measure on your own notes. qmd bench runs a fixture of your questions against keyword, vector, hybrid and full pipelines.

Local search makes the evidence easy to reach. Whether the agent reads it correctly and stays within it is still something to test.

Steal this

• Start with one collection and questions whose answers you already know.
• Give every collection a context line; give the agent one search tool over MCP.
• Make searching before answering a written rule, and require a cited file.
• If agents keep ignoring the search rule, add a hook that blocks the grep shortcut, fails open and logs every block.
• Reindex on every commit, through one worker and one lock.
• Make failure loud: a flag, a doctor, no silent temp logs.
• Measure latency and recall on your own corpus before you trust either.