LLM Wiki

The LLM Wiki is a pattern for building a personal (or team) knowledge base where a large language model — not the user — writes and maintains the knowledge. Instead of dropping files into a folder and searching them at query time, an LLM agent reads each source, extracts what matters, and incrementally builds and maintains a persistent, interlinked collection of Markdown pages that sits between the user and their raw sources. Source [llm-wiki.en.md]

The key idea is a shift in *when* the work happens:

• Query-time systems (e.g. RAG) retrieve relevant chunks and reconstruct an answer from scratch on every query.
• The LLM Wiki does the work at ingest time: the moment a source arrives, the model compiles it into the wiki — updating entity and concept pages, flagging contradictions, and strengthening the synthesis. Knowledge is compiled once and kept current.

The result is a knowledge base that compounds: cross-references are already there, contradictions have already been surfaced, and every new source or good question makes the wiki richer. It is the difference between a pile of notes and a living encyclopedia of your own material. [llm-wiki.en.md]

The pattern was popularized in 2026 by Andrej Karpathy — co-founder of OpenAI and former Director of AI at Tesla — who published a short "idea file" (a GitHub gist titled llm-wiki.md) meant to be copy-pasted into an LLM agent like Claude Code. The gist deliberately describes the *idea*, not a specific implementation, and invites users to instantiate their own version. [llm-wiki.en.md]

Karpathy framed it as spiritually close to Vannevar Bush's Memex (1945) — a personal, curated knowledge store with associative trails between documents, where the links are as valuable as the documents themselves. Bush's unsolved problem was *who does the maintenance*. The LLM solves exactly that: it never gets bored, never forgets to update a cross-reference, and can touch a dozen pages in a single pass. [llm-wiki.en.md]

His mental model:

"Obsidian is the IDE, the LLM is the programmer, and the wiki is the codebase."

You curate sources and ask questions; the LLM does the summarizing, cross-referencing, filing, and bookkeeping. [llm-wiki.en.md]

A working LLM Wiki has three layers:

[llm-wiki.en.md]

The core workflow is Ingest / Query / Lint: [llm-wiki.en.md]

• Ingest. A new source is dropped in; the agent reads it, writes a summary page, updates relevant entity and concept pages, refreshes the index, and appends a log entry. A single source might touch 10–15 pages at once.
• Query. A question is asked; the agent finds the relevant pages, reads them, and synthesizes an answer with citations. Good answers can be filed back into the wiki as new pages, so explorations compound just like ingested sources do.
• Lint. Periodically, the agent health-checks the wiki: finds contradictions, stale claims, orphan pages, missing cross-references, and gaps worth researching.

• index.md — A content-oriented catalog of every page with a one-line summary, organized by category. The agent reads the index first to route a query, then drills into relevant pages. This works at moderate scale (hundreds of pages) without any embeddings or vector database. [llm-wiki.en.md]
• log.md — An append-only chronological record of ingests, queries, and lint passes, giving a timeline of how the wiki evolved. [llm-wiki.en.md]

See the full comparison at LLM Wiki vs. RAG. The headline differences: [llm-wiki.en.md]

By 2026, around 85% of enterprise AI applications use RAG, yet analyses report 40–60% of RAG implementations never reach production, almost always due to knowledge-base quality rather than retrieval tuning. The LLM Wiki attacks that at the root with a single, curated, cross-referenced artifact whose quality is visible and editable. [llm-wiki.en.md]

The two are not mutually exclusive. For large corpora, a realistic architecture is a compiled wiki for hot, frequently-accessed context plus a RAG layer for broad retrieval over the long tail. [llm-wiki.en.md]

Three properties make the LLM Wiki more than a summary folder: [llm-wiki.en.md]

• Provenance. Every claim links back to the immutable raw source it came from, keeping the wiki verifiable rather than a hallucinated blob.
• Cross-links. Pages reference each other with wikilinks. Forward links are written by the LLM; backlinks and the graph are computed from them, so structure is always consistent. In Obsidian's graph view you can literally see which pages are hubs and which are orphans.
• Synthesis that accumulates. A wiki that has ingested 50 papers on a topic answers questions with far more depth and cross-reference than one that has ingested five. The thesis evolves; it isn't rebuilt from scratch each time.

The pattern is deliberately tool-agnostic, but a common stack has emerged: [llm-wiki.en.md]

• Obsidian — human-facing Markdown reader with wikilinks, backlinks, and graph view. The Web Clipper turns articles into Markdown sources.
• Claude Code (or Codex, OpenCode, etc.) — the agent that does compiling and maintenance, configured by the schema file.
• qmd — an on-device hybrid search engine for Markdown (BM25 + local vector embeddings + LLM re-ranking) created by Tobi Lütke (Shopify CEO), used as a memory/search backend for agents once the index file alone isn't enough at scale.
• A fast-growing set of open-source implementations — wiki "compiler" plugins, agent skill packs, and desktop apps — most citing Karpathy's gist directly.

At its heart, it is just a git repo of Markdown files, so version history, branching, and collaboration come for free. [llm-wiki.en.md]

The pattern has real edges: [llm-wiki.en.md]

• Scale. The index-first approach is comfortable up to roughly 1,000 files / hundreds of pages; beyond that, real search infrastructure and hierarchical indexing are needed.
• Single-player by default. Built for one person on one machine — no built-in multi-user access control, approval workflow, or concurrent editing.
• Setup and upkeep. The user assembles the agent, schema, search tooling, and backups — and owns the failure modes.
• Governance. With no confidence scoring or human-in-the-loop review, a wrong synthesis can quietly propagate. Quality is the ceiling on everything the wiki produces.

Those limits are exactly the gap a managed LLM Wiki closes. Cairni takes Karpathy's pattern and runs it as a hosted service — zero setup, with sources compiled into a living interlinked wiki where every claim is traceable. It adds scale beyond the ~1,000-file wall, team workspaces with per-notebook access control and approval queues, version history, contradiction surfacing, and public publishing. [llm-wiki.en.md]

Is the LLM Wiki the same as RAG? No. RAG retrieves raw chunks at query time and re-derives the answer each time. The LLM Wiki compiles sources into persistent, interlinked pages once and keeps them current, so knowledge accumulates instead of being rebuilt per query. [llm-wiki.en.md]

Do I need a vector database? Not at personal scale. An index.md catalog is enough for hundreds of pages. A local search engine like qmd becomes useful only as the wiki grows large. [llm-wiki.en.md]

Who maintains the wiki? The LLM does — writing summaries, updating entity and concept pages, fixing cross-references, and flagging contradictions. The user curates sources and asks questions. [llm-wiki.en.md]