← Docs hub

Architecture

llmwiki has two overlapping structures:

  1. The Karpathy three-layer wiki (conceptual): raw/ → wiki/ → site/
  2. The eight-layer build (implementation): how responsibilities are distributed across Python modules, HTML templates, scripts, CI, etc.

This document covers both. For why the product is shaped this way (MCP for agents, site for humans, scriptable add, human candidate gate), see Product principles.

Layer 1: Karpathy's three-layer wiki

From the original LLM Wiki gist:

raw/           IMMUTABLE source documents
    ↓          (llmwiki converts .jsonl → .md here)
wiki/sources/  synthesized session summaries (LLM)
    ↓          (llmwiki synth — default also harvests candidates)
wiki/candidates/  PENDING review stubs for entities/concepts
    ↓          (agent /wiki-candidates → promote|merge|discard; site UI in #97)
wiki/entities/ concepts/   TRUSTED hubs (after review — never auto-promoted)
    ↓
site/          GENERATED static HTML
               (llmwiki builds here via `llmwiki build` — not part of `synth`)

Trusted entity/concept hubs require human-or-agent review of candidates. Synthesis alone can leave Home looking “finished” (Raw → Synthesized) while the knowledge layer is still empty — Home’s Knowledge layer table (Candidates | Entities | Concepts) and Analytics Candidates to review make that backlog visible (#84).

raw/ — immutable layer

Everything under raw/ is treated as source-of-truth. The converter writes to it; nothing else should. If a source is wrong, fix the converter, not the output.

The converter writes one markdown file per session under raw/sessions/<project>/<date>-<slug>.md. Each file has YAML frontmatter (project, started, model, tools_used, gitBranch, etc.) and a Conversation body rendered turn-by-turn.

wiki/ — LLM-maintained layer

Your coding agent owns this layer entirely. It writes via the Ingest Workflow in CLAUDE.md:

wiki/
├── index.md          catalog of all pages, updated on every ingest
├── log.md            append-only chronological record
├── overview.md       living synthesis across all sources
├── sources/          one summary page per raw source (kebab-case slug)
├── candidates/       pending entity/concept stubs (harvest; review before promote)
├── entities/         people, products, tools (TitleCase.md)
├── concepts/         ideas, frameworks, patterns (TitleCase.md)
├── projects/         codebases and work streams (kebab-case slug)
└── syntheses/        saved query answers (kebab-case slug)

Pages interlink via [[wikilinks]]. Contradictions are recorded, not silently overwritten. Pages compound over time — every new source makes the wiki richer.

site/ — generated static layer

llmwiki build reads raw/ (and wiki/ if populated) and renders a complete static HTML site. Nothing here is hand-maintained. Safe to delete and regenerate any time.

Layer 2: The eight-layer build

Internally the code is organised into eight functional layers. Each layer has one clear responsibility.

┌──────────────────────────────────────────────────────┐
│  L7  CI / ops          .github/workflows/            │
│  L6  Adapters          llmwiki/adapters/             │
│  L5  Schema / docs     CLAUDE.md, AGENTS.md, docs/   │
│  L4  Distribution      setup.sh, .bat, .claude/      │
│  L3  Viewer            script.js in build.py         │
│  L2  Site              build.py (HTML + CSS)         │
│  L1  Wiki              CLAUDE.md workflows           │
│  L0  Raw               llmwiki/convert.py            │
└──────────────────────────────────────────────────────┘

L0 — Raw

Owner: llmwiki/convert.py

Reads .jsonl from the agent's session store (via an adapter), filters out noise records, runs redaction, normalises the output into markdown, and writes to raw/sessions/.

Key properties:

Vault state (v1.4+): one active llmwiki-state.json per process, configured at the CLI border (apply_default_vault → configure_state_file). Library modules call resolve_state_file() — they never re-read config.json for the state path. Import-time constants like DEFAULT_STATE_FILE were removed so tests and library callers cannot accidentally write into a developer's configured vault.

L1 — Wiki

Owner: your coding agent, following CLAUDE.md / AGENTS.md

llmwiki does NOT write to wiki/ directly. The agent does, via slash commands (/wiki-ingest, /wiki-query, /wiki-lint) that execute the workflows in the schema file.

L2 — Site (HTML generator)

Owner: llmwiki/build.py

Converts every file under raw/sessions/ (and any hand-authored files under wiki/) into static HTML. Uses python-markdown (the only runtime dep) — syntax highlighting runs in the browser via highlight.js loaded from a pinned jsdelivr CDN (v0.5, #73), so the build pipeline itself stays stdlib-only. Writes to site/.

Pages rendered (v0.9 surface):

Documents enter raw/docs/ either via the asynchronous producer queue path or synchronously via llmwiki add (llmwiki/add_doc.py, #16 / #311) — each import writes one complete Markdown file per document. Synthesis chunks long bodies in memory to the active backend budget and stitches one wiki source page; length-driven -NN series on disk are a legacy layout. The site build (#305) still groups any remaining multi-piece raw files into logical documents for reading, search, and the sidebar without rewriting raw/ or wiki/; operators merge legacy pieces offline with llmwiki migrate whole-document-storage (no mandatory mass re-synth).

L3 — Viewer (browser JS)

Owner: script.js (a string constant inside build.py)

Everything that happens in the browser, in vanilla JS:

Zero dependencies. No bundler. No framework. One file.

L4 — Distribution

Owner: the installable llmwiki package

How users install and run llmwiki:

L5 — Schema / docs

Owner: root-level markdown + docs/

Tells humans and agents how the system works:

L6 — Adapters

Owner: llmwiki/adapters/

One file per agent. Each subclass of BaseAdapter does three things:

  1. Knows where the agent writes its session store
  2. Walks that store to discover .jsonl files
  3. Derives a friendly project slug from the path

Everything else (record parsing, filtering, redaction, rendering) is shared in convert.py.

L7 — CI / ops

Owner: .github/workflows/ + tests/

Adding an adapter

See framework.md §5.25 Adapter Flow for the full contract. TL;DR: one new file at llmwiki/adapters/<agent>.py, one fixture, one snapshot test, one doc page, one README line, one CHANGELOG entry.

Design principles

  1. Stdlib first. Runtime dep: markdown only. Nothing else. Syntax highlighting runs client-side via a CDN-loaded highlight.js (v0.5, #73) so the build stays deterministic and offline-capable.
  2. Privacy by default. Redact everything sensitive before it hits disk.
  3. Idempotent everything. Re-running any command is safe and cheap.
  4. Localhost only. No network, no telemetry, no cloud. The user controls if/when to publish.
  5. One file per concern. build.py is one file, not a folder of templates. The whole HTML rendering lives there including CSS + JS.
  6. Agent-agnostic core. convert.py doesn't know which agent produced the .jsonl. Adapters translate.

Keyboard shortcuts

⌘K / Ctrl+KOpen command palette
/Focus search
g hGo to home
g pGo to projects
g sGo to sessions
j / kNext / prev row (tables)
?Show this help
EscClose dialogs

Structured queries

Mix key:value filters with free text in the palette:

type:sessionOnly session pages
project:llm-wikiFilter by project name (substring)
model:claudeFilter by model name (substring)
date:>2026-03-01Sessions after a date
date:<2026-04-01Sessions before a date
tags:rustPages mentioning a tag/topic
sort:dateSort results by date (newest first)

Example: type:session project:llm-wiki date:>2026-04 sort:date