llm-wiki-newsroom
alfadur7 · alfadur7/llm-wiki-newsroom
Biến tài liệu bạn bỏ vào thư mục raw/ thành wiki markdown có liên kết chéo, dùng nhóm agent mà người viết và người duyệt là hai agent khác nhau.
Harness engineering applied to knowledge production: a self-evolving multi-agent newsroom that turns your documents into a cross-linked markdown wiki. A "reground" loop pulls published pages back in before they go stale — writer ≠ reviewer, local-first, a structured alternative to RAG.
Xem trước
Bạn có thể làm gì với nó
- Xây dựng kho kiến thức cá nhân từ các bài viết và PDF đã lưu
- Sắp xếp ghi chú nghiên cứu thành wiki liên kết và đánh dấu các nhận định mâu thuẫn
- Thay thế một hệ thống RAG đơn giản bằng wiki markdown có cấu trúc chạy cục bộ
Kiểm tra an toàn
Không thấy dấu hiệu rủi ro trong các file đã quét.
Quét tự động các file trong repo. Luôn xem lại code trước khi chạy.
Trích từ README
LLM Wiki Newsroom
A multi-agent AI knowledge base run by a five-role "newsroom" — open-source, local-first, no vendor lock-in. Drop articles, documents, and PDFs into the raw/ folder, type a single command, and the newsroom — powered by an agent like Claude Code — reads them, extracts entities, concepts, and relationships, and organizes everything into a fully cross-referenced wiki, a structured and persistent alternative to RAG. Unlike most takes on the idea, the agent that writes a page is never the one that reviews it, and the authoring guidelines evolve themselves over time. Every new document you add also enriches the existing pages. This repo ships with a small example corpus — the debate over what "open source" means for AI — under wiki/, but the framework is domain-agnostic.
Most knowledge tools leave the finding to you. This project makes the AI read and understand your collected documents first, then organizes them into a wiki — with cross-references between pages, automatic flagging of conflicting claims, and per-topic synthesis built in from the start, so later retrieval is fast.
See the output before installing — the example corpus shipped in this repo is published as a browsable GitHub Wiki (no clone needed). It's a rendered static snapshot of the
wiki/folder; the interactive graph below runs locally.
The interactive knowledge graph (graph/graph.html) — every page a node, every wikilink an edge, color-coded by auto-detected cluster, with a live physics layout and filter/search built in. Shown here on a larger private deployment (~2,300 nodes) to convey how it scales; this repo ships a deliberately small example corpus you can browse the exact same way. (Interface shown in the optional Korean WIKI_LANG=ko mode.)
The concept
This project is one question worked into a running system: how far can you trust knowledge an AI wrote? Two ideas organize everything below.
1. The product is an LLM Wiki — Andrej Karpathy's three-layer pattern. The original documents you collect (Layer 1, raw/), the cross-linked wiki the agent maintains (Layer 2, wiki/), and the operating rules the agent follows (Layer 3, CLAUDE.md + .claude/) are kept strictly separate, so humans and AI don't trespass into each other's territory. As the operator you do exactly two things — feed Layer 1 and tune Layer 3; only the agent writes Layer 2. And ingesting one document doesn't just add a page: it refreshes the ~10–15 existing pages that mention the same entities and concepts, which is what makes the wiki compound instead of just piling up.
2. The factory is a newsroom running four loops. The wiki is produced by five roles modeled on a newspaper staff — and the agent that writes a page is never the one that reviews it:
| Role | What it does |
|---|---|
| Reporter | gathers material and drafts source pages + entity/concept stubs |
| Columnist | writes the deep cross-source analyses |
| Copy Editor | rule-based Python checks — not an LLM at all |
| Desk | re-reads finished drafts with fresh eyes; the only independent qualitative judgment in the system |
| Editor-in-Chief | routes work and gates publication — orchestration, not evaluation |
Four loops turn that division of labor into trust. The first three nest inside one another; only the fourth sits outside, feeding published pages back in:
%%{init: {"flowchart": {"wrappingWidth": 700}}}%%
flowchart BT
subgraph meta["Meta loop — mistakes become new rules"]
subgraph outer["Outer loop — two gates before publication"]
inner["Inner loop — the writer self-checks while drafting"]
end
end
%% the padding keeps this box as wide as the Meta loop box above
reground[" Reground loop — pages that go stale or inconsistent come back as input "]
reground -.-> meta
style meta fill:transparent,stroke:#58a6ff,stroke-width:2px
style outer fill:transparent,stroke:#3fb950,stroke-width:2px
style inner fill:transparent,stroke:#d29922,stroke-width:2px
style reground fill:transparent,stroke:#bc8cff,stroke-width:2px,stroke-dasharray:4 4
| Loop | When it runs | What it does |
|---|---|---|
| Inner | while drafting | the writer self-checks against the same yardstick the review gates will use later, and hands off instead of grinding |
| Outer | at publication | two gates — deterministic lint (Copy Editor), then a six-lens qualitative review (Desk) — and both must pass |
| Meta | when mistakes recur | repeat failures become proposals to amend the authoring rules themselves — blind measurement where it applies, operator sign-off always |
| Reground | after publication | published pages that have gone stale or inconsistent come back around as factory input |
One piece of the machinery is deliberately not a loop. A loop is feedback — something exists, gets checked, gets corrected. Before any loop runs, a GROUND Ladder governs the input side: how much the writer reads before drafting, widening from the page's declared dependencies toward the whole corpus only on a named signal that the evidence so far is insufficient. The loops correct what came out; the ladder disciplines what goes in — every authoring cycle enters through it. (Details under Key Features.)
The first three loops mirror the "software factory" playbook for AI-assisted coding; the fourth exists because knowledge, unlike code, keeps decaying after you ship it. The full argument for this design is in the companion article: The Knowledge Factory.
Everything else in this README — the commands, the tools, the feature list — hangs off this map.
What makes this different
Th