schliff

The Ruff for AGENTS.md — deterministic quality scores for the instruction files that drive your AI. Same input, same score, on every machine.

Your AI instruction files silently degrade — and nothing catches it. AGENTS.md is read by Cursor, Codex, Copilot, and Claude Code — one rotting file now quietly degrades four tools. A trigger phrase rots, an edge case slips, the file balloons past its token budget. No error, no red test — just agents that quietly get worse.

Schliff scores AGENTS.md — and the rest of the family (SKILL.md, CLAUDE.md, .cursorrules, system prompts) — against an explicit, versioned rubric. No LLM judge in the critical path. No network. No randomness. A rule engine you can read, pin, and gate CI on.

Inside Claude Code — wires up the /schliff:* commands (scoring, /schliff:auto, /schliff:doctor, ...):

/plugin marketplace add Zandereins/schliff
/plugin install schliff@schliff

From a terminal — for CI, pre-commit, or scripting outside Claude Code:

# no venv, no commitment — needs uv (https://docs.astral.sh/uv/):
uvx schliff score AGENTS.md   # or any SKILL.md / CLAUDE.md / .cursorrules
uvx schliff demo              # no instruction file handy? score a built-in bad one

# or install it:
pip install schliff
schliff score AGENTS.md

This is the real, current output of schliff score AGENTS.md on this repo's own AGENTS.md — clone and run it yourself:

schliff v8.12.0

  structure             ██████████  100/100  perfect
  operational_coverage  ██████████  100/100  perfect
  efficiency            ████████░░   78/100  good
  composability         ██████████   95/100  excellent
  clarity               ██████████  100/100  perfect

  Structural Score  ███████████████████░  95.6/100  [S]

  Tokens: 1,065 / 3,000 (ok)
  Format: agents.md (normalized)

No model produced that number. Run it on another laptop and you get 95.6 again — a score you can't reproduce isn't a measurement, it's a vibe. And we hold ourselves to that: this repo's own badge (scored in isolation) once disagreed with its own CLI (scored in-repo) by ~15 points on the same bytes, because structure was crediting an on-disk references/ neighbourhood instead of the file's content. We fixed the engine, not the file (#129) — now cp AGENTS.md /tmp && schliff score /tmp/AGENTS.md returns the same number as CI and the badge. (The agents.md headline weights structure·operational_coverage·efficiency at 0.4/0.4/0.2; composability and clarity are shown for information, not counted.)

The quick-start and case-study numbers here are reproducible from released schliff==8.12.0 (pip install schliff==8.12.0 or uvx schliff@8.12.0); the hydra field run below is dated to the version it was measured on.

---

A real catch

The SKILL.md for ShieldClaw — a real prompt-injection-defense skill, now archived — is Schliff's reproducible before/after. The fixtures ship in docs/case-studies/shieldclaw/; every number below is the current engine's output, reproducible with schliff score. (For the 28.7 row, score a copy of SKILL-before.md outside that directory — in place, the engine auto-discovers the sibling eval suite.)

	Score	Grade	Dimensions measured
Before, scored in isolation	28.7	F	4/7 — no eval suite; explicit ceiling warning
Before, with its eval suite	84.5	B	7/7
After fixes	94.6	A	7/7

Two separate effects, and Schliff refuses to conflate them. Adding the eval suite lifted the measurement ceiling (28.7 → 84.5) — that is coverage, not quality. The quality delta is 84.5 → 94.6, driven by composability 20 → 86 and efficiency 60 → 83 after adding scope boundaries, an I/O contract, and handoffs — structural gaps a linter can't see, caught as a number that was too low.

A second field run, against an external repo (hydra, measured on released schliff==8.4.0, 2026-07-03): 71.0 [C] → 76.5 [B] — edges 82→100, composability 56→81, fix merged upstream (Zandereins/hydra#34). Efficiency deliberately stayed at 47: a ~14k-token file against a 1,000-token budget was an informed decline, not a blind chase of the number. (External repo — not re-runnable from these fixtures.)

---

Why deterministic?

Most "AI quality" tools ask another LLM how good your prompt feels — a different answer every run. That makes the score non-reproducible (re-run it, get a different number), un-auditable (the rubric lives in a hidden prompt), and trivially gameable (write for the judge, not the user). You can't gate a release on a number that drifts. Schliff computes how good the file measurably is — the same answer every run.

Deterministic means reproducible and auditable — it does not automatically mean the number is right. Schliff's claim is narrower and checkable: the rubric is open source, every scorer is readable, the weights are a dict, and the case studies above show the score moving with real fixes. If you disagree with the rubric, you can read it and file an issue — you can't do that with a judge prompt.

Config linters tell you whether the file is valid — a list of pass/fail rules. Schliff tells you how good it is — one graded 0–100 score you can gate, diff across commits, and rank.

• Reproducible. The headline composite is computed from a canonical, versioned weight registry. Calibration is off by default, so verify and badge return the same score on your laptop and in CI.
• Auditable. Every dimension is a readable scorer in scripts/scoring/. The weights are a dict you can open. There is no hidden judge prompt.
• Anti-gaming, precisely scoped. A dedicated guard layer ([guards.py](skills/schliff/scripts/scor