The model, run against the case it came from
Daniel Blum's system supplied nine of this scale's thirteen gates. Scored against it, he clears every one of those nine — and stops at Level 3, two rungs below where the shape of his setup suggests he sits.
One gate does it. He is the one who describes the behaviour it tests, and he describes himself failing it: it'll give me a draft and then I'll change it and I'll just send it.
That is accept-or-rewrite, which is exactly what G9 is looking for the absence of.
Whether that is the instrument catching something real or the instrument being wrong is the interesting part, and it is treated below rather than resolved away.
This report scored Blum against v1 of the scale and returned Level 3. That scoring did its job: it exposed two defects in the instrument, and both are now fixed. Under the calibrated v2 he scores Level 4, blocked by a single gate — written, handable review criteria — which is the same underlying weakness this report identified, now sitting at the level where it belongs.
The page is kept as-is because it is the evidence for the calibration, not because the Level 3 verdict still stands. See the calibration record for what changed and why, and note that the “built high on a thin base” flag below turned out to be an artifact of the misplaced gate rather than a fact about him.
Blum never took this assessment. He was demonstrating a system on a podcast, not answering thirteen questions about it. Every score below is a third-party reading of a 46-minute transcript, assigned by me, against questions he was never asked. He would likely answer several of them differently — and on G9 in particular, he might reasonably say the criteria live in his skill files rather than in his head.
Every score below carries the verbatim lines it rests on, with timestamps, so you can overrule me. Three of the fourteen items are inferred rather than stated: he never discusses judging model output in his own domain (G2), never mentions choosing between model tiers (G6), and never describes a written definition of a good result (G12). I scored those from adjacent evidence, quoted the exact lines I used, and stated underneath each one what I inferred and where the inference could break. The sensitivity table at the end shows what happens if I got them wrong — and the answer for G2 is a lot.
Evidence grading
Reusable prompts, instructions or workflows rather than retyping.
Yes, saved somewhere I can find them10:12…one recurring task which is a composition of several skills called the weekly prep.
08:12I just created this skill on Cowork called demo and I just told it replace all the terminology and specifics with generic terms… in one prompt I was able to replace my manager's name with your manager.
28:58Many of my skills actually ended up being from this flow.
You notice when output is wrong in your own area, and can say what is wrong with it.
Reliably, and I know where it tends to fail30:32I've built a skill called improve which is basically like a very critical auditor for these… and I ask them like is this real, is this powerful, how does this adapt to what we have now, is this needed — and then it helps me review it and together we basically decide whether we want to build it or not.
22:24You're going to lose trust and you're going to be very skeptical and then you're going to double check everything.
17:30A term I didn't know — the settlement cap. This is some payments thing…
What I inferred from this: that a person who runs written tests against incoming claims, who describes a phase of systematically double-checking output, and who holds payments vocabulary the model lacks, can also judge model output in his own domain. He never says so. The three quotes are adjacent evidence, not the claim itself.
Written context about your work that outlives the session.
Yes, covering several areas and kept current09:02There's a context file around every topic that I have, and every area of my work and my goals and my colleagues.
09:42It updates itself on a recurring basis… every few weeks it runs and updates the knowledge files.
The strongest single item in the set.
Share of screen-based work inside one AI-connected system.
More than 40%19:32I manage about 70 to 80% of my time in front of the computer with Cowork alone.
Clears the top band by a wide margin; the threshold could be raised without troubling him.
The AI reaches your actual tools, not just what you paste in.
Most of the ones that matter10:35What this series of skills does, it pulls in information from my entire ecosystem — from my Notion, my calendar, Slack, from Granola transcripts of meetings.
20:17There's more and more MCPs and connectors… but the Chrome connector is a critical critical piece of the puzzle for when you don't have these connectors and integrations.
You choose deliberately between models or modes per task type.
Yes, it's a deliberate call each time29:07It's suggesting I create a Claude design handoff skill between Cowork, where I do the research, and design, where I actually build the prototype… I have a skill like this, by the way, that's handoff between Cowork, which I use kind of as the executing arm.
20:09In the beginning I was using Chrome a lot… as time proceeds and there's more and more MCPs and connectors then obviously those are comfortable.
What I inferred from this: that purpose-built handoff skills routing work between Cowork and Claude Code by task type, plus a stated shift from browser to connectors, counts as choosing modes deliberately. He never mentions choosing between model tiers — if the gate is read that strictly, there is no evidence for it at all.
Some AI work runs on a schedule, without you starting it.
Yes, at least one runs regularly10:59Weekly prep runs on Sunday mornings
. 13:05The morning brief is a scheduled task. 27:47The self-improvement loop is a scheduled task that runs weekly
. Three, not one.
The AI writes into a system you rely on, not just a chat window.
Yes, it writes there directly19:15My goal is to have Notion as something I only look at and Claude manages on my behalf.
07:17Cowork built the board itself after getting tired of
his messy Google Doc.
You review output against criteria set in advance, instead of accepting it or rewriting it wholesale.
Sometimes, informally28:10When I write with Claude, it'll give me a draft and then I'll change it and I'll just send it.
He names this as a problem and builds a loop to recover the lost signal afterwards — which is compensation for missing criteria, not criteria. The one item scored against him, and scored on his own words.
Your setup flags context it is missing instead of guessing.
Yes, it raises them unprompted17:30Live on camera: a term I didn't know — the settlement cap… I read the thread and understand it's a limit. Want me to save?
The agent raised it; he approved; it went into the context files.
Friction is captured automatically and reviewed on a cadence.
Yes, recorded and reviewed on a cadence29:40All of my skills and recurring tasks have lines in their files to collect feedback and friction… I don't need to do this proactively.
29:48Every week it surfaces the top frictions.
You could hand someone a written definition of what a good result looks like.
Yes, it's written and specific36:36About a year ago I built our spec writing gem, call it Spectacular… back in the day, holding up the template was something really hard to achieve, and making it not hallucinate in a long spec was something that's hard… after I built something that was really powerful and good for the way I was working, I was excited to distribute it.
30:46Is this real, is this powerful, how does this adapt to what we have now, is this needed.
What I inferred from this: that a distributed gem enforcing a template and suppressing hallucination, plus a skill carrying four written tests, amounts to a definition of a good result he could hand over — because he did hand it over. But the quote describes an artifact, and the lesson he draws from it is about onboarding UX, not quality criteria. This is the most generous of the three inferences.
Someone else is running a version of your setup.
Yes, someone else actually uses it36:13Workstation is a plugin so everybody has it in our library and anybody can access it
. 36:21My colleagues have taken it to the next level.
It started for PMs and spread to all Melio workers.
Did you go through a stretch where this felt slower before it felt faster?
Yes, and I came out the other side22:01The ROI is not going to look great.
22:24You're going to lose trust and you're going to be very skeptical and then you're going to double check everything. But the value tremendously compounds.
The result, as the instrument produces it
Scored at Level 3
What this exposes
Scoring an instrument against the case its markers came from is a stress test, and this one produced a result worth arguing with rather than accepting. The readings are not mutually exclusive.
This is the reading I find hardest to dismiss, because it comes from his own account rather than from my scoring. He describes the draft-and-rewrite habit as a problem he had to engineer around. Everything he built to address it — the loop that finds unanswered drafts and diffs them against what he actually sent — is retrospective. It recovers the signal after the fact instead of setting the standard before it.
That matters because the wiki's own claim, from three independent directions, is that evaluation is the terminal skill — the thing that separates people who direct fleets of agents from people who thrive doing it. On that claim, an otherwise exceptional system with a thin evaluation layer is exactly what the scale should flag. It caught the real weak point.
Level 4 is defined by delegation: work runs without you, and the agent writes to things you rely on. Blum does both, emphatically. Written evaluation criteria are a compounding-grade practice — they sit naturally beside friction logging and distribution, which are already L5 gates. Putting them at L4 makes the delegation level depend on a discipline that most people reach later.
Move G9 to L5 and he scores Level 5 with one open gate — which matches both the shape of his system and the intuition of anyone watching the episode. That is a real defect in v1 and I would fix it before anyone used this on themselves.
Twelve of thirteen gates met, and the level is 3. That is the ladder logic working as designed — you are at the highest level whose gates you have all passed — but it means a single "sometimes, informally" outweighs eleven emphatic yeses. For a diagnostic that names one next action, strictness is a feature. For anything resembling a rating, it is a liability.
The honest fix is not partial credit, which would blur the gates back into a score. It is to report the level and the shape — "Level 3, with all four Level 5 gates met" — which the thin-base flag already does, and which the headline number does not.
Sensitivity
Three items were scored from adjacent evidence rather than direct statement. Two of them barely matter. One would move the result by two levels — which is itself a finding about where this instrument is fragile.
| Item | What I inferred | If it were scored one step lower |
|---|---|---|
| G2 F | That his critical-audit habit and domain knowledge extend to judging model output in payments. | Level 3 → Level 1. G2 is a Level 2 gate, so failing it collapses the whole ladder. The single most load-bearing inference in the scoring, resting on the weakest evidence — a genuine fragility in the instrument, not just in my reading. |
| G6 F | That routing between Cowork and Claude Code counts as choosing modes deliberately. | Level 3 → Level 2. Defensible either way. If G6 is read strictly as model-tier selection, he has no evidence for it at all. |
| G12 F | That Spectacular and the Improve skill amount to a written definition of a good result. | No level change — it is a Level 5 gate and he is already blocked below it. Axis F would fall from 88% to 75%, widening the structure-versus-judgment gap the third flag describes. |