Outside review

Claims-ledger reframe check, notes for each item (2026-10-19)

A working document. It is one of the documents the site is built from, published as it is written for anyone checking the work, and it uses the project's own shorthand. The plain account of the same material is on the home page.

Claims-ledger reframe check, notes for each item (2026-10-19)

Source checks for pull request 27 at commit 93f1e6d by helpers of the checking session. Summary and verdict: claims-ledger-reframe-check-2026-10-19.md beside this file.

Undated words: the six hardest cases

Round 4, undated words, batch 1: six records checked at their sources

Checked 2026-10-19 against the pull request 27 checkout (read only; nothing edited there). Captions were fetched with yt-dlp into new folders dl4-u1-1 (Pichai), dl4-u1-2 (Suleyman) and dl4-u1-3 (LeCun); X posts through api.fxtwitter.com and X's own embed service (cdn.syndication.twimg.com) into dl4-u1-4.

A problem across the whole folder: quote length

The brief's rule is a quote of at most 25 words. The validator's check for undated words (scripts/validate.py, the undated-words function from line 784) never tests length; only timing claims are tested (line 688). Ten of the eleven files in data/claims_undated/ are over 25 words: Pichai 31, Mensch 39, Altman 41, Huang 29, Hassabis 41, LeCun 45 (48 counting the [...] markers), Suleyman 30, Zuckerberg 41, Nadella 29. Only Musk (5) and Legg (13) are within it. Either the quotes are trimmed and the validator gains the check, or the method says plainly that the limit does not apply to undated words. I count this once here, not as a fix on each item below, but trims are suggested where they help.


1. Pichai, "sooner rather than later" (Decoder, 2026-05-26): FIX NEEDED (small)

2. Nadella, "today's Super Intelligence systems" (X article, 2026-10-10): FIX NEEDED (drop the record)

3. Musk, "That will be Grok 5" (X reply, 2026-09-14): CONFIRMED, with one wording fix to about

4. LeCun, "it will take time" (Sciences Po, 2026-09-16): FIX NEEDED (the cut is not honest)

5. Suleyman, "superintelligence further out" (Decoder, 2026-09-17): CONFIRMED (suggest the uploader's track)

6. Huang, "AGI has arrived" (X reply, 2026-09-06): CONFIRMED


Counts

CONFIRMED 3 (Musk, with a one-phrase wording fix; Suleyman; Huang). FIX NEEDED 3. COULD NOT VERIFY 0. Cross-cutting: ten of eleven undated-words quotes are over the 25-word limit, and the validator does not check it.

Undated words: the other five, and the 'none found newer' searches

Check of the claims-ledger reframe (pull request 27): five undated-words files, and four "nothing newer" searches

Read only. Nothing in the pull request's checkout was changed. Checked on 2026-10-19. New downloads are in scratchpad folders dl4-u2-1 to dl4-u2-14, dl4-u2-web and dl4-u2-liang.

A problem that affects four of the five files: the quotes are too long. The brief, and the method's field table for claims, cap a quote at 25 words. The four spoken quotes run 41 words (Altman), 41 (Hassabis), 39 (Mensch) and 41 (Zuckerberg). Legg's is 13. The validator does not catch this: check_words in scripts/validate.py does not apply the 25-word limit (CLAIM_QUOTE_WORDS) that check_claims applies at line 688. The method's "Undated words" section does not state the limit either. Either cut the quotes and add the check, or have the method say plainly that the limit does not apply to undated words. Suggested cuts, each copied exactly from the source, are given under each item.

What the page shows. I ran the page's own selection logic (leaders_now in scripts/chart_style.py and the test at scripts/site.py line 1127) again over the data. All five files come out newer than the person's latest dated forecast of AGI or beyond, so all five are shown:

YouTube refused caption downloads with "429 Too Many Requests" for most of this session (I made more than six attempts per video, spread over about 30 minutes). Where that stopped a check, I say so.


Part A

1. Altman, 2026-09-01, "superintelligence may happen short-term": FIX NEEDED (quote length); exact wording COULD NOT VERIFY

2. Hassabis, 2026-09-11, "a few years from full AGI": FIX NEEDED (quote length); exact wording COULD NOT VERIFY

3. Legg, 2026-09-16, "AGI on the horizon": CONFIRMED (one point for the master)

4. Mensch, 2026-06-03, "AGI a direction, not an arrival": FIX NEEDED (quote length)

5. Zuckerberg, 2026-09-23, "personal superintelligence within reach": FIX NEEDED (quote length)


Part B

Dario Amodei, after his latest dated forecast of 2026-06-10: nothing found that qualifies; search log incomplete

His four "words" search rows (amodei-words-01 to 04) cover:

What I read:

Found but could not read (YouTube 429 on captions). These are newer than 2026-06-10 and are not in the log; each should get a log row:

Coverage of the Dreamforce session of 2026-09-15 (kioncentralcoast.com, cryptobriefing.com) describes it as about safety and slowing down. No timing words are reported.

Verdict: "none found newer" holds for what could be read. The log is incomplete, and five short clips remain unread.

Ilya Sutskever, after his latest dated forecast of 2025-11-25: CONFIRMED, nothing found

Liang Wenfeng, after his latest dated forecast of 2024-07-22: CONFIRMED, nothing primary found; leaving out the investor-meeting transcript is right

Kai-Fu Lee, after his latest dated forecast of 2026-09-23: CONFIRMED, nothing found (one borderline line, two small log fixes)


Counts

Why each FIX NEEDED:

The three possibly-missing dated claims

Three possibly missing dated claims (pull request 27 check, read only)

Checked 2026-10-19 against the checkout at scratchpad/pr27 (commit 93f1e6d). Rules applied: the admission rule in docs/claims-ledger-method.md (looks forward, about AI, carries a date or a length of time, primary source); John's readings of 2026-10-19 ("a few years" and "as little as X" are undated; a forecast that something will NOT happen by a date is a claim); and the page code in scripts/chart_style.py. In that code, a person's "what they say now" entry holds only the drawn forecasts (AGI, superintelligence or AI better than people; no chance tails) from the latest day they gave one. nearest ignores "not before" forecasts. The "latest dated prediction of any kind" line takes the newest claim of any kind by its date said.

Nothing in the checkout was edited.


1. Mustafa Suleyman, Decoder (The Verge), 2026-06-08: not in five years, the singularity decades away

Verdict: FIX NEEDED. A dated claim is missing, and it must go in before the page publishes.

Source. YouTube, channel "Decoder with Nilay Patel", upload date 2026-06-08. Title: "Microsoft AI chief thinks superintelligence is near, but won't take your job | Decoder". https://www.youtube.com/watch?v=Z4bwAjR7azM . I read the automatic captions (the original-language track) as text with timestamps on 2026-10-19. Captions are in scratchpad/dl4-m-1/, flattened copy in scratchpad/scripts-m/decoder0608.txt.

The question (01:08:04 to 01:08:30). The host cites Demis Hassabis saying "we are in the foothills of the singularity" and asks whether Suleyman agrees.

His words.

Does it meet the admission rule? Yes.

One judgement to flag. "This" in "I don't think this is something…" follows a sentence about society's job shifting to deciding what to invent. Its plainest reading is the singularity: that was the question, and the next sentence names it. The entry should quote both sentences and say so in its notes.

What it says arrives, and in what shape.

Is it his latest dated forecast of AGI or beyond? Yes. His drawn forecasts in data/claims/:

2026-06-08 is newer. His only later timing words are the undated record of 2026-09-17 ("superintelligence … further out into the future"). The 2026-09-25 claims are about Microsoft models (other).

Effect if it is left out. Because the page keeps only the latest day's drawn forecasts, adding this record changes his "now" entry:

Without the record, the page shows a February forecast as what he says now, when he spoke on a dated horizon four months later.

Side note for John: entering it also pushes his February AGI date off the "now" line, although the two do not contradict each other. Whether the "latest day" rule should keep the latest forecast for each kind of arrival is a question for him. Under the rule as written, the record must go in.

Other words in the same episode.

Search gap. The claims search log has no row for this episode. suleyman-claims-040 read the Decoder episode of 2026-09-17 and two CNBC uploads, but not Z4bwAjR7azM. Only the undated-words search (suleyman-words-03) found it, and it flagged the episode.


2. Mustafa Suleyman, The Rest Is Politics: Leading, 2026-09-28: healthcare in 18 months to 5 years

Verdict: CONFIRMED that a possible dated claim exists. It is not about AGI or beyond, so it can follow.

Source. YouTube, channel "The Rest Is Politics: Leading", upload date 2026-09-28. Title: "Can AI Stay Under Human Control? | CEO of Microsoft AI, Mustafa Suleyman". https://www.youtube.com/watch?v=_VyF0naZ4AI . YouTube refused the first caption download (HTTP 429, too many requests). The second try returned the original-language automatic track. Captions are in scratchpad/dl4-m-2/, flattened copy in scratchpad/scripts-m/trip0928.txt. The claims pass's row suleyman-claims-045 recorded this episode as not read.

His words.

Does it meet the admission rule? Borderline, leaning yes. The firm forecast is undated ("the next few years", reading 3). The dated words are a hope (18 months) and an outer possibility (5 years). Read together, they give a range of about 18 months to 5 years: due_early about 2028-03, due about 2031. If it is judged too hedged, the method says a truly borderline case stays out and is logged as borderline. The energy, materials and drug-discovery line ("5 to 10 years") is a second possible claim of the same kind.

What it says arrives. other. The thing forecast is a health prediction model, "a coding moment for healthcare". "Super intelligent healthcare" is his name for its effect in one field, not general superintelligence. This matches how suleyman-words-06 read his Bloomberg "medical super intelligence" (one field only). It is not drawn.

Is it his latest dated forecast of AGI or beyond? No, it is not about AGI or beyond. Two other effects:

Elsewhere in the episode:

Can follow. The figure and the home-page sentence are unaffected.


3. Shane Legg, 50% chance of minimal AGI by 2028, reported by the Financial Times, 2026-09-16

Verdict: COULD NOT VERIFY a primary source. It repeats a forecast already in the ledger, so it can follow.

What exists.

Does it meet the admission rule? Not as a primary-source claim. It is coverage only, with the forecast in the reporter's words. At most it could be noted as a coverage_only sighting.

What it says arrives, and in what shape. agi ("minimal AGI"). A 50% chance by 2028, which the ledger reads as year 2028.

Is it his latest dated forecast of AGI or beyond? It would be on its date, but it is an unchanged repeat. The ledger already holds "50% by 2028" in 2010-12-10-legg-human-level-agi-mean-2028. Its repeated_on list runs to 2025-12-11, including "are you still 50/50 by 2028?" "Yes" on Google DeepMind's podcast. That same 2025-12-11 episode gives his other current forecasts: minimal AGI in "about two or so" years (2027), a 1-to-5-year range, and full AGI within a decade.

So his "now" entry already shows 2028 among his latest forecasts, in his own words. His undated record of 2026-09-16 ("AGI is on the horizon") is already shown as newer words, and its notes record this FT report.

If 2026-09-16 were added to repeated_on from coverage, it would become his latest day. The page would then show only the 2028 record and drop the 2027 and full-AGI forecasts from "now". That would make his entry thinner, not more correct, and would rest on a paraphrase.

Can follow. Leaving it out does not make the "now" figure wrong.

A small fix to the search log. legg-claims-013 says the 2026-09-16 remark "carries no date". In the coverage it does carry one (the 50%-by-2028 paraphrase). It is simply not in his own words. legg-words-03 describes it correctly.


Counts

Reason for the FIX NEEDED: on Decoder of 2026-06-08 (01:09:12) Suleyman said the singularity is not coming "in the next 5 years". That is a dated "will not happen" forecast of beyond-superintelligence, and it is newer than his 2026-02-12 professional-AGI forecast. Under the page's latest-day rule, his "what they say now" entry and the computed headline are wrong until it is entered. The claims search also never read this episode.

Rendered from docs/reviews/claims-ledger-reframe-check-2026-10-19-notes.md at version a7bd17a.