The reader-and-inference review, a pilot for the next two research rounds
What it is
A second review before release, after the evidence review of AGENTS.md step 4, which it does not replace. The evidence review asks whether the figures and the reproducibility hold. This one asks: could these measurements be correct while the piece's explanation is wrong? The reviewer reads the public pieces first, as an ordinary reader would, writes down what that reader would come away believing, and only then reads the study to compare.
It runs for the next two research rounds, the rounds that release a piece or change the current account after 29 September 2026. The rehearsal of 29 September on LH012's corrected pieces (notes/R-0018.md) is not one of the two. At the first checkpoint after the second, the programme decides from the log below whether to keep it, change it or drop it.
Allowance
From 30 October 2026, the figures below are a guide, not a limit: the keeper set no spending bound for now and watches the total.
Up to seven dollars at list rates on Fable 5.1, inside the round's bound of about twenty dollars of subagent spend: about one for stage 1, about five for stage 2 and about one for the recheck. The driving session tells the reviewer its allowance, prices each stage from the transcript (harbour/hb tokens --latest) and writes the figures in the log. If the round cannot fit it beside the research and the evidence review, the driving session says in the log and the close-out what it cut to make room, or that the pilot was skipped and why. The allowance was first set at five dollars and raised to seven after the rehearsal of 29 September, which cost $13.34: $1.24 for stage 1, $6.46 for stage 2 (its own analysis of 37 libraries' version profiles, which found the problem that mattered) and $5.64 for a recheck sent to the same agent, whose whole context was re-read on every call.
How to dispatch it
- Dispatch it after the evidence review's fixes are in, on the text as it would be released.
- Give the reviewer the two task texts below, the repository path and the paths they name, and nothing else: not the working conversation, the conclusion the round hopes for, the evidence review's verdict or note, or any list of problems already known. Tell it not to read keys from the environment and to keep searches out of harbour/.session/.
- Stage 1 paths: the full piece, its short form, and the "Current focus" section of README.md. Stage 2 paths: the study's directory (record, brief, amendments, sources, data, scripts), the investigation page, programme.md's Next, and the next study's brief or, if it has none, its issue.
- The stages run in order. Where the harness can continue an agent (in Claude Code, SendMessage to the same agent), send stage 2 only after stage 1 has written its assessment and returned. Otherwise give both in one prompt and require that the stage 1 assessment be written to the note before any other file is opened. No tooling used so far can stop a reviewer opening other files, so the driving session checks the reviewer's transcript for files opened during stage 1 and says in the note what it found.
- A fresh agent is not an independent one: it runs on the same family of models and reads the same record. Its agreement is a review contribution, not independent confirmation, and no page says otherwise.
Stage 1, the task text
You are a reviewer for Lighthouse, an observatory kept largely by AI agents, which publishes what it finds in public records. This review has two stages; this is the first. Read only these files: [the full piece], [its short form], and the section "Current focus" of README.md. Do not open the study record, its brief or data, programme.md, the investigation page, any note in notes/, releases.md or the git history, and do not follow links out of these files. Then create [notes/R-nnnn.md] with a heading "What a reader would believe, written before reading the record", and under it, in no more than about 250 words: the principal finding as a careful ordinary reader would say it back; what appears to have been observed, what appears to have been explained, and what appears to have caused what; and the limits a reader would notice. Write what the pieces lead a reader to believe, not what you suspect the study says. Then stop and return the note's path; the second stage will follow.
Stage 2, the task text
Now read the study behind these pieces: [the study's directory], the investigation page, programme.md's Next, and [the next study's brief or issue]. Do not read other notes in notes/, releases.md or commit messages; they carry other reviewers' verdicts. The central question is: could these measurements be correct while the pieces' explanation is wrong? Compare what the reader would believe with what the evidence supports. Where they apply, and only if they matter, consider: a proxy treated as the thing of interest; a share read as a rate or a speed; aggregate counts read as people, machines or linked events; an assumption stated in the study that disappears in the summaries; compatibility, association or a modelled estimate presented as a demonstrated cause; a method fixed in advance that faithfully answers a different question from the one the piece appears to answer; a short form or headline implying something the full record contradicts. These are prompts for judgement, not categories to fill. You may challenge the brief itself, and you may conclude that there is no material problem. Then assess the next research decision: can the evidence it proposes distinguish the explanations in play, is that evidence available and when, and would another step materially improve understanding? Continuing, adjusting, waiting and changing direction are all valid answers. Add to the note, briefly, under these headings: What the round establishes; Material claim problems, each with the exact passage (file and quotation), the evidence (file and figure) and a proposed resolution, where a problem is material if a reader would come away believing something the evidence does not support, and "none" is a valid answer; Readability, at most three optional suggestions; Next step, a recommendation with its reasons. Keep any calculation you make as code and output under the study's review/ directory. Edit no file but the note.
After the review
Resolve every material claim problem before release, or write in the note why the evidence supports disagreeing, in a section of the note headed "Resolution, by the driving session". Readability suggestions are optional. After revising, give a fresh agent on the same model the note, the diff of the changed passages and the retained files they cite, for one recheck of the claims they affect; do not continue the stage 2 agent, which re-reads its whole context on every call, do not restart the review and do not start a second round of it. The recheck is one reading, not a chain: its own changes are listed in its note and not sent to another reader. It is not polish: it is the first reader of what the resolution writes. In the three rounds so far, 8 of its 10 changes to public claims corrected text written minutes before in response to a review, six of them sentences the resolution had introduced, and half of the eight a new reading stated more broadly or surely than its evidence, or mislabelled (studies/LH016, Finding 5). Whatever stays disagreed is written in the note and in the close-out. The next-step recommendation is weighed at the checkpoint in programme.md's Next, with the reasons for following it or not.
Log
One row per round. "Worth it" is the driving session's judgement in a line: did it change a public claim that the evidence review had passed, or improve the next research decision, for its cost?
| Round | Study and pieces | Reviewer and cost | Material problems raised | Changed before release | Disagreed, with reason | Also raised by the evidence review | Next-step recommendation, and what was done | Worth it |
|---|---|---|---|---|---|---|---|---|
| Rehearsal, 29 September 2026 (not one of the two) | LH012's pieces as corrected to version 1.1 (notes/R-0018.md) | Fable 5.1; $13.34 ($1.24, $6.46, recheck $5.64), over the $5 then set | 3: boto3's and aiobotocore's older downloads have the version profile of an installer's resolution walk, not of chosen versions; three of the four holds by identified bounds come through older dependent versions; the ten-point settling band is lenient for small shares | All 3, and the 3 readability suggestions; LH013's brief amended before its snapshot | None | None of the 3 (notes/R-0017.md passed the text) | Adjust, then continue: done (walk labelled post hoc in the record and pieces; version profile and a test of boto3's added to LH013; the installer's version named as a reader to seek; LH013 waits) | Yes, as a rehearsal: the walk changed what the piece's main case means and what LH013 measures. The known corrections were named in the pieces' own notices, so it says nothing of whether it finds problems like those; stage 1 read only what it was given, by its transcript |
| First research round, 29 September 2026 | LH014 and articles/rarer-further-down.md with its short form (notes/R-0020.md) | Fable 5.1; $5.18 ($0.55, $2.78, recheck by a fresh agent $1.85), within the $7 | 1: the short form's summary, the front page and the investigation page described 52 and 35 libraries as drawn, where 80 were drawn in each band and 52 and 35 released | The 1, on every surface that carried it (also AGENTS.md and two short notices); the 3 readability suggestions; the eve-share reading added, labelled post hoc, to the record (0.2) and the piece; the recheck's six wording fixes and one record sentence | None | None (notes/R-0019.md passed the text) | Adjust, then wait: taken (the eve share and the day-2 share as a fraction of it named for the next pace reading's brief, from about 1 November; LH012's measures for the band libraries after LH013) | Yes: the material problem was small, but the post hoc eve-share reading changed what the piece's main contrast means (a smaller share following the newest version at all, not slower movement) and tied it to the next question, which the evidence review did not do; stage 1's transcript shows it read only the piece, the short form and README.md, with two searches to find the section and one listing of changed files |
| Second research round, 29 September 2026 | LH015 and articles/recipes-follow-the-pin.md with its short form (notes/R-0022.md) | Fable 5.1; $3.95 ($0.48, $1.92, recheck by a fresh agent $1.55), within the $7 | 5: the share of projects at moments given as a share of recipes in the piece's and short form's summaries and on the front page; the lockfile split missing from the short form and the front page; the fresh-build reading given in part in the short form; a container build read as every build in the advice for a lockfile census; "more than half" true only at the count | All 5, on every surface that carried them; the 3 readability suggestions; record 0.3 with the review's two readings, made after the counts, and the per-build and per-recipe shares; the recheck's five wording fixes | None | None of the 5 (notes/R-0021.md passed the text after its own two fixes, one of which, the unread-lockfile count, sat near problem 4 but was not it) | Adjust, then wait: taken (the install steps of the same 469 pairs' CI workflows filed as issue #19 for a session before LH013 can run, in place of the record's registry study; LH013 waits) | Yes: it changed claims on every short surface that the evidence review had passed, the unit above all, and it replaced the record's proposed next study with one that bears on the frozen-list question, for about half the allowance; stage 1's transcript shows it read only the piece, the short form and README.md, with one search to find the section |
| Correction round, 29 September 2026, late evening | LH011's and LH012's pieces at 1.2, LH011's short form and about.md's "How the work is checked", after the evidence review's resolution (notes/R-0025.md) | Fable 5.1; $8.55 ($1.29, $3.93, and $3.33 for a recheck by a fresh agent of both reviews' resolutions), over the $7 by the recheck | 2: the pace piece's notice gave boto3's old-Python result without the reading that most of it looks like an installer's search; the date from which pip stops fetching each version it tries from the index was given as 22.3 (October 2022), where PEP 714 and pip's source give 23.2 (July 2023) | Both, on every surface that carried them (the synthesis's short form too, and LH013's brief by amendment 2), after the driving session read the sources itself; one readability suggestion; the recheck's five edits and the driving session's five after it | None | Neither (notes/R-0024.md passed both; the pip date sat in text the round had not changed) | Continue to issue #19, adjusted: taken (programme.md, Next; a comment on issue #19); the dip at 1.42.87 described in LH013's amendment 2 | Yes: both problems were in text that two evidence reviews had passed, and it sharpened the next study's brief; the recheck corrected three sentences written minutes before for the reviews; stage 1's transcript shows it read only the pieces, README.md and about.md, with two searches to find the sections |
| Third research round, 30 September 2026 | LH017 and articles/tests-read-the-lockfile.md with its short form (notes/R-0027.md) | Fable 5.1; $7.38 ($1.02, $2.06, and $4.30 for a recheck by a fresh agent of both reviews' resolutions, notes/R-0028.md), over the $7 by the recheck | 3: the figure and the investigation page compared the container recipes under their own study's every-build rule with the workflows under this study's any-job rule, so the workflows looked as if they install from the pin far more often (under one rule, 164 of 194 against 282 of 291, or 129 of 167 against 231 of 302); "N of M projects at those moments" counted moments; the short form stated a run | All 3, on every surface that carried them; 2 of the 3 readability suggestions; record 0.3 with two readings made after the counts, the one-rule comparison and what setup-uv's cache holds by version; the recheck's thirteen wording changes | None | None of the 3 (notes/R-0026.md passed the figure's comparison; it found other faults, three pair classes wrong) | Continue with LH013, its amendment sharpened by crossing the CI flag with Python admission, after reading what setup-uv's cache holds: taken (programme.md, checkpoint), with the cache read made at once, which showed the piece's cache caveat overstated for uv jobs, though not the review's premise for the current version | Yes: it found that the headline comparison with the recipes rested on two different pair rules, which both stages of the evidence review had passed, and its question on caches corrected a limit in the piece; stage 1's transcript shows it read only the piece, the short form, README.md's section (extracted with awk) and the rendered figure |
| PHP later-evidence round, 23 October 2026 | articles/php-emergency-first-reading.md version 1.1 and its short form, from studies/LH029/ (notes/R-0033-php-later-evidence-reader.md) | Fable 5.1; $3.98 ($0.63, $1.45, and $1.90 for a recheck by a fresh agent of both reviews' resolutions, notes/R-0034-php-later-evidence-recheck.md), within the $7 | None material; 2 wording points at the edge: "nobody has counted" stated of the world what the record says of itself; the 22 October Cloudflare edit read as a new deployment statement where it reworded one | Both, and the 2 readability suggestions | None | Neither (notes/R-0032-php-later-evidence.md passed both; its own two must-fix items were a clock sentence and the release log) | Continue LH030 as fixed: taken; of its three additions, one already in the classifier, one declined (a commit time would repeat PHP's stamp), one found moot (no 23 October LH024 read) | Yes, modestly: it found two overstatements the evidence review passed, for under a third of the allowance, and confirmed that the one inference on the public surfaces is labelled; stage 1 read README.md whole, not only its first section |
| PHP CISA round, 27 October 2026 | articles/php-emergency-first-reading.md version 1.2 and its short form, from studies/LH031/ (notes/R-0036-php-cisa-reader.md) | Opus 5.5, not Fable, at the keeper's caution about budget; $1.46 ($0.73 for both stages, $0.73 for a recheck by a fresh agent of both reviews' resolutions, notes/R-0037-php-cisa-recheck.md), within the $7 | 2: the CISA listing read as independent government confirmation of attacks, where the entries link no exploitation report and may rest on the same reports of attempts; the unlisted third flaw read as not exploited | Both, on every surface | None | Neither (notes/R-0035-php-cisa-evidence.md raised other points: an unsupported criterion phrase, the directive and ordering) | Continue LH033 without tying any change after 29 October to CISA's due date; a candidate read of CISA's criteria and 21 October alert: recorded in the continuation plan | Yes: it changed the main reading of the notice, which the evidence review had passed, for $1.46 |
| Week-one round, 30 October 2026 | articles/a-week-after-two-fixes.md version 1.0 and its short form, from studies/LH033/ and studies/LH034/ (notes/R-0040-week-one-reader.md) | Opus 5.5, not Fable, at the keeper's caution about budget; $1.42 ($0.75 for both stages, $0.67 for a recheck by a fresh agent of both reviews' resolutions, notes/R-0041-week-one-recheck.md) | 2: the July comparison left out the Node.js line that went the other way; the summaries and short form dropped the distribution caveat on the unfixed branches. The recheck changed the title's scope to the two counts followed daily, since a third count (W3Techs) was below half | |||||
| PHP compromise round, 30 October 2026 | articles/php-emergency-first-reading.md version 1.3 and its short form, from notes/php-leads-2026-10-30/ and LH031 amendment 3 (notes/R-0044-php-compromise-reader.md) | Opus 5.5, at the keeper's caution about budget; $2.75 ($1.15 for both stages, $1.60 for a recheck by a fresh agent of both reviews' resolutions, notes/R-0045-php-compromise-recheck.md); the evidence review, $2.11, is separate | 3: the reported compromise ran through firewalls carrying their own PHP on a branch PHP's fixes do not cover, which no surface said; Dream's AI sentence sat under the front page's "Before the fix" though Dream dates the attacks after it; Palo Alto's "impacted" and 0.02 per cent read as a count of compromised devices | All 3, on every surface | None | In part: the evidence review had the AI statements dated (R-0043 M1) but not moved | Adjust: a row to re-read Palo Alto's pages and weigh an appliance-vendor panel (PANW-reread-amendment) | Yes: the first problem changed what route the account says the compromise took, after the evidence review had passed it |
| LH013 later-evidence round, 3 November 2026 | the "Later evidence" notice on articles/old-versions-new-pythons.md version 1.3 and its short form, from studies/LH013/ (notes/R-0047-lh013-reader.md) | Opus 5.5; $1.72 ($0.79 for both stages, $0.93 for a recheck by a fresh agent, notes/R-0048-lh013-recheck.md); the evidence review is separate | 2: the short form's "the tests were of boto3 alone" read as though the library-wide shares were untested; the piece's notice left out that the other libraries' walk-like shapes it describes did not all keep | Both, on every surface; 3 readability suggestions; the recheck's six changes to public claims, among them "could help test" for the installer's version and 32 of 37 where a surface said the same libraries | None | Neither (R-0046 passed the record; it set the notice's limits but not these) | Adjust: a brief for one BigQuery day with a burst test beside the installer's version, row LH013-installer-brief | Yes: both changed what a reader would take from the notice, for under $2 |
| LH035 map round, 7 November 2026 | articles/who-watches-the-internet.md version 1.0 and its short form, from studies/LH035/ (notes/R-0050-lh035-map-reader.md) | Opus 5.5; $1.59 ($0.58 for both stages, $1.01 for a recheck by a fresh agent of both reviews' resolutions, notes/R-0051-lh035-map-recheck.md); the evidence review, $1.53, is separate | 2: the title, summaries and short form stated the post hoc vantage reading as the finding, where the piece's own gaps are as often about what is counted or who may read it; Epoch's completeness figure as the only one, dropping "on its own pages" (Censys's paper gives one) | Both, on every surface and in the record's opening; 1 of 2 readability suggestions; the recheck's eleven changes, seven to text the resolutions wrote (among them "about a third" recounted as about a quarter) | None | Neither (R-0049 softened the title's "single place" but kept the vantage reading in it) | Release, then widen the thin layers or move outward; the record's proposed vantage test would not separate the explanations: weighed at programme.md's checkpoint of 7 November | Yes: it changed the title's claim, which the evidence review had passed in softened form, for under $1.60. README.md has no "Current focus" section now, so stage 1 read only the piece, short form and figure |
| LH046 notices round, 9 November 2026 | "Later evidence" notices on articles/who-watches-the-internet.md and articles/what-we-can-see.md (both 1.1), from studies/LH046/ record 0.2 (notes/R-0062-lh046-reader.md) | Opus 5.5; $1.11 ($0.65 for both stages, $0.46 for a recheck by a fresh agent, notes/R-0064-lh046-recheck.md); the evidence review (R-0056) ran on 8 November and is separate; the resolution subagent, $1.09, is separate | 4: the proposed notices dropped the record's line between what the files hold and what happened on GitHub; absolutes without windows; "from mid-October" read as an onset; shares without counts or the four-a-day sample, and the August claim resting on two noon files | All 4, in the notices, the record (0.3) and both investigation pages | None | The evidence review had passed the notices' figures; none of the four was raised there | Run the December re-check with its reading fixed beforehand, and add a poll of GitHub's public events feed beside the files to tell feed from archive; weighed at LH046-recheck | Yes: every notice changed in meaning before release, at about a dollar |
| LH041 record round, 9 November 2026 | studies/LH041/ record 0.2 and its proposed lines for programme.md and investigations/ai-cost-and-spread.md, no piece (notes/R-0061-lh041-reader.md) | Opus 5.5; $0.70 for both stages; the recheck by a fresh agent (notes/R-0066-lh041-recheck.md) and the resolution subagent ($1.28) are separate | 4: the upper bound without its condition (OSV's records are a floor); maintainer groups readable as victims or worm steps though many began seconds apart; stale proposed lines (AntV as one account, no post hoc label); "thirteen waves dated" where nine have registry times | All 4, in record 0.3 and both public lines; partly disagreed on whether StepSecurity's full ChainDrop list is public (the saved page prints part; the rest needs a login) | 1, in part | The evidence review had corrected the AntV account reading in the record; the proposed lines had not followed | Read 2 cannot tell worm hops from one publishing run or one operator's token pool as written; amendment 2 (post hoc) adds the vendor lists, timing of simultaneous starts and other groupings, on the same waves | Yes: it changed what the counts are said to be before any public line |
| LH047 record round, 9 November 2026 | studies/LH047/ record 0.2 and its investigation-page line, no piece yet (notes/R-0065-lh047-reader.md) | Opus 5.5; review cost in the run's line; the resolution subagent and the recheck by a fresh agent (notes/R-0067-lh047-recheck.md) are separate | 5: 59 of the 172 fail only on a User-Agent class made after the result (113 under the brief as fixed); requests read as harm (90 got a 404, 146 from two practices); evidence on Anthropic's own bots given selectively, easier on the provider Lighthouse runs on; a TollBit breakdown the source does not give, and a cross-cohort comparison read as a trend; a stale 185 | All 5, in record 0.3, the catalogue, the investigation page, programme.md and proposals.md | None | The evidence review had corrected the Crawl-delay count (185 to 172) but not the post hoc class or the provider balance | Re-audit a later week after the shared fetcher, with an amendment first fixing the User-Agent class and the Crawl-delay count; the paper's must-not-say list is in the note | Yes: it changed the headline count's framing and the balance on Anthropic before any piece |
From 29 September 2026 a row also gives, where the diff lets the driving session count them, the changes to public claims made by stage 2 and by the recheck, and how many of the recheck's corrected text written during the round in response to a review. For the three research rows above they were 4 and 1, of which 1; 5 and 6, of which 4; and 8 and 3, of which 3, one of those three written partly in the corrections after the blind check (studies/LH016, data/changes.csv and data/origin_check.txt). For the correction round of 29 September, late evening, they were 3 and 3, of which 3, by the driving session's count from notes/R-0025.md. For the third research round of 30 September they were 0 and 13, of which 12, the other a unit left from the draft in a sentence the resolution renumbered, by the driving session's count from notes/R-0028.md.
Decision after the two rounds
Made by the driving session on 29 September 2026, evening, at the checkpoint after the second round, as the note's opening asks. Both rounds changed public claims that the evidence review had passed, and the second also changed the next research decision, at $5.18 and $3.95 against an allowance of $7. The review is kept as a standing step after the evidence review and before release, with the same dispatch, allowance and log, and AGENTS.md step 4 now says so. The log continues; the programme looks at it again after three more rounds, or sooner if a round finds nothing.
Known-answer replay, 29 September 2026 (studies/LH016)
LH016 replayed this review on LH011's and LH012's pieces as first released, at version 1.0, before the corrections that readings relayed by the keeper prompted, whose points were fixed in advance as known answers (studies/LH016/amendments.md, amendment 2): once with the task texts as they stand (LH011's piece) and twice with the stage 2 list of prompts removed (LH011's and LH012's). Each run found the relayed reading's central point on its piece: a low share of downloads read as a slow spread on LH011's, and boto3's older versions not chosen by its Python on LH012's. Of 16 known answers, 5 were found, 7 in part and 4 missed, among them the title's unbounded "never" and the first-day reading's missing label, which the evidence review's rules have targeted since the relayed reading of 28 September. On LH011's piece the list of prompts made no difference to what was found. It is kept, since it cost nothing either way; extending it after an escape is no substitute for a fresh reading. The runs also found two material problems still standing in released pieces (issues #21 and #22, corrected on 29 September 2026, late evening), and a reader review of the pieces that never had one is filed as issue #23. They cost $3.59, $4.98 and $4.82 at list rates (studies/LH016/data/costs.csv). Three runs on two texts are cases, one draw each, not a rate.
From notes/reader-review.md in the repository, last changed 9 November 2026.