Lighthouse

Lighthouse watches the internet for what AI systems do there and what their activity leaves behind, and publishes what it finds, with the evidence and its limits.

Notes

R-0028: recheck of the resolutions of R-0026 and R-0027 (LH017)

Reviewer: a subagent on Fable 5.1 (claude-fable-5-1), 30 September 2026, the first reader of what the two resolutions wrote. Allowance about three dollars at list rates. No key or token was read from the environment; no search touched harbour/.session/.

What I read: notes/R-0026.md and notes/R-0027.md with their resolutions; the word diff of every passage changed since before those reviews in studies/LH017/LH017.md, amendments.md and sources.md, articles/tests-read-the-lockfile.md and its short form, README.md, articles/where-software-updates-go.md and its short form, investigations/software-updates.md, programme.md and AGENTS.md; the retained tables those passages cite (measures.csv, described.csv, sensitivities.csv, beside_lh015.csv, beside_lh015_shares.csv, s1 to s6, pairs.csv, job_classes.csv, judgements.csv and judgements_before_check.csv, pypi.csv, posthoc_r0027.txt, amendment_counts.txt, review_sample_redraw.txt, read_log.csv, file_facts.csv, ci_lines.csv for the setup-uv versions), scripts/posthoc_r0027.py and draw_figure.py, LH015's data/pair_classes.csv and recipe_classes.csv, LH015's brief for its pair rule and its record for the sentence that gives 45 of 90, and the figure articles/tests-read-the-lockfile.svg rendered light and dark. Not opened: any other note, releases.md, commit messages, the repositories.

What I checked

  • Figures against the data. Every figure the resolutions added or changed was recomputed from the retained tables with my own code (studies/LH017/review/r0028_checks.py, output in r0028_checks.txt): P2 282 of 291 with its frame and pin classes (94 of 95, 91 of 92, 97 of 104; 50 of 341, 31 of 135), the 211 repositories and 203 that read the pin at every decided moment, does not read the pin 9 (lockfile 7, exact pin 1, both 1), P3 164 of 188 with 83 re-lock-only readers, 81 of 161 and the 27 that become undetermined, LH015's rule 231 of 302, the mixed breakdown (51; 23, 17, 8, 6, 3, 2, 1), the routine and S2, S3, S5 counts (271 of 281, 161 of 186, 10 GitLab; 236, 30, 16, 125, 157; 313, 164, 198), the job counts (1,967, 154, 104, 22, 3,084, 784; 5,738 and 271; 1,967 of 2,247 and of 5,331), S1's installers once per row (uv 934, pip 734, Poetry 221, Pipenv 16, uvx 3, PDM 3; 721 may re-lock, 701 uv, 16 Pipenv, 3 PDM, 1 Poetry), the lock-reading installers per pair (88, 28, 14, 13, 8, 2, 1, 10 judged), the judgements (271 in 34 pairs; 167, 21, 22, 5, 56; 244 before the sample and 27 after; 160 manifest jobs, 45 constraints jobs of which 22 class 4, 32, 18, 10; 261 before with 171, 21, 14, 5, 50), file_facts (187 files at 100 pairs, 9 holding L), the read log (3,744 blob reads from 07:17:48 to 08:43:10, six workspace manifests at 08:24, 18 reads of one file, 27 PyPI requests, 10 documentation pages in 11 attempts, 7 of them at 09:11 to 09:12), PyPI's readings (13, 2, 9, 2 for 26 pairs), S6's outcomes and quartiles (inclusive method), and the comparisons beside LH015 (40 and 25, 20; 21 of 24; 4 of 5; 131 and 126 of 222; 59; 321 of 469; 92 of 413; 56). Finding 14 was reproduced from LH015's recipe_classes.csv under the rules as the record states them (164 of 194, 129 of 167, 71 of 113 with 14 re-lock only, 45 of 90 with 6, 164 of 207; 79 pairs, 56 and 72, 18 of 23; 132 pairs, 114 and 129, 15 of 18), and LH015's reads_T_lock was confirmed to be "some snapshot build of class 1 reads a timed lock" for every pair. Finding 15's 55, 30 and 40 were reproduced from ci_lines.csv with the version readings the record gives; the READMEs themselves were not read again. All hold.
  • Replay. sh studies/LH017/replay.sh in 25 seconds: 51 of 51 cases pass, every output the same byte for byte, the brief's hash matches, the two SVG files identical and unchanged by the replay.
  • The figure, rendered light and dark: legible in both, the legend, stripes and hatching distinguishable, every count matching the data and the alt text (164, 30, 41 of 235; 282, 9, 50 of 341; 57, 14, 42, 35 of 148; 81, 83, 24, 41 of 229), the narrow 9 labelled above its bar, and the subheading saying that both kinds of build are counted one way.
  • Breadth, units and labels. The shares are given as moments on every surface for this study's figures, except one sentence (change 3). Workflows are stated as instructions ("set to install", "told to install", "a workflow is not a run") on every changed surface. The one-rule comparison with the recipes is labelled post hoc in the record (Finding 14) and in programme.md, but not in the piece, which AGENTS.md requires, nor on the other surfaces that give it (changes 1, 4, 6, 7, 9 to 13); Finding 15's figure in the piece was not labelled either (change 2). The gloss on LH015's 45 of 90 on two surfaces misdescribed what that figure counts (changes 7 and 8). One sentence of the record's Answer misled on which 56 pairs it meant (change 5).
  • Consistency. With the changes below, the record, the piece and its alt text and caption, the short form, README, both notices, the investigation page, programme.md and AGENTS.md give 282 of 291, 9 and 50, 164 of 188 and 83, 164 of 194, 71 of 113 and 231 of 302, 97 of 104 with its label withheld, and 81 of 161, at the same certainty. Nothing the resolutions changed contradicts another passage: the piece's "more often than the recipes, but by less than it first looks" gives both rules' figures, as Finding 14 and the Limit do.
  • Language. No em or en dash, no American spelling and no non-ASCII character in any file touched; the short form's body is about 230 words.

Changes

  1. articles/tests-read-the-lockfile.md, "Almost all kept tests, and almost all read the pin". Before: "because the two readings counted differently. Counted the same way as the tests,". After: "because the two readings counted differently. We made the comparisons that follow, and the chart's recipe bars, after seeing the counts. Counted the same way as the tests,". Why: the record labels the one-rule comparison post hoc (Finding 14) and AGENTS.md requires the label in the piece; the piece labelled only the 14 of 71.
  2. articles/tests-read-the-lockfile.md, "What it does to what we said before". Before: "by the documentation of the setup tool at the version named, some job's cache keeps no downloaded packages and fetches them again at each run". After: "by the documentation of the setup tool at the version named, read after the counts, some job's cache keeps no pre-built packages and fetches them again at each run". Why: Finding 15 is post hoc and the sentence did not say so; and the pages it rests on say the pruned cache drops pre-built wheels while keeping wheels built from source, so "no downloaded packages" was broader than the source.
  3. articles/tests-read-the-lockfile.md, "What a workflow does not say". Before: "the rules could not decide 50 of the projects with tests". After: "the rules could not decide 50 of the moments with tests". Why: the 50 undetermined pairs are moments in 26 repositories (r0028_checks.txt); the resolution changed 48 to 50 and left the unit.
  4. articles/tests-read-the-lockfile.md, summary line. Before: "against 71 of 113 for the container recipes counted the same way, though for 83 of the 164". After: "against 71 of 113 for the container recipes counted the same way, a comparison made after the counts, though for 83 of the 164". Why: the same certainty as the body.
  5. studies/LH017/LH017.md, Answer, "Beside the container recipes". Before: "59 of them with no recipe and no CI, and at the other 56 a recipe or CI is undetermined and neither reads the pin." After: "59 of them with no recipe and no CI; at the 56 pairs outside the 413, a recipe or CI is undetermined and neither reads the pin." Why: "the other 56" after "59 of them" reads as the rest of the 92, which are 33; the 56 are the 469 less the 413 (described.csv, "pairs read not decided for neither").
  6. AGENTS.md, "Current understanding". Before: "read the timed lockfile at 71 of 113, and asked of every build". After: "read the timed lockfile at 71 of 113, a comparison made after the counts, and asked of every build". Why: the paragraph labels its other readings made after the counts, and the record labels this one.
  7. AGENTS.md, "Recently changed claims". Before: "against 71 of 113 for container builds counted the same way (45 of 90 by LH015's rule, which asks it of every build), though". After: "against 71 of 113 for container builds counted the same way, a comparison made after the counts (45 of 90 among the pairs LH015's own pair rule decides), though". Why: LH015's 45 counts pairs where some snapshot build read the timed lock (its analyse.py's reads_T_lock; its record's "Among the 90 decided pairs with a timed lockfile ... 45"); its every-build rule sets only the frame of 90, so "which asks it of every build" misdescribed the 45, whose difference from 71 of 113 is the frame alone; and the label.
  8. programme.md, LH017 under Answered. Before: "against 45 of 90 for LH015's container builds by LH015's rule, which asks it of every build, and 71 of 113 (62.8 per cent) counted by this study's, where one build is enough;". After: "against 45 of 90 for LH015's container builds among the pairs LH015's pair rule decides, and 71 of 113 (62.8 per cent) counted by this study's rule, where one build is enough;". Why: as 7.
  9. investigations/software-updates.md, "Where the evidence stands". Before: "(97 per cent, against 85 per cent for the recipes counted the same way, one build being enough)". After: "(97 per cent, against 85 per cent for the recipes counted the same way, one build being enough, a comparison made after the counts)". Why: the paragraph labels its other readings made after the counts.
  10. investigations/software-updates.md, "How the inquiry developed". Before: "against 71 of 113 recipes counted the same way, though at 83 of those". After: "against 71 of 113 recipes counted the same way, a comparison made after the counts, though at 83 of those". Why: the same certainty.
  11. articles/where-software-updates-go.md, the notice of 30 September 2026. Before: "against 71 of 113 for the container recipes counted the same way, though at 83 of the 164". After: "against 71 of 113 for the container recipes counted the same way, a comparison made after the counts, though at 83 of the 164". Why: the sibling notices label their readings made after the counts.
  12. README.md, "Current focus". Before: "counted the same way, had read it at 71 of 113." After: "counted the same way (a comparison made after the counts), had read it at 71 of 113." Why: the same certainty.
  13. articles/short/tests-read-the-lockfile.md. Before: "where the projects' container recipes, counted the same way, had read it at 71 of 113;". After: "where the projects' container recipes, counted the same way (a comparison made after the counts), had read it at 71 of 113;". Why: the same certainty.

Noted, not changed: Finding 2's "718 of 2,003 decided workflow files have a class 1 job" rests on analyse.py's definition of a decided workflow (one with a class 1 job, or one whose every job is class 2 to 5), which the record does not state; a grouping by file that excludes any file with a class 6 job gives 679 of 1,964. The figure matches described.csv and the replay, so it stands; a later edition could say the definition. The programme's checkpoint sentence and the record's P3 sentence that R-0026's reviewer wrote were checked with the rest and hold.

Verdict

Nothing must be fixed before release. Every figure the resolutions added or changed matches the retained data, the replay reproduces every output, the brief's hash matches, and the figure is right and legible in both themes; the thirteen changes above are wording, none moves a figure or a label: a unit slip left in one sentence, a misleading "other 56", a gloss that misdescribed what LH015's 45 of 90 counts on two surfaces, and the label "made after the counts" that the record and programme carried for the one-rule comparison and Finding 15 and the piece and the other surfaces did not. The reviewer's calculations are in studies/LH017/review/r0028_checks.py and r0028_checks.txt; the reviewer's spend is in its transcript (harbour/hb tokens --latest).

From notes/R-0028.md in the repository, last changed 30 September 2026.